How good are ChatGPT, GPT-4, and other AI writing tools? Here are the results of our sales email writing experiment.
by
Xier Dang
UPDATED Oct 8, 2024
10Min Read
As a product marketer at a sales tech company, I think about sellers a lot.
Specifically, sales development reps (SDRs), the unsung heroes of the sales world. SDRs play a crucial role in a company’s success — often working between 200 and 500 contacts at a time to surface new opportunities.
A host of AI-powered tools have been released recently, all promising to make an SDR’s life easier. If true, this could be transformative to businesses and SDRs everywhere. Given the potential of these products, I wanted to cut through the hype to understand how good these tools actually are.
Sure, they can write an email for you, but is it a GOOD email? How personalized can it truly be if it’s not connected to the data? And how well do these tools scale?
This article documents my experiment with 4 AI tools, and includes my honest opinion on each.
In the end, I’ll try and arrive at an answer to the question that every SDR has likely been wondering about.
Is AI here to take my job?
To arrive at an answer, I tried four tools: ChatGPT, GPT-4, Copy.ai, and Apollo’s own AI writing tool.
There are three things I graded on a scale of 1 to 10.
To run this experiment, I picked my coworker Josh Garrison as my pretend prospect. He’s the Head of Content Marketing at Apollo.
Let’s pretend that I work at a content management software company called BestCMS. I want to get in touch with Josh and book a meeting with him. To do this, I’ll ask each AI tool to create a sequence of personalized emails that will address Josh’s specific challenges.
Using Apollo, I’m able to find all the relevant information about Josh I need pretty quickly.
My emails should address these insights about Josh. I also know that Josh doesn’t have time to read long, dry emails. So I need the emails to be concise, punchy, and straight to the point.
Now that we’ve got all that out of the way, let’s dive into the four tools I’m testing for this experiment: ChatGPT, GPT-4, Lavender, and Copy.ai.
My ratings:
Let’s look at ChatGPT.
ChatGPT is a great tool, and I love the fact that it’s free. It does everything from giving you synonyms of a word to passing the US Bar Exam. But for the sake of this blog, I just want it to write me a few personalized emails I can use in a sequence to get Josh to book a meeting with me.
Let’s start with a basic prompt and see what it does.
Prompt to ChatGPT:
Write a three-step email sequence that will convince Josh Garrison to book a meeting with me. Here’s all the information you need. Make the emails concise. Include a catchy subject line for the first email.
The first emails ChatGPT gives me are a good start, but they’re too long. So I ask for them to be shortened.
Then it gives shorter emails, but the personalization isn’t as strong.
I do some back-and-forths with ChatGPT, asking it to make things “more concise but still personalized” so that they are mobile-friendly. Finally, I get something I like.
These are good, but are they the best emails I’ve ever seen? Not quite. I’d still take another 5 minutes to edit these before sending them out.
Although I am satisfied with the quality and personalization, and I love that it was able to give cohesive emails, it took me a few tries to get here. I had to research Josh and provide ChatGPT with all of the necessary information. In total this took about 10 minutes for just one lead.
This would never work at scale. Safe to say, ChatGPT is not taking your job.
My ratings:
While I was in the middle of writing this blog, GPT-4 was announced. Naturally, I had to try it.
I’ve already been a big fan of ChatGPT so I couldn’t wait to see the improvements the team at OpenAI made with GPT-4.
I give it the same prompt as ChatGPT and click enter.
The quality of GPT-4 is (unsurprisingly) as good as one can expect from an AI writing tool, and it includes all of the relevant information I provided. The tone is friendly and conversational, which I hadn’t prompted it to do. It also follows up on every email in a logical way.
The downside is, like ChatGPT, the emails are too long. Nobody likes reading chunks of text and they’re not friendly on mobile devices. I need concise and better-formatted emails to capture Josh’s attention.
So I ask GPT-4 to remove large chunks of text and make the emails concise. After a few back-and-forths with GPT-4, I get emails that I like.
Honestly, I can use these out of the box. GPT-4 not only writes personalized copy, it recognizes Josh is a manager and includes value propositions for his team as well. I could do without the line introducing me in the first email, and I may tweak a few small things, but man - this is good.
But, like with everything, it has its drawbacks. For starters, it needs to be fed any and all data to create content that’s personalized.
It also took me about 7 minutes to get to the final result. If I’m an SDR working 200 to 500 leads, that’s anywhere from 1,400 to 3,500 minutes of time spent in GPT-4.
My takeaway: with the right request, GPT-4 can provide high-quality, personalized copy that gives SDRs a launching-off point to create killer messaging. But in terms of replacing SDRs themselves? I wouldn’t quite count on it.
My ratings:
Lastly, let’s look at Copy.ai.
I notice right out of the gate that Copy.ai has a wide range of email templates to work with in their free version. As an SDR sending more than just cold emails, this is beneficial.
For this experiment, I try their Cold Outreach Email tool. It gives me a blank slate to fill out, so I plug in all the relevant information.
After clicking “Create Content” it spits out a few different options.
To be frank, I wouldn’t send any of these. Some are too long, most have big chunks of text that won’t look good on mobile, and they all start by introducing me.
I doubt Josh would be interested in BestCMS if I used these emails.
Where’s the data? Where are the specific suggestions? The final product is not personalized and, ultimately, full of copy that would take a significant amount of time to revise. Copy.ai could be a good starting point, but of the four, I’d say it’s the weakest option.
I can confidently say Copy.ai isn’t taking your job.
Now it’s time to put our own AI writing assistant to the test!
Rating our own feature feels a bit subjective, so I’ll skip that part.
The Apollo AI writing tool helps SDRs save time by:
- Josh Gosnell, Lead Consultant at MediaDev
How does it produce value-based emails with no typing required?
The first time you use the tool, Apollo AI looks up your corporate domain and matches it with all the rich company data Apollo has on file. Then, Apollo AI automatically generates your company’s unique offering and the value propositions directly within the email. You’ll have the option to update the messaging and it will be saved for the next time you use the AI writing assistant. Simple, right?
Let’s continue with the experiment. I’ll go ahead and add BestCMS' pain point, value prop, and CTA (as it’s a fictitious company that Apollo wouldn’t have data on).
I hit generate and I get two subject lines and body copies to choose from. If I want more options, I can hit the “refresh” button, but I choose the first subject line and body copy – short and directly to the point.
Now for personalization.
Apollo AI auto-personalizes the first sentence of my email based on the specific prospect and the data Apollo has on for them. It comes up with five unique personalized openers for Josh.
Within the AI writing tool itself, I can preview AI-generated personalized openers for anyone in just a few seconds. Unlike GPT-4 and company, it requires no manual research to find something attention-grabbing, personalized, and built for scale.
A note — The openers are written for Apollo's value prop (not BestCMS) because I'm using the tool with my Apollo.io email. Don’t fret! With your company email, you will have personalized copy that aligns with the values of your product.
So, will any of these tools, including Apollo’s AI writing assistant, be taking your job? No, they won't, and they probably never will.
But, with the right tool, AI can be a powerful way for you to save time AND get better results.
While the Apollo AI writing tool hasn’t been in the market as long as the others mentioned above, what it offers is unparalleled:
And we’re not stopping here! Expect to see more exciting AI-powered features being launched on Apollo in the coming months. We have some pretty grand plans.
- Louis Fischer, SDR @ Superb.ai
Receive insider stories and data-backed insights for elevating your work and staying ahead of the curve
You can unsubscribe at any time using the link in our emails. For more details, review our privacy policy.
Prospect on LinkedIn
Gather verified email addresses & phone numbers directly from LinkedIn, reach out, and see when they open your emails.