How good are ChatGPT, GPT-4, and other AI writing tools? Here are the results of our sales email writing experiment.
by
Xier Dang
PUBLISHED Aug 11, 2023
10Min Read
As a product marketer at a sales tech company, I think about sellers a lot.
Specifically, sales development reps (SDRs), the unsung heroes of the sales world. SDRs play a crucial role in a companyâs success â often working between 200 and 500 contacts at a time to surface new opportunities.
A host of AI-powered tools have been released recently, all promising to make an SDRâs life easier. If true, this could be transformative to businesses and SDRs everywhere. Given the potential of these products, I wanted to cut through the hype to understand how good these tools actually are.
Sure, they can write an email for you, but is it a GOOD email? How personalized can it truly be if itâs not connected to the data? And how well do these tools scale?
This article documents my experiment with 4 AI tools, and includes my honest opinion on each.
In the end, Iâll try and arrive at an answer to the question that every SDR has likely been wondering about.
Is AI here to take my job?
To arrive at an answer, I tried four tools: ChatGPT, GPT-4, Copy.ai, and Apolloâs own AI writing tool.
There are three things I graded on a scale of 1 to 10.
To run this experiment, I picked my coworker Josh Garrison as my pretend prospect. Heâs the Head of Content Marketing at Apollo.
Letâs pretend that I work at a content management software company called BestCMS. I want to get in touch with Josh and book a meeting with him. To do this, Iâll ask each AI tool to create a sequence of personalized emails that will address Joshâs specific challenges.
Using Apollo, Iâm able to find all the relevant information about Josh I need pretty quickly.
My emails should address these insights about Josh. I also know that Josh doesnât have time to read long, dry emails. So I need the emails to be concise, punchy, and straight to the point.
Now that weâve got all that out of the way, letâs dive into the four tools Iâm testing for this experiment: ChatGPT, GPT-4, Lavender, and Copy.ai.
My ratings:
Letâs look at ChatGPT.
ChatGPT is a great tool, and I love the fact that itâs free. It does everything from giving you synonyms of a word to passing the US Bar Exam. But for the sake of this blog, I just want it to write me a few personalized emails I can use in a sequence to get Josh to book a meeting with me.
Letâs start with a basic prompt and see what it does.
Prompt to ChatGPT:
Write a three-step email sequence that will convince Josh Garrison to book a meeting with me. Hereâs all the information you need. Make the emails concise. Include a catchy subject line for the first email.
- Josh is currently the Head of Content Marketing at Apollo, a sales intelligence and sales engagement platform
- He spent 2022 as the Head of Revenue at Teamflow and 2021 as a Sr. Growth Marketing Manager at Autodesk
- Apollo raised $110M Series C in 2022
- His first ever job was Inside Sales Rep
- My name is Xier Dang
- My company is BestCMS
BestCMS streamlines processes for content teams. Our solution simplifies content creation, management, and delivery, enabling teams to deliver better content faster.
The first emails ChatGPT gives me are a good start, but theyâre too long. So I ask for them to be shortened.
Then it gives shorter emails, but the personalization isnât as strong.
I do some back-and-forths with ChatGPT, asking it to make things âmore concise but still personalizedâ so that they are mobile-friendly. Finally, I get something I like.
These are good, but are they the best emails Iâve ever seen? Not quite. Iâd still take another 5 minutes to edit these before sending them out.
Although I am satisfied with the quality and personalization, and I love that it was able to give cohesive emails, it took me a few tries to get here. I had to research Josh and provide ChatGPT with all of the necessary information. In total this took about 10 minutes for just one lead.
This would never work at scale. Safe to say, ChatGPT is not taking your job.
My ratings:
While I was in the middle of writing this blog, GPT-4 was announced. Naturally, I had to try it.
Iâve already been a big fan of ChatGPT so I couldnât wait to see the improvements the team at OpenAI made with GPT-4.
I give it the same prompt as ChatGPT and click enter.
The quality of GPT-4 is (unsurprisingly) as good as one can expect from an AI writing tool, and it includes all of the relevant information I provided. The tone is friendly and conversational, which I hadnât prompted it to do. It also follows up on every email in a logical way.
The downside is, like ChatGPT, the emails are too long. Nobody likes reading chunks of text and theyâre not friendly on mobile devices. I need concise and better-formatted emails to capture Joshâs attention.
So I ask GPT-4 to remove large chunks of text and make the emails concise. After a few back-and-forths with GPT-4, I get emails that I like.
Honestly, I can use these out of the box. GPT-4 not only writes personalized copy, it recognizes Josh is a manager and includes value propositions for his team as well. I could do without the line introducing me in the first email, and I may tweak a few small things, but man - this is good.
But, like with everything, it has its drawbacks. For starters, it needs to be fed any and all data to create content thatâs personalized.
My takeaway: with the right request, GPT-4 can provide high-quality, personalized copy that gives SDRs a launching-off point to create killer messaging. But in terms of replacing SDRs themselves? I wouldnât quite count on it.
My ratings:
Next, letâs look at Copy.ai.
I notice right out of the gate that Copy.ai has a wide range of email templates to work with in their free version. As an SDR sending more than just cold emails, this is beneficial.
For this experiment, I try their Cold Outreach Email tool. It gives me a blank slate to fill out, so I plug in all the relevant information.
After clicking âCreate Contentâ it spits out a few different options.
To be frank, I wouldnât send any of these. Some are too long, most have big chunks of text that wonât look good on mobile, and they all start by introducing me.
I doubt Josh would be interested in BestCMS if I used these emails.
Whereâs the data? Where are the specific suggestions? The final product is not personalized and, ultimately, full of copy that would take a significant amount of time to revise. Copy.ai could be a good starting point, but of the four, Iâd say itâs the weakest option.
I can confidently say Copy.ai isnât taking your job.
Now itâs time to put our own AI writing assistant to the test!
Rating our own feature feels a bit subjective, so Iâll skip that part.
The Apollo AI writing tool helps SDRs save time by:
- Josh Gosnell, Lead Consultant at MediaDev
How does it produce value-based emails with no typing required?
The first time you use the tool, Apollo AI looks up your corporate domain and matches it with all the rich company data Apollo has on file. Then, Apollo AI automatically generates your companyâs unique offering and the value propositions directly within the email. Youâll have the option to update the messaging and it will be saved for the next time you use the AI writing assistant. Simple, right?
Letâs continue with the experiment. Iâll go ahead and add BestCMS' pain point, value prop, and CTA (as itâs a fictitious company that Apollo wouldnât have data on).
I hit generate and I get two subject lines and body copies to choose from. If I want more options, I can hit the ârefreshâ button, but I choose the first subject line and body copy â short and directly to the point.
Now for personalization.
Apollo AI auto-personalizes the first sentence of my email based on the specific prospect and the data Apollo has on for them. It comes up with five unique personalized openers for Josh.
Within the AI writing tool itself, I can preview AI-generated personalized openers for anyone in just a few seconds. Unlike GPT-4 and company, it requires no manual research to find something attention-grabbing, personalized, and built for scale.
A note â The openers are written for Apollo's value prop (not BestCMS) because I'm using the tool with my Apollo.io email. Donât fret! With your company email, you will have personalized copy that aligns with the values of your product.
So, will any of these tools, including Apolloâs AI writing assistant, be taking your job? No, they won't, and they probably never will.
But, with the right tool, AI can be a powerful way for you to save time AND get better results.
While the Apollo AI writing tool hasnât been in the market as long as the others mentioned above, what it offers is unparalleled:
And weâre not stopping here! Expect to see more exciting AI-powered features being launched on Apollo in the coming months. We have some pretty grand plans.
- Louis Fischer, SDR @ Superb.ai
Learn more about Apollo AI and what weâre building this year here.
Thanks for reading!
Receive insider stories and data-backed insights for elevating your work and staying ahead of the curve
You can unsubscribe at any time using the link in our emails. For more details, review our privacy policy.