Can AI write good marketing copy? Part 3: Our comparison

This post is part of a series looking at the capabilities of generative AI for content marketing:

• Part 1 was written by our team using prompts from leadership.
• Part 2 is the same white paper based on the same prompts provided to our writers, except given to ChatGPT.
• Part 3 (below) is a comparison of our work to that of ChatGPT.

Content wise, all the points ChatGPT raises are valid. Generative AI can produce a lot of copy fast, generate idea after idea, and process tonnes of data in moments. But it isn’t great at producing content that evokes human emotions. It struggles to emulate unique brand voices. It can’t cover niche or complex topics with sufficient depth.

Where generative AI stumbles: Our analysis

One significant point missing from ChatGPT's output is the inability of generative AI to offer fresh insights, which makes it unsuitable for producing true thought leadership. ChatGPT’s white paper is also light (among other areas) in its discussion of the legal and ethical implications of generative AI. Granted, these are developing conversations that would be absent from the model’s training data. (While GPT-4 can browse the web, this feature proved buggy in our testing.) But they’re also topics with major implications for businesses. For instance, as our paper notes, it isn’t even clear who, if anyone, should own the copyright of AI-produced content.

We do have some minor mechanical gripes, like list items not being parallel in their construction. But the biggest issue with the AI-generated white paper is the tone.

Through the custom instructions feature, we gave ChatGPT directions that would be instinctual to an experienced writer. Write in a lively, engaging and conversational style (neither too casual nor too formal). Use plain language, writing clearly and concisely while maintaining technical precision. Ensure your sentences, paragraphs and subsections flow seamlessly from one to the next. Avoid repetition, clichés and overused turns of phrase.

The AI ignored many of these instructions, or else was incapable of following them. While we asked for a more natural writing style, the tone veers too casual rather than striking the middle ground between conversational and professional as our version does. And despite directions to avoid clichés, ChatGPT includes in its intro: “Generative AI, with its ability to churn out content at an astounding pace, might just be that game-changer. But hold your horses! It's not all sunshine and rainbows.” That’s just one example; later on ChatGPT channels Voltaire (or Stan Lee), advising the reader that “With great power comes great responsibility.”

It’s the kind of writing that’s more likely to elicit eyerolls and guffaws than convert customers. Less like a real human, more like a caricature of a content marketer.

And that’s exactly what the AI is doing: emulating rather than truly embodying the capabilities of a content marketing specialist. The result is writing loaded with overwrought metaphors and clichés, which would be overrepresented in the training data. ChatGPT concludes: “In the dynamic dance of content marketing, let AI be your partner, not your proxy, and together, we can choreograph a narrative that not only sells but also tells the story of who we are and what we stand for.”

Prompting the AI to write to the audience resulted in the crude (and somewhat unsettling) first line: “Welcome to the age of Generative AI, fellow marketers!” Another attempt produced a similar result, but with the audience now “marketing mavens.” Neither is a great way to open a white paper.

It’s also worth noting the struggle of getting ChatGPT to produce a long-form piece like a white paper. We asked for a white paper roughly the length of ours, around 1,200 words, yet this yielded a white paper about half that even after repeated attempts. This was despite ChatGPT’s own assurances that it could in fact output as many as 4,096 words per response. We only managed to generate a white paper of comparable length by prompting ChatGPT to write it in two parts, which introduced some repetition and structural issues.

Final word

Ultimately, the best role for generative AI at the moment is precisely what both white papers suggest: to support human content writers in the earliest stages of writing, such as with research and outlining. But even that’s debatable. Generative AI is famous for making stuff up, even stats or references you might be looking for to substantiate your content.

How Ascribe can help

At Ascribe, we’re known for producing high-quality original content, but many of our clients also rely on us for editing services. We’ve taken long, complex technical reports and turned them into concise, accessible documents for a general audience. We’ve rewritten brochures, blog posts and other copy to incorporate a strong brand voice without altering the basic content. And we’ve made countless documents more engaging and scannable with basic line-by-line editing.

Whether we’re writing or editing, our end goal is the same: to produce highly appealing, readable content that delivers the right message in the right way for each client. And we can do the same for AI-generated content. We can do line-by-line editing for mechanics and fact checking, a stylistic update to ensure your brand is well represented, or a complete overhaul if the text isn’t quite what you had in mind.

To learn more about how we can help meet your content marketing needs, contact us today.