Client Profile

• Industry: Direct-to-Consumer Online Wine Retail
• Client: A leading online wine retailer known for its captivating storytelling in email marketing
• Solution: Seeking to streamline content creation while maintaining brand voice, they partnered with Kenway to explore the potential of generative AI

Our client cultivated a loyal following of 27,000 subscribers through captivating email marketing that goes beyond product descriptions. Each email is an immersive experience, transporting the reader on a journey centered around a specific wine. Evocative storytelling is penned by a dedicated writing staff who meticulously follow a defined style guide. This unique voice, both informative and engaging, is a key driver of sales and a differentiator in the competitive DTC wine market. However, crafting these compelling narratives was a costly and time-consuming process.  Recognizing the potential of AI to streamline content creation, our client approached Kenway with the objective of maintaining their brand's unique voice and storytelling magic while exploring new avenues for efficiency.

The Problem

While captivating narratives were the backbone of our client's email marketing success, crafting them presented a significant challenge. Each email, meticulously crafted by a dedicated writing team, aimed to be more than just a product description; it was an immersive journey centered around a specific wine. This commitment to storytelling yielded impressive results, fostering brand loyalty and driving sales. However, the very process that fueled their success became a hurdle to further growth.

Maintaining brand consistency and crafting these narratives required a dedicated writing team to research each wine, develop the story, and meticulously adhere to a clearly defined style guide. This meticulous approach, while ensuring brand identity, limited the volume of content our client could produce. Scaling their email marketing efforts with the current process proved to be cost-prohibitive and resource-intensive.

• Storytelling: These narratives weave stories around the featured wine, incorporating details about origin, vintner philosophy, and tasting notes – all delivered in a lively and engaging style.
• Consistent Brand Voice: A clearly defined style guide ensured a consistent brand voice across all emails, a key differentiator for subscribers. Specific elements like sentence structure, tone, and figurative language were meticulously followed.
• Costly & Time-Consuming: While the human writing team consistently produced high-quality content, the current approach presented a challenge in terms of cost-efficiency. The meticulous attention to detail required for each email, including research, writing, and style guide adherence, presents a high fixed cost of content creation.
• AI for Efficiency: Recognizing the potential of AI to streamline content creation, our client approached Kenway with the objective of maintaining their brand's unique voice and storytelling while exploring new avenues for efficiency.

While the captivating narratives were effective, crafting them was a resource-intensive process. The dedicated writing staff's attention to detail, while ensuring brand consistency, was expensive and limited the volume of content that could be produced.  Our client saw potential in AI to bridge this gap. They envisioned a system where AI could create content or at a minimum generate the initial drafts, freeing up their writing team to focus on editing, polishing, and exploring new creative avenues. However, their primary concern was maintaining the unique brand voice that fueled their email marketing success.

The Solution

Before addressing any technical aspects, our first step was to convert dreams and ideas into a workable vision that garnered consensus.  Successful AI implementation requires a shared vision, and for our client, this meant fostering collaboration across various stakeholder groups beyond just the technology team.

We began by thoroughly assessing their current email marketing processes. This in-depth analysis allowed us to map out the necessary workflow adaptations and considerations that would be essential for successfully integrating an AI solution into their content creation pipeline.  This collaborative approach ensured everyone involved had a clear understanding of the project's goals and the impact on their roles.

With a well-defined vision in place, we could then turn our attention to exploring the technical solutions that would bring this vision to life.

Kenway explored two approaches to achieve our client's goals. The first approach involved leveraging a pre-trained, publicly available generative AI model.  We hypothesized that by feeding the model with the style guide, descriptions of successful past narratives, and specific wine details (name, vintage, winery, etc.), we could guide the model towards replicating the brand's desired voice and style.

Our team experimented with various prompt engineering techniques:

1. To guide the model towards replicating our client's brand voice, we experimented with incorporating the style guide directly into the prompts we fed it. This essentially provided the model with a roadmap for writing content that adhered to the specific elements outlined in the guide, like sentence structure, tone, and figurative language. 
2. We further evaluated the model's output by assigning scores based on how well it adhered to these elements from the style guide. This scoring system allowed us to gauge the model's progress in capturing the brand's desired voice and style.
3. We also explored providing the model with examples of past successful narratives and prompting it to analyze and describe their writing styles. Then, we incorporated those descriptors into the prompts for new narratives, hoping to generate content that mimicked the tone and style of those successful examples.
4. Finally, we established an iterative feedback loop. We generated content samples using various prompt engineering techniques and presented them to our client's Chief Marketing Officer and content editor. Their feedback was thoroughly documented and incorporated into the prompts, further refining the model's output.

While the pre-trained model generated factually accurate content, it lacked the precise storytelling elements and brand-specific nuances that were crucial for our client's email marketing. The narratives, while grammatically correct and informative, often fell flat, failing to capture the engaging and evocative style of the human-written narratives.

Recognizing this limitation, we shifted our approach to fine-tuning a custom AI model specifically tailored to replicate their unique writing style.

Fortunately, our client maintained a comprehensive archive of past email narratives, along with corresponding performance metrics such as click-through rates and conversion rates.  

• This rich data provided a valuable foundation for training. Each past narrative was paired with a prompt based on the original wine details, essentially transforming the archive into a vast prompt-response dataset.
• This dataset was then used to fine-tune a Mistral 5B model chosen for its cost-effectiveness, performance, and accessibility.
• The fine-tuning process involved training the model to recognize patterns and relationships within the existing narratives, allowing it to generate new narratives that mimicked the style and voice of the human-written originals.
• Ultimately, the fine-tuned model was deployed on secure AWS servers, ready to generate narratives based on user input.

What We Delivered

With the fine-tuned AI model humming in the background, our focus shifted to user experience. We wanted to ensure the writing staff could seamlessly integrate AI-generated content into their existing workflow.

Here's what we delivered:

• User-Friendly Interface: We designed a single-page web application with a minimalist interface. This streamlined approach minimized distractions and learning curves for the writing team.
• Simple Data Input: Users could easily input essential wine details like name, vintage, winery, and style (e.g., Cabernet Sauvignon, Pinot Noir). These details were then automatically integrated into a prompt sent to the model.
• Direct Integration with Email Drafting: Upon receiving the prompt, the model generated its narrative. This content was then displayed directly on the screen, ready for review.  A convenient copy function allowed users to easily transfer the generated text to their email drafts.

While developing the interface, we explored additional functionalities like:

• Style Guide Adherence Scoring: This feature would have rated the generated content based on how well it adhered to the style guide.
• User-Defined Revisions: We explored an additional feature where the web app offered a secondary input window after displaying the initial content. This allowed users to provide the model with additional instructions on how to modify its response, facilitating a more iterative approach to content creation.

However, for this project, the simple approach was chosen. The focus was on evaluating the raw performance of the AI-generated narratives with only the potential for controlled influence of human editing. This clean data allowed for a more objective assessment of the model's effectiveness.

The Result

Our client conducted a two-week experiment to assess the effectiveness of the AI-generated narratives. Their email subscribers were divided into four groups:

1. Control Group: This group received no email during the experiment, acting as a baseline for measuring the overall impact of email marketing on revenue generation.
2. Human-Written Group: This group received emails written by our client's dedicated writing staff, maintaining the existing content creation process.
3. AI-Written Group: This group received emails where the narratives were entirely generated by the fine-tuned AI model.
4. AI/Human Hybrid Group: This group received emails where the AI model generated the initial narrative draft, which was then edited by the writing staff before being sent to subscribers.

The experiment yielded valuable insights:

• As expected, the control group, which received no emails, performed the worst in terms of revenue and profit. This confirms the importance of email marketing in generating revenue for our client.
• The human-written and AI-only groups performed similarly in terms of key metrics, with the human-written group having a slight edge in total revenue. This suggests that the AI model was able to successfully replicate the core elements of our client's brand voice and storytelling approach.
• When factoring in operational costs, the AI-written group emerged slightly ahead. While there were initial costs associated with model development and infrastructure setup, these were significantly lower compared to the ongoing costs of employing a dedicated writing staff.
• An intriguing finding was the underperformance of the AI/Human hybrid group. This suggests that the human editing process may have inadvertently altered the AI-generated content in a way that diminished its effectiveness. This highlights the importance of further research into the optimal integration of human and AI content creation processes.

Conclusion

While further refinement and experimentation are needed, our client is now positioned to leverage AI to streamline their email marketing content creation process. The fine-tuned model can generate the narratives, freeing up their writing staff to focus on higher-level tasks like concept development, strategic content planning, and human-centric editing.

This case study demonstrates the potential of generative AI to optimize content creation in the marketing and advertising space. With AI Powered content creation, businesses can create compelling and brand-consistent content while improving efficiency and reducing costs.  The success of this collaboration between Kenway and our client paves the way for further exploration of how AI can be used to enhance content creation across various industries.

Ready to unlock the full potential of Artificial Intelligence? Connect with our experts today to get started.