4 Conversational AI Metrics: How to Measure AI Chatbot Performance

AI-powered chatbots accelerate the buying cycle, build relationships with customers while you sleep, and generate new revenue for your business.

At least, that’s the goal.

Kyle Bastien, Drift’s Director of Sales Enablement, said this about AI-powered chat: “At the end of the day, what matters is that this is producing revenue for the business...Unless more people are buying because you made it easier for them, none of this matters.”

In the emerging field of Conversational Marketing, you need to know if it’s worth the investment. Start with four types of conversational AI metrics that funnel up to overall business performance and ROI.

1. AI-Specific Metrics

Those who own the chat experience need to know: when someone talks to an AI-based chatbot, did they get the right answer?

Accuracy is the baseline requirement of your AI-powered chatbot. Without it, hopes of reducing customer friction and accelerating revenue aren’t realistic.

Metrics include:

• Clusters supported. This is the number of question and answer pairs you’re capable of responding to.
• Questions per cluster. For each cluster or topic, how many variations of questions is your chatbot able to answer?
• Accuracy rate. Within those known clusters and questions, how often is the bot giving a helpful and correct answer? We recommend aiming for 80% accuracy to be top quality.

These metrics are best analyzed at an individual level, assessing each conversation so you can check the bot’s work. The model learns through this exercise as you correct mistakes made in each conversation.

2. Core Chat Metrics

Core chat metrics will be familiar to marketers. These build on the accuracy of the model discussed above and include:

• Site coverage: How many people saw the chat?
• Engagement rate: How many talked to it?
• Lead capture rate (i.e. email address provided) or meetings booked: What were the outcomes of those chats?

While AI-specific metrics tell us how accurate the bot is, core chat metrics show what it’s doing for our marketing. We start to learn if the bot is responding with more than just accuracy, but with the humanity and empathy buyers expect from a digital conversation.

3. Revenue Growth Metrics

Revenue growth metrics build on core chat metrics. Are chat conversations turning into business?

The four primary metrics to assess here are:

• Chat influenced pipeline. How many new customers engaged with the chatbot during the buying process?
• Chat influenced revenue. Of new customers that engaged with the chatbot, how much revenue did that create?
• Chat sourced pipeline. How many new opportunities were sourced by the chatbot?
• Chat sourced revenue. Of the new opportunities that were sourced by the chatbot, how much revenue was created?

This is where we’ve gone beyond vanity metrics or typical KPIs and have started to show ROI from Conversational Marketing.

4. Revenue Efficiency Metrics

The most effective conversational AI programs don’t just generate revenue, they accelerate it when compared to other marketing and sales activities.

For example, one Drift customer has a 4% rate of conversion from lead to customer across all marketing channels. However, if a lead engages with AI chat, they convert at a 32% rate, or eight times higher than the average lead. For this company, not only do contacts that engage with AI chat convert more often, they also spend 33% more and purchase eight days faster.

These are the three metrics to look at:

• Average cycle length: The amount of time from the first touch to closed-won.
• Average contract value: The average spend of the customer, either on a monthly or annual basis.
• Win rate. The conversation rate from lead to new customer.

When compared to other lead sources or against contacts that don’t engage with chat, you can identify the true efficiency of your AI-enabled Conversational Marketing and chat program.

To learn more, check out the Conversational Marketing Blueprint, which tells you everything you need to know about Conversational Marketing and how to use it to accelerate revenue and achieve maximum results.