AI Conversational Agents. Wow!

by Jim December 11, 2024

Welcome back, Rankers! This week I dive into the exciting possibilities unlocked by Eleven Labs’ new Conversational Agents. You’ll learn how these advanced AI tools can transform the way businesses interact with their audiences, whether it’s through websites, phone calls, or personalised assistants. I share my own experience using Eleven Labs to train a voice clone and explore how this technology can be customised for various use cases. If you’re curious about how AI is reshaping communication, this is a must-read!

What I learned:

Introduction to Eleven Labs’ Conversational Agents: Discover how these AI-driven voices can be tailored to various business needs.
Customisation and Cost Management: Learn how to integrate and optimise these tools while keeping budgets in check.
Multilingual Capabilities and Language Model Choices: Insights into choosing the right large language model and exploring multilingual options.
Practical Applications and Future Potential: Explore real-world use cases, from customer service to professional advice, and the rapid evolution of this technology.

Transcript

Hey, welcome back, Rankers! How are you going? I wanted to quickly share something exciting with you. Eleven Labs has introduced a new tool that some of you might find fascinating. Many of you know that I used Eleven Labs to train my voice for my audiobook, creating a professional voice clone. Now, they’ve released something they call “Conversational Agents,” and it’s quite incredible.

Exploring Eleven Labs’ Conversational Agents

So, what are these Conversational Agents? Essentially, you can take any of their voices and turn them into AI assistants. It’s remarkable! These voices can perform various roles—be an agent on your website, handle Interactive Voice Response (IVR) systems, or even make and answer phone calls, including outbound calls.

For example, my own voice, which I trained using their tools for my audiobook, is just one of many voices available. You can configure these voices with different criteria depending on the interaction’s goal. Imagine using them for customer service, data collection, or simply engaging users.

Integration is seamless too. For phone-related tasks, you can connect to Twilio, a popular SMS and phone service. This allows you to customize interactions, collect data like strings or numbers, and handle it all through a large language model (LLM). It’s efficient and straightforward.

Another thing to note is that Eleven Labs charges based on voice interaction length. To manage costs, you can set parameters like conversation duration. Embedding these agents on your website is as easy as copying and pasting code. You can also customize widgets, colours, and more to fit your branding.

Language Capabilities and AI Configurations

While I trained my voice clone in English, we tested it to see if it could handle other languages. It managed to speak in Urdu, though our native speaker noted it wasn’t perfect. Since my voice isn’t trained for languages like Hindi, I’ll stick to English for now.

You can also choose your preferred large language model. I’ve been using the 3.5 Sonnet from Anthropic, which I find most efficient. Interestingly, you could plug in an open-source LLM, though I haven’t tried that yet. This could help save costs further.

A cool feature is retrieval-augmented generation (RAG), which allows the AI to access specific data during interactions. This makes it possible to provide highly relevant information. There are also tools and parameters you can add for more functionality. I’m still exploring these possibilities, but the potential is exciting.

A Test Conversation

To demonstrate, I ran a test conversation. Here’s how it went:

Agent (my voice): Hi, I’m Jim. How can I help you today?
User: G’day Jim. I’ve been dealing with a bald patch for years. Can you help?
Agent: I’m sorry to hear about your bald patch, but don’t worry—I’m here to help. Hair loss can be tricky, but there are options to explore. Could you share more details, like how long you’ve noticed it and whether you’ve tried treatments?
User: Well, it started about 40 years ago. A friend suggested shaving it off to make it grow back thicker, but that hasn’t worked.
Agent: Thank you for sharing. Unfortunately, your friend’s advice is a myth. Shaving doesn’t make hair grow back thicker. It sounds like male pattern baldness, which is natural and common. While there’s no miracle cure, treatments are available to slow hair loss or encourage regrowth. Have you consulted a dermatologist or trichologist?
User: I don’t even know what a trichologist is, but thanks for your help.

It’s a bit surreal hearing my own voice, but this showcases the possibilities—whether for outbound sales calls, inbound inquiries, or even niche professional advice.

Practical Applications and Final Thoughts

Small business operators could find this tool invaluable. For example, I could use my expertise in SEO—gathered over the past 25 years—to train a conversational agent that provides advice to clients. This space is evolving rapidly, and Eleven Labs’ setup is impressively user-friendly.

I’ve also experimented with multilingual models, and while I haven’t tested them extensively, the potential is immense. The system charges 500 credits per minute, and I’ve already burned through thousands just playing around. Still, it’s worth exploring.

Closing Remarks

That’s it for today. If you’re intrigued, I encourage you to give it a try. By the way, last week I had a great conversation with Gabriel Lanari, another experienced SEO, about the intersection of SEO, AI, and search. Check it out here if you haven’t already.

I’m planning more interviews soon, so if there’s someone you’d like me to chat with, let me know in the comments. Don’t forget to like, share, and subscribe. Thanks for tuning in, and I’ll see you next week! Bye!

And don’t forget! We’re excited to remind you of our two groundbreaking reports that will catapult your business into the AI-driven future: The SAMScrum Report—a comprehensive analysis that will provide actionable insights to optimise your website for the AI era, and the Brand Intelligence Report – that reveals brand sentiment, competitive positioning, and product associations in the AI realm.

The SAMScrum Report: Your Website’s Secret Weapon

Analyses your entire online presence: SEO, Google Ads, UX/UI
Identifies revenue-killing issues in your digital strategy
Implements changes that have DOUBLED revenue for our clients in months

The Brand Intelligence Report: Your AI Visibility Booster

Examines your brand through the lens of leading AI models (Claude, ChatGPT, Google Gemini)
Reveals brand sentiment, competitive positioning, and product associations in the AI realm
Uncovers hidden opportunities to dominate your market in the age of AI

These aren’t just reports. They’re your battle plan for the AI revolution. (And you get a FREE Brand Intelligence Report with every SAMScrum purchase!)

Check out the reports here https://research.stewartmedia.com.au/