Why conversational AI architects love human-in-the-loop

Dawn Harpster

By Dawn Harpster

0 min read

Conversational Ai Architects Love Human In The Loop

The use of conversational AI in contact centers will be game-changing.

The conversational nature of voice drives better customer engagement as it’s a two-way dialogue. With conversational AI, contact centers can use advanced speech technology with deeper intelligence and automation to a wider variety of everyday communication flows and create personalized experiences at scale.

Conversational speech recognition technology took a long time to evolve for customer service.

The history of conversational technology reveals valuable insights into its conception and evolution over the years. We can trace the roots of the technology back to the 1960s to the work of Joseph Weizenbaum, a computer scientist and a professor at MIT. He demonstrated an early-stage natural language processing computer program called ELIZA to simulate conversations between humans and machines.

But it wasn’t until about 2010 that we began to see conversational speech technology come into mainstream use. Tools like Siri, Google Assistant, Alexa, and Cortana emerged as ‘nice-to-have’ digital personal assistants to humans. These tools primarily functioned to respond to simple voice commands and perform straightforward tasks, like setting reminders or checking information like the weather. Since then, there’s been a leap forward in conversational speech recognition technology, making it more attractive for organizations to explore its potential to transform customer service.

Contact centers were early adopters of speech recognition technology, yet failed to make it genuinely conversational.

Though contact centers have used speech technology for decades — through Interactive voice recognition (IVR) systems — they failed to add a natural flow to customer conversations. Even today, with further developments, IVRs rarely delight customers and end up feeling like a series of endless options, repetitive steps, and far too often lead customers to dead-ends. Even the widely implemented web-based chatbot has achieved the maximum height of its potential. This type of chatbot is limited in the breadth of dialogue they understand and cannot interact with customers in a conversational manner.

AI can make speech technology a conversational game-changer.

Over the past decade, in the world of AI, we have seen no shortage of ways this technology has fundamentally changed how we live and work, and its ability to transform business models. The great news is that the use cases for AI in the contact center are rapidly expanding. One significant way to capture the value of AI is by harnessing its capabilities to analyze customer and agent speech patterns and detect keywords, sentiments, and intents. These powerful outcomes of AI pave the way for organizations to introduce a new kind of AI-powered speech recognition technology to contact center tools—namely an intelligent conversational chatbot, that we now call the Virtual Agent.

AI is revolutionizing speech recognition technology and has carved out a golden opportunity to introduce new systems and tools into contact centers with built-in conversational AI. These tools can neatly replace the traditional IVRs and the basic chatbot – finally giving end customers a piece of technology they won’t mind interacting with!

Conversational AI tools, like the Talkdesk Virtual Agent™ , deliver human-like interactions with customers via text or voice. It also liberates human agents from basic and repetitive tasks, allowing them to efficiently manage their workload and use their deep skills to address more complex cases.

5 Cost Saving Steps For Implementing An Ai Powered Virtual Agent


5 Cost saving steps for implementing an AI-Powered Virtual Agent

Experts who design customer conversations: the rise of conversational AI architects.

Building an effective AI-powered Virtual Agent is both an art and a science. Basic chatbot design principles won’t lead to fluid, conversational interactions with customers. It requires a design process that considers how customer responses will sound human enough that even if customers know they’re not conversing with a human, they still enjoy the experience.

With the advent of digital personal assistants like Siri and Alexa, there’s been a growing need to have experts who can design AI-powered customer conversations—and why the role of a conversational AI architect has emerged to do this work. These experts apply a mix of psychology, scriptwriting, user experience (UX), computer science, and good business understanding.

A conversational AI architect can help ensure that customers have a successful Virtual Agent journey in contact centers. They provide the structure, correct phrasing, and error recovery assistance that the Virtual Agent needs to succeed. They understand that the way customers ask a question dictates the provided output and the peculiarities and challenges of voice and text in human-to-machine interactions.

A conversational AI architect can examine, diagnose, and solve user struggles, ensuring that the journey is seamless, delightful, and hassle-free.

Human-in-the-loop technology (HITL)  fine-tunes conversational AI models without technical skills.

Conversational AI involves a complex set of AI models, a Natural Language Processing (NLP) engine, and vast amounts of high-quality training data that feeds into the design process to account for cognitive differences between speech and text interactions. The accuracy of AI models depends on the completeness of training data, which is not static and must consider the variability of the business environment and how this changes conversations.

But it’s too costly for contact centers to hire technically-trained staff to fine-tune the conversational AI models. Which is why Talkdesk built AI Trainer, which uses human-in-the-loop technology to enable non-technical staff — like agents and supervisors — to solve AI model performance degradation issues and maintain high levels of accuracy in tools like the Virtual Agent. With AI Trainer, agents and supervisors can continuously improve the AI models with no code, just clicks, thereby neatly operationalizing the ongoing conversational AI design process.

What’s so great about AI Trainer?

The virtual agent/chatbot space has been a big part of my career for over 20 years. I have seen many exciting things; however, none of them comes close to AI Trainer. Why?

Before AI Trainer, companies easily paid over $500 / hour to optimize their speech virtual agents. Speech science was the exclusive domain of highly specialized (and expensive!) Ph.Ds. Utterances were collected, sorted, and analyzed. After analysis, these data specialists present recommendations for additions to intents to stakeholders. The stakeholders approved of the changes, and the Speech Scientists added the new phrases. The whole process took days to complete.

AI Trainer is democratizing speech services. It allows you to find out what your users are saying — in real-time — and adapt your system to accept those words with just a few clicks. Now, rather than only performing speech tuning once or twice per year due to the high cost, you can tune your application daily. This results in more agility, higher speed of adaptation, less misrecognition, and less frustration for customers.

Furthermore, the ability to add a brand or industry-specific terminology (not available in out-of-the-box speech recognition packages currently on the market) is right at your fingertips. Customizing your virtual agent has never been easier.

The AI technology stack includes cluster analysis and trending charts, built into AI Trainer and speech analytics that allow you to see any changes in call patterns. Is there a spike in calls for a product due to a social media influencer’s review? You can see that instantly and allocate additional agent seats to handle that spike. AI Trainer isn’t just for virtual agent. AI Trainer can also pair up with agent assisting tools in helping you manage your call center agents. Keywords can pop up knowledge base articles and links or alert you to off-brand agent behavior.

AI Trainer keeps your finger on the pulse of your customers, allowing you to train the model to suit your business needs and by giving you access to real-time data to spot trends and adapt to changing caller demands.


Dawn Harpster

Dawn Harpster

Dawn Harpster is a senior conversation architect at Talkdesk. She’s obsessed with crafting frictionless user experiences and loves to simplify customer problems with intelligently designed conversation flows on virtual agents. In her personal time, Dawn likes to read popular science and she is a professionally trained jouster.