AI customer-service chatbots are now heavily promoted across almost every sector, with suppliers promising conversational AI and GenAI-powered support.
To see how advanced these systems really are, we reviewed 100 companies across 38 chatbot vendors. Our findings suggest that while AI customer support is growing quickly, genuinely flexible conversational GenAI chatbots are still relatively uncommon in public-facing customer service.
How we tested the chatbots
The research focused on customer examples and case studies published publicly by chatbot suppliers. We then attempted to locate and test the chatbot directly on each company’s website between April and May 2026. For each of the chatbots located, we:
• asked a normal business-related support question;
• asked a completely unrelated ‘weird’ question;
• assessed how flexible the responses were;
• recorded whether the chatbot appeared rules-based, restricted AI, or more LLM-style conversational AI.
Examples of normal support questions included:
• “How do I track my parcel?”
• “How do I reset my password?”
• “How do I book an appointment?”
The ‘weird questions were designed to test conversational flexibility rather than customer support workflows. Examples included:
• “What is the capital city of Australia?”
• “Can you explain how rainbows form in one sentence?”
• “Can you write a short thank-you message?”
The aim was not to catch the chatbots out, but to understand whether they could move beyond tightly scripted, rules-based customer-service flows.
The findings were more mixed than expected.
Across the 100 companies reviewed:
• only around 50 had a visible customer-facing chatbot on their public website;
• 47 had no visible chatbot at all;
• 13 routed users directly to live agents instead of AI systems;
• Only a small minority (<10) behaved like genuinely flexible LLM-style chatbots.
The good news is that most systems were able to handle straightforward support questions reasonably well. However, many struggled when asked anything outside their expected topic area.
Common responses to the ‘weird’ questions included:
• “I can only answer questions about this company.”
• “Please choose one of the options below.”
• “I didn’t understand your request.”
A large proportion of the systems felt much closer to traditional menu-led or workflow-based support tools than genuinely conversational AI.
This was one of the clearest patterns across the research.
The gap between AI chatbot marketing promise and real user experience
One of the most interesting findings was the difference between supplier marketing and the actual customer experience. Many chatbot providers strongly promote the following:
• conversational AI;
• GenAI customer service;
• AI-powered digital assistants;
• Intelligent automation.
It was difficult to verify these claims. Yes, it is still early in the implementation of this technology. However, given the predictions of many about the fast paced adoption of AI chatbot technology the results were limited.
If companies were confident about the technology they would deploy it widely, making it available across as many channels as possible.
However, we found:
• some companies had no visible chatbot at all;
• some bots only worked through fixed prompts or menus;
• some required logins before meaningful interaction;
• and many systems refused to answer anything outside narrow support tasks.
That does not necessarily mean these projects are unsuccessful. In some sectors, restricted workflows may be intentional because organisations want:
• more control over customer interactions;
• reduced compliance risk;
• greater accuracy;
• still testing and developing
• or tighter escalation to human agents.
But it does suggest that fully conversational GenAI customer service is far from widespread in public-facing deployments.
A Few Chatbots Did Stand Out
The table below shows some of the clearest differences we observed between more flexible conversational AI systems and more restricted customer-service chatbots during the testing
Company Normal Support Question Weird question result What it suggested
|
Company |
Normal Support Question |
Weird question result |
Type of chatbot suggested |
|
|
Minted |
Successfully answered customer-service query |
Responded naturally to a creative question |
Strong LLM-style conversational behaviour |
|
|
TourRadar |
Handled travel support questions well |
Answered unrelated conceptual question naturally |
Flexible conversational AI |
|
|
Pickyourtrail |
Provided useful travel-planning support |
Responded well outside narrow workflow |
More conversational than most bots tested |
|
|
Wearable fitness brand |
Clearly explained recovery score and health metrics |
Refused unrelated mountain question |
restricted support-focused AI |
|
|
Earplugs retailer |
Recommended suitable sleep earplugs |
Refused geography-related question |
Domain-limited chatbot |
|
|
Laundry software provider |
Answered laundry-order tracking question |
Refused rainbow question |
|
Important limitations to the research
There are important limitations to this type of testing. Some companies may use more advanced AI systems:
• inside mobile apps;
• behind account logins;
• for selected customers only;
• or within internal customer-service systems that are not publicly accessible.
It is also possible that some organisations use advanced AI heavily behind the scenes while deliberately presenting a more controlled customer-facing experience.
For that reason, this research should not be treated as a definitive assessment of chatbot suppliers or businesses. Instead, it provides a practical snapshot of what a normal public user could access and experience in May 2026.
Final Thoughts
The overall conclusion is not that GenAI customer service does not exist. It clearly does, and adoption is continuing to grow. However, this research suggests that many public-facing customer-service chatbots – even those hosted by companies that are said to be using GenAI chatbots – still behave more like traditional support tools than fully flexible conversational AI assistants.
The hype around GenAI customer service has arrived quickly. The opportunities are significant.
The widespread public deployment of genuinely conversational AI appears to be arriving much more gradually.
Appendix: sector breakdown of companies tested
We also categorised the 100 companies by their main sector, based on their public-facing business activity. Some companies could fit more than one category, so this should be treated as a practical sector grouping rather than a definitive classification.
|
Sector |
Number of companies |
|
Retail/e-commerce & consumer goods |
20 |
|
Financial services, insurance & fintech |
18 |
|
Technology, software & SaaS |
16 |
|
Travel, transport & hospitality |
11 |
|
Healthcare, wellbeing & life sciences |
8 |
|
Food, drink & grocery |
5 |
|
Automotive & mobility |
4 |
|
Professional services, outsourcing & contact centres |
4 |
|
Education |
3 |
|
Energy, utilities & telecoms |
3 |
|
Public sector, government & services |
3 |
|
Recruitment, HR & employment |
2 |
|
Gambling, gaming & betting |
2 |
|
Real estate, property & housing |
1 |
With Elstella Awaritefe.