- Monitoring 1M+ AI agent interactions and understanding why some escalate to humans.
- Reducing response latency by 50% with a phenomenal 4.5/5 CSAT across 40K+ weekly conversations.
- Accurately predicting a ~5% deflection rate of potential support cases.
Agentforce, Salesforce’s platform for building and deploying AI agents, went live on the Salesforce Help Portal in October 2024 and already has handled over 1 million support requests. The Salesforce Cloud Success BI Team uses Tableau to monitor the performance of Agentforce based on 35 key metrics. By harnessing advanced analytics and integrating data from multiple sources on Tableau, the team can understand feature usage, track how customer interactions contribute to successful resolutions, offer input to product teams, and ultimately improve user experience at scale.
By harnessing advanced analytics, integrating data from various sources and bringing it all together to develop a sophisticated dashboard in Tableau, we create an end-to-end funnel for feature usage and gain deep insights into user interactions with Agentforce.
Ankur SahDirector, Analytics and Insights, Salesforce
The Challenge
How do you measure AI agent effectiveness?
The goal of launching Agentforce on the Salesforce Help portal in October 2024 was to set the standard for world-class, agentic service. Powered by Data 360 , Agentforce analyzes both structured data — like account history and product usage — and unstructured content — like knowledge articles and product documentation — to deliver precise, personalized answers. It also frees human support engineers to focus on more complex issues.
Right away the Salesforce analytics and insights team realized that monitoring Agentforce’s performance would be crucial to its success with customers and executives. With no industry blueprint for how to measure AI agent effectiveness, Salesforce integrated Tableau’s powerful data visualization into its Agentforce initiative to create its own.
The objective of bringing Tableau and Agentforce together was to answer critical performance questions such as:
- How effective is Agentforce in resolving issues or handing off to humans?
- Where is Agentforce succeeding, where is it falling short, and what are the KPIs they should be monitoring to measure success?
- Are customers finding value in Agentforce, and is it improving service quality?
How Tableau Helps
Precisely Measuring Self-Service on Agentforce with Tableau
To collect and harmonize the most relevant, accurate information, the analytics and insight team drew from impression, rerouting, latency, and customer satisfaction data in a variety of sources. They then deployed Tableau as the primary visualization and report building tool, defining 35 key metrics that evaluate the Agentforce’s performance and its hand-offs to humans.
These metrics–which include everything from conversation rate and median latency to escalation rate and customer satisfaction–are reported on a centralized Tableau scorecard that is refreshed every two hours.
The team also built an AI-powered analytics ecosystem to provide actionable insights for Agentforce optimization and to scale as capacity needs grew. One AI model in this ecosystem looks at customer conversation sentiment, another derives the relevancy of conversations so they can better measure a true resolution to a problem versus a deflection.
If you can measure it, you can improve it
After the team standardized which key metrics to track, they searched for insights in the Agentforce performance data they had brought together. Latency in response time was one metric that stood out, at an average of 14 seconds. Now that they were aware of this result, the Agentforce team was able to take action to address it. In less than a year, Agentforce response times have improved by 50%.
The team also has identified some best practices that customers can use to roll out AI service agents, including Agentforce:
- Know what you want to measure. Determine what’s most important–queue time, escalations, abandonments, human handoffs, customer-confirmed resolutions–and why.
- Launch in phases. Don’t flip the switch to AI agents all at once. Slowly turn off legacy channels to move traffic from defaulting to a human agent to your new AI agent, but allow for escalation to human agents when needed. Another possible path is to flip the switch to all AI agents, but only for a short period such as a week, and then collect customer input on the experience.
- Once you start to measure, be open to learning and change. Some metrics may not cover all the user behaviors you’ve anticipated, such as bookmarking specific help pages.
The Tableau Difference
Using Tableau for its Agentforce service metrics, the Cloud Success BI Team team was able to leverage its highly interactive visual analytics, strong data governance and scalability, and live + extract data model flexibility. The result is a single source of truth to understand the performance of Agentforce on the Salesforce Help site. But the team also has established a precedent that they can spotlight with customers. As more customers decide to add Agentforce as their own digital agent, Salesforce is sharing support metrics and best practices for evaluating its effectiveness in places like The 360 Blog and in a detailed case study .
The Results
As a core component of Salesforce’s self-service strategy, Agentforce is already playing a vital role in improving customer experience at scale. Today, Agentforce resolves more than 85% of support requests autonomously and recently passed a major milestone: over 1 million support requests handled on Salesforce Help, delivering a resolution rate of nearly 80%. And even with a 2% rise in site traffic, support case volume since launch has dropped by 5%, freeing human agents to focus on complex issues while AI expertly manages high-volume, low-effort tasks.
The team continues to pursue its ultimate goal, that all customers talk to Agentforce first. Tableau remains a key part of this equation, providing primary data visualization and report building tools for the Cloud Success BI Team.