What is analytics?
There are a number of web sites and vendors that can address analytics at length. We’ve provided a short description here as it relates to IVR. Analytics usually is performed in a business setting, often uses rich and disparate forms of data, makes extensive use of graphics to convey findings, and can be used for predictive modeling.

What’s the difference between analysis and analytics?
Analytics is really just the science or methodology of analysis. Most of the time, you can treat the terms as interchangeable if it reduces the intimidation factor.

So what does that mean for UI design?
When we perform analytics for IVRs or mobile devices, we are interested in improving the caller experience, but primarily we want to make the IVR a better, more efficient business tool. The primary goal is to save money, with improvements to the caller experience being a natural result of efficiency improvements. There are three fundamental questions that you want to answer as part of any analytics endeavor. It’s not dissimilar from performing triage in the ER. First, where is the most bleeding (i.e., caller trouble spots) occurring? Second, can it be improved and how? Third, will it cost us more to fix it than we would ultimately save by fixing it? Analytics will help you answer those questions, so you can zero in on where to get the most bang for your buck with regard to improvements. And because you’re usually using information based on tens of thousands of calls, you can feel certain that the issues that you uncover are statistically significant.

What kind of data can/should you use?

• Logs – event logs, recognition engine logs (for information about what data to log and how to use it, see Logging strategy)
• Any available reports or dashboards
• Voice of the Customer and any other survey results
• Call Reason and Disposition data, i.e., what was the reason for the caller to call and how was the call handled? You’ll want this for both the IVR and for agents
• Call recordings
• Any existing trending analysis

What tools are available?
There are a number of tools available that allow the user to import multiple sources of data, indicate the relationship between them, mine the data for patterns, and generate visual representation of the data. Adobe Insight, IBM Cognos, and Clickfox are well-known, but aren’t necessarily endorsed by this committee. Be sure to look at what’s available, and make a choice based on what’s right for your organization.

What kind of results can/should you expect to get?
What you get out of it truly depends on what you put into it. The quality of the logging and other data sources will determine how much you can glean about your application’s performance. It’s a bit of a Catch-22: without any data, you don’t know what questions to ask; without any questions to ask, you don’t know what data to collect. A good place to start is collecting menu selections, error outs and opt-outs at each input state. Then you can start asking more elaborate questions, such as:

• What percentage of my callers are super-users?
• How many of my platinum customers are actually using the IVR?
• Does this lengthy explanation in the IVR help or hurt task completion?
• What percentage of callers who hear about our new product in the IVR go on to request more information from an agent?

Every IVR is different, so it’s hard to quantify what you “should” be able to find in terms of opportunity; the lower the containment rate of an IVR, generally speaking the more opportunity there is for analytics to uncover something really useful. Even if an organization isn’t particularly interested in increasing containment, analytics is a useful exercise for understanding how current users utilize an application. The awareness may inform future business decisions.

Can’t we just use usability and tuning?
Usability doesn’t typically test every possible scenario before implementation, and it’s performed in a lab. Results are necessary and informative, but they don’t provide the full picture. For more information, see our section on Usability (need link). Tuning also tells us something about application performance, but is intended to optimize recognition performance at each input state. Prompt improvements may be proposed based on out-of-grammar utterances from callers, but tuning doesn’t typically take a whole-call perspective. And yet, it’s this whole-call perspective from analytics that yields the most interesting findings. For example, one large insurance company had a large number of callers hitting the main menu, selecting “more options”, hitting the main menu again, and then opting out to a representative, resulting in a proposed redesign of the main menu and options sub-menu. Tuning typically would not uncover this caller behavior.

What about speech analytics?
Speech analytics is the process whereby speech recognition technology is applied to recorded agent calls to isolate calls of interest. For example, perhaps a call center analyst is interested in calls in which the caller is upset. Some speech analytics tools may come with a pre-packaged option to search for calls of this nature, or if the analyst wants to do something custom, he or she may create what’s essentially a grammar consisting of search terms such as, “frustrated,” “speak to a supervisor.” Or, if the supervisor is asked what the top three reasons are that callers cancel service, he could search on “cancel my service.” The recognition engine “listens” to calls within the indicated time frame for the search terms, and then indicates what calls contain the search term and where within the call. This is a very useful input into a full analytics process, but doesn’t provide the same depth and breadth of information that a full analytics effort would.