Appropriateness as a criterion for speech IVR

Someone who is putting together a conference symposium on "the impact of speech technology on society" pinged me and a big bunch of other opiniated people on this topic and asked for input. My quick effort to be an IVR/speech industry forward-thinker follows.
--
Where the IVR industry will go is away from homegrown, do-everything speech recognition systems running on a company's own platform to highly specific, appropriate applications running on hosted platforms. A lot of companies have spent a ton of money on speech development tools and handed them to IT people who figure that speech is like every other technology - you just hack on it until you get it right. When it doesn't work the companies blame the technology, and the IT department (or communications department) makes excuses.

Examples of highly specific, appropriate use applications are things like name capture and city and address capture. The applications that do this well are created by speech rec companies that understand what they are doing and design and test them for a long time, refining them for a long time. These small, self contained applications may be embedded in larger apps that may be DTMF only.

Hosting allows the speech vendor to gather an enormous amount of data from various companies' callers and use that data to improve their product. IT departments at individual companies don't have the resources or expertise to do that. Another specific, useful application is speech to text transcription. Again, a few speech vendors will succeed at doing this, but most companies don't have the resources.

The idea of speech recognition systems as giant, expensive opportunities to push an auditory brand at customers who call will go away.
--
"Appropriateness" is really something that is hard to talk to companies about. Once they've made up their minds about speech, based on some conversations you weren't at, it's very hard to pull them back and get them to think about whether speech is an appropriate modality for the IVR app they want to develop. I've tried to engage a number of people at companies, pointing out that a three-item menu of easily discriminable labels is a lot easier to implement and use if done in DTMF. No sale. After the speech decision has been made then it's full steam ahead, and anyone who says otherwise is simply engaging in "analysis paralysis."

Just because you can write VXML...

...doesn't mean you should. That's what the head of an IVR and system integration company has been known to tell people. There's a lot of specific skills and knowledge required to implement speech systems correctly, beyond simply being able to write code.

When you get it wrong, people post amusing videos of your system on YouTube. Didn't I just say the "press or say" construct won't die? Thanks to Phil Shinn for forwarding this.

"First, tell the caller to do this...."

There's nothing that demonstrates the differences in assumptions between VUI designers and IT or business people than the way they express their ideas for how prompts should be written.

• "First, tell the caller to do this. Then tell them to do that. Then, they'll pick this option and they'll be where they should be."

That's IT/business speak. "Telling" callers to do things over the phone is a losing proposition. Assuming that they'll follow your cryptic commands isn't reasonable. Usually, they aren't your subordinates, and even if they are they usually aren't compelled to follow your instructions. In fact, you're often dealing with customers who may have the funny idea that YOU should be responsive to THEM.

The assumption that you can tell callers what to do leads to a logical fallacy.

• If callers obey instruction n they'll obey instruction n + 1.

This leads IT/business types to propose endlessly long dialogs, another assumption being that the caller on the other end of the line has no alternative but to obey instructions. VUI designers know that's not how people operate. At each step in the dialog callers evaluate whether they're getting closer to their goal, and whether it makes more sense to press the magic zero key (or say "agent," or play possum, or press the pound key three times, etc). People aren't calling to make the company's life easier, they're calling to get something--and that thing may be something they've already paid for and you owe them.

You can present the business people with call statistics that show you lose a percentage of callers on every question asked in a dialog, but often the message doesn't sink in. It's a matter of assumptions, and getting people to change unvoiced assumptions is a difficult thing indeed.

The "press or say" construct will not die

The "press or say 1" construct for IVR menus was rolled out by AT&T about 15 years ago, when speech recognition was new. There was a "wow" factor at the time, but those days are long gone. Still, I work with companies that feel a need to use this in their own IVRs and I can't understand why. When I ask why this is needed usually the response is "someone else decided this, we're not sure who, we just do project work." So someone high up in the food chain made the decision and we, the lowly consulting company, can't talk to them and try to get the decision changed.

There's no IVR design guild but if there were one it would dedicate itself to iradicating this practice.

Telephone etiquette and VUI guidelines

A lot has been written on the topic of persona and best practices for designing voice user interfaces. Here's an early Bell Systems manual that tells you just about all you need to know. Thanks to Crispin Reedy for forwarding this.

Wordsmithing: putting band-aids on gunshot wounds

I get this request frequently. "This prompt is giving us trouble. Can you wordsmith it?" It often comes up when a call flow is out of order in some fundamental way, but the client company doesn't want to redesign it. The wordsmithed prompt is supposed to alert the caller that something strange is going to happen and to prepare them for it.

IVRs have to work really well to work at all. Customers' tolerance for ambiguity and confusion in an IVR is far less than for a web form with similar functionality. Wordsmithing a broken call flow is about as helpful as treating gunshot wounds with band-aids.

Lessons Learned: Don't ask for the moon

Listening to recorded answers to IVR prompts is a necessary part of tuning a speech IVR application for the purpose of improving recognition performance. It also sometimes reveals what people think about the IVR. Here's a verbatim quote taken from a tuning exercise.

"Oh you gotta be kidding me how the hell am I supposed to know that?"

The IVR had prompted the caller for the last four digits of the credit card number that was used to set up and pay for a service. The service may have been set up as long as a year previously, and many customers have more than one credit card. The question of how the hell the customer was supposed to know that had at least occurred to the IVR designer of this application. There were extra steps placed in the IVR to mitigate this scenario, but none were very effective. Lesson learned: ask for things that you know are easily available to the caller, or prepare them in advance for unusual requests that take time to track down.

Demo: "Please listen carefully..."

"Please listen carefully as our menu options have changed." I've complained about this useless little IVR nugget before. The IVR script writer believes that misroutes are solely caused by callers' inattention to the recorded prompt - certainly not because the menu options are confusing or misleading.

The phrase is really annoying. Some brilliant individual created a working demo that makes that point far better than I can in writing on my blog. Call 888-583-2801 and enjoy. Thank you Brad Lehman for the pointer.

Political IVR demo

Here's a fine speech-enabled IVR that's being used to collect feedback from real Americans like you and me. This is the best use of technology for democracy building that I've seen in a while.

Thanks to Todd Chapin for forwarding this.

Design anti-patterns: context-free design

I recently presented some training to a group of IT folks who were trying to improve the performance of an existing IVR. Completion rate was only 40%, not high enough to justify the expense of the system. The task seemed easy enough. The caller enters some identifying information, then data about amount of miles driven, fuel used, perhaps a vehicle number, all numeric.

I had started by preaching the importance of writing good scenarios - descriptions of customer behavior in the context of their environment and their tasks. Then I presented some general guidelines for designing prompts. This is harder than it appears because there are often contextual factors that make a suggested prompt worthless in a given case, so every guideline must be wrapped with disclaimers.

Back to the problem with the existing IVR. After cautiously making a few suggestions about prompting I asked the participants to back up and describe the scenario in detail. Then it became clear. The caller was a cashier standing at a check out counter talking to a driver. The driver supplied the information, who sometimes had to pull out a notebook to get the information. Two obvious problems. One, if the system is converted to speech then you'll get a lot of sidetalk recognition errors, where the caller isn't speaking directly to the IVR, but to someone else. Two, speech timeout values are by default pretty short. If you don't have the information the IVR is asking for close at hand, the IVR goes into its Silence error routines.

These are nice design issues and potentially solvable, but not unless you understand the context and the likely outcomes of the caller's behavior. In this case, not having information at hand and needing to talk to someone during the call. An expert can look pretty silly and helpless if he's offering recommendations without understanding the scenario.

Design anti-patterns: Extraordinary rendition

Some years ago I was asked to review and suggest improvements to a touchtone call routing IVR for a financial institution. One of the submenus had seven options, and it wasn't clear to me how callers would be able to select the correct product option. One of the business analysts told me, "Don't worry about redesigning that part. All of those calls go to the same unit. The options are just for reporting purposes." The management was just using the menu to gather statistics on what customers were calling about. Since that time I've seen a number of applications that require callers to select a number of options for the sole purpose of generating automated reports on usage.

Instrumenting your applications is usually a pretty good idea. It allows you to identify and fine tune hotspots and learn all sorts of things about how the application is being used. Requiring customers to wade through several menus of inscrutable options before they reach a representative causes several problems, however. One, it generates bad data, because callers can't be counted on to successfully navigate a poorly designed application. Or even a well designed one, for that matter. There are better ways to collect the data businesses are looking for. Two, it annoys the hell out of people to answer a bunch of questions before being allowed to talk to someone and receive service.

If fact, if the IVR is too disagreeable then people will simply answer prompts randomly or press zero (if it's available). It's like torture: if the experience is painful enough people will say anything in order to make the pain stop. Businesses shouldn't torture their customers (or allow its IVRs to torture customers by proxy). And they shouldn't build multi-layered IVRs full of inscrutable prompts in the misguided idea that reports are more important than happy customers.

A simple request: CSS for TTS

Text-to-speech produces output that has relatively flat affect, that is, it's mostly free of emotion. TTS engines account somewhat for end of sentences by changing the inflection and pause of the last word before a period or question mark. There are slight pauses for other punctuation in a sentence, but for the most part TTS doesn't do much to interpret sentences.

I'd love to be able to separate text from the presentation of the text, in the same way cascading style sheets allow web designers to separate written text from the presentation of the text. I'd like CSS for TTS. I'd like to be able to control the speed, pitch, intensity and stress of TTS text by tagging the text, and then writing style sheets that recognize the tags and control the TTS output accordingly. It would be a big step towards being able to define the persona of an agent implemented fully in TTS.

This should be perfectly feasible. I'm surprised it hasn't been done. If anyone with a technical background wants to work on this and needs some direction drop me a line.

Design anti-patterns: What's one more option?

You have just finished a solid draft of an IVR call routing application, and you're pretty happy with. The menus follow conventional wisdom for numbers and types of options, and the design has done well in early usability testing. You're looking forward to sign-off on the design, and one of the business analysts pipes up, "Hey, we need to add one more option to the main menu. What's one more option?"

"What's one more of anything," like records in a database or fields on a form, usually isn't a big deal. Computers are supposed to be able to iterate any number of things with a minimum of additional effort, so the same principle should apply to IVR menu options. That's the logic, anyway. Unfortunately, the same logic applied to IVR menus is a disaster, because it's the callers who have trouble sorting through and remembering and verbally reproducing multiple menu options.

The logic of "What's one more option" is very seductive, and can lead to main menus with ten or more options. Big main menus (or "Maim menus," for what they do to callers) just scream "I don't know what I'm doing! Hit the zero key now!"

IVR menus structures are still actively researched. For an excellent recent study on IVR menu options, see "A comparison of broad versus deep auditory menu structures," Commarford et al., (2008), Human Factors, Vol. 50, pp. 77-89.

Alligator in the kitchen

The Holy Grail for IVR designers is a natural language application that can handle a significant percentage of calls to a service desk. 911 call centers are a kind of mission-critical service desk: you call with your problem and address and hope that help arrives before things get much worse. In order to get natural language to work you'd need to anticipate what callers will say when they call (or at least have categorized a number of previously recorded similar calls).

Unfortunately, there is just no way a VUI designer is going to craft a system that will recognize "alligator in my kitchen" as a legitimate problem call. Even human 911 operators have difficulty with the phrase, as you can hear in this recording.

Design anti-patterns: Possession by hobgoblins

"A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines." Ralph Waldo Emerson.

Consistency in the design of a user interface is often a good thing. Re-using a small set of salient, simple interactions within a single application, or across a suite of applications, allows users to apply what they've learned during one interaction to many others. It makes the user interface predictable, thereby making it easier to use. Scott Berkun made a similar point in an extended meditation on consistency way back in 1999, a point that still stands.

I'm a proponent of consistency. I've written about the usefulness of standards and guidelines, one way of supporting consistency among applications. However, it's misapplied when the benefit of a local optimization outweighs the benefit of enforcing consistency throughout an application. It's also misapplied when a presentation style developed for one domain (like radio advertising or visual web) is applied to a wholly different domain (like VUI dialogs). I was recently handed a dialog by a company's branding expert that read like this: "Press 1 to do x, or press 2 to do y," where x and y were rambling treatises on what services could be obtained. It was thought that this dialog was "consistent" with the company's auditory brand derived from web and advertising dialogs. Unfortunately, it violated one of the really useful consistency rules in IVR design, which orders options like this: "For (goal), (press/say) (action)."

At times when I'm arguing against the misuse of consistency I say (with a nod to Emerson) that I'm practicing exorcism of possession by hobgoblins.

GetHuman in the news

The GetHuman standard for call center customer service were covered in a recent BusinessWeek story. The gist of the story was that the GetHuman "movement" has lost steam, and gave some reasons why.

There's no doubt that customer service should be a top priority for companies, and that automated phone attendants are a frequent source of poor service. The GetHuman standard falls short because a company's call center needs to commit to a far higher level of service than is embodied by the standards. Companies that are committed to good service don't need the GetHuman standards; they have managers in charge who understand where the company needs to be and the amount work it will take to get it there. Companies whose call center managers think they can offer quality service by following a checklist are lost in the woods, and won't find their way out without some serious help.

I think standards and guidelines have their place in promoting good design practice if followed properly; I've said so before. I give GetHuman a lot of credit for raising the issue of poor customer phone service. However, standards can only take one so far; delivering great service is a matter of a company's planning, desire, and execution.

Is your IVR persona Devo?

Distinctiveness and originality count for much when establishing a brand. Brand managers often push designers to create unique, memorable products, which designers are usually happy to do. A speech IVRs persona, the image created in the mind of the customer by the system's voice, vocabulary, and interaction style, is a tempting target for marketers who want to create a differentiated experience for callers.

When distinctivess and originality are pushed far enough, as in this fine video, the audience may be left puzzled, confused, even outraged. I call highly idiosyncratic, over-the-top IVR personas Devo IVRs, in honor of the band's memorably distinctive songs and videos.

Nearly all IVRs are intended to help callers accomplish a work goal. Highly idiosyncratic personas rarely help callers achieve goals, and may even hurt if the presentation is distracting enough. Sometimes this is a hard message for branding experts and marketers to hear.

"Please listen carefully as our menu options have changed..."

There's always some unnecessary verbiage in any IVR, but this little message seems to be the favorite. I've listened to some systems present this nugget for over a year without changing anything. It signals to me that no one is really minding the IVR store at whichever company I'm trying to get some service from.

When all IVRs were DTMF only the good ones enabled key ahead, the ability to press several keys in succession, without waiting for the prompt to play. Frequent callers to an IVR learned the keypress sequence to get them to their preferred transfer or function very quickly without having to listen to all the prompting. If the menu options changed, however, the speed dialers would up transferring to the wrong department, generating complaints back to the IVR manager. I think this was how the "menu options have changed..." phrase was born. The message may also reflect IVR managers' beliefs about why callers misroute. If the callers would only listen carefully to the options, the belief goes, then misroutes would be eliminated. It doesn't occur to some IVR managers that the prompting could be confusing or unclear. Thus, the phrase "please listen carefully."

Unfortunately, it's easier (process wise) to put phrases into IVRs than take them out. Power users, the ones who learn to key ahead, call frequently. And they don't listen to prompting. Playing the message for longer than a week is nearly useless, because the power users have already adjusted after a week, and it provides no information to occasional users. In speech systems it's even more useless, since you can leave the original menu grammars in place even if you change the prompting.

The message lives on, however, a legacy of another era and some magical thinking about how callers behave.

Ordering your IVR menu options

You might think that, after all these years, putting menu options in order in an IVR would be a solved problem. It's not. There are some general rules for ordering menus, like "order by frequency of use," and exceptions to the rules, like the "specific-to-general rule." However, the observed rules don't account for an important part of the IVR design, and that's the caller's needs for accomplishing tasks.

You have some top-of-flow menus in which the caller is offered several services. Main menu is always a top-of-flow menu. The order of options can influence what the caller selects, and you have some latitude in ordering those items. Let's consider a bottom-of-flow menu, like the presentation of a payment amount. In any pay-by-phone system the caller is given a payment amount and then offered the option to pay using the IVR. Since the company is taking money from a customer, they'll usually make it pretty easy to talk to a representative at this point. How do you order the menu options?

I noticed when I wrote the options for this menu that I followed an unnamed convention that I call psychological distance from the task. I've seen this done so often, and I do it so often, that I don't even give it much thought. In the pay by phone system, from the caller's perspective, they need to select an account to pay, get the payment amount, decide how much they want to pay, indicate their method of payment, pay some or all of the bill, and make sure the company understood their request. Then the caller continues with another task or set of tasks. Here's the ordering of this bottom-of-the flow menu: Repeat amount, Make payment, Main menu, Talk to agent.

How is this related to psychological distance from task? The options are, abstractly, Re-do this step, Go to next step, Leave this task, Leave the IVR. That maps pretty closely to decision the caller makes when presented with an amount. If a caller intends to pay a bill they have to be sure how much the bill is for. They are at the "get payment amount" step. They won't move to the next step until they have the amount. If they didn't hear the amount the only thing they are thinking about is hearing that amount again.

The ordering is also consistent with what the company prefers the caller to do--serve themselves using the IVR before asking for an agent. Some terminal state bottom-of-flow menus include the phrase, "if you're finished you may hang up." Where does that belong in the menu? Obviously, at the end, since it means Stop Contact with the company.

The missing piece in all of this, and the thing that makes ordering menus difficult sometimes, is knowing what the caller wants to do next. In the pay-by-phone example it's obvious. In many others it isn't so easy. The lack of a good task analysis and understanding of why people call leads to the disagreement over ordering menus and menu options. That's why "best practices" rules for ordering menus will never be enough.

Tricking the caller to stay in the IVR

I don't think I've worked on an IVR project when the business people didn't suggest using "tricks" to keep callers in the system. You know what I mean by tricks: disabling the zero key, or playing a routing menu when zero is pressed, or using a non-zero key for transfers, messages that falsely state long queue times, or putting the caller back in the system after they request a transfer. Often the tricks are used in IVRs that are pure misery to try to use. Businesses have a reason to try to increase automation rates - it saves them money. The best way to increase automation rates is to ensure that your automation is useful and usable to your customers. Unfortunately, getting your IVR to that point takes a lot of work, and keep-'em-in tricks are simple to implement. Some business folks take the easy path, and insist on tricks.

The tricks almost never work, or don't work the way the owners intend them to work. If a caller really wants to talk to a representative they'll figure out a way to do it. Eventually. And once they get to a real CSR after they've been plagued by IVR tricks they often aren't very happy. I listen to a lot of calls between customers and IVRs and then follow the calls into the call center. Some customers remain calm with the CSR after a poor experience in the IVR. Others do not, and take out their frustration on the CSR. No one goes away thinking better of a company after a miserable IVR experience.

My recommendation to companies considering using tricks to keep customers in the automation: work on the quality of your IVR first. Monitor, survey, read the reports, improve. Once you're satisfied that the IVR operates flawlessly you can consider using some small inducements (a nice way of saying a subtle trick) to keep callers in the IVR. If it's done properly you might be able to increase your automation rate slightly with no cost to the user experience. However, it all depends on first getting the IVR right. Do the hard stuff first, worry about the tricks later.