Speaking from experience, I can tell you it's virtually impossible to test intent. When you are building a mostly spoken UI, such as an Alexa or Google Home interface, it is imperative to test your conversations early and often, but the kicker is that it can be really difficult to do with more traditional user testing techniques. This is especially true if you're developing your conversations at the same time as whatever physical device will contain it, is being designed and manufactured. Can you set up remote user testing sessions when there is no tangible thing or site to interact with? How do you perform in-person tests that mimic the "real experience" with lo-fi prototypes?
Having recently spent six months researching, prototyping, testing, and building some conversational UI components for a robot, I plan to share what worked well, including specific tools, techniques (use an object similar to the "real one" if the source of your conversation is not yet built or too complex to build a working prototype, like a robot), and tips for success (test with literally as many people as possible, focus on the JTBD).

The workshop will consist of interspersed short lectures and small group activities (4-5 people), and will include time at the end for sharing what folks learned with each other, and for asking questions. While the workshop will touch on testing considerations and techniques, the main thrust of the workshop will be on the designing of these conversations, and how to go about the process, where to begin, and how constant, guerilla testing can make a world of difference. I will also include a overview of the difference between chatbots and AI-based conversational UIs, and why those differences matter to what conversations you design and how you design them. We will touch on specific tools that can help the design and testing process, a few testing techniques that work well with these tools, as well as tips for success and pitfalls to avoid.

By the end of this workshop, attendees will have an understanding of what it takes, broadly, to design conversations for conversational UI-based experiences, whether that's an Alexa, or a robot, or a smart home device. Attendees will also gain an understanding of some tools and techniques to use in the early discovery and design stages, as well as for research throughout the lifecycle of the design and development process. Furthermore, folks will gain an understanding of what unique questions to ask their business partners or tech teams, in order to build the right (CUI) thing, the right way.

Attendance Requirements:

Attendees do not need to possess any prior conversational UI experience or knowledge; your background in user experience design, research, strategy, and management will provide you with ample background knowledge, as well as a basis for comparison to see how this is similar and different from current practices for non-conversational UIs. Please bring a laptop, if possible. However, it is not mandatory, and the workshop can certainly be completed without having your own laptop with you, as much of the work will be done in small groups, in which it isn't necessary that all members of the group have a laptop at all times.