Reducing Deployment Delays in IBM Watson Assistant with Multiple Staging Environments
Overview
My role:
Lead UX Designer on a team of 2 designers, 5 developers, and 1 product manager
Co-led workshop with PM to gain alignment amongst team.
Participated in customer calls to gain clear understanding of their needs
Created designs and interactive prototypes using Figma and the IBM Carbon Design System
Developed design language for identifying each environment throughout the product UI
Users:
Enterprise customers referenced in this project include ADP, Bradesco, Vodafone, and Nokia.
Techniques used:
Design thinking workshop and virtual collaboration with distributed team utilizing Mural, Figma, and WebEx
UI components sourced from IBM Carbon Design System
Timeframe:
Approximately 1 year from start to finish
Challenges:
Changes in staffing caused disruptions and delays.
Our sponsor user, ADP, backed out of the agreement to participate in usability testing as we neared deployment, which left us without approval to test the functionality of the designs externally before releasing the feature.
Understanding the Problem Space
Meet our user, Tanya:
Tanya is a conversation designer and subject matter expert (SME) at an enterprise-level company.
She works as part of a team of conversation designers within a large business unit using Watson Assistant to build and deploy AI Assistants on the company website.
Her end user is a customer who interacts with the chatbot, so it’s important to her that her content be accurate and deployed quickly.
Tanya’s Problem:
Tanya and her team have been unable to deploy their assistants quickly enough because they don’t have a reliable way to test it within the product. They are slowed down by the need for third party tools or custom-built workarounds.
IBM risks losing Tanya’s company as a customer if we do not address these issues
Planning the architecture
After a workshop, the team aligned on the need for Tanya to test her work in an additional staging environment, I worked on some process flows to figure out how we could achieve this. We determined that 3 additional environments would meet the needs of most of our enterprise customers.
Design Iterations
As-is State - Draft and Live environments on separate pages:
Watson Assistant had two environments – Draft and Live, each on their own page within the navigation. When Tanya published (deployed) her Draft assistant (from the Publish page), it automatically went to the Live environment where her end user could see it.
Sketches and wireframes of new concepts:
These are some early concepts showing ways that Tanya could access her environments as well as publish to them from her “Draft” environment. My design partner and I decided that the tabs component from our design system would enable us to keep all the environments on one page, rather than clutter the main navigation with additional buttons for new pages.
Final design of new Environments page:
In the interest of time, we ended up leveraging the existing design for the “Draft” and “Live” environments pages and the tabs component in our design system, combining them into one page with each environment in its own tab.
Integration of Concepts
Distinguishing between environments in the UI
Working within the confines of our design system and product color palette, I devised a color-coded system of tags to use throughout the product to show which environments were in use. Special consideration was given to address issues with accessibility, as well.
A new way to publish an assistant
Tanya can now deploy her assistant to any environment or create a version to access later. She previously made no selection at all, but now she chooses a destination for her version in a dropdown modal and “assigns” her assistant version to an environment.
Feedback and Revisions
Due to the nature of the product, testing had to be done post-release over a longer period of time than a typical usability test.
We received customer feedback that they had trouble understanding the new publishing model of “assigning” a version, so I worked extensively with our UX content designer to rename key areas and refine the flow to simplify this.
We also discovered no one was using the multi-select feature in the dropdown, so we removed it. An addition we made was to allow for the creation of a version without publishing to an environment.
Insights
Despite all the early validation we did for this feature, it doesn’t get the level of usage we expected. I suspect that we were too late for it to really help the people who needed it most. Many have built their own workarounds to solve this problem, which they prefer to keep using.
I continue to advocate for multiple staging environments as new features are brought into the product, reminding designers to account for both the draft/live and multiple environments use cases.
This feature is only available in the newest version of the product, and we’d hoped it would incentivize our enterprise customers to migrate from the legacy version. Unfortunately, as of 2024, several are still reluctant to migrate due to the amount of content they have and the time involved in recreating it in the new format.
See it in action!
This is a tutorial I made after the new publishing flow with multiple environments was live in the product. We had new team members who were relying on outdated help docs, so I thought this would both help get them up to speed and serve as a guide for the new documentation.
Recording of flow for publishing and versioning with multiple staging environments in Watson Assistant