Is UX on Board the Scrum Train?

Joshua Candamo, Ph.D., Accusoft QA Manager

Welcome aboard the Scrum train. My name is Scrum Master Sam, and I am the conductor of this train. I trust you will be happy traveling with us for the next two weeks. I hope you enjoyed your stay in our state-of-the-art facilities at the Backlog train station. On behalf of the entire Scrum staff, I apologize to those of you who had to wait two extra weeks in the station. You have to understand that Product Owner extreme weather conditions are out of our control.

I’m glad to inform you that our Scrum team diligent staff has created a tentative itinerary for our trip. However, I must also inform you that our initial estimates could be a little bit off. We are typically poor estimators, so we might have to make some modifications along the way. If we happen to miss your stop, please be patient. I’m sure we can get to it later, but in the near future. Please understand that we have sprint goals important passengers that absolutely need to reach their destination on time.

Scrum as a Business

If we evaluate the Scrum train as a business model, at first glance it’s unclear if it represents an effective model. In general, companies succeed when they “do the right things” and “do things right,” regardless of what process they use. However, most would agree that there is some correlation between processes and results.

Success comes from achieving one goal: adding enough meaningful and timely value to your products. This goal is a natural byproduct of following the core philosophy behind the Scrum software development process. After all, the first principle behind the Agile Manifesto is: “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”

Value is just like the different passengers on a train. Theoretically, they should all be treated equally (I have a dream). But in reality, they are not. For instance, you have staff, coach, and first-class passengers. All of these represent different business value propositions for a company. Let’s say you are the one “paying the bills” for the Scrum train. If there’s a supply issue, and there is only food for 10% of the passengers, which passengers would you feed? Now, let’s say you know that the U.S. president is traveling among the coach passengers. Would that change your previous answer?

The Product Owner

To do, or not to do, that is the question. In fact, in Scrum there is a role dedicated to answering that question: the Product Owner (PO). A PO is in charge of maximizing the value of work produced by the Scrum team. However, important things are not necessarily easy to do.

Nevertheless, “If something is important enough, even if the odds are against you, you should still do it.” Or, at least that’s what Elon Musk would say about it. The problem is not that we don’t want to do something we think is important; it’s often the case that we have constraints, and the PO makes the “wrong” choice (either voluntarily or involuntarily).

Common constraints are a lack of expertise (not only technical) and deficient resource allocation that doesn’t maximize the value Scrum teams can deliver. Clearly, these are not problems typically caused by the PO. I’ve seen organizations that have the right resources, but don’t benefit fully from the full range of their technical abilities or expertise. However, even when resource allocation is a factor, you could still argue that there could be a lack of expertise in resource management or planning.

UX in Scrum

Scrum is just a vehicle to provide value. On the Scrum train, value comes from delivering the correct passengers along the way. One particularly important passenger that tends to be treated unfairly or left behind is User Experience (UX).

How important is UX? “User experience is everything.” Or at least, that’s what Don Norman (the person who coined the phrase user experience) would tell you.

Most would agree that there is a direct relationship between UX and value that Scrum teams deliver when public-facing functionality or infrastructure is involved. However, the concept of “everything” in the Scrum process is something difficult to grasp. So, evaluating something that is difficult to define and manage is naturally a challenging task. Never fear. There are two tips that will allow your Scrum team to deliver more value by improving your releases’ user experience.

 

Tip #1: Add UX Reviews for Stories

If there are many passengers on a train, we can wait until all of them reach their destination and then send a survey to gather satisfaction data from all passengers. Alternatively, we can survey all or a sub-sample of passengers during their trip to get interim data. In Scrum, we can use the same idea of sampling UX data gathered as the team closes stories throughout the sprint.

In an ideal world, UX experts are people who have comprehensive and authoritative knowledge of user behavior relevant to an application’s domain. UX experts that meet that definition are not easy to find. It would be even less likely to expect UX experts to be a part of every Scrum team.

In a practical world, a typical Scrum team needs to find people who have strong critical thinking skills and general knowledge of user behavior relevant to an application’s domain. In simple terms, a UX expert puts themselves in the user’s shoes to experience the product as an outside user would. This is what the Scrum team needs to be able to do when a UX expert is not available to them 24/7.

Don’t be intimidated by the word expert. In fact, calling somebody an expert (even if it is only for a story), can be very powerful. You are empowering a team member to think like an expert—to get out of their comfort zone and think about usability problems with the deliverable produced by other team members.

In software development, developers are used to doing code reviews for source code and QA professionals regularly perform test reviews for automated tests. Similarly, some stories in a sprint—for instance, design, front-end development, public-facing API development, and documentation—should include UX reviews to be performed by UX experts. Having UX review sub-tasks or stories in your sprint allows the team to quickly incorporate necessary adjustments to obvious usability problems early on.

A common problem among product managers, executives, and POs is the misunderstanding of the Minimum Viable Products (MVP) concept. An MVP product is a version of a new product that allows an organization to collect the most meaningful and largest amount of customer information with the least amount of effort. The key is that the customer information that you collect be meaningful and plentiful (in that order). When your MVP provides a poor user experience, the amount of meaningful information you can get from the market decreases. Clearly, the law of diminishing returns would dictate that we should be conscious of UX efforts and their marginal investment returns for an MVP.

Remember your days in school? Remember the strategy of proofreading papers? Pretty much everybody (who wanted to get good grades) used proofreading. Why? It works! The UX review strategy works the same way and is consistent with the decrease of marginal returns when we invest more time into our efforts. If you have an important paper that you want a colleague, professor, or parent to review, you would first glance at the entire paper yourself, and perhaps have a close friend look at it also. This first proofreading strategy would allow you to identify obvious problems quickly. The goal of this story is not to find all usability problems, just like the goal of a paper’s first proofreading is only to find all obvious problems with a quick glance.

If you have ever been part of large usability studies or behavioral research studies, you probably already know (you can also read about it) that subjects lose sensitivity in their attentional set right after identifying certain type of patterns or targets. In other words, let’s say you are looking for a “target” within a scene. After you’ve identified one target, your performance for finding other targets markedly drops right afterward (Wait… What?).

Let’s illustrate with an example. Say someone asks you to provide feedback for a new login screen. You take a quick glance (breadth-first search) and find a bunch of typos, broken branding element images, and a “login” button that contrasts so little with the background that you can’t easily find it on the screen. All of these trivial finds are clear signals that the developer didn’t “proofread” his interface. Each of these finds distracts and can even discourage the reviewer from digging deeper into more complex functional exploratory tests.

In Scrum, tasks are story-pointed or time-bound. So, team members will typically establish personal benchmarks for performance (personal goals). When they reach that benchmark, they’ll stop and consider their task completed. UX reviews for relevant stories will allow us to quickly identify important (and often obvious) problems before engaging in more detailed (and costly) usability studies.

Tip #2: Add a Usability Study Story Before Releases

I mentioned earlier that if there are many passengers on a train, we can survey passengers during their trip to get an idea of their experience so far. However, there’s also significant value in surveying passengers after they reach their destination. In fact, some passengers would be more willing to offer more and perhaps meaningful feedback after the fact, and not during their trip.

Few strategies will allow a Scrum team to maximize value delivered to users more than usability studies. A usability study is a detailed investigation and analysis of user testing results. User testing is a technique used to evaluate a product by testing it with users.

A common criticism of comprehensive usability studies is that they are not very “Agile.” Theoretically, a particular challenge is to balance when and how to do usability studies. Nowadays you have fantastic wireframing tools (e.g., balsamiq) that allow you to perform usability studies early in the design process effectively.

Performing UX reviews early in the development process will increase the value usability studies can provide. In other words, a polished user experience will allow the targets of a usability study to spend time on the functional tasks they have been given without distractions from obvious usability flaws. Fewer distractions mean more valuable feedback.

Consider that paid participants have limits in terms of attention span, natural human capacity, and willingness to perform user testing. Also consider that the main goal of a usability study is to understand how typical users would interact with a product. It’s not the goal of the participant to find usability flaws. Finding bugs and flaws are just a natural byproduct of having a better understanding of how users think and operate.

Until Our Next Trip

The more you involve potential users along the software development process, the better each product’s user experience will ultimately be. Some companies are more mature than others in terms of their processes and their view of the value UX adds to the development process. Nevertheless, it’s no secret that great user experience leads to customer conversions, and as your software development matures, so will your user experience strategy.

Scrum is a particularly challenging environment where UX can be underappreciated and underused, due to the fast-paced nature of Agile methodologies. In Steve Jobs’ words, “You’ve got to start with the customer experience and work backwards for the technology,” (not the other way around).

If you need a place to improve your current development process and maximize the user experience of your products, the two tips presented in this article are a great place to start. First, you can add UX reviews for your stories that will help you identify early-on obvious usability flaws. Second, perform usability study stories to get more and better user testing data that will allow you to maximize the value that Scrum teams deliver.

There are many topics we could talk about to help you perform effective usability studies in Scrum environments, like sprint planning, setting measurable goals, and determining sample size. But these are all topics for another article. In the meantime, remember: Always make sure UX is on board the Scrum train.

---

Joshua Candamo, Ph.D., is the QA Manager for the SDK division of Accusoft Corporation. Dr. Candamo is a UX expert with over 20 years of experience in front-end software development, design, systems integration, and business management of technology services.