Riskstorming to establish priorities and identify threats

A guest post from Fearless Software Test Engineer Alex de los Reyes

Risk isn’t something people usually want to associate with a product or service they’re developing. No one wants to hear that their application is risky, or there is a list of risks linked to their project.

But risk is inherent in everything. Certain things are inherently higher risk than others, but it’s better to address risk during development than when the risks are turning into problems for users.

As a test engineer, I don’t want people to view software testing as an annoying and unnecessary step. Testing should pique your curiosity. It is an opportunity to learn and your interaction should be as organic as possible.

Why do we test?

Software shouldn’t be developed in a vacuum, but it sometimes feels that way. Teams can spend weeks, months, or even years creating software, and if they don’t take the time to think about how it will meet the needs of end users, the product will fail.

Testing helps prevent failure. By identifying the risks before anything is delivered to an end-user, we can mitigate or eliminate the risk or problem.

Examples of risk:

• A user is unable to log into their bank account
• Usability is threatened, a user can’t get to the basic functions of the system
• Availability is threatened, a user can’t get to the system when they need to

A user is going to remember that experience, and it’s going to affect the reputation of that bank. Trust in the product and bank is diminished. When trust is diminished, we also lower the quality of our software.

What is Riskstorming

You can test before a line of code is even written. In fact, I recommend testing at the earliest stages.

Riskstorming is a collaborative activity. We use it to help identify 3 key points that help us develop software with the Fearless soul:

1. Quality Criteria: What are the most important things to the application?
2. Related Risks: What risks are related to the important criteria?
3. Mitigation Strategies: Plan to lower or eliminate the risk

Example of the 3 at play: If broad usability is one of your quality criteria, then a risk could be that the application does not work with older versions of Windows. The mitigation strategy could be developing the application so it works with older versions of Windows, or deciding to not support older versions.

Riskstorming wasn’t developed by Fearless, although we use the activity a lot in our projects. The tactic was created by the Ministry of Testing. Traditionally, you use a deck of cards to go through the riskstorming activity, but there is also a free online version when everyone can’t be in the same room together.

Running a riskstorming activity for a DoD project

As part of Fearless’ work with the Air Force software factory, BESPIN, we worked with an Air Force team on JOCAS II, a cost accounting system used by thousands of personnel within the Air Force. Before anyone wrote a line of code, we brought together (Pre-COVID) tech leads, product managers, stakeholders, and developers to run a riskstorming activity.

By talking about the most important things to JOCAS and the critical pieces everyone needed to understand, we were able to build a quality development plan and set expectations across the team.

How the process played out:

1. Identify the quality criteria
2. Identify 6 quality criteria cards from the stack for your team to work with
3. Spend 15–20 minutes working with the team to narrow down all of the potential criteria for your application to the 6 most important ones
4. Everyone needs to agree which 6 are the ultimate end focus so there needs to be debate and open dialogue
5. For the JOCAS exercise, some of our chosen quality criteria were: functionality, user-friendliness and data
6. Identify risks and threats
7. Using sticky notes, write down every risk you can think of related to each chosen quality criteria your own risks related to each criteria
8. A risk for the user-friendliness criteria could be a long onboarding time or reluctance from users to do something different
9. Identify risk mitigation strategies
10. You’ve identified the risks so now it’s time to develop a plan to tackle the risk while developing the product. Use the remaining cards to help come up with tests or mitigation strategies
11. A risk mitigation strategy for user reluctance could be developing a communication strategy to engage users about the updates and changes

Fearless ran this riskstorming exercise for the JOCAS team at the beginning of the project and many of the identified criteria and risks are still valid to the team. We are still using this information to help everyone understand what we’re doing and shape the way we develop this application.

What are the benefits of riskstorming assessments

• Deeper understanding of the overall product
• This exercise gets people talking about the application: what do we need, what do we have and what do we want
• Agreement on the important aspects of the project
• Riskstorming is an opportunity to get everyone on the same page
• Rather than assuming everyone is on the same page, this ensures everyone is on the same page because we have collaboration to get there.
• When everyone is working on the same page, everyone has a shared understanding and expectation and can focus on building a quality application
• Develop a foundation for working with risks and threats to the project
• Acknowledging the risks your project has does not make it weaker
• By addressing threats head on, your team will have a stronger project in the end and your users will have a better experience

This sounds great, how can I try riskstorming

• Riskstorming only takes about 45 minutes so it is a great way to facilitate conversations with people without having everyone commit to a day of meetings and brainstorming
• Look through the available cards on RisktormingOnline and pick a handful to start the conversation between your team
• Go through the 3 step process with your team to:
• Identify the quality criteria
• Identify risks
• Formulate a design/development strategy to mitigate the risks

It’s important to come into riskstorming with an open mind. This is a time to explore concepts, rather than validate an idea you have. You shouldn’t view riskstorming as a way to prove your predetermined criteria. Riskstorming is a way to find and discover the input of everyone involved in the project.

While riskstorming is best done at the beginning of a project, it’s still important to revisit your selected quality criteria from time to time to make sure it is still relevant and you are building the right thing. If you find your criteria has changed then run the exercise again to determine and address your updated criteria.

Learn more about how Fearless uses context-driven testing to identify the unique skills and techniques needed to deliver successful software solutions.