Skills
Quantitative research, Usability evaluation, Synthesis.
Background
- A new design (mainly focused on visual uplift had been created
- Given large scale usage of product, needed to prove that interface changes improved task completion/usability.
Challenge
- It was the first quant research undertaken by the chapter and there was a feeling that it would only slow things down.
- A quant approach was important – to prove the new design would give improvements or at least not make things worse.
- Wanted to test the visual design in a quantifiable way – not an easy thing to do
Actions
- I created a shared research plan within the tribe to make sure the research was aligned with stakeholder/business needs.
- I set up, ran and reported on the quantitative research using Userzoom to over 200 bank customers.
Whiteboard test plan
Study details:
- 2 main customer segments
- Probing questions on channel – web vs app – selection.
- Random A/B split with series of 1st click testing on existing and planned UI designs with SEQ (Single Ease Question)and follow up questions for lower SEQ scores.
- Product reaction analysis.
Results
- Study showed although most tasks performed significantly better or the same, one key task was statistically worse in the new design due to button placement change.
- New visual design accepted by business subject to changes driven by the research.
Lessons learned
Using a whiteboard test plan is a good way to allow for multiple edits and to share the plan outside the team
When using Userzoom – or any online tool – it’s important to have the study plan/questions completed before building the study.