Agile Usability Testing


Making usability testing part of your agile process agile usability testing

Usability24/7 has developed a highly adaptable agile methodology to enable you to include usability testing into your rapid development cycles. The service we deliver typically includes some or all of the following components:

  • 5 x 1-hour or 6 x 45-minute sessions in a single day
  • End of day wrap up so you can start making changes immediately
  • High level report close of play day after testing
  • Full report three days after testing
  • The location can be in a viewing facility or in your offices

We don’t ‘make’ our clients have a full report if they don’t want or need one and we will establish a test protocol and set of deliverables that fit your project, time-frame and budget

Our methodology

Agile testing tends to differ from full scale usability testing in that the scope of what we are testing may not be fully understood until the day before the testing takes place. Working with minimum viable products (MVP’s) means we have to be flexible and ready for any thing although one of the benefits is that sprint deadlines tend to be more fixed than larger development projects. Our methodology reflects these differences as follows:

Step 1 – Recruitment, location and timing

The first step in a agile usability testing is to confirm the critical path items upfront. These are:

  • When will the sprint finish and the MVP be ready for testing?
  • Where will the testing take place – do you want to view or can it be carried out in your office with remote viewing?
  • What is the recruitment profile?
  • What is the overall goal of the project and broad subject matter?
  • What deliverables will be required and when – so we can commit the right resource

This information is normally captured [in bits] via email or on the phone so that we can kick the project off and initiate recruitment. Recruitment generally takes 5 to 7 working days to organise and fulfil.

Step 2 – Calm

Pretty much nothing happens until the day before the testing other than monitoring recruitment.

Step 3 – Test Prep

The day before the usability testing sessions we generally meet the client or have a Webex in order to finalise the final details of the project and get our hands on the MVP or prototype. As with any other usability testing project we do use a Research Plan to capture all the information about the project and the client still has the opportunity to review and iterate it but it all happens very quickly. A level of trust is therefore required and it is another reason our team are all experienced UX’ers. The items we will go through during the briefing session are as follows:

  • Creating the Research Plan – this documents all the details of the research including:
  • Specific project objectives and goals
  • Platforms to be used for testing (i.e. desktop, tablet PC, smartphone) – note recruitment often defines users that are familiar with all three if this isn’t known upfront.
  • testing protocol to be used
  • tasks and scenarios including success criteria and purpose
  • test url / location of app
  • technology configuration
  • deliverables – timing and shape

Step 4 – Running the usability testing

On the day/s of the agile usability testing the following process is adopted for each participant session:

  • The participant is welcomed and invited to read the instructions and to read and sign the informed consent form that allows us to video record them.
  • The participant is invited into the research room/ office, offered refreshments and made to feel comfortable. Generally there is a short interview to gather background information and enable the participant to relax.
  • The moderator will then ask the participant to attempt the tasks or scenarios that have been agreed, in order if necessary and will observe their progress. The moderator, who is a highly experienced usability consultant, will delve into areas that are causing the participant issues and gather data for later analysis.
  • After the tasks are completed the participant may be asked some closing questions and then invited to leave the room and receive and sign for their incentive.

Between sessions the moderator will spend time with the client team to ask if there are any additional areas they wish to delve into and also to ensure they are happy with the way the research is going.

Step 5 – Deliverables

We scope the deliverables based on the client requirements. We provide picture in picture video of all the sessions free of charge and there are various debriefing options.

Some clients want a detailed report as a record of the testing and for audit purposes. Others simply want a verbal debrief in workshop format at the end of the test day. The typical deliverables are any or all of the following:

  • high level verbal debrief
  • high level report
  • detailed report
  • presentation of findings

Step 6 – quality assurance

If we have been asked to provide a report it will be quality assured before being released to the client. The quality check is designed to correct any typographical errors and to review the analysis and recommendations. The report will then be supplied to you in PDF format and is a highly visual document.

We have created an example usability testing report which is free to download and demonstrates one of the deliverables we can offer. Download the report here.

The benefits

There are a number of benefits to our agile usability testing service including:

  • Because we use only experienced user experience experts we are able to cope with the last minute nature of agile development and provide the maximum insight available
  • Our mobile equipment allows us to be extremely flexible, even switching between devices during individual user sessions without loss of momentum or picture-in-picture video recording
  • Our pragmatic approach to deliverables allows us to be customer centric and deliver only what you want and need.

Whose it for?

Agile usability testing is for design teams that don’t want to lose out on testing with real users simply because they are working in an agile way.

How much will it cost?

We believe in transparent pricing and have been running usability testing long enough to understand how the cost models work. We can run agile usability testing with 6 people in one day with a viewing room, high level report and detailed report for under £6,000 plus VAT. A similar test run in an office would come in under £5,000 plus VAT.

Ready to get started? Get in touch by phone on +44(0)800 0246 247, or email or contact us