Competitive Analysis

A competitive analysis can be an audit, evaluation or a hybrid of both. It allows your product or service to be benchmarked against those of your main competitors. The same set of heuristics is applied to all products under test and all users perform the same tasks across your product or service and those of your competitors.

Using quantitative measurements raw baseline data can be collected against which future tests can be benchmarked to show how much improvement has been made. The qualitative data can be used to ascertain which enhancements to the service or product that your users would most like to see, or conversely, which features they dislike.

• Obtain both quantative and qualitative feedback on how your product or service fairs with your competitors’
• Benchmark the usability of your product or service against your competitors’ offerings
• Identify design improvements and added features that will set you apart in the marketplace

• Determine what features are unique to your product or service
• Determine how your target customers respond to your competitors’ offerings
• Determine which elements of your product or service your customers prefer.

This service can be used at the following stages in the product development cycle:

• Design
• Prototyping
• Test
• Deployment.

Evaluation Method

1. A research plan is devised in consultation with the client, to determine which areas of the product or service need to be tested. The Usability Specialist then analyses the other competitor products to be tested and ensures a common set of tasks can be accomplished across all of the sites to give a fair benchmark. A set of tasks is then devised by the Specialist to extract information from the participants
2. One participant at a time is interviewed, typically for a maximum of 75 minutes. Each participant carries out the same tasks across all of the products or services. The task order for each participant is rotated to avoid tiredness or familiarity becoming a factor in the results
3. Before the test begins a questionnaire is administered to the participant to discover any assumptions, opinions and preferences held by the participant about each of the products or services to be tested. Each participant is given access to the product or service and a list of tasks that he/she is required to complete in order
4. The participant ‘thinks-aloud’ while performing the tasks verbalising his/her thoughts, feelings, and opinions while interacting with the system. After quantative measurements such as speed have been noted, the Specialist prompts the participant by asking direct questions about the product or service, in order to understand the participant’s mental model of the system and the tasks, and where she/he has trouble in understanding and using the system
5. The participant’s on-screen interactions, facial expressions and verbalisations can be captured and recorded for in-depth analysis if required. This is also useful for compiling video-highlights for stakeholders who were not able to attend the evaluation sessions
6. Following the completion of all of the tasks the participant is interviewed to measure overall perceptions and to understand particular observed behaviours. This allows the evaluator to retrace selected behaviours with the participant and fathom why certain actions were taken, without disturbing the user’s flow too much. This also provides the participants with an opportunity to compare and contrast all the services or products tested

7. After the interview the participant is given a questionnaire to complete. The questionnaire asks the participant about features/functionality that were considered most useful, desired features/functionality, problem areas and general likes and dislikes. This allows both quantative and qualitative measurements to be captured
8. Analyse findings: The results are gathered and analysed. Any behavioural trends are noted as well as any interesting outcroppings. Qualitative data in the form of participants’ verbalisations, post-test questioning and responses to open-ended questions within the questionnaire is gathered and any common threads are highlighted. The relevant quantitative data is graphed and presented along with the qualitative data and a psychological analysis of the reasons behind the behaviour and recommendations as to how to remove the problems encountered by the participants.

Audit method

1. A briefing meeting is held with the Usability Specialist and the client to determine what the priorities for the site are
2. Each product/ service is assessed against the main heuristics and the results noted
3. Performance criteria is analysed and graphed to quickly show the ratings for each site. The strengths and weaknesses of the sites are pointed out and recommendations are made on how to correct any usability issues on your product or service
4. Potential problems in your product or service are identified and explained, while practical recommendations are provided to correct the issues.

For a typical project covering a hybrid Competitive Analysis of one e-commerce website and 3 competitor websites, evaluating 8 target users:

Duration: 12.5 days (+ 7 days lapsed time for participant recruitment)

Cost: £5,000 – £7,000

Activities

1. Briefing meeting
2. Recruit participants
3. Research plan
4. Conduct audit
5. Conduct evaluations
6. Analyse findings
7. Deliver report.

Deliverables

• Research plan
• Competitive Analysis report (view sample Competitive Analysis report), including:
  • Analysis of audit results
  • Analysis of evaluation results
  • Graphs of quantative information
  • Comparison matrices and graphs to illustrate how your product or service compares to your competitors’
  • Documentation of the main usability issues
  • Rating of the severity of these issues on your product or service to assist in implementing alterations
  • Recommendations on how to eradicate usability issues
  • Best practice information
  • Results of questionnaires administered to participants
• Video highlights of the evaluation sessions (optional)
• PowerPoint presentation (optional)
• Preparation & delivery of video highlights (optional).