Studio 2 - Applying Quantitative Usability Methods

The aim of this studio is to design a usability evaluation on a Samsung E840 (FIG 1). The purpose of this evaluation will be to measure the Efficiency of Visibility.

FIG 1 – Samsung E840

The plan below shows the scope for the design of the usability evaluation.

Device –	SAMSUNG E840 (FIG 1)
Attribute -	Measuring the Efficiency of Visibility
Metrics -	Efficiency measures
Heuristic -	Visibility of system

Efficiency Measures (metrics)

Listed below, shows the three efficiency metrics that will be measured against in each task when carrying out the test. This is what we are going to measure.

1. Time to complete a task
2. Time spent on correcting errors
3. Relative efficiency compared with an expert user

Heuristic – Visibility

The heuristic that we will use is measuring the Visibility of system. How the user interacts with the phone in the quickest/appropriate way.

Tasks

The usability is measured relative to the users' performance on a given set of test tasks. Listed below, shows the three specific tasks that are focused on the attribute measurement.

4. Task 1 – Find the Games section on the phone

5. Task 2 – Take a photo of you hand using the phones camera

6. Task 3 – Set the alarm on the phone to 6pm

Now that the usability evaluation had been designed we will now look at the Best Case and Worst Case level. An expert user will be used to do the best case test and a novice user will be used to carry out the test in the worst case. These results will be compared with the other users to differentiate the comparisons.

Best Case (expert user)

Below shows the results:

Task 1 – Find the Games section on the phone
Time -	10 sec
Errors -	0

Task 2 – Take a photo of you hand using the phones camera
Time -	9 sec
Errors -	0

Task 3 – Set the alarm on the phone to 6pm
Time -	15.5 sec
Errors -	0

Worst Case (novice user)

Below shows the results:

Task 1 – Find the Games section on the phone
Time -	22.5 sec
Errors -	1 – Pressing the back button to go out of a submenu

Task 2 – Take a photo of you hand using the phones camera
Time -	13 sec
Errors -	1 – Pressing the back button to go out of a submenu

Task 3 – Set the alarm on the phone to 6pm
Time -	1 min 10 sec
Errors -	6 - Pressing the back button to go out of a submenu (x5) - Inputting the wrong time(x1)

Now that we have these results, they will be compared with the other users to differentiate the comparisons. There will be 3 who will carry out the usability test. The test will be done individually through each task and their actions will be record and measured against the usability design. A designed test form will be used to record what happens in the test. The usability test forms and given instruction to the users as show below.

User Instructions

“Hi. We have a usability test that will be tested on a Samsung E840. We will give you three specific task and you use this mobile device to carry the test on. We will be doing a live analysis on this test Are you ready? Ok.

1. Find the Games section on the phone

The task is to:

2. Take a photo of you hand using the phones camera

And the final task is to:

3. Set the alarm on the phone to 6pm

Ok that’s the end of the test. Thank you for you time and support.”

Usability Test Form