People have been tying to argue the ROI (return on investment) of usability for years. There are books and articles on the subject. Many arguments used are theoretical and some are real world examples. As both a practitioner in the field and a professor, this most recent story was both sad and infuriating to hear.
A student I know well went to take his first college midterm. It was computer-based and conducted at the University’s computer center. The exam was twenty multiple choice questions. After signing in, he used his mouse to select answers.
After answering twelve of the questions, he thinks he inadvertently hit the Enter key. As a result, the exam got submitted. No warning message saying that some questions had not been answered. No screen confirming his submission. Nothing. Feedback from the system did tell him that he had scored 50% on his midterm, though this was primarily because of the interface design flaw. (His score on the questions he was able to complete before this happened can be shown to be 83%.)
The exam proctor said she had no ability to reset the exam. She did hit the Back button (which the test instructions said not to do), and the student tried to complete the exam. But he was told by the system that a completed exam was already submitted for him. He was afraid the professor could do nothing about this even if he saw the email in time (the exam was only open for 24 hours). He was devastated that his first midterm, in a subject he knows well, would be compromised by a design flaw instead of being his fault.
When he returned to his dorm, frustrated and angry, he found that several people in his dorm reported that this same thing has happened at some point OVER THE LAST SEVERAL YEARS!
Luckily, the professor did reset the exam the next morning. This student went to take the exam again. This time he got fifteen minutes in and the exam got submitted again, though he swore he didn’t even hit the Enter key this time. Now the system reported he got 30% on his midterm.
If he was angry and frustrated before, now he was beside himself. He assumed it unlikely that his professor would give him another chance, and feared that his grade would be even worse. The professor did fortunately agree to reset the test one more time, saying it was the student’s last chance.
On the third try, he was more than careful. Thinking proactively, he removed the keyboard from the computer after logging into the exam since the questions were all answered with the mouse. As a result, the student was finally able to complete the exam without it getting submitted prematurely. He even got a score of 85%. And while he took the test, three other people went to the exam proctor and reported the same issue happened to them—-their exam was submitted before they had completed it.
There was NO message on any part of the exam telling students that pressing Enter would submit the exam. The professor didn’t warn him, nor did the exam proctor. This design flaw has apparently existed for years and many students have been caught by it. Many reported getting no opportunity to retake the exam. This student was lucky, though the stress of two days dealing with this probably took a heavy toll on him, both emotionally and on his opinion of the University.
Certainly someone could fix this design flaw. How could it even have survived functional testing if not usability testing, either user-based or through a heuristic evaluation? And how could it remain uncorrected for years? How many students have suffered over the years from professors who were unwilling or unable to reset the exam? How many students never even tried getting another chance and just accepted this situation? Barring fixing it, professors could warn students. The computer center could warn students. Signs could be placed at each terminal warning students. Even better, the exam proctor could move or remove the keyboard after a student logs in. So far, no attempts of any kind appear to have been made to fix or mitigate this problem.
We all have taken exams. We all have probably stressed over them, complained they didn’t reflect our actual knowledge, or were somehow unfair or deceptive. But imagine failing an exam because of a design flaw like this—-something very easy to fix—-which has been present and known about for years. Imagine a student having to retake a class because of this. (Imagine the parent’s frustration with having to pay for the class again over something like this.) This is wrong on so many levels.
It’s so sad, and infuriating, to keep hearing stories like this.
