In July of 2016 I worked at Jaguar Land Rover, where we worked toward the in-vehicle infotainment system that would ultimately launch with the 2021 Land Rover Defender. Multiple teams from the UK and Portland were working on versions to test. The Portland office was testing two main versions, and as the User Experience Research team lead I was asked to test two head to head. This test had 12 participants and used a standard think aloud, moderated methodology. Participants were asked to complete the System Usability Scale, and graded using task completion rate. The test was conducted on early click-through prototypes using a laptop.
After the test, the data was compiled and compared, a short report was generated and shared with the design teams involved as well as local management. A PDF of the test is available here:
View The A/B Test
The design that did better out of two in the first round of testing was upgraded to a higher fidelity and tested again, this time with five participants. The second test was more complicated, testing additional features that weren’t included in the first test. Once again, this test measured task completion rate and the System Usability Scale. This test was also conducted on an early click-through prototype on a laptop.
As before, after the test the data was compiled and shared with the teams who worked on the prototype and local management as a PDF document. The test is available here:
View the Test
Many smaller tests were completed on this system, as well as additional design and development work. By October the teams were prepared to test the prototypes in a driving environment. Conducting a user test while driving a real vehicle on the street would be extremely dangerous, so tests were conducted under simulated driving. A Range Rover Sport was converted into a test vehicle, hooked into computers and used as a realistic controller to a video game based driving simulator. Linux computers were installed in the rear of the vehicle to power a touch screen prototypes could be loaded directly to.
Additional rounds of testing happened throughout 2016 and 2017. The prototypes were tested using occlusion goggles, which simulate glances off the road to the screen. Occlusion is a method used to test safety. Eye tracking tests were also conducted, in addition to more rounds of moderated user testing. My team found specific metrics that worked well for our moderated and simulated driving user tests: The System Usability Scale, task completion rate, and the Single Ease Question became favorites, especially when paired with driving data from the simulator or occlusion data.
In 2017 I left JLR to pursue other career options. The final product we worked on launched with the 2021 Land Rover Defender, seen above.