CTFL-UT ISTQB Certified Tester-Foundation Level - Usability Testing Questions and Answers

The primary goal of usability testing is to observe and record user behavior while interacting with the system. A location where user actions cannot be seen or recorded (option A) completely undermines the test's purpose and is therefore not suitable.

While options B, C, and D are not ideal in all cases, they can still be valid settings depending on the context. Testing in a user’s natural environment (B) increases ecological validity. Remote testing via video link (C) is widely accepted. Testing in a public setting (D) can be useful for mobile apps or services intended for real-world use.

A usability test script is a structured document used by the moderator to ensure consistency across test sessions. It typically includes pre-session briefing text, pre- and post-session interview questions, and the specific usability tasks to be performed. However, it does not include answers to the test questions, as usability tests focus on observing user behavior and performance—not testing users’ knowledge. Providing answers would bias results and invalidate findings. Thus, option D is incorrect and the best choice.

Common mistakes in usability testing include:

Ignoring critical results (ii): A significant issue if findings are not acted upon.

Scheduling the test too late (iv): This can lead to usability problems being discovered too late to be addressed effectively.

Having the wrong focus (v): For instance, focusing on aesthetics instead of usability goals.

Option i ("user is too active") is not a problem—active participation is necessary. Option iii ("no observers involved") is not a typical mistake, since tests can still be valid even with recorded sessions and later review. Hence, the correct answer is A: ii, iv & v are true.

Option B represents a realistic, goal-oriented scenario that reflects how an actual user would interact with the website. It incorporates context, user intent, constraints, and desired outcome—all characteristics of well-designed usability tasks. Option A is vague and lacks real-world motivation. Option C is too prescriptive and limits insight into user behavior, while D is a survey question, not a usability task. According to Nielsen Norman Group and ISO 25062, the best usability tasks are scenario-based, realistic, and outcome-driven—making B the correct answer.

The Web Content Accessibility Guidelines (WCAG) define three levels of conformance for web content accessibility:

Level A (Single A) is the minimum level of compliance.

Level AA (Double A) is the standard recommended for most websites, balancing accessibility and design.

Level AAA (Triple A) is the highest and most comprehensive level of accessibility, covering the widest range of needs.

There is no such level as “Quadruple A (AAAA),” making option D invalid.

Thus, WCAG Triple A (AAA) promises the most accessible content, even though it may not always be practically achievable for all types of content.

Usability is defined in ISO 9241-11 as the extent to which a system can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use. Without knowing the context—such as product complexity, user familiarity, device type, or purchasing habits—it is impossible to judge whether five minutes is good or bad. For complex items, five minutes may be reasonable, while for one-click purchases, it may be excessive. Thus, usability cannot be assessed solely based on one metric like time—it must be evaluated within its full usage context.

Usability labs provide a controlled environment that offers several advantages for observing and recording usability test sessions. Specifically, they allow for easy observation of participants (ii), standardized conditions for each session (iv), and facilitate audio and video recording for detailed analysis (v). However, the context is not natural (i), because the lab is an artificial setting rather than the user’s actual environment. Also, setting up a professional usability lab is not inexpensive (iii); it typically involves significant costs for equipment, space, and staffing. Therefore, options ii, iv, and v are true, while i and iii are false.

Positive usability findings play a key role in reinforcing what is working well in the user interface. They help:

A: Build developer confidence and soften criticism when pointing out issues.

C: Provide a complete picture of usability strengths and weaknesses.

D: Inform developers what should remain unchanged during redesign.

However, B is incorrect because justifying usability test costs should not rely solely on positive findings. Cost justification should come from the overall impact of usability on user satisfaction, performance, and business outcomes—not just good results.

The usability test moderator plays a crucial role during the execution phase of a usability test. Their principal task is to facilitate the test sessions by guiding participants through predefined testing tasks, observing behaviors, and ensuring that the test environment is neutral and free from bias. Defining testing tasks, however, is primarily done during the test design phase by the usability test designer or analyst, but the moderator ensures these tasks are clearly communicated and administered to participants during the session. Logging usability problems (option B) is often done by observers or analysts reviewing session recordings or notes, not directly by the moderator during live testing. Creating a survey plan (option C) and writing the usability test plan (option D) are preparatory activities generally done before the test sessions and are not the moderator's main focus. Therefore, the principal role of the moderator centers on conducting the sessions and managing task delivery effectively to obtain authentic user interaction with the system under test.