Skip to main content

Module 5: Evaluation and Communication

Results that Drive Business

Learning objective: At the end of this module, you will be able to apply Usability Tests and Heuristic Evaluation to evaluate products, and communicate findings effectively and actionably to stakeholders.

Estimated time: 2 - 2.5 hours

Module 5 is the culmination of research, where the focus moves from understanding (Discovery) to the practical validation of solutions and the conversion of data into actionable business strategies.

5.1. Usability Testing

Usability testing is one of the most important and frequently used methods in UX Research.

Definition and Purpose

Usability Testing is a collection of techniques used to measure the characteristics of a user's interaction with a product.

Main objectives:

  • Evaluate how well users can complete specific tasks
  • Identify problems they encounter when interacting
  • Validate the effectiveness of design decisions

Important: Usability tests serve to evaluate, not to create. They evaluate the effectiveness of ideas already conceived.

Types of Usability Tests

Type Characteristics When to use it
In-person moderated Researcher and user in the same space When depth and detailed observation are needed
Remote moderated Via video conference in real time Geographically distributed users
Remote unmoderated User completes tasks alone, recorded Large samples, quantitative validation
With prototype Evaluates a design in development Early stages, low cost of change
With final product Evaluates the product in production Post-launch, continuous improvement

Step by Step: Conducting a Usability Test

1. Planning

  • Define objectives (aligned with research question)
  • Select tasks to evaluate (typically 5-7 tasks)
  • Recruit representative participants (5 users detect ~85% of problems, according to Nielsen)
  • Prepare prototype or product
  • Create session script

2. Task Design

Tasks should reflect the user's real objectives and goals.

Fundamental principle: Tell the user the objective they have to achieve, not the steps to follow.

❌ Poorly designed task ✅ Well designed task
"Click on the menu and select 'My orders'" "You want to know what status the order you placed yesterday is in"
"Find the help button" "You have a question about returns, how would you resolve it?"

Write a brief scenario that gives context to each task:

"Imagine you bought a product 3 days ago and it hasn't arrived yet. You want to know where it is."

3. Think-Aloud Protocol

The Think-Aloud Protocol requires participants to verbalize what they're thinking while completing a task.

Instruction to participant:

"While you perform the tasks, I'd like you to think aloud. Tell me what you're seeing, what you're thinking, what confuses you, what you expect to happen. There are no right or wrong answers, we're evaluating the design, not you."

Advantage: Captures the thought process and decision-making.

Variant - Retrospective Think-Aloud (RTA): The participant performs the task silently and then reviews the recording commenting on their thoughts. Useful when precise time data is needed.

4. During the session

  • Don't guide or help (unless the user is completely stuck)
  • Observe non-verbal behaviors (frustration, confusion)
  • Take notes of specific problems and verbatim quotes
  • Ask at the end about the overall experience

5. Analysis and Reporting

  • Identify problem patterns among users
  • Prioritize by severity and impact
  • Document with evidence (videos, quotes, metrics)

5.2. Heuristic Evaluation

Heuristic Evaluation is a usability inspection method that doesn't require users. Experts review the design comparing it with established principles.

History and Origin

The method was developed by Jakob Nielsen and Rolf Molich in 1990. Nielsen refined the list in 1994 in the paper "Enhancing the Explanatory Power of Usability Heuristics" presented at the CHI conference.

Advantages

  • Fast and relatively inexpensive
  • Doesn't require recruiting users
  • Can be done at any stage (even with low-fidelity prototypes)
  • 3-5 expert evaluators can find most problems

Nielsen's 10 Heuristics

1. Visibility of system status
The system should always keep users informed about what is going on, through appropriate feedback in reasonable time.

Example: Show a progress bar during loading, indicate how many steps remain in a form.

2. Match between system and the real world
The system should speak the users' language, with words, phrases and concepts familiar to the user. Information should appear in a natural and logical order.

Example: Use "Shopping Cart" instead of "Acquisition Basket".

3. User control and freedom
Users need a clearly marked "emergency exit" to leave unwanted states without having to go through an extended process.

Example: "Undo" option after deleting an item, visible "Cancel" button.

4. Consistency and standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

Example: If "Save" is used in one section, don't use "Store" in another.

5. Error prevention
Even better than good error messages is a careful design that prevents problems from occurring in the first place.

Example: Disable "Submit" button until all required fields are complete; confirm before permanently deleting.

6. Recognition rather than recall
Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another.

Example: Show recently viewed products; autocomplete in search fields.

7. Flexibility and efficiency of use
Accelerators (invisible to the novice) can speed up the interaction for the expert user. Allow customization.

Example: Keyboard shortcuts, advanced configuration options.

8. Aesthetic and minimalist design
Dialogues should not contain irrelevant or rarely needed information. Every extra unit of information competes with relevant units.

Example: Remove unnecessary text, prioritize primary actions over secondary ones.

9. Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), indicate the problem precisely, and suggest a constructive solution.

Example: Instead of "Error 404", say "We didn't find this page. You can return to home or search for what you need."

10. Help and documentation
Even though it's better if the system can be used without documentation, it may be necessary to provide help. This should be easy to search, focused on the user's task, and offer concrete steps.

Example: Contextual FAQs, tooltips, help chat.

Step by Step: Conducting a Heuristic Evaluation

  1. Select evaluators: Ideally 3-5 usability experts.

  2. Define scope: Which flows or sections will be evaluated?

  3. Each evaluator reviews independently: Avoid group biases.

  4. Document each problem found:

    • Heuristic violated
    • Location of problem
    • Problem description
    • Severity (scale 0-4)

  5. Consolidate findings: Combine individual evaluations.

  6. Prioritize by severity and impact.

Severity Scale

Level Description
0 Not a usability problem
1 Cosmetic problem - fix if there's time
2 Minor problem - low priority
3 Major problem - high priority
4 Usability catastrophe - must fix before launch

Heuristic Evaluation Template

HEURISTIC EVALUATION

Product evaluated:
Evaluator:
Date:

| # | Location | Violated Heuristic | Problem Description | Severity (0-4) | Recommendation |
|---|----------|-------------------|---------------------|----------------|----------------|
| 1 |          |                   |                     |                |                |
| 2 |          |                   |                     |                |                |

5.3. Inclusion, Accessibility and Latin American Context

Web Accessibility

Web accessibility ensures that products and services can be used by people with the widest range of characteristics and capabilities.

Why does it matter?

  • Inaccessible products exclude a significant percentage of the population
  • Accessibility is usability for users with disabilities
  • In many countries it's a legal requirement

WCAG Standards

The Web Content Accessibility Guidelines (WCAG) are the international standard. They are organized into three conformance levels:

  • Level A: Minimum required
  • Level AA: Recommended (common goal)
  • Level AAA: Optimal

Recommendation: Include users with disabilities in usability tests when possible.

Regionalization in Latin America

Cultural and social context influences design practices. There are no universal "best practices".

Aspects to consider in Latam:

Communication:

  • Formal vs. informal treatment varies by country (tú/usted/vos)
  • Local idioms and expressions
  • Sensitivity to cultural topics

Payment Processes:

  • Great diversity of methods (local cards, transfers, cash payments in stores)
  • Variable penetration of digital money
  • Different legislations by country

Example of cultural adaptation:
A travel site can promote the same destination (Santiago, Chile) differently depending on country of origin: highlighting shopping for Argentinians vs. snow and wines for Brazilians.

5.4. Communicating Findings

Communicating findings is the pinnacle of research effort. It's not enough to inform; the objective is to persuade audiences to act.

The Strategic Role of Communication

Researchers must be good communicators and have business vision to express how their work contributes to business objectives.

Key principles:

  • Research is useless if no one acts on it
  • Deliverables must be actionable
  • UX research is a team sport

Avoiding the "Reports Cemetery"

Extensive 10-20 page reports risk collecting dust. If stakeholders need to turn the page, the report is too long.

More effective alternatives:

  • 1-page executive summaries
  • Posters or infographics (hallway evangelism)
  • Brief presentations with key findings
  • Videos of key moments from sessions

Storytelling: Telling the Data Story

Storytelling is the most powerful approach to transmit insights and persuade the audience to act.

Elements of storytelling in UX:

  • Characters: The Personas from research
  • Conflict: The identified problems and frustrations
  • Resolution: Design recommendations

Generate empathy:

  • Use videos of users expressing frustration
  • Include verbatim quotes with real user language
  • Show, don't just tell

Structure of an Effective Report

  1. Executive Summary (1 page maximum)

    • Research question
    • Methodology in 2-3 lines
    • 3-5 main findings
    • Priority recommendations

  2. Context and Methodology (brief)

    • Study objectives
    • Participants
    • Methods used

  3. Detailed Findings

    • Organized by topic or severity
    • Each finding with evidence (quotes, videos, data)
    • Associated specific recommendations

  4. Next Steps

    • Concrete actions
    • Suggested responsible parties
    • Prioritization

Prioritizing Findings

Not all problems have the same urgency. A simple matrix:

High Impact Low Impact
Easy to Resolve Do first Do if there's time
Difficult to Resolve Plan Consider if it's worth it

Final Course Exercise

Integrating Project:

Select a commonly used website or app in your country (bank, supermarket, public service). Perform:

  1. Heuristic Evaluation: Identify 5 problems using Nielsen's heuristics.

  2. Mini Usability Test: Ask 2-3 people to complete 3 tasks while thinking aloud.

  3. 1-Page Report: Summarize main findings and 3 actionable recommendations.

Module 5 References

  • Nielsen, J. (1994). Enhancing the Explanatory Power of Usability Heuristics. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '94).
  • Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
  • Krug, S. (2014). Don't Make Me Think, Revisited. New Riders.
  • W3C. (2018). Web Content Accessibility Guidelines (WCAG) 2.1.

Course Conclusion

A good UX researcher equips themselves with a compass and a map. The compass is User Experience (UX), which points to human emotions and needs. The map is User-Centered Design (UCD), an iterative process that tells us we shouldn't start building until we know where the user stands and why they want to go where they're going.

Ignoring initial research is like building a house without foundations: it may look attractive, but it will collapse when faced with the reality of users.

Recommended Additional Resources

Fundamental Books

  • Norman, D. (2013). The Design of Everyday Things. Basic Books.
  • Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
  • Cooper, A., Reimann, R., & Cronin, D. (2007). About Face 3. Wiley.
  • Portigal, S. (2013). Interviewing Users. Rosenfeld Media.
  • Krug, S. (2014). Don't Make Me Think, Revisited. New Riders.

Reference Websites

Free Tools

  • Prototyping: Figma (free version)
  • Card Sorting / Tree Testing: Optimal Workshop, UXtweak
  • Surveys: Google Forms, Typeform (free version)
  • Remote session recording: Zoom, Google Meet
  • Analytics: Google Analytics, Hotjar (free version)

About This Course

This course was developed for the UX community in Latin America and is available for free at uxr.cl.

The content is based on recognized academic and professional sources in the field of User Experience and user research.

Last updated: 2025