Operationalizing Product Legibility

Research State Snapshot

• Theory Version: Product Genome preliminary theory after RP-PROD-001
• Knowledge Base Version: Product Genome KB 0.2
• Highest Confidence Areas: Task-specific usability measurement; representative-user testing; motor target tradeoffs; critical-task analysis; feedback and error observation
• Lowest Confidence Areas: Cross-product aggregation; long-term learning; emotional meaning; weighting among legibility dimensions
• Largest Remaining Unknown: Whether measurements from different products and contexts can be normalized without erasing critical differences
• Active Research Streams: Product Legibility, Complexity Allocation, Rams/Sapper case coding, repairability, product longevity
• Recently Invalidated Ideas: Minimalism as a universal law; visual simplicity as a proxy for usability; additive legibility scoring without critical gates
• Priority Changes: Product case-study coding should use the protocol in this REP before designer-genome scores are assigned

Executive Summary

What Was Accomplished

This research run addressed the highest-ranked open question from RP-PROD-001:

How should Product Legibility be measured?

Five investigation cycles were completed:

1. Determine what kind of construct Product Legibility is.
2. Test whether its six components can be measured independently.
3. Test whether a single composite score is defensible.
4. Examine whether classic predictive laws can supply quantitative foundations.
5. Build an executable evaluation protocol and falsification plan.

The resulting model is no longer merely descriptive. It now specifies units of analysis, required context, observable measures, failure criteria, and reporting rules.

Major Discovery

Product Legibility is not an intrinsic property of an object.

It is a measured relationship among:

Product × Task × User Population × Environment × Time

A control can be highly legible for a trained technician standing in good light and illegible for an older first-time user wearing gloves under time pressure. Therefore, statements such as “this product is intuitive” or “this control is legible” are incomplete unless the task and population are named.

Strongest Conclusion

The Product Legibility Model should use a gated profile, not a simple arithmetic average.

A product interaction must pass six gates:

1. Actual Action Possibility
2. Discoverability
3. Interpretability
4. Executability
5. Feedback Closure
6. Consequence Comprehension

A severe failure in a critical gate can invalidate the interaction even when the other scores are high. A dangerous control can be easy to find, quick to operate, and pleasant to use while still failing because users misunderstand its consequence.

Confidence

Moderate-High.

Confidence is high that usability must be evaluated for specified users, goals, and contexts, and that effectiveness, efficiency, errors, and satisfaction require direct observation. Confidence is moderate in the exact proposed scoring and thresholds because they have not yet been validated across product categories.

Remaining Uncertainty

The protocol can reliably structure measurement, but three major questions remain:

• How should scores be normalized across products with radically different tasks?
• How should learning over repeated use alter the legibility profile?
• What thresholds should define pass, caution, and failure for non-safety-critical consumer products?

Original Objective

Operationalize MODEL-001 from RP-PROD-001:

L = f(A, D, I, E, F, C)

Where:

• A = actual action possibility
• D = discoverability
• I = interpretability
• E = executability
• F = feedback quality
• C = consequence comprehension

The prior model proposed these components but did not specify measurement units, aggregation rules, thresholds, or experimental procedure.

Scope

Included

• Physical products and hybrid physical-digital products
• Discrete and continuous controls
• First-use and repeated-use testing
• Representative users and contexts
• Performance, error, confidence, and biomechanical measures
• Safety-critical and ordinary consumer tasks

Excluded

• Purely aesthetic judgment
• Brand preference
• Manufacturing quality except where it affects interaction
• Full emotional durability assessment
• Full repairability assessment
• Longitudinal ownership beyond learnability and retention measures

Repository Context

RP-PROD-001 established that:

• minimalism is not a universal law;
• affordance is relational rather than visual alone;
• feedback closes a control loop;
• compatibility is more precise than “intuitiveness”;
• complexity can be transferred rather than removed;
• Product Legibility is a promising model but lacked operational metrics.

This REP advances that theory and should be used before large-scale Rams and Sapper product coding begins.

Current Understanding

Product Legibility concerns whether a person can:

1. recognize that an action is possible;
2. locate the means of action;
3. predict what the action will do;
4. physically perform it;
5. detect and interpret the system response;
6. understand the resulting state and consequences.

These stages resemble a control loop, but they are not always strictly sequential. Experienced users may act through stable spatial memory before consciously interpreting a label. Physical constraints may prevent an invalid action before consequence comprehension is required. Feedback can update interpretation during execution.

The model is therefore best treated as a network of necessary interaction conditions, not a universal linear pipeline.

Key Discoveries

1. Product Legibility is relational and conditional. It must be reported for a named task, population, environment, and exposure level.
2. Outcome metrics and diagnostic metrics are different. Completion rate tells whether the interaction succeeded; the six gates help explain why.
3. Averages conceal catastrophic weaknesses. Critical failures require gating or minimum rules.
4. Discoverability is not equivalent to visual salience. Search time, first action, and unaided discovery are stronger measures.
5. Interpretability must be tested before action. Asking users to predict results separates comprehension from trial-and-error learning.
6. Executability requires capability distributions, not an “average user.” Reach, force, precision, posture, and sensory capability must be population-specific.
7. Feedback quality is multidimensional. Latency, detectability, discriminability, compatibility, persistence, and proportionality should be measured separately.
8. Confidence is useful only when calibrated against correctness. High confidence paired with wrong predictions is a dangerous form of illegibility.
9. Fitts' and Hick's laws are local models. They predict constrained aspects of movement and choice, not overall product quality.
10. Learnability can improve performance while preserving exclusion. A product that becomes usable after training may still have poor first-use legibility.

Research Log

Cycle 1: Define the Construct

Objective

Determine whether Product Legibility can be treated as an intrinsic product attribute.

Hypothesis

HY-PROD-008: Product Legibility is a stable property of product form and can be scored without specifying a task or user population.

Supporting Evidence Sought

• Stable rankings across tasks and user groups
• Usability standards defining usability independently of context
• Product features that predict success regardless of population

Evidence Found

ISO 9241-11 defines usability in relation to specified users achieving specified goals with effectiveness, efficiency, and satisfaction in a specified context of use. NIST usability testing likewise emphasizes representative users performing representative tasks and collecting completion, error, time, and satisfaction data. FDA human-factors guidance requires analysis of intended users, uses, use environments, and critical tasks rather than generic judgments about interface quality.

NASA human-integration material similarly treats anthropometry, biomechanics, strength, environment, equipment, and task as coupled design inputs.

Evidence Against

The same control can vary in usability with gloves, darkness, vibration, impairment, expertise, urgency, posture, and divided attention. No strong source supports context-free usability scoring.

Attempted Falsification

Visual form sometimes produces stable expectations across users, suggesting that product-level properties matter. However, stable cues do not eliminate dependence on bodily capability, task, and environment. Product properties influence legibility but do not contain it independently.

Analysis

Product Legibility should be indexed as:

PL(product, task, population, environment, exposure)

The exposure term is required because first use, learned use, and expert use may have different profiles.

Conclusion

HY-PROD-008 rejected.

Product Legibility is a relational measurement.

Confidence

High.

Next Step

Identify measurable variables for each legibility gate.

Cycle 2: Operationalize the Six Gates

Objective

Determine whether each component can be associated with observable measures.

Hypothesis

HY-PROD-009: Each Product Legibility component can be measured using at least one behavioral or biomechanical variable rather than subjective rating alone.

Evidence Found

Human-factors practice supplies multiple direct measures:

• successful task completion;
• time on task;
• use errors and close calls;
• assistance required;
• repeated actions;
• deviations from expected sequence;
• workload and satisfaction;
• reach, force, posture, and precision requirements;
• state-identification and prediction accuracy.

Fitts' work demonstrates that movement time varies systematically with movement amplitude and target width under constrained aiming tasks. Hick's experiments show that choice reaction time can vary with information uncertainty under specific stimulus-response conditions. Card, Moran, and Newell's Keystroke-Level Model demonstrates that expert error-free task time can be estimated by decomposing a known procedure into operators.

These do not validate the whole Product Legibility model, but they show that important subcomponents can be operationalized.

Evidence Against

Some variables are difficult to isolate. Discoverability and interpretability interact. A user may fail to find a control because the task representation is wrong, not because the control lacks salience. Satisfaction can be influenced by brand, aesthetics, expectations, and prior experience. Anthropometric accommodation does not ensure comfortable or safe execution.

Attempted Falsification

A purely questionnaire-based scale would be cheaper and could produce a single score. It fails because users may report confidence after incorrect actions, may not recall near-errors, and may adapt to poor design. Direct observation remains necessary.

Conclusion

HY-PROD-009 provisionally supported.

All six gates can be associated with observable variables, but no gate should be inferred from a single measure.

Confidence

Moderate-High.

Next Step

Test aggregation.

Cycle 3: Test the Single-Score Hypothesis

Objective

Determine whether the six components should be averaged into one Product Legibility score.

Hypothesis

HY-PROD-010: A weighted arithmetic mean of the six component scores provides a valid overall Product Legibility score.

Evidence That Would Support It

• Weak components can be compensated by strong components without unacceptable outcomes
• Similar average scores predict similar task success and risk
• Weightings remain stable across product categories

Evidence Against

FDA critical-task analysis contradicts simple compensation. A low-frequency but severe use error may dominate risk even when most actions are fast and satisfactory. ISO's context-specific definition also cautions against assuming stable weights across goals and environments.

A control can score highly on discoverability, executability, and immediate feedback while producing a confidently incorrect state interpretation. An arithmetic mean can label that interaction acceptable.

Counterexample

Consider an emergency shutoff:

• Discoverability: 95
• Interpretability: 40
• Executability: 95
• Feedback: 90
• Consequence comprehension: 20

The average is 68, but the interaction is unsafe because users do not understand what will stop or whether shutdown occurred.

Attempted Falsification

A weighted mean with very high weights on safety-relevant dimensions could reduce this problem. However, weights would need to vary by task consequence, effectively reintroducing gates and context-specific rules.

Analysis

The correct architecture is hierarchical:

1. Critical gate rule: Any critical gate below threshold causes failure.
2. Task outcome rule: Critical-task completion and serious-use-error rates are reported independently.
3. Profile rule: Remaining dimensions are displayed as a profile.
4. Composite rule: A summary score may be calculated only for comparisons within the same task, population, environment, exposure, and consequence class.

Conclusion

HY-PROD-010 rejected.

A context-free additive Product Legibility score is not defensible.

Confidence

High.

Next Step

Determine how classical quantitative laws can strengthen individual gates without being overgeneralized.

Cycle 4: Test Classical Predictive Laws

Objective

Assess whether Fitts' law, Hick's law, and task-decomposition models can serve as universal mathematical foundations for Product Legibility.

Hypothesis

HY-PROD-011: Established human-performance laws can be combined to predict overall product interaction quality.

Evidence Found

Fitts' original experiments established a reliable speed-accuracy relationship for constrained aimed movement. Hick found a relationship between reaction time and stimulus uncertainty in choice tasks. The Keystroke-Level Model predicted expert, routine, error-free execution time by decomposing a known method.

These models offer useful local predictions:

• larger or nearer targets generally reduce aiming difficulty;
• more uncertain choices can increase selection time;
• longer action sequences generally increase expert execution time;
• system response delay adds directly to task time and can disrupt control.

Evidence Against

Each model has assumptions that Product Genome must preserve:

• Fitts' law does not measure whether users know which target to select.
• Hick's law is affected by familiarity, probability, grouping, stimulus-response compatibility, and search structure.
• KLM assumes skilled users, a known correct method, and error-free routine action.
• None directly measures safety, meaning, physical exclusion, attachment, repairability, or long-term trust.

Attempted Falsification

A product interaction could be decomposed into movement, choice, mental preparation, and response operators. This may predict task time after the correct method is known. It still cannot predict discovery, mistaken mental models, or whether the user chooses the correct goal.

Conclusion

HY-PROD-011 rejected in strong form and retained in narrow form.

Classical laws should be embedded as gate-specific models, not promoted to universal product laws.

Confidence

High.

Next Step

Build an executable protocol with explicit separation between outcomes and diagnostics.

Cycle 5: Construct and Stress-Test the Protocol

Objective

Create a protocol another research agent can apply without additional context.

Hypothesis

HY-PROD-012: A standardized sequence can produce comparable Product Legibility profiles while preserving context and critical failures.

Protocol Stress Tests

The draft protocol was tested conceptually against four interaction classes:

1. A single-purpose mechanical switch
2. A continuous lamp-positioning mechanism
3. A multifunction appliance control
4. A safety-critical medical-device task

Evidence Found

The six gates apply to all four classes, but their measures and thresholds differ. Continuous controls require trajectory quality and proportional feedback. Multifunction controls require mode awareness and state persistence. Safety-critical tasks require use-error severity and consequence comprehension to dominate.

Evidence Against

Full numerical comparability across categories remains weak. A discovery time of two seconds may be excellent for a hidden maintenance latch and unacceptable for an emergency control. The protocol must preserve raw measures and context rather than only normalized scores.

Conclusion

HY-PROD-012 provisionally supported.

The protocol is suitable for research execution, but cross-category ranking is not yet justified.

Confidence

Moderate-High.

Next Step

Apply the protocol to a pilot set of matched Rams and Sapper products and examine inter-rater reliability.

Proposed Model

MODEL-PROD-001 Revision A: Gated Product Legibility Model

Unit of Analysis

A defined interaction, not an entire product.

Example:

Incorrect: Braun SK 4 legibility
Correct: Ability of a first-time adult user to select and begin radio playback in a quiet home environment

Model

PL = Profile(A, D, I, E, F, C | T, P, X, R)

Where:

• A: Actual Action Possibility
• D: Discoverability
• I: Interpretability
• E: Executability
• F: Feedback Closure
• C: Consequence Comprehension
• T: Task
• P: Population
• X: Context/environment
• R: Repetition or exposure level

Gate Rule

Interaction Pass =
  Task Outcome Pass
  AND all Critical Gates >= task-specific thresholds

Noncompensation Rule

A critical failure in one gate cannot be canceled by high performance in another.

Summary Score Rule

A composite may be used only when:

• the task is identical;
• the user population is identical;
• the environment is identical;
• exposure level is identical;
• consequence class is identical;
• critical gates have passed;
• raw measures remain available.

Measurement Architecture

Layer 1: Outcome Measures

These determine whether the interaction worked.

Layer 2: Gate Measures

A. Actual Action Possibility

Question: Can the target user perform the action under the target conditions?

Measures:

• percentage of population physically capable;
• reach accommodation percentile;
• required force relative to capability distribution;
• required range of motion;
• grip or pinch type;
• sensory requirement;
• prerequisite equipment or posture;
• successful manipulation under environmental constraints.

Failure examples:

• control cannot be reached from required position;
• force exceeds capability for part of intended population;
• action requires visual acuity unavailable in intended context;
• gloves prevent grip or touch recognition.

D. Discoverability

Question: Can the user locate the action opportunity without inappropriate help?

Measures:

• unaided discovery rate;
• time to first fixation or first contact;
• time to correct control selection;
• incorrect-control approaches;
• requests for help;
• search path length;
• first-action correctness.

Important distinction:

Visual salience is an input variable. Discoverability is a behavioral result.

I. Interpretability

Question: Before acting, can the user predict what the control will do?

Measures:

• predicted-effect accuracy;
• predicted-direction accuracy;
• mode identification accuracy;
• label/icon comprehension;
• mapping accuracy;
• confidence in prediction;
• confidence calibration.

Calibration measure:

Calibration Error = |Confidence - Accuracy|

High confidence plus low accuracy is a severe warning.

E. Executability

Question: Can the user carry out the intended action accurately and efficiently?

Measures:

• task movement time;
• miss or slip rate;
• correction count;
• overshoot;
• force variability;
• posture deviation;
• sequence length;
• hand or visual transitions;
• workload;
• fatigue across repetition.

Applicable local models:

• Fitts-type target analysis;
• movement trajectory analysis;
• anthropometric accommodation;
• task decomposition;
• force and precision testing.

F. Feedback Closure

Question: Can the user detect, distinguish, and interpret the product's response?

Measures:

• response latency;
• feedback detection rate;
• feedback identification accuracy;
• state-identification accuracy;
• repeated-action rate;
• overcorrection rate;
• feedback persistence;
• cross-modal agreement;
• perceived versus actual state.

Candidate dimensions:

Latency
Detectability
Discriminability
Compatibility
Proportionality
Persistence
Redundancy

C. Consequence Comprehension

Question: Does the user understand the resulting state, risk, reversibility, and downstream effect?

Measures:

• final-state identification;
• hazard awareness;
• reversibility knowledge;
• downstream-effect prediction;
• recovery-path knowledge;
• delayed-state recall;
• discrepancy between perceived and actual state.

This gate is especially important for mode changes, automation, destructive actions, medication delivery, vehicle controls, and emergency systems.

Layer 3: Experience Measures

These explain acceptance but do not replace performance measures.

• perceived effort;
• satisfaction;
• trust;
• comfort;
• perceived control;
• perceived complexity;
• aesthetic coherence;
• preference.

Product Legibility Evaluation Protocol

PLP-001

Step 1: Define the Interaction

Record:

• product;
• task goal;
• start state;
• successful end state;
• critical and noncritical errors;
• expected action sequence;
• possible alternate valid sequences.

Step 2: Define Population

Record distributions, not just averages:

• age range;
• experience;
• relevant disability or impairment;
• handedness if material;
• strength and reach considerations;
• language and literacy;
• professional training;
• familiarity with product conventions.

Step 3: Define Environment

Record:

• illumination;
• noise;
• vibration;
• posture;
• mobility;
• divided attention;
• gloves or protective equipment;
• time pressure;
• social conditions;
• environmental hazards.

Step 4: Define Exposure

At minimum:

• first encounter;
• after brief instruction;
• after repeated practice;
• retention after delay, when relevant.

Step 5: Conduct Pre-Action Elicitation

Before permitting interaction, ask participants to identify:

• what can be acted upon;
• what they expect each relevant action to do;
• the current product state;
• likely consequences.

This prevents trial-and-error success from being mistaken for initial legibility.

Step 6: Observe Action

Record:

• first action;
• path;
• errors;
• hesitation;
• corrections;
• repeated actions;
• assistance;
• completion;
• time;
• physical difficulty.

Step 7: Test Feedback and State Understanding

Immediately after action, ask:

• Did the product register the action?
• What state is it now in?
• What changed?
• Is the action complete?
• Can it be reversed?

Compare answers with actual state.

Step 8: Repeat Under Relevant Stressors

Use only realistic stressors:

• low light;
• noise;
• gloves;
• divided attention;
• fatigue;
• urgency;
• reduced mobility;
• repeated use.

Step 9: Apply Critical Gate Rules

Define thresholds before testing. Do not lower thresholds after observing results.

Step 10: Report Profile and Raw Data

Required output:

Task outcome
Critical errors
A profile
D profile
I profile
E profile
F profile
C profile
Experience measures
Population and context
Exposure level
Uncertainty and limitations

Confirmed Findings

CF-PROD-011

Usability and Product Legibility cannot be evaluated independently of specified users, goals, and context.

Evidence: EV-PROD-011, EV-PROD-012, EV-PROD-013

Confidence: High.

CF-PROD-012

Representative-user task testing supplies stronger evidence than expert aesthetic judgment alone.

Evidence: EV-PROD-011, EV-PROD-012

Confidence: High.

CF-PROD-013

Task completion, time, errors, assistance, and satisfaction are distinct outcome dimensions and should not be collapsed without preserving raw values.

Evidence: EV-PROD-011, EV-PROD-012, EV-PROD-014

Confidence: High.

CF-PROD-014

Motor target size and movement amplitude can predict movement time under constrained aiming tasks.

Evidence: EV-PROD-015

Confidence: High within the law's boundary conditions.

CF-PROD-015

Choice uncertainty can affect reaction time, but the effect is conditional on task structure, probability, familiarity, and compatibility.

Evidence: EV-PROD-016, EV-PROD-017

Confidence: High.

CF-PROD-016

Expert task execution can be decomposed into operators for time prediction, but such models do not measure discoverability or novice mental-model errors.

Evidence: EV-PROD-018

Confidence: High.

CF-PROD-017

A severe use-related risk cannot be made acceptable by strength in unrelated usability dimensions.

Evidence: EV-PROD-012

Confidence: High for safety-critical contexts; moderate-high as a general design principle.

Rejected Hypotheses

RH-PROD-008

Product Legibility is an intrinsic object property.

Rejected because performance depends on task, population, environment, and exposure.

RH-PROD-009

A questionnaire can replace behavioral testing.

Rejected because confidence and satisfaction can remain high despite incorrect action or state understanding.

RH-PROD-010

The six gate scores can always be averaged.

Rejected because critical failure can be hidden by compensation.

RH-PROD-011

Fitts' law is a general law of ease of use.

Rejected because it models aimed movement, not target selection, comprehension, safety, or product value.

RH-PROD-012

Hick's law proves that fewer options are always better.

Rejected because structured, familiar, probable, and compatible choices do not behave like arbitrary equiprobable alternatives.

RH-PROD-013

Successful task completion proves the interaction was legible.

Rejected because users may succeed by guessing, repeated action, external cues, or recovery from error.

Emerging Patterns

EP-PROD-007: The Success Mask

A final successful outcome can conceal a poor interaction path.

A user may complete a task after multiple errors, accidental discovery, excessive force, or repeated activation. Product analysis that records only success will overestimate legibility.

EP-PROD-008: Confidence Inversion

The most dangerous interaction is not always one users find difficult. It may be one they perform confidently while misunderstanding the result.

This connects Product Legibility to calibration research, automation bias, medical-device use error, and mode confusion.

EP-PROD-009: Legibility Migration Through Learning

Repeated use can move performance from conscious interpretation toward recognition, spatial memory, and motor routine.

The product has not necessarily become more legible in its first-use state. The user has absorbed part of the product's complexity.

EP-PROD-010: Measurement Must Follow Consequence

The importance of a gate depends on what failure means.

A delayed state indication may be mildly annoying in a lamp and dangerous in medication delivery. Universal dimensions can exist without universal thresholds.

EP-PROD-011: Product Families Can Externalize Learning

Stable mappings and control grammars may reduce discovery and interpretation costs across products. This suggests that legibility can exist at the family or ecosystem level, not only within one product.

Candidate Laws

LAW-PROD-008: Contextual Legibility Law

Hypothesis

An interaction is legible only relative to a specified task, population, environment, and exposure level.

Prediction

Changing any of these conditions can change the Product Legibility profile without altering the physical product.

Supporting Evidence

EV-PROD-011 through EV-PROD-014.

Counter Evidence

Some strong physical cues remain stable across many contexts, but no evidence shows complete independence from context.

Confidence

High.

LAW-PROD-009: Critical Gate Law

Hypothesis

For consequential tasks, failure in one necessary interaction gate cannot be offset by high scores in other gates.

Prediction

Additive scores will misclassify some interactions that produce rare but severe use errors.

Supporting Evidence

FDA critical-task and use-related-risk framework.

Confidence

High in safety-critical contexts; moderate-high generally.

LAW-PROD-010: Pre-Action Prediction Law

Hypothesis

The difference between users' predicted outcomes before action and actual outcomes is a direct indicator of interpretive legibility.

Prediction

Products with low prediction accuracy will show more exploratory actions, reversals, and mode errors.

Status

Testable candidate law.

Confidence

Moderate.

LAW-PROD-011: Feedback Closure Law, Revised

Hypothesis

Feedback improves control only when it is detectable, discriminable, timely enough for the task, compatible with the action, and informative about state.

Prediction

Decorative or ambiguous feedback may increase perceived responsiveness without improving state accuracy.

Confidence

Moderate-High.

LAW-PROD-012: Learning Transfer Law

Hypothesis

Stable control mappings across a product family reduce discovery and interpretation cost for users with prior family exposure.

Prediction

Experienced family users will show lower first-action error and faster prediction than equally experienced users of unrelated products.

Confidence

Preliminary.

Evidence Registry

EV-PROD-011

Citation

International Organization for Standardization. ISO 9241-11:2018, Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts.

Source Type

Standard.

Summary

Defines usability in relation to specified users, specified goals, and a specified context, expressed through effectiveness, efficiency, and satisfaction.

Supports

• LAW-PROD-008
• MODEL-PROD-001 Revision A

Quality

High.

URL

https://www.iso.org/standard/63500.html

EV-PROD-012

Citation

U.S. Food and Drug Administration. Applying Human Factors and Usability Engineering to Medical Devices. 2016.

Source Type

Regulatory guidance.

Summary

Requires intended-user, use-environment, task, use-error, and critical-task analysis. Emphasizes designing out use-related hazards and validating safe and effective use with representative users.

Supports

• LAW-PROD-008
• LAW-PROD-009
• PLP-001

Quality

High.

URL

https://www.fda.gov/media/80481/download

EV-PROD-013

Citation

NASA. Human Integration Design Handbook, Revision 1 and OCHMO-HB-004 Anthropometry, Biomechanics, and Strength.

Source Type

Engineering and human-factors documentation.

Summary

Treats user capability, anthropometry, biomechanics, environment, task, and system design as integrated variables. Provides population-based physical-capability data and design rationale.

Supports

• Actual Action Possibility gate
• Executability gate
• LAW-PROD-008

Quality

High.

URLs

https://www.nasa.gov/wp-content/uploads/2015/03/human_integration_design_handbook_revision_1.pdf

https://www.nasa.gov/wp-content/uploads/2023/12/ochmo-hb-004-rev-a-dec2023.pdf

EV-PROD-014

Citation

National Institute of Standards and Technology. Usability Testing.

Source Type

Government research guidance.

Summary

Describes representative-user task testing using quantitative measures such as time, errors, and successful completion, combined with qualitative satisfaction data.

Supports

• PLP-001
• Outcome measurement architecture

Quality

High.

URL

https://www.nist.gov/programs-projects/usability-testing

EV-PROD-015

Citation

Fitts, P. M. The Information Capacity of the Human Motor System in Controlling the Amplitude of Movement. Journal of Experimental Psychology, 47(6), 1954, 381–391.

Source Type

Original empirical research.

Summary

Demonstrated a systematic speed-accuracy relationship in constrained aimed movement based on movement amplitude and target tolerance.

Supports

• Executability gate
• Local target-design prediction

Challenges

• Use of Fitts' law as a complete usability model

Quality

High within scope.

DOI

10.1037/h0055392

EV-PROD-016

Citation

Hick, W. E. On the Rate of Gain of Information. Quarterly Journal of Experimental Psychology, 4(1), 1952, 11–26.

Source Type

Original empirical research.

Summary

Related perceptual-motor choice time to stimulus information under controlled conditions.

Supports

• Interpretability and choice analysis

Challenges

• The simplistic claim that every additional option produces the same burden

Quality

High within scope.

DOI

10.1080/17470215208416600

EV-PROD-017

Citation

Proctor, R. W., and Schneider, D. W. Hick's Law for Choice Reaction Time: A Review. Quarterly Journal of Experimental Psychology, 71(6), 2018.

Source Type

Academic review.

Summary

Reviews the law's history, evidence, interpretations, and boundary conditions, including uncertainty and stimulus-response conditions.

Supports

• Narrow use of Hick-type models

Quality

High.

DOI

10.1080/17470218.2017.1322622

EV-PROD-018

Citation

Card, S. K., Moran, T. P., and Newell, A. The Keystroke-Level Model for User Performance Time with Interactive Systems. Communications of the ACM, 23(7), 1980, 396–410.

Source Type

Original modeling and validation research.

Summary

Predicts expert routine task time from low-level operators, mental preparation, and system response time. Explicitly addresses only one dimension of performance.

Supports

• Executability analysis
• Task decomposition

Challenges

• Using execution-time models for novice discovery and comprehension

Quality

High within scope.

DOI

10.1145/358886.358895

EV-PROD-019

Citation

NIST. Human Engineering Design Criteria Standards, Parts 1 and 2. NISTIR 7889 and NISTIR 7934, 2014.

Source Type

Government engineering guidance.

Summary

Provides criteria and practices intended to improve personnel performance and integrate human capabilities into equipment and system design.

Supports

• Gate-level design variables
• Environmental and task specificity

Quality

High.

URLs

https://nvlpubs.nist.gov/nistpubs/ir/2014/NIST.IR.7889.pdf

https://nvlpubs.nist.gov/nistpubs/ir/2014/NIST.IR.7934.pdf

Hypothesis Registry

Failed Assumptions

FA-PROD-001: Product-level scoring is the correct unit

Failed because whole products contain many tasks with different users, consequences, and contexts.

FA-PROD-002: More numerical compression means more scientific rigor

Failed because a single score can remove the very failure information needed for diagnosis and safety.

FA-PROD-003: Successful use proves correct understanding

Failed because users can succeed by chance, imitation, recovery, or repeated action.

FA-PROD-004: Classic laws automatically become design laws

Failed because predictive models are conditional and limited to specific portions of interaction.

Theory Impact Assessment

Affected Theory Records

• MODEL-PROD-001 Product Legibility Model
• MODEL-PROD-003 Feedback Closure Index
• LAW-PROD-002 Relational Affordance Principle
• LAW-PROD-003 Closed Feedback Loop Principle
• LAW-PROD-005 Eyes-Free Control Principle

New Principle Candidates

• LAW-PROD-008 Contextual Legibility Law
• LAW-PROD-009 Critical Gate Law
• LAW-PROD-010 Pre-Action Prediction Law
• LAW-PROD-012 Learning Transfer Law

Deprecated Principles

• Product Legibility represented as a context-free scalar
• Any additive formula that permits critical-failure compensation

Confidence Changes

• Product Legibility as a useful research construct: Moderate → Moderate-High
• Product Legibility as a single score: Preliminary → Rejected
• Fitts/Hick as universal design laws: Low → Rejected
• Fitts/Hick as local predictive models: Moderate → High

Predictions Created

1. Pre-action prediction error will correlate with exploratory actions and reversals.
2. High confidence paired with low prediction accuracy will predict more consequential errors than low-confidence uncertainty.
3. Stable family mappings will reduce first-action errors among previously exposed users.
4. A gated profile will identify unsafe interactions missed by additive scoring.
5. Stressors will degrade different gates selectively rather than lowering all dimensions equally.

Predictions Invalidated

• A visually simple product will produce a uniformly high legibility score.
• A low action count will predict high Product Legibility.

Required Theory Registry Updates

• Replace MODEL-PROD-001 with Revision A.
• Mark composite scoring as context-limited.
• Add PLP-001 as the canonical evaluation protocol.
• Add critical-gate annotations to all future case studies.

Open Questions

Ranked by importance.

1. What thresholds define gate failure?

Importance: Critical
Needed: Empirical distributions and consequence-specific acceptance criteria.

2. Does pre-action prediction accuracy reliably forecast use errors?

Importance: Critical
Needed: Controlled studies across mechanical, digital, and hybrid products.

3. How reliable are independent raters when coding the six gates?

Importance: Critical
Needed: A coding manual, training set, and inter-rater reliability analysis.

4. How should learning curves be represented?

Importance: High
Needed: First-use, repeated-use, and delayed-retention measurements.

5. Can population exclusion be summarized without averaging it away?

Importance: High
Needed: Distributional reporting, subgroup analysis, and worst-case critical-task rules.

6. Which measures transfer from digital interfaces to physical products?

Importance: High
Needed: Matched physical/digital interaction experiments.

7. How should continuous manipulations be evaluated?

Importance: High
Needed: Trajectory, stability, overshoot, force, and proportional feedback measures.

8. Can product-family consistency be measured as transfer rather than sameness?

Importance: Medium-High
Needed: Cross-product learning and transfer studies.

Recommended Next Research

Priority 1: Pilot the Protocol on Matched Product Pairs

Select six products:

• three Rams-associated products;
• three Sapper-associated products;
• matched by broad task where possible;
• at least one discrete control, one continuous adjustment, and one multifunction interaction per designer.

Expected Value

Very high. This tests whether the protocol produces useful distinctions rather than merely formal structure.

Effort

Moderate-High.

Priority 2: Build the Coding Manual

Define examples, decision rules, and evidence requirements for every gate and metric.

Expected Value

Very high. Required for inter-rater reliability and autonomous-agent consistency.

Effort

Moderate.

Priority 3: Test Pre-Action Prediction

Run a small controlled study comparing predicted action effects with actual interaction behavior.

Expected Value

High. This may become one of the strongest direct measures of interpretive legibility.

Effort

Moderate.

Priority 4: Build a Population Accommodation Layer

Integrate anthropometric, strength, sensory, and mobility distributions into Actual Action Possibility and Executability.

Expected Value

High. Prevents “average user” scoring from concealing exclusion.

Effort

High.

Research Backlog

1. Matched Rams/Sapper pilot set
2. Product Legibility coding manual
3. Inter-rater reliability study
4. Pre-action prediction experiment
5. Confidence-calibration metric validation
6. Continuous-control protocol extension
7. Stressor matrix for low light, gloves, divided attention, and time pressure
8. Product-family learning transfer study
9. Accessibility and aging-user integration
10. Relationship between Product Legibility and Complexity Allocation

Suggested Specialized Research Agents

Agent A: Human-Factors Measurement Specialist

Focus on task metrics, critical errors, thresholds, and validation design.

Agent B: Ergonomics and Anthropometry Specialist

Focus on reach, force, posture, grip, mobility, and population accommodation.

Agent C: Cognitive Modeling Specialist

Focus on prediction, mapping, mode awareness, confidence calibration, Fitts, Hick, KLM, and learning curves.

Agent D: Industrial Design Historian

Select well-documented Rams and Sapper products and reconstruct original design intent and constraints.

Agent E: Research Methods and Reliability Specialist

Develop coding manual and inter-rater reliability procedure.

Parallel Research Opportunities

• Complexity Allocation Ledger can be developed while Product Legibility case coding proceeds.
• Repairability scoring can share task decomposition and population/context definitions.
• Architecture and interface branches can test whether the gated model transfers across media.
• “Intuitive is familiar” research can directly inform exposure and learning dimensions.

Risks

1. False precision: Numerical profiles may appear more validated than they are.
2. Context loss: Aggregation may detach results from the tested task and population.
3. Sampling bias: Young, able, technically experienced participants can inflate scores.
4. Observer inference: Researchers may code intention rather than behavior.
5. Historical incompleteness: Museum artifacts may not be operable or tested in original contexts.
6. Survivorship bias: Famous products may have better documentation and stronger cultural familiarity.
7. Training contamination: Instructions can conceal poor first-use discoverability.

Cross-Discipline Opportunities

Cognitive Psychology

Mental models, choice, confidence calibration, learning, memory, and transfer.

Biomechanics

Reach, force, precision, fatigue, posture, and capability distributions.

Control Theory

Feedback latency, signal quality, correction, stability, and state estimation.

Information Theory

Local uncertainty measures, not whole-product quality scores.

Safety Engineering

Critical gates, severity, probability, hazard controls, and failure analysis.

Architecture

Wayfinding and threshold recognition can test discoverability and consequence comprehension at environmental scale.

Typography and Visual Composition

Signal hierarchy and grouping affect discoverability but should be validated through behavior.

Knowledge Relationships

Relational Affordance
    ↓
Actual Action Possibility + Discoverability

Population Compatibility
    ↓
Interpretability + Executability

Closed Feedback Loop
    ↓
Feedback Closure + Consequence Comprehension

Fitts / Hick / KLM
    ↓
Local predictive models inside gates

Critical-Task Analysis
    ↓
Noncompensating gate rule

“Intuitive Is Familiar”
    ↓
Exposure, learning, and product-family transfer

Complexity Allocation
    ↔
Legibility costs may be transferred to learning, maintenance, or infrastructure

Repository Updates

Required

• Add this REP under product-genome/research-packages/.
• Replace MODEL-PROD-001 with Revision A in the theory registry.
• Add HY-PROD-008 through HY-PROD-012 to the hypothesis registry.
• Add EV-PROD-011 through EV-PROD-019 to the evidence registry.
• Add PLP-001 under methods/product-legibility/.
• Mark Product Legibility operationalization as complete but unvalidated.

Suggested File Names

research-packages/RP-PROD-002-Operationalizing-Product-Legibility.md
methods/PLP-001-Product-Legibility-Evaluation-Protocol.md
theory/MODEL-PROD-001-Product-Legibility-Revision-A.md

Website Updates

Add a Product Legibility page with:

• the six-gate model;
• explicit task/population/context fields;
• a profile visualization rather than one score;
• critical-gate warnings;
• raw metrics;
• evidence links;
• first-use versus learned-use comparison;
• case-study filters.

Do not display rankings across unrelated product tasks until validation exists.

AI Consumption Notes

Agents using this package must:

1. Never call an entire product “legible” without naming a task.
2. Never infer Actual Action Possibility from photographs alone.
3. Preserve raw outcome and gate measurements.
4. Treat questionnaires as supplementary evidence.
5. Record first-use separately from trained use.
6. Never average a critical failure away.
7. State the tested population and environment.
8. Apply Fitts, Hick, and KLM only within their assumptions.
9. Distinguish observed behavior from inferred design intent.
10. Mark all untested numerical thresholds as provisional.

Handoff Instructions

The next agent should:

1. Read RP-PROD-001 and RP-PROD-002.
2. Select a six-product Rams/Sapper pilot set based on documentary quality and task comparability.
3. Build a coding sheet directly from PLP-001.
4. Code documentary evidence separately from empirical user evidence.
5. Identify where live product testing is impossible.
6. Conduct at least two independent coding passes.
7. Report disagreements and calculate inter-rater agreement where possible.
8. Produce RP-PROD-003 with case profiles, failed comparisons, and protocol revisions.

Research Quality Metrics

Research Debt

Missing Evidence

• Direct validation of the six-gate model
• Empirical thresholds
• Cross-category normalization
• Disabled and aging-user datasets linked to specific case products
• Longitudinal learning and retention data

Missing Experiments

• Pre-action prediction versus error study
• Inter-rater coding study
• Stressor sensitivity study
• Product-family transfer study

Missing Disciplines

• Occupational therapy
• Rehabilitation engineering
• Inclusive design research
• Psychometrics
• Safety-case engineering

Replication Needed

• Fitts and Hick effects in selected physical-product interactions
• Confidence calibration in product-state prediction

Tool Limitations

• No direct access to original physical products during this research run
• Some standards provide only limited public previews
• Historical case records may omit failed use and maintenance experience

Bibliography

Academic: Original Research

• Card, S. K., Moran, T. P., and Newell, A. “The Keystroke-Level Model for User Performance Time with Interactive Systems.” Communications of the ACM 23(7), 1980: 396–410. DOI: 10.1145/358886.358895.
• Fitts, P. M. “The Information Capacity of the Human Motor System in Controlling the Amplitude of Movement.” Journal of Experimental Psychology 47(6), 1954: 381–391. DOI: 10.1037/h0055392.
• Hick, W. E. “On the Rate of Gain of Information.” Quarterly Journal of Experimental Psychology 4(1), 1952: 11–26. DOI: 10.1080/17470215208416600.

Academic: Reviews

• Proctor, R. W., and Schneider, D. W. “Hick's Law for Choice Reaction Time: A Review.” Quarterly Journal of Experimental Psychology 71(6), 2018. DOI: 10.1080/17470218.2017.1322622.

Standards

• International Organization for Standardization. ISO 9241-11:2018: Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts.

Government and Regulatory Guidance

• U.S. Food and Drug Administration. Applying Human Factors and Usability Engineering to Medical Devices. 2016.
• National Institute of Standards and Technology. Usability Testing.
• Furman, S. et al. Human Engineering Design Criteria Standards, Part 1. NISTIR 7889, 2014.
• Furman, S. et al. Human Engineering Design Criteria Standards, Part 2. NISTIR 7934, 2014.
• NASA. Human Integration Design Handbook, Revision 1.
• NASA. OCHMO-HB-004 Revision A: Anthropometry, Biomechanics, and Strength. 2023.

Books and Foundational Works

• Card, S. K., Moran, T. P., and Newell, A. The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates, 1983.

Historical

• Fitts, Hick, Card, Moran, and Newell original publications listed above.

Research Journal

JR-PROD-002-01

The initial plan was to create numerical scales for all six gates and combine them. Review of ISO, FDA, and task-risk frameworks showed that this would create false compensation. The research direction changed from scalar scoring to gated profiling.

JR-PROD-002-02

Classic “laws” initially appeared to offer the mathematical foundation. Closer inspection showed that each predicts a narrow part of skilled performance. They were retained as local models and rejected as universal quality measures.

JR-PROD-002-03

The distinction between pre-action understanding and post-action success emerged as a high-value measurement principle. It creates a way to test interpretability without allowing trial and error to conceal poor initial comprehension.

JR-PROD-002-04

The most important safety insight is confidence inversion: a product can feel clear while producing an incorrect mental model. Confidence must therefore be calibrated against correctness rather than treated as positive evidence alone.

Appendix A: Minimal Coding Sheet

case_id:
product:
designer:
year:
interaction:
task_goal:
start_state:
success_state:
critical_errors:
noncritical_errors:
population:
environment:
exposure_level:

outcomes:
  completion_rate:
  correct_completion_rate:
  critical_error_rate:
  noncritical_error_rate:
  assistance_rate:
  median_time_seconds:
  abandonment_rate:

actual_action_possibility:
  observations:
  measures:
  critical_gate: true|false
  pass: true|false|unknown

discoverability:
  observations:
  unaided_discovery_rate:
  first_action_correct_rate:
  median_discovery_time_seconds:
  critical_gate: true|false
  pass: true|false|unknown

interpretability:
  predicted_effect_accuracy:
  predicted_direction_accuracy:
  confidence:
  calibration_error:
  critical_gate: true|false
  pass: true|false|unknown

executability:
  median_execution_time_seconds:
  error_rate:
  correction_count:
  physical_difficulty:
  critical_gate: true|false
  pass: true|false|unknown

feedback_closure:
  latency_ms:
  detection_rate:
  state_identification_accuracy:
  repeated_action_rate:
  critical_gate: true|false
  pass: true|false|unknown

consequence_comprehension:
  final_state_accuracy:
  reversibility_accuracy:
  hazard_awareness_accuracy:
  critical_gate: true|false
  pass: true|false|unknown

limitations:
evidence_ids:
confidence:

Completion Checklist

• Executive Summary
• Original Objective
• Scope
• Repository Context
• Current Understanding
• Key Discoveries
• Evidence Registry
• Hypothesis Registry
• Failed Assumptions
• Open Questions
• Recommended Next Research
• Research Backlog
• Suggested Specialized Research Agents
• Parallel Research Opportunities
• Risks
• Cross-Discipline Opportunities
• Knowledge Relationships
• Repository Updates
• Website Updates
• AI Consumption Notes
• Handoff Instructions
• Research Journal
• Appendix
• Theory Impact Assessment
• Research Quality Metrics
• Research Debt
• Bibliography

Revision History

Agent Instructions

When extending this document:

1. Separate observation from interpretation.
2. Never strengthen a conclusion beyond the available evidence.
3. Preserve contradictory findings.
4. Prefer measurable variables over subjective descriptions.
5. Reference candidate laws and genome nodes whenever possible.
6. Use stable IDs for observations, evidence, laws, experiments, metrics, and case studies.
7. Record assumptions explicitly.
8. Record confidence explicitly.
9. Keep the YAML header valid.
10. Do not delete revision history; append to it.