Evaluation and Measurement Research Journal — Cycle 1

Cycle objective

Start the approved roadmap's first Wave 0 stack and leave a reconstructable research state. Test VE-EVL-H1 and VE-EVL-H2 far enough to define the next high-information investigations.

Repository searches and material

• Enumerated project artifact directories and repository registry placeholders.
• Read the approved master roadmap, priority matrix, dependency map, VE-EVL roadmap, VE-GOV roadmap, VE-EVL foundation prompt, and metadata standard.
• Read EX-COMP-011, EX-COMP-012, EX-WC-0001, and the research-methodology index.
• Observed that the two Composition Science experiments are proposals, while the Web Components probe report is verified despite mixing executed, source-backed, and deferred probes.

External search strategy

Queries covered:

• ISO usability definitions and context;
• APA/JARS and trial-reporting minimums;
• CONSORT 2025 and open-science fields;
• NIST human-centered measurement;
• CHI reproducibility, replicability, and transparency;
• questionable measurement practices and construct validity;
• generalizability across participants, stimuli, tasks, sites, and samples;
• heterogeneous treatment effects and transport.

Accepted sources prioritized official standards, venue guidance, peer-reviewed methodological work, and empirical generalizability studies. Search summaries, blogs, Wikipedia, Reddit, commercial advice, and unrelated AI benchmark papers were rejected as evidence carriers. They may be used only to locate primary sources.

Decisions

• H1 is weakened and narrowed; common metrics are contributions, not generic evidence of visual-engineering success.
• H2 is rejected as written; a multidimensional validity profile is the current rival.
• The first protocol artifact is a minimum experiment record, not a universal score.
• Registry updates remain proposed until integration review.
• Human-participant execution is explicitly gated on ethics, consent, privacy, and accessibility requirements.

Negative results and access limits

• The public ISO page supports the framework and scope but does not expose the full paid standard text.
• APA JARS discovery surfaced official references but the first pass did not obtain a complete authoritative field-level table suitable for a crosswalk.
• Current evidence does not establish numerical promotion thresholds.
• No independent reviewer test of the validity profile has been run.

Files changed in this cycle

• content/projects/evaluation-measurement/research-execution-package/rep-ve-evl-001-foundation-and-falsification.md
• content/projects/evaluation-measurement/research-journal/2026-07-24-evaluation-measurement-cycle-1.md

Exact resume point

Stack A completed on 2026-07-25 as AUD-VE-EVL-001. Thirteen predecessor artifacts were coded. The most consequential finding is the absence of a common execution-state field separating plans, proposed experiments, repository probes, expert analyses, source syntheses, secondary-data analyses, and human studies.

Resume with Stack B, the reporting-framework crosswalk. It must produce a small common record plus method-specific extensions and must not silently assume that all repository evaluations are participant experiments.