Conformance against Ensembl VEP

Official Ensembl Variation release VCFs provide a separate product-audit lane. Their indexed VE relation can be compared in ordinary CI without starting Perl VEP; CSQ is a lossy presentation of those stored rows. This is not an executable-VEP oracle: in release 116, X/Y:276322 G>A is published as intergenic_variant, while cache-mode VEP with --distance 0 emits three path-specific 5_prime_UTR_variant rows on each chromosome. Pinned release shards are therefore useful lineage evidence, while the executable/cache combination remains the semantic compatibility authority. Matching full models belong in external versioned artifacts, not in git or the network-free extension build step.

Declared conformance closure

The independent-event conformance contract covers the declared model and event surfaces: admitted Ensembl transcript, mature-miRNA, RegulatoryFeature, and MotifFeature objects; independent literal small alleles; exact typed DEL, DUP, tandem-DUP, INV, INS, and CNV events; structural tandem repeats (STR); paired breakends; supported BioPerl codon tables and exceptional Ensembl peptide edits; and the separately declared VEP NMD-plugin result. DEL/DUP/tandem-DUP/INV/INS/CNV and BND have generated executable-VEP differentials. Structural STR has source-derived VEP-116 semantics plus fixed SQL/R and randomized C coverage; raw repeat reconstruction is a separate input-preparation operation. The evidence spans GRCh38, GRCh37, and P. falciparum, executable witnesses, indexed-cache corpora, generated state exploration, sanitizer runs, and pure-C oracle properties.

VEP 116 parses CIPOS/CIEND into inner/outer structural coordinates, but its registered consequence predicates use nominal POS/END; DuckVEP therefore annotates that nominal span while the surrounding relation preserves the uncertainty metadata. The checked-in 12-record GRCh38 confidence witness records this directly: nominal and IMPRECISE;CIPOS;CIEND forms of CNV, DEL, DUP, tandem DUP, INV, and INS produced 466/466 exact transcript pairs, and both engines had equal nominal/imprecise consequence multisets for all six event-kind pairs. VEP can also expand a bounded <CNV:TR> from RN, RUS, and RUC or RB into a literal allele before consequence calculation. Implementing that lossless expansion and mapping VEP’s finite supported symbolic vocabulary into the typed event API are narrower input- preparation tasks, not missing consequence predicates. VEP itself rejects unrecognised types such as CPX, so this closure does not promise arbitrary symbolic parsing. Untested species/releases and phased multi-record haplotypes remain outside this contract. Phased coding consequences have a separate correctness and throughput contract; compound HGVS and broader domain composition remain open in #11 and #50. Any newly observed fixed-event mismatch fails this contract rather than being relabelled as unsupported.

Evidence units and statistical interpretation

The release evidence has three distinct units. A complete-corpus audit establishes exactness only for every event in that named corpus. The generated state-exploration campaign deliberately over-samples rare splice, coding, strand, length, and structural states to discover defects and demonstrate transition coverage; it is not a sample of deployment prevalence. A population error-rate statement would require a separately specified probability sample whose primary outcome is whether the complete output multiset for each independently sampled input event differs from VEP.

The append-only history retains its original upper95 field for schema compatibility. Tables below label that value as a descriptive independent-pair Clopper–Pearson calculation. Transcript and other annotated-object pairs are clustered within input events, transcripts, and genes, and the targeted generators are intentionally not deployment samples. These calculations therefore are not general engine error-rate bounds; separate stratum values are also not simultaneous confidence intervals. The release gate itself is deterministic and stricter: any discordance, unresolved state, extra emission, or missing emission fails the audited run.

Independent-event HGVS differential

This is exact string agreement for independent transcript events with VEP 116 invoked using --hgvs. A comparison is exact when both engines emit the same string or both omit that HGVS field. Unresolved, missing, extra, and unequal strings remain discordant; none is removed from the denominator. The checked ledger accepts only a pair artifact produced from the current clean source revision by a vendored-htslib distclean followed by an in-tree release build. The table retains complete SHA-256 receipts for the extension, model, FASTA and index, source VCF, exact sampled VCF passed to VEP, and pair-level Parquet; shortened digests are rendered above. Historical HGVS rows recorded before build receipts were introduced remain in the append-only CSV but are not presented as checked evidence.

The b7c7237ee686 ClinVar HGVS and NMD runs use a freshly acquired vep116_grch38_cache_chr21. Its complete 27,644,657,162-byte source archive was verified against registry SHA-256 014c7dd9bb5ad06665866d62eb80f31ca761197bcb9b59280300676f996e600d before publication. The cache contains 98 files / 326,767,268 bytes; every retained file matches the preserved earlier cache byte-for-byte. Earlier HTTP-identity-only acquisition receipts remain historical observations, not content-checksum evidence. The refreshed HGVS run retains the same 1,864-variant input and all 56,998 transcript pairs, with no unresolved, missing, extra or discordant pairs.

The 6ce2ddd85df7 rerun again retains that exact source/input VCF and all 56,998 pairs, with exact consequences and HGVSc/HGVSp agreement. It resolves the model, reference and checksum-verified chromosome-21 cache through the artifact registry. Its physical model-file receipt differs from the older run; this is not a claim of byte-identical model files. The full GIAB conformance campaign has not been refreshed to this revision. A separate chromosome-21 attempt found no eligible model joins because the raw callset uses chr21 and the model uses 21; it stopped before VEP, was not counted as a pass, and did not replace the full GIAB gate.

Paired-breakend differential

The generated seed-31 corpus spans chromosomes 1, 2, 7, 21, and X; intra- and interchromosomal mates; all four VCF bracket orientations; and transcript, exon, intron, CDS, and directional-flank endpoint states. The comparison is the union of consequences produced by both breakend endpoints for each transcript, which is VEP 116’s transcript-level paired-breakend contract.

VEP 116’s buffered BND path inserts mate coordinates into a chromosome-blind interval tree. A multichromosome batch can therefore omit valid transcript pairs even when the input chromosomes are contiguous and position-sorted. The oracle command uses --buffer_size 1 in one Perl process so every event is evaluated independently. This is oracle isolation, not a DuckVEP compatibility rule; the ledger records breakend_buffer_size=1 and the artifact hash.

Core regulation and motif differential

The GIAB run checks ordinary alleles against transcript and core funcgen objects. The generated structural run deliberately crosses, contains, exactly matches, and partially overlaps RegulatoryFeature and MotifFeature intervals under DEL, DUP, TDUP, INV, INS, and CNV operations. Structural STR is covered separately by the source-derived VEP-116 rule, fixed SQL/R adapter tests, and randomized C oracles; this generated executable-VEP run does not reconstruct raw repeat metadata. The resident model contains only VEP-admitted core funcgen objects: VEP 116 removes epigenetically_modified_region rows before constructing RegulatoryFeature overlap objects, so DuckVEP excludes them during deterministic model preparation rather than filtering output after candidate traversal.

Prepared model receipts

These are complete model-build receipts, not counts inferred from a differential. The ledger retains the full source-manifest, reference, and model SHA-256 values, the exact VEP transcript filter, every count above, CDS/flank base totals, and the external artifact name. The Plasmodium row is an Ensembl Genomes release-63 cache paired with the VEP/core-116 executable libraries, which is why both release numbers are recorded.

History

Randomized executable-VEP state exploration

This is the anti-overfitting lane against the VEP executable, not an internal property test. Each seed contains the fixed predicate witnesses plus 100,000 unique alleles. Three quarters of random positions are within six bases of splice sites, exon endpoints, and translation starts and stops; one quarter is uniform across the transcript. SNVs, MNVs, insertions, deletions, and delins are sampled with equal probability, with differing alleles up to 49 bases.

The combined denominator counts pair comparisons across seed runs, not distinct alleles: the 268 fixed witnesses are deliberately shared and random draws may overlap. All compared runs use the same generator and acceptance rules.

Repeat-clipping witness

Seed 27182818 at 7d40756adc75 matched every consequence pair but emitted one extra HGVSc: chrDuck:250 CGT>CCC, transcript DUCK1-201, yielded c.*10[3] where VEP emitted no HGVSc. The pure-C properties passed on that revision too. VEP skips transcript allele clipping only for two-copy duplication; larger multiplications must undergo clipping and coordinate projection before HGVSc repeat formatting. The ledger retains the failing observation and its matching regression run under the same generator and oracle.

The same three frozen 100,268-pair corpora pass consequences and HGVS at 6ce2ddd85df7. A separate four-record executable witness retains the discovered event, its two-copy positive control and larger-copy variants. Native regression tests additionally enumerate 120 transcript-end projection cases: four bases, both strands and copy counts 2 through 16. Those finite cases exercise the shared edit/HGVS path; they are not 120 additional executable-VEP comparisons or proof of whole-haplotype SO/HGVS semantics.

The same campaign covered the following SO terms. Counts are term memberships, not distinct transcript pairs, because one pair may carry several terms and seed runs deliberately retain the same fixed witnesses.

This distribution deliberately stresses local alleles and positions near exon, splice-site, and CDS endpoints on one engineered transcript. It does not replace the indexed-cache corpora, which add real transcript density, imported flags, exceptional peptide edits, codon tables, assemblies, and species. The revision is shown explicitly because this expensive campaign is not silently attributed to later code.

Official Ensembl release corpus in Parquet

The official release consequence VCF is already BGZF-compressed. This table measures its complete typed DuckHTS reader projection and the narrower VE plus CSQ projection used by the bulk oracle lane. It is a storage comparison, not a claim that the Parquet projection can reproduce the original VCF byte-for-byte.

The consequence projection is also the natural CI payload: retain deterministic shards with source URL, Ensembl release/species/assembly, artifact digest, row cardinalities, and CSQ schema. A scheduled full-release job may pair the complete projection with a published receipt-hashed DuckDB model. Neither form broadens the supported consequence contract; it only makes a large known-variant regression cheap to replay.

The ledger records the official source URL, SHA-256 of every input and output, DuckHTS and DuckDB versions, compression, row-group size, thread count, machine, and exact byte sizes.

Randomized pure-C properties

The property ledger is separate from the VEP differential. It records successful runs of each randomized oracle, including the exact seed and duplicate count. A failed suite does not append rows.

Passing the requested number of trials is necessary but does not prove that a generator visited the states named by its contract. Randomized properties therefore emit distribution counters, and the recorder stores each counter as a separate numeric row. The table below is the latest complete run’s state distribution; the long-form CSV remains the machine-readable authority. Zero is evidence too: it identifies a state that the declared seed did not exercise and must not be hidden by the suite-level pass count.

The run observed all 314 required nonzero counters. The other 3 counters have named fixed witnesses in the coverage manifest; their absence from a random draw is not counted as statistical coverage. These counters describe the declared generators, not an exhaustive enumeration of biological configurations.

The recorded runs executed 6,300,000 property trials on the shown revision. Millions of passing trials do not make a counter with single-digit observations densely explored, and marginal counters do not establish coverage of their cross-products. The fresh-seed HGVS failure above is direct evidence of this limit. Dedicated rare-state strata and retained counterexamples complement broad draws; they do not justify a population error-rate claim. Compound phased HGVS and broader domain composition are separate contracts in #11 and #50. Real-corpus evidence is bound to the named revision, model and oracle.

Phased replay with noncoding contributors

Sources eb83f6ff1d03d05a3c9f8135c8ef355b7f431ee7, d1c591b76f8a9a07036736ac0666a004eb58e0eb, 8f9987e3826018cfa73c155eebac7a956b6dc024, 7f4a4e28bff31a13f14ee1cab25049408740ef65, 9b1b8a0d4e12c99f8f08d9c013d840abf5eb7ec1 were built from clean checkouts, including an HTSlib clean rebuild. The ledger retains the extension hash, input/run receipt hashes and pinned VEP/variation revisions. DuckDB used four threads; these are correctness counts, not timing or memory measurements.

Each unchanged 1,000-transcript corpus first passes its original public replay checks: 6,000 complete lanes, 4,000 occupied carriers, 3,000 output leaves, 22,000 oracle comparisons and 4,000 first-stop/frame comparisons per policy. The supplemental corpus adds one homozygous intronic SNV per transcript while preserving every original VCF record. Running unmodified Haplosaurus on those inputs leaves its complete observations unchanged. DuckHTS must retain the intronic contributors, including on implicit reference lanes, with unchanged literal CDS/protein and an outside_cds contributor status.

Both policies pass all 120,000 carrier comparisons and 286,980 provenance memberships. The same biological lanes are counted separately under each policy and revision; these are not independent statistical trials. Five deliberately corrupted outputs per policy/seed are rejected. Fixed SQL/R tests additionally cover UTRs, insertions, missing calls and coding-overlapping projection failures; a native two-strand span enumeration supplies 6,774 assertions of contributor retention.

This certifies the declared literal-replay cases, not altered splicing, combined SO/HGVS, broad phase compatibility or exhaustive rare configurations. The conformance driver keeps this augmentation opt-in and does not replace the original corpus or verifier. The phased replay benchmark records sorted native and public SQL execution separately, with workspace and process memory. The current SQL benchmark includes local coding-block SO; these Haplosaurus comparisons do not certify those masks or evaluate whole-haplotype SO/HGVS.

Raw genotype compatibility audit

Source 0857ec1faf5736299a4341f73ffa557d8a9691ee records a failing decoded-call/raw-parser comparison: 6990 disagreements in 7020 profiles. It is not a population error rate or a replacement for the passing literal-sequence corpus. This lane uses public input_mode := 'alt_events'; the separate raw source-record input is checked below.

The R driver enumerates every GT over 0, 1, 2, . at ploidies 1–4, every intervening separator pattern, and absent, /, or | leading prefixes. No profile is sampled or excluded. Each has one multiallelic site plus a homozygous second site in a different phase set, using the registered 180-base reference. There are 14,040 source records/genotype calls, 21,060 source ALT events/candidate calls, and 54,168 input allele slots. Every REF is checked before execution. These are correctness denominators, not timing measurements.

The pinned, unmodified Haplosaurus runner and decoded vep_compat executor consume the same VCF/GFF/FASTA. Comparisons retain complete CDS/protein multisets, source-record contributors and carrier counts. Eighteen ordinary called diploid profiles agree. The audit rejects 12 deliberate corruptions: four check sequence/protein/provenance/duplicate handling, and eight check grouped-observation fields, identities and row preservation. All other disagreements remain failures, including missing-input NULL sequences and the difference between explicit source ploidy and Haplosaurus’s file-input diploid fallback. The command exits nonzero after writing full observations, comparisons and source-bound receipts.

There are 1471 groups in which distinct raw GT spellings have identical HTSlib alleles and phase flags but different Haplosaurus outputs. A witness is 0|1 versus |0|1: both decode to alleles [0,1], phase flags [true,true]. VEP-116’s parser retains a leading empty split field as REF before the container consumes its allele slots, changing the result. The same model, remaining call and phase sets are used on both sides.

Thus raw-parser compatibility cannot be recovered from typed calls alone. This does not justify changing HTSlib-faithful genotype decoding or treating an unknown call as biologically known. Exact raw-input emulation needs retained source GT and source-record allele context, with upstream conditional sequence explicitly distinguished from strict-phase evidence. These raw-input measurements do not assess whole-haplotype SO/HGVS or typed structural composition.

The constant-space native raw-GT parser separately records 0 disagreements across 14040 source calls. An optional observer sidecar reads actual Haplosaurus genotype objects and its file-profile ploidy; it does not override parsing or sequence construction. The comparison checks retained/omitted calls, parsed slot counts, the two consumed allele ordinals, source ploidy and missingness, with seven rejected field corruptions. The standard output on the 1,000-transcript seed-173 corpus matches its recorded oracle output. Parser code, bridge, compiler identity, binary and observations are hashed in the same clean-build receipt.

The native source-record stream records 0 disagreements across 7020 complete CDS/protein, carrier-count and physical-edit provenance comparisons, and 0 disagreements across 28080 per-lane source-record observations. The latter checks selected allele ordinals, record and aggregate evidence, and sequence availability against the upstream object sidecar; four deliberate field corruptions are rejected.

Raw replay keeps full source REF spans and actual ALT ordinals. An undefined consumed slot follows the pinned runner’s full-REF deletion, with conditional sequence status and evidence distinct from a called ALT. Missing REF observations retain provenance without inventing physical edits. Reference validation and coding projection failures still prevent sequence availability.

This native result covers the enumerated two-site, single-exon grammar, not overlapping replacements, splicing or combined SO/HGVS. Public decoded-call comparison objects, including all 6990 failures, match the preceding audit exactly. The passing native lane does not waive those failures or make conditional sequence biologically known.

Public input_mode := 'source_records' with phase_policy := 'vep_compat' records 0 disagreements across 7020 complete CDS/protein multiset, carrier-count and physical-edit provenance comparisons, plus 0 disagreements across 28080 per-lane record observations. It consumes original GT text and complete ALT lists from read_geno(..., raw_gt := true) on the same VCF fixture. An independent text read checks every source GT spelling and physical record ordinal; it does not supply the native input. The verifier additionally checks exact global record IDs, regions, positions and source REF/ALT bytes; repeated local site labels cannot substitute for record identity.

The public lane exercises input materialization, sorting, transcript closing, DuckDB vector transitions and nested output. Fixed SQL/R controls reject duplicate calls, NULL keys, inconsistent record/GT identities, invalid ALT lists and GTs, and ploidy-limit exhaustion. This establishes the declared finite raw-input contract, not overlapping replacements, broader phase conformance, whole-haplotype SO/HGVS or biological certainty for conditional sequence.

Rare source-record geometry

Source 4119d55c43fe0649ffe8325135bbdf26c1f37e94 runs the public source_records/vep_compat path against pinned, unmodified Haplosaurus. Across the two seeds there are 73,504 profiles, 220,512 source records and 147,008 oracle file lanes; DuckHTS returns 147,008 carrier memberships. The full comparison retains CDS/protein multisets, counts and applied-record identity sets within each equal-sequence group. It does not certify physical-edit multiplicity or per-lane sequence association. Carrier-key uniqueness and totals are checked separately. Available includes explicitly conditional replay; a NULL sequence remains a disagreement, not a waiver. All failure columns count profiles; 0 profiles have unavailable sequence. Availability does not imply agreement. The receipt ledger retains every run, generator, coverage and artifact hash.

The identical-input comparison against receipt-named source 47eaa7b1460a6b29049cdb50928956a055d6abb9 has 15,953 resolved failures, with 0 regressions and 0 remaining failures. These cumulative transitions apply only to the declared generated inputs; they do not establish absence of regression outside this campaign. Every earlier failing comparison remains in its revision-labelled artifact.

Each seed includes 144 fixed profiles and 512 general-random overlapping pairs. The rare lane requires 32 draws in each of 1,128 cells: 12 geometries × two strands × 47 valid GT-pattern/source-ploidy combinations. Ploidies are 1, 2, 4, 8, 16 and 64. Patterns include mixed and leading separators, first/last missing slots, all-missing calls and ALT calls beyond the two consumed file lanes. Positions, replacement lengths and bases are seeded; actual GT strings and all cell quotas are checked. A pair shares its GT-pattern/ploidy class; this does not enumerate every pairing of different classes. Each profile has two tested records and a separate homozygous anchor. Both policies’ existing decoded-input corpora and the finite raw-GT audit remain independent evidence lanes.

All 36,752 generated inputs per seed match source 47eaa7b1460a6b29049cdb50928956a055d6abb9 exactly, and each complete oracle output is byte-identical. The comparison rules and all preceding comparison artifacts are preserved. Four deliberate output corruptions are rejected; the 24 fixed disjoint/adjacent controls per seed pass. Fixed profiles repeat across seeds, and all profiles use the same registered 180-base, single-exon reference and standard codon table. These are quota-controlled generated configurations, not independent biological observations, population error rates, strict-phase certification, combined SO/HGVS or structural-event conformance. Four DuckDB threads are used; these counts are not performance measurements.

Full-span replacement witness

At source 6bcff3353d7c9a2609996aae9828f9da3fb5f238, the fixed positive-strand mnv_retained_middle cis profile has these records:

On the changed lane, CDS bases 25–40 are GCTGCAGCTGCAGCTG in Haplosaurus. DuckHTS source 6bcff3353d7c9a2609996aae9828f9da3fb5f238 returns GCTGCAGCAGCAGCTG; source 47eaa7b1460a6b29049cdb50928956a055d6abb9 matches Haplosaurus’s complete sequence and applied-record provenance. The pinned _mutate_sequences implementation uses the Haplo parser’s raw spans and applies full REF spans in descending mapped-CDS order: record a restores the retained middle base after b changes it. Applied-record provenance retains b even when its sequence change is overwritten. Ordered raw replacements use bounded native storage and preserve net component spans; local SO and displaced-frame facts are unavailable for that path. Source-buffer ordering and duplicate selection are exercised by the quota-controlled campaigns in this report. Multi-transcript and cross-exon mapping require separate conformance. These observations are not evidence that VEP is wrong.

Paired source-context configurations

Source 4119d55c43fe0649ffe8325135bbdf26c1f37e94 includes 38,016 paired context profiles from 1152 seeded edit templates. The table also includes 656 fixed/general-random controls per seed; their inputs and complete oracle observations match the corresponding profiles in the rare-GT campaign. The ledger retains both the one-draw smoke runs and the 8-draw runs.

Every one of 2,376 geometry × GT-pattern × strand × placement × neutral-count cells receives 8 draws per seed. Each edit template is reused in 33 contexts: reference-only records occur before, between or after the tested edits, with counts including the neighbourhoods of the pinned interval tree’s root changes. The generator verifies unchanged edit geometry, alleles and GTs across those contexts, the complete upstream source buffer, and the absence of neutral records from retained genotype objects.

In 10,752 profiles, the oracle’s complete sequence/count/applied-record-set observation differs from the zero-neutral-record case; DuckHTS records 10,752 such changes. The full per-profile comparison, not equality of those two totals, determines agreement. These are paired observations, not independent biological samples or population error-rate estimates. The single-exon, one-sample reference and comparison limitations above also apply here; rare GT classes and neutral contexts are separate campaigns, not their complete cross-product.

Shared-transcript rare configurations

Source afb41c8688fbc058df3687160328f0c1c554bf67 has 69,120 transcript cases, including 33,792 shared-transcript quota cases, 32,256 variable-exon/UTR cases and the complete 3,072-case diagnostic baseline. Each fixed-model region has three diploid samples, one full-exon transcript and an overlapping two-exon transcript (exons 11–70 and 101–190). All models use the registered 180-base reference; variable-exon models distribute it across genomic exons with introns and optional UTRs.

The fixed-model cohort supplies 32 draws per seed in every geometry × cohort-GT-pattern × strand × neutral-count cell: four record geometries, three sample configurations, two strands and eleven neutral source counts through 36. This crosses shared-transcript mapping with source-context pressure; it is not a distribution of independent biological samples, arbitrary transcript models or an estimate of population error rates.

Against source 504dc785a1850d5f5c244ee5630f0da26b23fb81, 0 failures resolve, 0 passing cases regress, and 36,864 cases remain passing. Every matched input column and complete keyed oracle JSON record is unchanged. Constructor observations retain every field and duplicate, preserving record order within each sample; sample enumeration order is not a biological observation. All matched grouped, input-provenance, mapper and lane verdicts agree with the baseline. Earlier ledger entries retain the 4,608 failures resolved at source 4119d55c43fe0649ffe8325135bbdf26c1f37e94, including all 384 failures in the 3,072-case diagnostic baseline.

Raw replay marks validated coding/noncoding source-span omissions source_unmapped and replays the other mapped sources with conditional evidence. The independent artifact audit checks omission status, evidence bit 8 and absence from physical edits across 110,592 leaves / 162,816 carrier memberships. Model/REF errors still withhold sequence, and strict decoded replay retains its projection-failure policy. This is pinned Haplosaurus sequence-mechanics conformance, not evidence that VEP is wrong or that a conditional sequence describes biology.

The upstream mutator’s original return values provide 414,720 sample/file-lane observations before equal-sequence grouping. Each lane compares complete CDS/protein and applied-source identity sets, including full source keys and transcript-oriented alleles. A homozygous alternate anchor puts every sample through this observed path; reference-only samples handled separately upstream are not synthesized.

All 26 corruption controls pass per seed, including a changed reference-CDS control. Lane swaps, changed allele keys and a source removed from one shared-sequence lane are rejected even when grouped sequence, sample-count and source-set comparisons still agree. Native contributor checks retain IDs, regions, positions, REF/ALT interpretations and carrier keys. Upstream checks retain source buffers, genotype multiplicities and constructor-owned mappings, including exon-repeated and unselected duplicate sources.

Variable-exon and UTR configurations

The geometry cohort adds 32,256 models, with 0 disagreements. Each seed requires 32 draws in each of 504 cells: 2/3/5/7 coding exons × first coding split phase 0/1/2 × absent, intra-exon or separate-exon UTRs × both strands × seven source geometries. Exon, intron and intra-exon UTR lengths vary. Edits cover coding substitutions, CDS start/end crossings, exon entry/exit crossings, a whole exon with flanks and exon-end anchored insertions. Three samples carry opposite lanes and a compacted missing call, plus a homozygous ALT anchor.

The audit verifies 137,088 coding exons, including 4,580 observed one- or two-base microexons. Microexon frequency is an observation of this generator, not a forced per-cell quota or a population rate. Upstream’s constructed reference CDS equals every loaded model; the mapper check clips exons to the model’s CDS coordinates. Model, mapping, sequence, count, source and lane failures remain separate counters and all contribute to the verdict.

Older rows have unmeasured geometry/model/lane metrics, not inferred passes. Applied-source sets do not certify physical-edit multiplicity. This geometry grammar uses complete standard-code CDSs and one transcript per new region; arbitrary biological models, shared-variable-exon transcript interactions, reference-only sample routes, full phase/PS behavior, whole-haplotype SO/HGVS and structural composition remain outside this campaign’s tested scope.

Interacting exon/UTR diagnostics

These are local diagnostic runs, not source-bound release certificates. The receipt ledger retains the captured runtime revision, extension hash, generator and comparator hashes, complete-input/output hashes, comparison hashes and each failure count. The retained run directories contain the full observations; this ledger is not a portable corpus pack.

The one-draw pair has identical source inputs and native outputs. Its 98 baseline failures arise from the R comparator deleting an empty JSON contributor-array field while retaining an empty native character vector. Both are empty applied-source sets. Explicit empty-set normalization resolves that representation mismatch; eight additional controls reject absent/null fields, invalid identities, lost or invented sources and dropped or duplicated rows. No source record, sequence or lane comparison is excluded, and the failing baseline remains in the table.

The 3 four-draw campaigns contain 78,696 models / 472,176 sample/file lanes, with 0 failed model comparisons. Each seed adds four draws in every one of 6,048 cells: the 504 exon/UTR geometry cells crossed with four partner allele shapes and three partner start locations (overlapping the first source span, in the same coding exon or in another coding exon). All source spans intersect their selected transcript. The original 1,536 fixed-model and 504 geometry cases precede the interaction draws without changing their inputs or RNG stream. Complete CDS/protein, sample counts, input provenance, source mapping, reference-model sequence and per-lane applied-source identities have separate gates.

This remains a standard-code, 180-base reference-CDS grammar with three diploid samples. Quota coverage is not an estimate of population error rates. Grouped upstream flags, arbitrary genetic codes, phase sets, structural composition and complete phased SO/HGVS require their own evidence; the publication failures below are not waived by these diagnostic comparisons.

Short and long CDS diagnostics

These are local diagnostics, not source-bound release certificates. The receipt ledger names the source revisions, measured binary hash and retained inputs, observations and comparisons. The initial query error is a retrospective capture from its terminal log and retained files, not a completed runner receipt. Missing comparison counts in the two aborted runs mean not computed, not zero failures.

The one-draw length cohort crosses CDS lengths 36/37/38, 2,047/2,048/2,049 and 6,143/6,144/6,145 with all 504 exon/UTR geometry cells. The 4,536 models use the registered reference’s ATG start, repeated internal codons and a raw TAA suffix; partial terminal codons remain in the comparison. The 1,536 fixed-model cases remain included. Source e58f18ac3820c298af11694ee09282ecae93a01e compares complete CDS/protein, sample counts, input provenance, source mapping, reference CDS and per-lane applied-source identities for all 6,072 models and 36,432 sample/file lanes. All seven failure counts are zero, and all 42 corruption controls pass.

At the default 16,777,216-cell limit, transcript 4,561 requires 32,274,432 cells for exact CDS traceback and the query fails. The successful retry explicitly uses 67,108,864 cells per sequence axis with byte-identical generated inputs. It does not drop records, omit differences or substitute approximate alignment. The separate one-cell control fails at a 26,969-cell requirement and records the executed SQL, native error and full input/oracle hashes without a comparison verdict.

This generated standard-code cohort does not estimate a population error rate or certify arbitrary reference models, grouped upstream flags, phase sets, structural composition or complete phased SO/HGVS. The grouped-metadata publication failures below remain unresolved.

The clean source-bound campaign above combines 32 draws in each fixed-model rare stratum with one draw in every geometry, interaction and CDS-length stratum. It stops at transcript 28,010, a 6,143-base CDS, because exact traceback needs 37,742,592 cells against the declared 16,777,216-cell limit. All 29,520 input profiles and upstream observations remain in its receipt. Native output and comparison counts were not produced. This is a failed execution, not zero disagreements. The chained seed 20260906 did not execute; neither seed is certified by this campaign. No input, alignment algorithm or limit was changed to pass it.

Complete model campaigns with an explicit alignment budget

The model-history publisher requires a CI-authenticated execution receipt and reopens retained artifacts. It regenerates the declared seeded inputs, reconstructs comparisons from keyed upstream/native observations, checks every summary and control, and derives alignment bounds from inputs. Existing rows must match the reconstruction exactly. Publication verifies the recorded verdict; it does not convert disagreements into passes. The publication regression test uses retained pinned observations and rejects rehashed comparison forgeries and coherent carrier-lane swaps while accepting honestly recorded failures. The historical campaigns below were locally executed and their complete observations were reconstructed at 7d7d0bc62806f9104db9d59911ebf62f9a3ae4e1. They have no CI-issued signatures and do not satisfy the authenticated-publication gate. Their recorded counts and verdicts remain unchanged.

Source 0857ec1faf5736299a4341f73ffa557d8a9691ee completes both full campaigns with all 59,040 models, 388,896 source records and 354,240 sample/file lanes. Every declared stratum meets its quota. Sequence, count, reference-model, source, mapping and lane comparisons have zero disagreements. Each seed also passes exact nominal-length and raw lane-flag comparisons, the observed upstream group-owner checks, 42 corruption controls and 18 metadata controls. The ledger retains all input, output, comparator and receipt identities. These results cover the generated standard-code grammar, not a population error rate or complete phased annotation.

The per-call budget is 83,886,080 exact alignment cells per sequence axis; the default remains 16,777,216 and the query workspace limit remains 268,435,456 bytes. For reference length n, a conservative alternate-length bound is n plus the sum of each source record’s longest ALT. The complete matrix (n + 1) * (bound + 1) requires at most 75,479,026 cells for seed 173 and 75,454,442 for seed 20260906. This bound uses retained inputs, not native results. Seed 173’s inputs are byte-identical to the failed campaign above. Its failed receipt remains a failed execution; neither missing output nor a capacity error is counted as agreement.

The separate 6,990 decoded/raw phase disagreements and four grouped-metadata publication disagreements remain retained. Per-run group-owner validation does not make order-dependent group metadata reproducible across upstream executions. Whole-haplotype consequences, compound HGVS and structural composition require their own implementation and conformance evidence.

Repeated-model publication audit

Source 16afccd0fd795df318bbd8b3e66f89b8321da022 repeats all 69,120 models above with identical inputs. The sequence/count, model, mapping, source-provenance and complete-lane gates pass. The full keyed oracle-record comparison with afb41c8688fbc058df3687160328f0c1c554bf67 fails for 4 records. They remain in the complete disagreement pairs; the failed publication audit is not promoted into the passing model-history table.

The four changed records are TG13940, TG14331, TG14521 and TG14573 from seed 173. Only their grouped CDS flags differ: empty versus frameshift, indel. Complete sequences, counts and contributing-source identities agree. This classifies the differences; it does not remove those fields from the audit.

The fixed reproducer uses TG13940’s seven-exon model and both original same-position records, including the mixed missing call. It invokes the original VEP-116 Runner JSON path in 64 fresh processes: 32 Perl hash seeds, each repeated twice, with PERL_PERTURB_KEYS=0. A second 64-process lane observation must preserve every byte of the original JSON within each seed and repeat. The per-sample/file-lane sequences and raw flags agree across all seeds.

Run the reproducer with Rscript test/duckvep/conformance/haplotype_grouped_flags.R. It stages the registered reference without network access; the pinned VEP environment and tools must already be installed. Complete JSON, lane observations, environment and input hashes remain in its receipted artifact directory. Receipt SHA-256: 4e4dd34fcf5ca14f1ecdcd9b31730a1c141561f3a84dd0a9ce47c415ebd1a49c.

For the 180-base CDS group, has_indel is zero in 11 seeds and one in 21; both repeats agree. All groups retain the same three sample memberships. Observed group metadata matches the first contributing mutation lane. The pinned container implementation iterates samples in hash order, initializes each sequence-keyed object from its first lane, and adds later samples without combining their flags.

This establishes order-dependent grouped metadata for the pinned implementation and fixture. It does not establish a biological error, validate whole-haplotype consequences or supply a consensus flag rule. Native flags describe occupied edit paths; equal final sequences need not have equal edit histories. Full grouped-metadata conformance remains unresolved.

Reference-only sample routes

Source c69e0e0cf124c65cc8e23c25ca80691bd446a2cd passes 23,040 generated models. Each seed supplies 32 draws in all 360 cells: three start codons (ATG/CTG/TTG), internal-stop presence, terminal-stop presence, both strands, three all-missing GT spellings and five genotype-retention routes. Internal-stop codons, stop positions, source positions and intron lengths vary. The routes cover missing-only samples, retained reference lanes, later retained calls, and calls in 1–12-base or 13–120-base introns. Models have one or two exons, a complete 180-base standard-code CDS, and three samples.

The original upstream container JSON is the oracle. CDS and protein groups are compared separately by sample and count: one CDS can belong to both a curated reference protein and a mutation protein. The observer must preserve every byte of each complete keyed canonical JSON record, including array order. It records the original mutator’s 55,296 sample/file lanes, with complete sequences and applied-source identity sets. The independent audit also verifies exon-based genotype admission.

Each model has six upstream carrier memberships. DuckVEP emits four memberships for the tested samples and keeps the two pure-reference memberships implicit. All 46,080 implicit memberships are checked upstream and must be absent from native output; no oracle sequence is inserted into the native comparison. Native contributors retain exact source IDs, regions, positions, REF/ALT bytes, allele ordinals, evidence and projection status. All 29 corruption controls pass per seed. Each run also passes 25 metadata and 16 output controls. Metadata comparisons retain the exact signed nominal edit-length sum for each mutation lane and the observed upstream owner of each sequence group. They do not infer nominal length from the final CDS length or from frame bits. Older history rows leave these unmeasured metadata fields empty rather than implying a pass.

A sample without retained exon-overlapping genotypes uses the curated reference peptide, preserving internal stops and legitimate-start methionine. Retained exonic reference lanes use mutation translation even with no coding edit. Intronic context retains provenance without selecting mutation translation or changing literal CDS replay, including short introns with distinct SO semantics. Missing evidence remains conditional; model/REF failures still withhold sequence. These are compatibility rules, not a claim that VEP is biologically wrong.

This stratified grammar does not estimate population error rates or certify arbitrary genetic codes, biological models, ploidy/PS inference, splicing, whole-haplotype SO/HGVS or structural composition. Earlier failing diagnostics and the separate decoded/raw phase disagreements remain retained.

Individual Sequence Ontology terms

For each transcript pair, this compares the union of terms emitted by either engine. A missing or extra term is therefore visible under its own SO name. Rows must not be summed across terms because one pair can carry several terms.

term_mismatches counts an SO term that is missing or extra on a transcript pair. unresolved_term_memberships is reported separately: an unresolved pair can still carry the exact VEP term set, and a multi-term pair appears once under each term.

No SO-term mismatch or unresolved membership remains in the latest declared runs.

Terms absent from this frontier table were exact and resolved everywhere they were observed. The complete zero and nonzero strata remain in the CSV ledger.

VEP impact classes

This table uses full consequence sets, so each transcript pair is counted once within each corpus.

The source artifact hash and exact Ensembl core/variation build remain in test/duckvep/conformance/data/conformance_history.csv for audit and reruns.

Variant-induced NMD

This is a separate executable differential against the pinned VEP Plugins release/116 NMD.pm. It compares triggering, escaping, and unresolved for every eligible transcript pair; it does not infer NMD from the core NMD_transcript_variant biotype consequence.

The b7c7237ee686 run uses registered variantkey_clinvar_20260706 on chromosome 21: 49,937 source records yield 49,781 eligible alleles under the existing 50-base limit, with no duplicate eligible alleles removed. The oracle uses registered vep116_grch38_cache_chr21, containing every chromosome-21 cache file and root metadata; the native model is duckvep_ensembl116_model. Its 1,353,288 native pairs all match. The 70,521 NMD classifications include 29,954 unresolved by both engines, not resolved predictions. These reproduce the a84ff1500149 counts with the checksum-verified cache. The older run with 1,331,664 native pairs and 68,554 NMD classifications has different denominators; comparison with that run is not an identical-workload comparison.

The ledger keeps the prediction confusion matrix rather than only the total:

VEP projects the complete uploaded VariationFeature for the plugin’s CDS and exon-position rules. DuckVEP therefore retains both geometries: minimized edit coordinates drive consequence and sequence changes, while the original feature endpoints drive NMD. A padded and a minimal allele can encode the same sequence edit but cross the plugin’s inclusive positional threshold differently.

Species product matrix

The receipt column is the SHA-256 of the model’s current-definition receipt in duckvep_model_receipts.csv. Exact pairs count normalized (variant ID, transcript ID, SO-term set) agreements with the indexed-cache oracle; the corpus links specify the sampling and comparator. Human ledger rows predate their most recent receipt rebuilds, so their receipt hashes identify the current product definition, not the binary artifact used by those historical differential runs.

Issue #5 completion evidence

• Deterministic receipts and exact indexed-cache differentials: the matrix above and each product’s source/model/corpus records, including the table-6 Tetrahymena pins.
• Genetic-code translation or failure before publication: coding-table counts appear in the receipt ledger; every admitted pair is exact against VEP 116. Unsupported table IDs fail projection in test/sql/duckvep_projection.test.
• Actual alternative codons: table-6 TAA/TAG, fly table-5 TGG/TGA, Pf table-4 and table-11 witnesses and plant table-11 initiation witnesses are exercised by the Docker comparisons. The offline excerpts compare source codons under table 6 and table 1 in both SQL and installed-package R.
• Distinct GRCh37 / no fabricated MANE: the GRCh37 row retains its own receipt and differential. test/sql/duckvep_mane_grch37.test enforces GRCh37 MANE ownership; missing mappings remain absent.
• Coexistence, packaging and ownership: test/sql/duckvep_species_coexistence.test and r/Rduckvep/inst/tinytest/test_species_coexistence.R load three assembly-specific models at once, check local ordinals and receipts, and exercise independent model dropping and FASTA pinning. The SQL/R release tests, randomized coverage and sanitizer evidence are recorded in the preceding conformance sections. The offline fixtures are source excerpts rather than whole-model performance or oracle substitutes.