BM ↔ MMDVMHost conformance corpus — format, provenance & PII/licensing policy

Status: policy DECIDED, corpus EMPTY. This file exists so the rules are settled before any capture lands in git (Bench Spike 0, leg (b) — epics.md:334). The capture itself is operator-run on the bench; nothing is committed here until it conforms to the policy below. This corpus is the test oracle for the Epic 1 engine port — the M1 mitigation (risk-seam-map.adoc, row M1).

What this corpus is for

The Epic 1 MMDVMHost frame-relay engine (crates/engine) is the critical path (risk M1). Its correctness is judged against real BrandMeister ↔ MMDVMHost traffic rather than against our own assumptions. A small, anonymised golden subset captured from a live session becomes the replayable oracle: feed the recorded frames at their recorded timing, assert the engine produces the expected relayed output.

That means a record must carry two things, not one:

  1. The frame bytes (DMRD data frames + the control frames around them), and

  2. The inter-frame arrival timing — leg (a) and the engine’s slot/hang-timer logic are timing-sensitive, so a corpus of bytes alone is not a sufficient oracle.

Format (decided)

Encoding: JSON Lines (.jsonl) — one JSON object per frame, newline-delimited. Chosen over length-prefixed binary because the committed subset is small, must be human-reviewable for PII before it lands, and diffs legibly in git. One file per captured session.

Each line is one frame:

{"seq": 0, "t_us": 0,       "dir": "bm->host", "kind": "DMRD",    "len": 55, "hex": "444d52440000…"}
{"seq": 1, "t_us": 60120,   "dir": "bm->host", "kind": "DMRD",    "len": 55, "hex": "444d52440100…"}
{"seq": 2, "t_us": 120080,  "dir": "host->bm", "kind": "control", "len": 11, "hex": "52505449…"}
Field Meaning

seq

0-based frame index within the session (monotonic).

t_us

Arrival timestamp in microseconds, relative to the first frame (seq 0 ⇒ t_us 0). Inter-frame delta = t_us[i] − t_us[i−1]. Same unit the jitter-stats.sh tool consumes — extract the t_us column to one integer per line first (e.g. jq -r '.t_us' session.jsonl), then feed that to the leg-(a) analysis (the tool wants bare integers, not the full JSONL).

dir

bm->host or host->bm (transport direction).

kind

Frame class — DMRD (voice/data), control, or another stock-MMDVMHost tag.

len

Byte length of the original frame (pre-redaction).

hex

Frame bytes, lowercase hex — after the redaction in the PII policy below.

A capture also commits a sibling PROVENANCE.md (template at the end of this file) recording who/what/when/where, so the oracle’s origin is auditable.

PII / RF-capture-licensing policy (decided — the conservative posture)

Real amateur DMR traffic carries other operators' callsigns and DMR-IDs. Those are personal data (PII), and capturing/redistributing third-party RF may carry licensing and consent implications that vary by jurisdiction. Therefore:

  • No raw RF captures in git — ever. Raw captures stay on the operator’s machine. They are .gitignore-guarded (docs/interop/bm-corpus/raw/ + *.dmrcap) so they cannot be committed by accident. (Micro-decision #3.)

  • Only a documented, anonymised oracle subset is committed, and only after manual review. Permitted sources for the committed subset, in order of preference:

    1. Synthetic frames hand-built to exercise a conformance case (no third-party PII at all); or

    2. The operator’s own callsign/DMR-ID traffic (self-captured, self-owned); or

    3. Permission-cleared third-party frames (explicit consent recorded in PROVENANCE.md).

  • Anonymisation rule for any non-synthetic frame: third-party DMR-IDs and callsigns in the hex payload are replaced with reserved placeholder IDs (e.g. the DMR-ID test range) before commit; the substitution is noted in PROVENANCE.md. Timing (t_us) and frame structure are preserved — those are what the oracle tests; identities are not.

  • The committed subset is minimal — only enough frames to pin the conformance cases the engine must satisfy, not a bulk recording.

If the operator’s licensing/consent situation differs, they may adjust this posture — but the change is recorded here in writing before any affected capture is committed.

Layout

docs/interop/bm-corpus/
├── README.md            ← this policy (committed)
├── raw/                 ← raw captures live here; .gitignore-guarded, NEVER committed
└── <session>.jsonl      ← anonymised oracle subset (committed by hand) + <session>.PROVENANCE.md

Capture procedure (operator runbook — leg (b))

  1. Establish a live BM ↔ stock-MMDVMHost session on the rig (real board, real TG, your DMR-ID).

  2. Capture frame bytes with inter-frame arrival timing into raw/ (e.g. a UART/UDP tap or stock-MMDVMHost frame logging; the .dmrcap/raw/ path is git-ignored). Record enough of a QSO to cover the conformance cases (voice superframe start/continue/end + control).

  3. Convert the raw capture to the .jsonl format above, applying the anonymisation rule.

  4. Manually review every line for residual PII. Write <session>.PROVENANCE.md.

  5. Commit only the reviewed .jsonl + its PROVENANCE.md to docs/interop/bm-corpus/.

  6. Record in the spike report (leg (b) result slot) that the oracle landed and where.

PROVENANCE.md template

# Provenance — <session>.jsonl
- Captured by   : <callsign / operator>
- Date (UTC)    : <when>
- Rig           : <MMDVM board model> on <Pi 4 | Pi 5>, MMDVMHost <version>
- Source class  : synthetic | self-owned | permission-cleared
- Consent       : <n/a (synthetic/self) | who consented, when, how>
- Anonymisation : <which IDs/callsigns were replaced, with what placeholders>
- Notes         : <conformance cases this subset is meant to pin>