ADR-0009: On-device OCR & local report-missing

Status

Accepted

Area

Features / privacy

Context

Two new flows need to capture a photo and scan a model label, but the app must honour the LAN / no-telemetry pledge and avoid requesting a camera permission.

Photo capture happens in the WEBVIEW, not a camera plugin. Both new flows use a plain <input type="file" accept="image/*" capture="environment">; the wasm layer (ui/src/capture.rs) reads the chosen File via a FileReader wrapped in a JS Promise we await, yielding a data:<mime>;base64,… URL. This means NO camera permission (the system camera app returns the image) and the preview <img src="data:…"> is already allowed by the existing CSP (img-src 'self' data: — left untouched). Manifest declares android.hardware.camera required="false" only.

Decision

OCR is Google ML Kit on-device text recognition via a Kotlin plugin (OcrPlugin.kt, @TauriPlugin("OcrPlugin")) mirroring IrPlugin.kt. Rust ocr::ocr_image(app, image_base64) strips the data:…, prefix and run_mobile_plugin("recognize", …); Kotlin Base64.decodeBitmapFactory.decodeByteArrayInputImage.fromBitmapTextRecognition.getClient(DEFAULT_OPTIONS).process() → resolve {text}. Gradle dep com.google.mlkit:text-recognition:16.0.1 (bundled Latin model, offline, needs no permission; adds a few MB to the APK). On non-Android ocr_image returns "OCR needs the Android app" — the model field stays editable so the search still works by typing.

tv_core::ir::find_by_model(query) normalizes the query with the same normalize used for button-name matching (lowercase, alphanumerics only) and returns every code set whose normalized id or label contains it, across all device types/brands (so AA59-00741A, aa59 00741 a, etc. all hit TV/Samsung/Samsung_AA59-00741A.ir). Empty/punctuation-only query → no match (never match-all). Exposed as ir_find_by_model; a picked match opens the normal find-code wizard as a single-set NewRemote (brand = id’s middle segment), reusing the existing confirm/save path.

Report-missing is a purely local export. report::save_report(app, name, model, notes, image_base64) writes <app data>/reports/<sanitized-name>/ (collision-suffixed -2, -3, …) containing report.json ({name, model, notes, image_file}) and, if a photo was captured, photo.jpg (base64 decoded via the base64 crate, added to fernweh-tauri). Returns the folder path, which the UI toasts. Timestamp-free by design.

Consequences

No network path exists in the report-missing module (LAN / no-telemetry pledge kept); OCR runs entirely on-device with a bundled offline model that needs no permission.

The OCR flow is untestable without the APK + a phone; verified only that Rust/UI compile and the command is registered.