Recorded time tags and exact CSV conversion in r25

The standalone and configured SaaS research worker use the same pf_integrations.time_tags decoder and coincidence analysis. Recorded binary replay uses the installed Swabian SDK. It does not contact an instrument. CSV replay works without that SDK. Converting Swabian binary recordings to CSV requires the SDK.

Install and inspect the SDK

The vendor currently provides the Swabian-TimeTagger package and Swabian.TimeTagger Python module. PF also accepts the legacy TimeTagger module. Install into the interpreter that runs the research worker, then inspect that interpreter:

.venv_research\Scripts\python.exe -m pip install Swabian-TimeTagger==2.22.6
.venv_research\Scripts\python.exe convert_time_tags.py --check-sdk

This diagnostic reports the module and SDK version, while explicitly reporting recording_validation: not_run. An SDK import is not a recording or instrument accuracy test. Confirm current platform/Python support in the vendor installation instructions. The optional hosted worker must expose this SDK to its research Python interpreter.

Inputs and exact arithmetic

A time_tags study supplies exactly one of events, input_file, or input_files. CSV files have exactly two columns: timestamp_ps,channel. Use decimal integer timestamps; floating-point timestamps are rejected. JSON outputs encode large timestamps as strings. Neither conversion nor analysis rescales timestamps, estimates missing events, or invents an acquisition duration.

For binary recordings, set format: "swabian" and use an explicit ordered list, such as:

"input_files": ["recording.1.ttbin", "recording.2.ttbin"]

The entries must have the same recording name and contiguous ascending positive segment numbers. A single numbered file can instead use input_file. The unnumbered sequence header, wildcards, symlinks, directories, duplicate segments, and skipped/reordered numbers are rejected. This is an explicit contiguous subset; PF cannot establish that additional first/last recording segments do not exist. Every file is snapshotted in a separate temporary directory before decoding, preventing implicit reads of undeclared siblings. A timestamp reset, rollover, regression, or configuration change between segments fails with an error.

The limits are 256 segments, 64 MiB combined input, 200,000 decoded records, and 2,000,000 candidate event pairs. Partition large acquisitions explicitly. Partitioning boundaries affect cross-boundary pair counts and must be declared when combining results.

examples_r25/config_swabian_segments.json is an editable acquisition template; its recording files are not included. Replace its acquisition identifier, detector channels and full exposure window before use. config_r25_time_tags_exact_csv.json is runnable software verification data, explicitly marked synthetic and containing timestamps above JavaScript's exact numeric range.

SaaS upload controls turn each declared binary segment or CSV into a tenant-managed asset. Submit input_files as the ordered list of asset references. Preserve numbered basenames. Only the service sets input_root; a caller-supplied filesystem path is not permission to read a server file.

Convert for upload or independent inspection

Run from the PF package directory, where relative input paths resolve:

.venv_research\Scripts\python.exe convert_time_tags.py --config examples_r25\config_swabian_segments.json --output converted_recording

The output directory must not already exist. Successful clean conversion writes:

• events.csv: exact decoded detector records in original order.
• provenance.json: original file SHA-256 values, sizes, per-segment record counts, instrument configuration, SDK module/version, event counts, acquisition declaration, and output checksum.
• replay_config.json: the analysis controls and embedded conversion provenance. Its CSV checksum is verified on replay.

For command-line replay, change into converted_recording, then invoke the parent runner so events.csv resolves in that directory:

..\.venv_research\Scripts\python.exe ..\run_research_integrations.py --config replay_config.json --output replay_results

For hosted replay upload events.csv, retain the generated conversion_provenance object, and replace input_file with its managed asset reference. Raw and calibrated timestamps remain distinct; declared channel offsets only affect subsequent analysis.

Losses and acquisition qualification

Binary event types and missed-event counters are inspected. Any non-time-tag marker or nonzero missed-event count blocks coincidence analysis, regardless of loss_policy. The decoder preserves all such marker records with their original integer timestamps, channels, event types and missed counts. The missed-event total is a lower bound because an overflow does not guarantee a complete missing-count estimate.

The default conversion loss_policy: "reject" refuses a recording containing these markers. For forensic inspection only, loss_policy: "report" writes events_with_losses.csv, loss_markers.csv, and provenance. It emits no replay configuration and the diagnostic CSV has an extra loss column that ordinary PF CSV replay rejects. No corrected exposure, g2, or experimental-validity result is asserted for that output.

Absence of recorded loss markers does not prove that source photons were all detected. Instrument dead time, selected channel recording, filtering, and calibration remain in the scientific interpretation. The acquisition window is explicitly supplied, including empty exposure; PF does not infer it from the first/last photon. Source identifiers and measurement classification are declarations, not authentication.

Qualification gates

test_r25_time_tags.py exercises deterministic conversion, picosecond preservation above 2^53, original-file immutability, metadata hashes, malformed segment rejection, cross-segment ordering, loss handling and CLI operation. Tests using a fake SDK are explicitly software contract tests.

A separate opt-in test uses an installed real SDK and an independently exported reference CSV. Set PF_SWABIAN_VALIDATION_MANIFEST to a JSON file containing:

{
  "reference_kind": "independent_vendor_export",
  "config": {"schema_version": 1, "integration": "time_tags", "experiment": "coincidence_replay", "parameters": "REPLACE WITH THE FULL ACQUISITION PARAMETERS OBJECT"},
  "reference_csv": "independent_export.csv",
  "sha256": {"recording.1.ttbin": "REPLACE_WITH_SHA256", "independent_export.csv": "REPLACE_WITH_SHA256"}
}

The parameters object must use explicit relative input_files. All inputs resolve inside the manifest directory and all binary segments/reference CSV must have declared checksums. The test compares every timestamp/channel and all resulting coincidence/histogram values. Once the manifest is supplied, SDK or data errors fail the test instead of skipping it. Without it the independent-recording test is explicitly skipped. A public binary decoding check and agreement with another invocation of the same vendor SDK establish I/O parity, not an independent detector calibration.

The vendor FileReader documentation specifies the segment and event APIs. Its library units documentation defines integer picoseconds. Refer to validation_r25 and the release report for the actual execution environments and tests completed in this release.