# Simulation Report (archive) — Metadata & Data Format

This document describes the JSON report files created by the application and stored in the `archive_simulations/` folder. The goal is to provide a stable reference for post-analysis tools and map visualisation.

Filename and location
- Folder: `archive_simulations/`
- Filename pattern: `YYYYMMDD_HHMMSS_<scenario_name>.json` (example: `20251105_102118_test.json`).
- Encoding: UTF-8, pretty-printed JSON (indent=4).

Top-level JSON structure

- `metadata` (object)
  - `scenario_name` (string)
    - Name of the scenario used for the run.
  - `start_timestamp_utc` (string, ISO-8601)
    - Wall-clock UTC timestamp when the archive was saved (e.g. `2025-11-05T10:21:18.123456`).
  - `duration_seconds` (number, seconds)
    - Duration of the recorded session (end_time - start_time), in seconds.
  - `client_update_interval_s` (number, seconds) — optional
    - The update interval the client used when sending updates to the server (from UI `update_time`). Example: `1.0` means one update per second.
  - `client_update_rate_hz` (number, Hz) — optional
    - The computed send rate (1 / client_update_interval_s). Example: `1.0` Hz.
  - `estimated_latency_ms` (number, milliseconds) — optional
    - One-way estimated latency (server → client) computed by the `ClockSynchronizer` model, rounded to 2 decimal places.
  - `prediction_offset_ms` (number, milliseconds) — optional
    - Optional manual prediction offset read from connection settings.
  - `latency_summary` (object) — optional
    - Summary statistics over recent latency samples (milliseconds). Keys:
      - `mean_ms` (number)
      - `std_ms` (number)
      - `var_ms` (number)
      - `min_ms` (number)
      - `max_ms` (number)
      - `count` (integer)
  - `latency_samples` (array of arrays) — optional
    - Array of `[timestamp, latency_ms]` pairs where timestamp is the monotonic time (seconds) and latency_ms is the measured latency in milliseconds. Contains all available samples collected during the simulation (up to 10000 samples, sufficient for ~2.7 hours at 1Hz sampling).
    - Example: `[[3255.123, 1.2], [3255.456, 1.5], ...]`
    - Note: Old archives using `latency_samples_ms` format are no longer supported.

- `scenario_definition` (object)
  - The scenario data returned by `Scenario.to_dict()`. Structure depends on the scenario model (targets, durations, etc.). Keep in mind some fields may be ctypes-derived values when taken from live packets.

- `ownship_trajectory` (array of objects)
  - Time-ordered list of ownship samples recorded during the run. Each entry is an object with fields such as:
    - `timestamp` (number) — monotonic clock timestamp used by the app (seconds)
    - `position_xy_ft` (array `[x_east_ft, y_north_ft]`) — position in feet relative to the local origin used by the simulation
    - `altitude_ft` (number)
    - `velocity_xy_fps` (array `[vx_east_fps, vy_north_fps]`) — feet per second
    - `heading_deg` (number) — degrees (0 = North)
    - `latitude` (number) — decimal degrees (WGS84)
    - `longitude` (number) — decimal degrees (WGS84)

  Note: `latitude`/`longitude` may be absent for some ownship samples if the upstream source doesn't provide them.

- `simulation_results` (object)
  - A mapping of `target_id` → object with two arrays: `simulated` and `real`.
  - For each target id (string keys in JSON but integer in code):
    - `simulated` — array of tuples/lists: `[timestamp, x_ft, y_ft, z_ft]`
      - `timestamp`: monotonic timestamp (seconds)
      - `x_ft`, `y_ft`, `z_ft`: positions in feet. Convention used: `x` = East, `y` = North, `z` = altitude (feet).
    - `real` — same format as `simulated` but represents states reported by the radar (reception time used for `timestamp`).

- `simulation_geopos` (object)
  - A mapping of `target_id` → array of geo-position samples computed for each recorded real state when possible.
  - Each geo sample object contains:
    - `timestamp` (number) — monotonic timestamp associated with the real sample
    - `lat` (number) — decimal degrees (WGS84)
    - `lon` (number) — decimal degrees (WGS84)
    - `alt_ft` (number | null) — altitude in feet (if available)
  - How geopos are computed (implementation notes):
    - The archive looks for the nearest `ownship_trajectory` sample in time that includes `latitude`, `longitude` and `position_xy_ft`.
    - It computes the east/north offsets between the target and ownship (units: feet), converts them to meters, and then converts meters to degrees using a simple equirectangular approximation:
      - lat offset = delta_north_m / R
      - lon offset = delta_east_m / (R * cos(lat_rad))
      - R used: 6,378,137 m (WGS84 sphere approximation)
    - The result is suitable for visualization on maps at local/regional scales. For high-precision geodesy or very large distances, replace this with a proper geodetic library (e.g. `pyproj` / `geographiclib`).

Extra notes and best practices
- Units and coordinate conventions
  - Positions reported in `simulation_results` are in feet (ft). Velocities in feet/second (fps). Time values are seconds (monotonic timestamps used internally); `start_timestamp_utc` is wall-clock UTC ISO string.
  - Georeferenced coordinates are decimal degrees (WGS84) and altitude remains in feet.

- Sample timestamps
  - `timestamp` values saved with target states are the application's monotonic timestamps (used consistently for relative timing). When correlating with wall-clock time, use `start_timestamp_utc` and `duration_seconds` as anchors.

- Size considerations
  - `latency_samples_ms` and `simulation_geopos` are intentionally limited in length to avoid huge files. If you need full-resolution traces, consider enabling raw persistence (the application can persist raw SFP packets separately) or adjust retention in the router/archive code.

Example (excerpt)

```json
{
    "metadata": {
        "scenario_name": "test",
        "start_timestamp_utc": "2025-11-05T10:21:18.123456",
        "duration_seconds": 12.34,
        "client_update_interval_s": 1.0,
        "client_update_rate_hz": 1.0,
        "estimated_latency_ms": 12.34,
        "latency_summary": {"mean_ms": 12.3, "std_ms": 0.5, "count": 100},
        "latency_samples_ms": [11.2, 12.1, 12.4]
    },
    "ownship_trajectory": [
        {"timestamp": 1.0, "position_xy_ft": [0.0, 0.0], "latitude": 45.0, "longitude": 9.0}
    ],
    "simulation_results": {
        "1": {
            "simulated": [[1.0, 100.0, 200.0, 30.0]],
            "real": [[1.1, 100.0, 200.0, 30.0]]
        }
    },
    "simulation_geopos": {
        "1": [{"timestamp": 1.1, "lat": 45.0009, "lon": 9.0012, "alt_ft": 30.0}]
    }
}
```

If you want, I can:
- Add a small unit test that verifies `simulation_geopos` computation for a controlled synthetic case (recommended), or
- Replace the equirectangular approximation with `pyproj`/`geographiclib` for improved accuracy (adds dependency), or
- Add this document text into another doc or README section.

---
Document created by the development tooling. If you'd like this expanded with a field-level table or example scripts that load and plot the archive on a map (GeoJSON output), tell me which format you prefer and I will add it.