S1005403_RisCC/doc/report_metadata.md

Simulation Report (archive) — Metadata & Data Format

This document describes the JSON report files created by the application and stored in the archive_simulations/ folder. The goal is to provide a stable reference for post-analysis tools and map visualisation.

Filename and location

• Folder: archive_simulations/
• Filename pattern: YYYYMMDD_HHMMSS_<scenario_name>.json (example: 20251105_102118_test.json).
• Encoding: UTF-8, pretty-printed JSON (indent=4).

Top-level JSON structure

• metadata (object)

  • scenario_name (string)
    • Name of the scenario used for the run.
  • start_timestamp_utc (string, ISO-8601)
    • Wall-clock UTC timestamp when the archive was saved (e.g. 2025-11-05T10:21:18.123456).
  • duration_seconds (number, seconds)
    • Duration of the recorded session (end_time - start_time), in seconds.
  • client_update_interval_s (number, seconds) — optional
    • The update interval the client used when sending updates to the server (from UI update_time). Example: 1.0 means one update per second.
  • client_update_rate_hz (number, Hz) — optional
    • The computed send rate (1 / client_update_interval_s). Example: 1.0 Hz.
  • estimated_latency_ms (number, milliseconds) — optional
    • One-way estimated latency (server → client) computed by the ClockSynchronizer model, rounded to 2 decimal places.
  • prediction_offset_ms (number, milliseconds) — optional
    • Optional manual prediction offset read from connection settings.
  • latency_summary (object) — optional
    • Summary statistics over recent latency samples (milliseconds). Keys:
      • mean_ms (number)
      • std_ms (number)
      • var_ms (number)
      • min_ms (number)
      • max_ms (number)
      • count (integer)
  • latency_samples (array of arrays) — optional
    • Array of [timestamp, latency_ms] pairs where timestamp is the monotonic time (seconds) and latency_ms is the measured latency in milliseconds. Contains all available samples collected during the simulation (up to 10000 samples, sufficient for ~2.7 hours at 1Hz sampling).
    • Example: [[3255.123, 1.2], [3255.456, 1.5], ...]
    • Note: Old archives using latency_samples_ms format are no longer supported.

• scenario_definition (object)

  • The scenario data returned by Scenario.to_dict(). Structure depends on the scenario model (targets, durations, etc.). Keep in mind some fields may be ctypes-derived values when taken from live packets.

• ownship_trajectory (array of objects)

  • Time-ordered list of ownship samples recorded during the run. Each entry is an object with fields such as:
    • timestamp (number) — monotonic clock timestamp used by the app (seconds)
    • position_xy_ft (array [x_east_ft, y_north_ft]) — position in feet relative to the local origin used by the simulation
    • altitude_ft (number)
    • velocity_xy_fps (array [vx_east_fps, vy_north_fps]) — feet per second
    • heading_deg (number) — degrees (0 = North)
    • latitude (number) — decimal degrees (WGS84)
    • longitude (number) — decimal degrees (WGS84)

  Note: latitude/longitude may be absent for some ownship samples if the upstream source doesn't provide them.

• simulation_results (object)

  • A mapping of target_id → object with two arrays: simulated and real.
  • For each target id (string keys in JSON but integer in code):
    • simulated — array of tuples/lists: [timestamp, x_ft, y_ft, z_ft]
      • timestamp: monotonic timestamp (seconds)
      • x_ft, y_ft, z_ft: positions in feet. Convention used: x = East, y = North, z = altitude (feet).
    • real — same format as simulated but represents states reported by the radar (reception time used for timestamp).

• simulation_geopos (object)

  • A mapping of target_id → array of geo-position samples computed for each recorded real state when possible.
  • Each geo sample object contains:
    • timestamp (number) — monotonic timestamp associated with the real sample
    • lat (number) — decimal degrees (WGS84)
    • lon (number) — decimal degrees (WGS84)
    • alt_ft (number | null) — altitude in feet (if available)
  • How geopos are computed (implementation notes):
    • The archive looks for the nearest ownship_trajectory sample in time that includes latitude, longitude and position_xy_ft.
    • It computes the east/north offsets between the target and ownship (units: feet), converts them to meters, and then converts meters to degrees using a simple equirectangular approximation:
      • lat offset = delta_north_m / R
      • lon offset = delta_east_m / (R * cos(lat_rad))
      • R used: 6,378,137 m (WGS84 sphere approximation)
    • The result is suitable for visualization on maps at local/regional scales. For high-precision geodesy or very large distances, replace this with a proper geodetic library (e.g. pyproj / geographiclib).

Extra notes and best practices

• Units and coordinate conventions

  • Positions reported in simulation_results are in feet (ft). Velocities in feet/second (fps). Time values are seconds (monotonic timestamps used internally); start_timestamp_utc is wall-clock UTC ISO string.
  • Georeferenced coordinates are decimal degrees (WGS84) and altitude remains in feet.

• Sample timestamps

  • timestamp values saved with target states are the application's monotonic timestamps (used consistently for relative timing). When correlating with wall-clock time, use start_timestamp_utc and duration_seconds as anchors.

• Size considerations

  • latency_samples_ms and simulation_geopos are intentionally limited in length to avoid huge files. If you need full-resolution traces, consider enabling raw persistence (the application can persist raw SFP packets separately) or adjust retention in the router/archive code.

Example (excerpt)

{
    "metadata": {
        "scenario_name": "test",
        "start_timestamp_utc": "2025-11-05T10:21:18.123456",
        "duration_seconds": 12.34,
        "client_update_interval_s": 1.0,
        "client_update_rate_hz": 1.0,
        "estimated_latency_ms": 12.34,
        "latency_summary": {"mean_ms": 12.3, "std_ms": 0.5, "count": 100},
        "latency_samples_ms": [11.2, 12.1, 12.4]
    },
    "ownship_trajectory": [
        {"timestamp": 1.0, "position_xy_ft": [0.0, 0.0], "latitude": 45.0, "longitude": 9.0}
    ],
    "simulation_results": {
        "1": {
            "simulated": [[1.0, 100.0, 200.0, 30.0]],
            "real": [[1.1, 100.0, 200.0, 30.0]]
        }
    },
    "simulation_geopos": {
        "1": [{"timestamp": 1.1, "lat": 45.0009, "lon": 9.0012, "alt_ft": 30.0}]
    }
}

If you want, I can:

• Add a small unit test that verifies simulation_geopos computation for a controlled synthetic case (recommended), or
• Replace the equirectangular approximation with pyproj/geographiclib for improved accuracy (adds dependency), or
• Add this document text into another doc or README section.

---

Document created by the development tooling. If you'd like this expanded with a field-level table or example scripts that load and plot the archive on a map (GeoJSON output), tell me which format you prefer and I will add it.