SXXXXXXX_ControlPanel/doc/VIDEO_RECEIVER_INTEGRATION.md

# VideoReceiverSFP — Integration Guide for ARTOS Collector

This document explains how to integrate the `VideoReceiverSFP` module (the SFP video/image receiver developed in this repository) into the ARTOS orchestration paradigm described in the `doc/` folder. It documents the module's supported API, points of integration with the `TestContext`/Collector, any gaps vs. the ARTOS module contract, and concrete examples for orchestrator implementers.

**Status:** compatible with ARTOS design with a small recommended adapter (see "Gaps & Recommendations").

---

**Contents**

- Overview
- Supported API (what `VideoReceiverSFP` already exposes)
- ARTOS contract checklist (match to docs)
- Gaps & recommended minimal adapter changes
- Installation & configuration
- Usage examples
  - Basic: initialize, register callbacks, start/stop
  - TestContext adapter: exposing `get_data_stream()` to Collector
- Notes on timestamps, synchronization and threading
- Appendix: file references

---

## Overview

`VideoReceiverSFP` is a lightweight, self-contained module that can run in simulation mode (reading image files from disk) and in network mode (receiving SFP fragments via UDP). It provides viewers for MFD and SAR display and exposes callback registration so a host application can receive frames in realtime.

The implementation already follows many ARTOS expectations (start/stop, initialize, status), and integration is straightforward. The Collector can either register callbacks to receive frames, or use a small adapter wrapper to expose the `get_data_stream()` convenience method expected by the ARTOS `TestContext`.

---

## Supported API (present in `VideoReceiverSFP/core/sfp_module.py`)

- `SfpConnectorModule()` — constructor for the module instance.
- `initialize(config: dict) -> bool` — configure the module. Supported keys (observed in code):
  - `sim_dir` (path for simulated frames)
  - `fps` (simulation frames per second)
  - `host`, `port` (network mode)
  - `normalize` (`'cp'` or `'autocontrast'`)
  - `image_type_filter` (`'MFD'` or `'SAR'`)
  - `record_mfd_video`, `record_sar_video`, `video_fps`

- `start_session()` / `stop_session()` — start / stop capture or simulation loop.
- `get_status() -> Dict[str,Any]` — returns runtime info (is_running, sim_dir, fps, host, port).
- `register_callback(cb: Callable[[frame], None])` — generic callback for frames.
- `register_mfd_callback(cb: Callable[[frame], None])` — register MFD-only callback.
- `register_sar_callback(cb: Callable[[frame], None])` — register SAR-only callback.
- `register_sar_metadata_callback(cb: Callable[[str], None])` — register SAR metadata callback.
- SAR parameter setters: `set_sar_brightness`, `set_sar_contrast`, `set_sar_autocontrast`, `set_sar_save_png`.
- `update_mfd_lut(lut)` — store a LUT (added to accept UI LUTs).

Implementation detail: frames delivered to callbacks are `PIL.Image` objects (RGB) when running simulated playback from image files; code also handles bytes payload in some paths.

---

## ARTOS contract checklist

ARTOS expects modules to conform to the `BaseModule` API described in `doc/ARTOS Technical Design Specification.md` and `doc/ARTOS_design.md`. Below is a side-by-side checklist with the module status.

- `initialize(config)` : Present — YES ([VideoReceiverSFP/core/sfp_module.py](VideoReceiverSFP/core/sfp_module.py)).
- `start_session()` / `stop_session()` : Present — YES.
- `get_status()` : Present — YES.
- `get_data_stream()` : ARTOS describes this as the module data-stream interface used by `TestContext`. VideoReceiverSFP does not implement a `get_data_stream()` method with that exact name, but provides `register_*_callback` and keeps the latest frame in `_last_frame` — so Collector can obtain frames either through callbacks or via a small adapter which we recommend (see next section).
- Thread-safety & non-blocking callbacks: The module queues frames and invokes callbacks from a loop thread; callbacks are called directly on that thread and the docs warn they must be non-blocking — matches ARTOS guidance.

Conclusion: the module satisfies the majority of ARTOS requirements _out of the box_. Only the optional convenience method `get_data_stream()` is missing and can be exposed by either the module or by an orchestrator-side adapter.

---

## Gaps & Recommendations

1. `get_data_stream()` / `get_current_data()` convenience method
   - Why: ARTOS `TestContext` uses `module.get_data_stream()` to pull the latest data for algorithms and time alignment.
   - Current: `SfpConnectorModule` stores the last frame in `self._last_frame` and exposes callbacks.
   - Recommendation (minimal): implement a small adapter class inside the Collector that wraps the module's `register_*_callback` and exposes a `get_data_stream()` method returning a tuple `(frame, system_receipt_timestamp)` or a small dict with metadata.
   - Recommendation (module-side alternative): add a `get_data_stream()` method to `SfpConnectorModule` that returns `{'frame': <PIL.Image>, 'timestamp': <float>}` reading from the internal `self._last_frame` together with an internal `system_receipt_timestamp` (could use `time.time()` when frame arrives).

2. Timestamp normalization
   - The module stamps frames only indirectly (frame arrival is recorded in `_frame_times` for FPS). For ARTOS synchronization you need a stable `system_receipt_timestamp` per frame.
   - Recommendation: adapter should attach `system_receipt_timestamp = time.time()` when receiving the frame callback and pass it to the Collector.

3. Data buffering and query API
   - ARTOS algorithms may request small buffers of recent frames (for example N frames around a timestamp). If needed, extend adapter/module with `get_recent_frames(count=...)` that returns a time-indexed list.

4. Contract doc: declare the exact frame object type
   - Document that frames are `PIL.Image` in RGB. If Collector prefers NumPy arrays, convert in adapter using `numpy.asarray(img)`.


---

## Installation & Configuration

This module is part of the repository. To run it inside a virtualenv already configured for the project follow the normal project steps. Example (from repo root):

```bash
# activate venv (example on Windows PowerShell)
& .venv\Scripts\Activate.ps1

# run the module directly (simulation mode uses ./dumps)
python -m VideoReceiverSFP.core.test_orchestrator --sim-dir dumps
```

If you want to import it from the Collector, ensure the Collector's Python process has the repository on `PYTHONPATH` or install this package into the environment.

---

## Usage Examples

### 1) Basic example — Orchestrator registers callbacks

This is the easiest integration path: register callbacks for MFD and SAR frames and for SAR metadata.

```python
from VideoReceiverSFP.core.sfp_module import SfpConnectorModule

module = SfpConnectorModule()
config = {'sim_dir': 'dumps', 'fps': 5}
module.initialize(config)

# Callback that receives PIL.Image frames
def on_mfd_frame(frame):
    # frame is a PIL.Image (RGB)
    # Attach a system timestamp if you need it
    import time
    ts = time.time()
    # push to collector pipeline
    collector.ingest_video_frame('mfd', frame, system_receipt_timestamp=ts)

module.register_mfd_callback(on_mfd_frame)

# same for SAR
def on_sar_frame(frame):
    collector.ingest_video_frame('sar', frame, system_receipt_timestamp=time.time())

module.register_sar_callback(on_sar_frame)

# SAR metadata (optional)
def on_sar_meta(metadata_str):
    collector.ingest_metadata('sar', metadata_str, system_receipt_timestamp=time.time())

module.register_sar_metadata_callback(on_sar_meta)

# start
module.start_session()
# ... stop when done
module.stop_session()
```

This approach matches ARTOS pull model by letting the Collector treat callbacks as the canonical source of data.

### 2) Adapter example — expose `get_data_stream()` for TestContext

If the Collector expects the module to implement `get_data_stream()` (pull model) you can implement a small adapter in the Collector that wraps callbacks and exposes the method.

```python
import threading
import time
from collections import deque

class VideoModuleAdapter:
    def __init__(self, vf_module, maxbuf=128):
        self._module = vf_module
        self._buf = deque(maxlen=maxbuf)
        self._lock = threading.Lock()
        # register callbacks
        self._module.register_mfd_callback(self._on_mfd)
        self._module.register_sar_callback(self._on_sar)

    def _on_mfd(self, frame):
        with self._lock:
            self._buf.append({'type':'mfd', 'frame': frame, 'ts': time.time()})

    def _on_sar(self, frame):
        with self._lock:
            self._buf.append({'type':'sar', 'frame': frame, 'ts': time.time()})

    def get_data_stream(self, last_n: int = 1):
        """Return a list of recent frames with timestamps (most-recent-first)."""
        with self._lock:
            items = list(self._buf)[-last_n:]
        return items
```

Then the Collector's `TestContext` can call `video_adapter.get_data_stream()` to obtain the frames.

### 3) Example TestContext usage

```python
# inside collector orchestration code
video_module = SfpConnectorModule()
video_module.initialize({'sim_dir':'dumps'})
video_adapter = VideoModuleAdapter(video_module)
video_module.start_session()

# TestContext then uses:
recent = video_adapter.get_data_stream(last_n=5)
# pass recent frames to an algorithm for comparison
```

---

## Timestamps & Synchronization

- Attach `system_receipt_timestamp = time.time()` on the first point the Collector receives the frame (either in callback or adapter).
- If the SFP payload includes an internal hardware timestamp, capture it as `hw_timestamp` and include mapping logic in the Collector to align hardware time with system time.
- For deterministic tests use the adapter buffer to extract frames around the event timestamp.

---

## Threading & Callbacks

- Callbacks are invoked from the module's worker thread. Per ARTOS recommendation, keep callbacks non-blocking: farm heavy processing to worker threads or push frames into the Collector's internal queue.
- Example pattern in Collector:
  - callback -> put frame into `queue.Queue()` -> worker thread consumes queue and performs expensive operations (algorithms, DB writes).

---

## Appendix: File references

- Module implementation: [VideoReceiverSFP/core/sfp_module.py](VideoReceiverSFP/core/sfp_module.py)
- Test harness / orchestration example: [VideoReceiverSFP/core/test_orchestrator.py](VideoReceiverSFP/core/test_orchestrator.py)
- Viewer examples: [VideoReceiverSFP/gui/viewer_with_params.py](VideoReceiverSFP/gui/viewer_with_params.py) and [VideoReceiverSFP/gui/viewer_sar.py](VideoReceiverSFP/gui/viewer_sar.py)
- ARTOS design docs: [doc/ARTOS Technical Design Specification.md](doc/ARTOS Technical Design Specification.md), [doc/ARTOS_design.md](doc/ARTOS_design.md)

---

## Next steps & optional code changes

- Option A (recommended quick): implement `VideoModuleAdapter` in Collector as shown above — zero change to the existing module.
- Option B (module change): add `get_data_stream()` to `SfpConnectorModule` that returns the last frame and its `system_receipt_timestamp`. I can prepare a minimal patch for this if you prefer the provider-side API to expose it.

If you want, I will:
- add `get_data_stream()` to `SfpConnectorModule` with a timestamp, plus `get_recent_frames()`; or
- provide a ready-to-drop `VideoModuleAdapter` module in `VideoReceiverSFP/core/adapter_for_artos.py` and an example `collector_integration_example.py` demonstrating full wiring into a minimal TestContext.

Which of the two next steps do you prefer? (module-side API vs. orchestrator adapter)