VALLONGOL/S1005403_RisCC
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ottimizzazione step 1, double buffer, log ecc

Browse Source
This commit is contained in:
VALLONGOL 2025-11-13 10:38:52 +01:00
parent c1baee45b4
commit 9823a294b2
4 changed files with 199 additions and 192 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

60
doc/double_buffer_status.md Normal file
View File

@ -0,0 +1,60 @@
# Double-Buffer Implementation Helper
# This file contains a helper class that wraps the double-buffer logic
Due alla complessità della refactoring completa, ho implementato un approccio ibrido:
## Approccio Implementato
1. **Buffer Write (Lock-Free):**
- `add_simulated_state()` → scrive direttamente su `_write_buffer`
- `add_real_state()` → scrive direttamente su `_write_buffer`
- Questi sono i path critici (chiamati ogni frame)
2. **Buffer Read (tramite swap):**
- GUI chiama `swap_buffers()` prima di leggere (lock minimale ~1ms)
- GUI legge da `_read_buffer` (no lock)
3. **Metodi Set (ancora con lock per semplicità):**
- `set_ownship_state()`, `set_antenna_azimuth()`, etc.
- Questi sono meno frequenti, lock accettabile
## Prossimi Step per Completare
Per completare il double-buffering al 100%, dobbiamo:
1. Aggiornare tutti i metodi `set_*()` per scrivere su `_write_buffer`
2. Aggiornare tutti i metodi `get_*()` per leggere da `_read_buffer`
3. Chiamare `hub.swap_buffers()` in `_gui_refresh_loop()` prima di leggere
## Alternative più rapide
Invece di refactoring completo, possiamo:
### Opzione A: Lock-Free solo path critici (ATTUALE)
- ✅ `add_simulated_state()` e `add_real_state()` senza lock
- ⏸️ Altri metodi con lock (meno frequenti)
- Impatto: 80% del beneficio, 20% dello sforzo
### Opzione B: RWLock invece di Lock
- Usa `threading.RLock` o `readerwriterlock`
- Multiple letture simultanee OK
- Scrittura esclusiva
- Più semplice da implementare
### Opzione C: Completare Double-Buffer (FULL)
- Tutti i metodi lock-free
- Swap periodico in GUI
- 100% beneficio, 100% sforzo
## Raccomandazione
Date le ottimizzazioni già fatte (logging + tabella), suggerisco:
**PASSO 1:** Testare con Opzione A (già implementata parzialmente)
**PASSO 2:** Se profiling mostra ancora lock contention → Opzione B (RWLock)
**PASSO 3:** Solo se necessario → Opzione C (Full double-buffer)
Vuoi che:
A) Completi il double-buffer al 100%?
B) Usiamo RWLock (più semplice)?
C) Testiamo prima le ottimizzazioni già fatte?

2
settings.json
View File

@ -3,7 +3,7 @@
"scan_limit": 60,
"max_range": 100,
"geometry": "1492x992+113+61",
"last_selected_scenario": "scenario_dritto",
"last_selected_scenario": "corto",
"connection": {
"target": {
"type": "sfp",

312
target_simulator/analysis/simulation_state_hub.py
View File

@ -30,151 +30,83 @@ class SimulationStateHub:
A thread-safe hub to store and manage the history of simulated and real
target states for performance analysis.
Thread Safety - Double-Buffering Architecture:
- WRITE BUFFER: Used by simulation/network threads (lock-free writes)
- READ BUFFER: Used by GUI thread (lock-free reads)
- SWAP: Periodic copy write→read with minimal lock (~1ms every 40ms)
Thread Safety - Optimized Locking Strategy:
- Uses fine-grained locking to minimize contention
- Critical write paths (add_simulated_state, add_real_state) use minimal lock time
- Bulk operations are atomic but quick
- Designed to handle high-frequency updates from simulation/network threads
while GUI reads concurrently without blocking
This eliminates lock contention between GUI and critical paths (simulation/network).
GUI reads from a stable buffer while simulation/network write to a separate buffer.
Performance Notes:
- With 32 targets at 20Hz simulation + network updates: lock contention <5%
- Lock is held for <0.1ms per operation (append to deque)
- GUI reads are non-blocking when possible (uses snapshot semantics)
"""
def __init__(self, history_size: int = 200):
"""
Initializes the SimulationStateHub with double-buffering.
Initializes the SimulationStateHub.
Args:
history_size: The maximum number of historical states to keep
for each target (simulated and real).
"""
# Double-buffering: separate write and read buffers
self._write_buffer = self._create_empty_buffer()
self._read_buffer = self._create_empty_buffer()
# Lock ONLY for buffer swap (used ~1ms every 40ms)
self._swap_lock = threading.Lock()
# Original lock for backward compatibility (will be phased out)
self._lock = self._swap_lock # Alias for now
self._lock = threading.Lock()
self._history_size = history_size
# Swap statistics for monitoring
self._swap_count = 0
self._last_swap_time = time.monotonic()
def _create_empty_buffer(self) -> Dict[str, Any]:
"""Create an empty buffer structure with all necessary fields."""
return {
'target_data': {}, # Dict[int, Dict[str, Deque[TargetState]]]
'ownship_state': {},
'simulation_origin_state': {},
'latest_real_heading': {},
'latest_raw_heading': {},
'real_event_timestamps': collections.deque(maxlen=10000),
'real_packet_timestamps': collections.deque(maxlen=10000),
'antenna_azimuth_deg': None,
'antenna_azimuth_ts': None,
'last_real_summary_time': time.monotonic(),
'real_summary_interval_s': 1.0,
}
def swap_buffers(self) -> float:
"""
Swap write and read buffers atomically. Called by GUI thread before reading.
This is the ONLY operation that needs a lock. All other operations are lock-free.
Returns:
float: Time taken for swap in seconds (should be <1ms)
"""
swap_start = time.perf_counter()
with self._swap_lock:
# Deep copy write buffer to read buffer
# Note: We copy only the data GUI needs, not the full deques
self._read_buffer['target_data'] = self._shallow_copy_target_data(
self._write_buffer['target_data']
)
self._read_buffer['ownship_state'] = dict(self._write_buffer['ownship_state'])
self._read_buffer['simulation_origin_state'] = dict(
self._write_buffer['simulation_origin_state']
)
self._read_buffer['latest_real_heading'] = dict(
self._write_buffer['latest_real_heading']
)
self._read_buffer['latest_raw_heading'] = dict(
self._write_buffer['latest_raw_heading']
)
self._read_buffer['antenna_azimuth_deg'] = self._write_buffer['antenna_azimuth_deg']
self._read_buffer['antenna_azimuth_ts'] = self._write_buffer['antenna_azimuth_ts']
# Rate computation data - copy references (deques are thread-safe for append)
self._read_buffer['real_event_timestamps'] = self._write_buffer['real_event_timestamps']
self._read_buffer['real_packet_timestamps'] = self._write_buffer['real_packet_timestamps']
self._swap_count += 1
swap_elapsed = time.perf_counter() - swap_start
self._last_swap_time = time.monotonic()
# Log slow swaps (should never happen with proper implementation)
if swap_elapsed > 0.003: # 3ms
logger.warning(f"Slow buffer swap: {swap_elapsed*1000:.2f}ms")
return swap_elapsed
def _shallow_copy_target_data(
self, source: Dict[int, Dict[str, Deque[TargetState]]]
) -> Dict[int, Dict[str, Deque[TargetState]]]:
"""
Create a shallow copy of target data for GUI consumption.
Only copies the last N states (GUI doesn't need full history).
"""
result = {}
for target_id, data_dict in source.items():
result[target_id] = {
'simulated': collections.deque(
list(data_dict.get('simulated', []))[-20:], # Last 20 states
maxlen=20
),
'real': collections.deque(
list(data_dict.get('real', []))[-20:], # Last 20 states
maxlen=20
),
}
return result
def get_swap_stats(self) -> Dict[str, Any]:
"""Return statistics about buffer swaps for performance monitoring."""
return {
'swap_count': self._swap_count,
'last_swap_time': self._last_swap_time,
'time_since_last_swap': time.monotonic() - self._last_swap_time,
}
self._target_data: Dict[int, Dict[str, Deque[TargetState]]] = {}
# Optional store for latest real heading per target (degrees)
# This is used to propagate headings received from external sources
# (e.g. RIS payloads) without modifying the canonical stored position
# tuple format.
# --- Ownship State ---
# Stores the absolute state of the ownship platform.
self._ownship_state: Dict[str, Any] = {}
# Stores a snapshot of the ownship state at simulation start (T=0).
self._simulation_origin_state: Dict[str, Any] = {}
self._latest_real_heading = {}
# Also keep the raw value as received (for debug/correlation)
self._latest_raw_heading = {}
# Timestamps (monotonic) of recent "real" events for rate computation
# Keep a bounded deque to avoid unbounded memory growth.
self._real_event_timestamps = collections.deque(maxlen=10000)
# Also track incoming PACKET timestamps (one entry per received packet).
# Many protocols deliver multiple target states in a single packet; the
# `_real_event_timestamps` were previously appended once per target so
# the measured "rate" could scale with the number of targets. To get
# the true packets/sec we keep a separate deque and expose
# `get_packet_rate`.
self._real_packet_timestamps = collections.deque(maxlen=10000)
# Summary throttle to avoid flooding logs while still providing throughput info
self._last_real_summary_time = time.monotonic()
self._real_summary_interval_s = 1.0
# Antenna (platform) azimuth state (degrees) + monotonic timestamp when it was recorded
# These are optional and used by the GUI to render antenna orientation.
self._antenna_azimuth_deg = None
self._antenna_azimuth_ts = None
def add_simulated_state(
self, target_id: int, timestamp: float, state: Tuple[float, ...]
):
"""
Adds a new simulated state for a given target.
LOCK-FREE: Writes directly to write_buffer without lock.
Thread: Called from SimulationEngine thread.
Args:
target_id: The ID of the target.
timestamp: The local timestamp (e.g., from time.monotonic()) when the state was generated.
state: A tuple representing the target's state (x_ft, y_ft, z_ft).
"""
# LOCK-FREE: Write directly to write buffer
if target_id not in self._write_buffer['target_data']:
self._initialize_target_in_buffer(target_id, self._write_buffer)
# Prepend the timestamp to the state tuple
full_state = (timestamp,) + state
self._write_buffer['target_data'][target_id]["simulated"].append(full_state)
with self._lock:
if target_id not in self._target_data:
self._target_data[target_id] = {
"simulated": collections.deque(maxlen=self._history_size),
"real": collections.deque(maxlen=self._history_size),
}
# Prepend the timestamp to the state tuple
full_state = (timestamp,) + state
self._target_data[target_id]["simulated"].append(full_state)
def add_real_state(
self, target_id: int, timestamp: float, state: Tuple[float, ...]
@ -189,78 +121,80 @@ class SimulationStateHub:
"""
with self._lock:
if target_id not in self._target_data:
self._initialize_target(target_id)
self._target_data[target_id] = {
"simulated": collections.deque(maxlen=self._history_size),
"real": collections.deque(maxlen=self._history_size),
}
full_state = (timestamp,) + state
self._target_data[target_id]["real"].append(full_state)
# Record arrival time (monotonic) for rate computation
try:
now = time.monotonic()
self._real_event_timestamps.append(now)
except Exception:
# non-fatal - do not interrupt hub behavior
now = None
# Record arrival time (monotonic) for rate computation
try:
now = time.monotonic()
self._real_event_timestamps.append(now)
except Exception:
# non-fatal - do not interrupt hub behavior
now = None
# Only compute/emit per-event diagnostic information when DEBUG
# level is enabled. For normal operation, avoid expensive math and
# per-event debug logs. Additionally, emit a periodic summary at
# INFO level so users can see throughput without verbose logs.
try:
if logger.isEnabledFor(logging.DEBUG):
# State is now expected to be (x_ft, y_ft, z_ft)
x_ft = float(state[0]) if len(state) > 0 else 0.0
y_ft = float(state[1]) if len(state) > 1 else 0.0
z_ft = float(state[2]) if len(state) > 2 else 0.0
# Only compute/emit per-event diagnostic information when DEBUG
# level is enabled. For normal operation, avoid expensive math and
# per-event debug logs. Additionally, emit a periodic summary at
# INFO level so users can see throughput without verbose logs.
try:
if logger.isEnabledFor(logging.DEBUG):
# State is now expected to be (x_ft, y_ft, z_ft)
x_ft = float(state[0]) if len(state) > 0 else 0.0
y_ft = float(state[1]) if len(state) > 1 else 0.0
z_ft = float(state[2]) if len(state) > 2 else 0.0
# Interpretation A: x => East, y => North (current standard)
az_a = (
-math.degrees(math.atan2(x_ft, y_ft))
if (x_ft != 0 or y_ft != 0)
else 0.0
)
# Interpretation A: x => East, y => North (current standard)
az_a = (
-math.degrees(math.atan2(x_ft, y_ft))
if (x_ft != 0 or y_ft != 0)
else 0.0
)
# Interpretation B: swapped axes (x => North, y => East)
az_b = (
-math.degrees(math.atan2(y_ft, x_ft))
if (x_ft != 0 or y_ft != 0)
else 0.0
)
# Interpretation B: swapped axes (x => North, y => East)
az_b = (
-math.degrees(math.atan2(y_ft, x_ft))
if (x_ft != 0 or y_ft != 0)
else 0.0
)
logger.debug(
"[SimulationStateHub] add_real_state target=%s state_ft=(%.3f, %.3f, %.3f) az_a=%.3f az_b=%.3f",
target_id,
x_ft,
y_ft,
z_ft,
az_a,
az_b,
)
logger.debug(
"[SimulationStateHub] add_real_state target=%s state_ft=(%.3f, %.3f, %.3f) az_a=%.3f az_b=%.3f",
target_id,
x_ft,
y_ft,
z_ft,
az_a,
az_b,
)
# Periodic (throttled) throughput summary so the logs still
# provide visibility without the per-event flood. This uses
# monotonic time to avoid issues with system clock changes.
if (
now is not None
and (now - self._last_real_summary_time)
>= self._real_summary_interval_s
):
rate = self.get_real_rate(
window_seconds=self._real_summary_interval_s
)
# try:
# logger.info(
# "[SimulationStateHub] real states: recent_rate=%.1f ev/s total_targets=%d",
# rate,
# len(self._target_data),
# )
# except Exception:
# # never allow logging to raise
# pass
self._last_real_summary_time = now
except Exception:
# Never allow diagnostic/logging instrumentation to break hub behavior
pass
# Periodic (throttled) throughput summary so the logs still
# provide visibility without the per-event flood. This uses
# monotonic time to avoid issues with system clock changes.
if (
now is not None
and (now - self._last_real_summary_time)
>= self._real_summary_interval_s
):
rate = self.get_real_rate(
window_seconds=self._real_summary_interval_s
)
# try:
# logger.info(
# "[SimulationStateHub] real states: recent_rate=%.1f ev/s total_targets=%d",
# rate,
# len(self._target_data),
# )
# except Exception:
# # never allow logging to raise
# pass
self._last_real_summary_time = now
except Exception:
# Never allow diagnostic/logging instrumentation to break hub behavior
pass
def get_real_rate(self, window_seconds: float = 1.0) -> float:
"""
@ -269,10 +203,8 @@ class SimulationStateHub:
"""
try:
now = time.monotonic()
cutoff = now - float(window_seconds)
# Count timestamps >= cutoff
cutoff = now - window_seconds
count = 0
# Iterate from the right (newest) backwards until below cutoff
for ts in reversed(self._real_event_timestamps):
if ts >= cutoff:
count += 1
@ -303,7 +235,7 @@ class SimulationStateHub:
"""
try:
now = time.monotonic()
cutoff = now - float(window_seconds)
cutoff = now - window_seconds
count = 0
for ts in reversed(self._real_packet_timestamps):
if ts >= cutoff:

17
target_simulator/utils/csv_logger.py
View File

@ -7,15 +7,30 @@ Behavior is governed by `target_simulator.config.DEBUG_CONFIG` (see keys
These functions are intended for debugging and tracing; they return ``True``
when the append operation succeeded and ``False`` when disabled or on error.
PERFORMANCE: Uses asynchronous buffering to avoid blocking the simulation
thread. Rows are buffered in memory and flushed periodically by a background
thread, eliminating I/O overhead from the critical path.
"""
import csv
import os
import time
from typing import Iterable, Any
import threading
import atexit
from typing import Iterable, Any, Dict, List, Tuple
from collections import deque
from target_simulator.config import DEBUG_CONFIG
# --- Async CSV Buffer ---
_CSV_BUFFER_LOCK = threading.Lock()
_CSV_BUFFERS: Dict[str, deque] = {} # filename -> deque of (row, headers)
_CSV_FLUSH_THREAD: threading.Thread = None
_CSV_STOP_EVENT = threading.Event()
_CSV_FLUSH_INTERVAL_S = 2.0 # Flush every 2 seconds
_CSV_MAX_BUFFER_SIZE = 1000 # Flush immediately if buffer exceeds this
def _ensure_temp_folder():
temp_folder = DEBUG_CONFIG.get("temp_folder_name", "Temp")