Data Engineering

Datasets & Telemetry Schemas

Standardizing heterogeneous UAV flight controller logs into clean, reproducible analytical formats.

Why Telemetry Normalization Matters

Unmanned aerial vehicles record sensor topics at asynchronous rates—high-frequency IMU samples (400 Hz–1 kHz), medium-rate EKF attitude estimates (50–100 Hz), and lower-rate GPS or battery telemetry (5–10 Hz). Comparing flights across airframes or autopilots requires consistent schemas, units, and monotonic timestamps.

Core Schema Conventions

  • Monotonic Timebase: Unified timestamp_us (microseconds since boot) paired with UTC epoch alignment when GPS lock is valid.
  • SI Physical Units: Meters, meters/second, radians, volts, amperes, and degrees Celsius enforced across all adapters.
  • Columnar Storage: Exportable to Apache Parquet and Arrow tables for zero-copy analytics in Python, DuckDB, and R.

Example Normalized Telemetry Record

Sample JSON representation of an aligned telemetry frame produced by the Flight Data Lab pipeline:

{
  "flight_id": "fdl-2026-uav-0104",
  "autopilot": "ardupilot-4.5",
  "timestamp_us": 184520000,
  "imu": {
    "acc_x_ms2": 0.14,
    "acc_y_ms2": -0.08,
    "acc_z_ms2": -9.81,
    "vibration_rms": 1.42
  },
  "gps": {
    "lat": 43.761539,
    "lon": -79.411079,
    "alt_rel_m": 48.6,
    "hdop": 0.88,
    "satellites": 16
  },
  "battery": {
    "voltage_v": 22.84,
    "current_a": 18.6,
    "consumed_mah": 1420,
    "remaining_pct": 71.6,
    "est_remaining_sec": 860
  },
  "anomaly_flags": []
}

Open Research Datasets Roadmap

Alongside our analysis libraries, Flight Data Lab curates anonymized, schema-validated UAV telemetry datasets to benchmark anomaly detection algorithms and battery discharge models across multi-rotor and fixed-wing unmanned platforms.

Dataset Track Primary Signals Target Use Case Format
FDL-Nominal-Flight IMU, Baro, GPS, RC Out, Battery Baseline flight profile & trajectory modeling Apache Parquet
FDL-Power-Discharge Pack Voltage, Current Draw, Temp, mAh Battery endurance & voltage sag prediction Parquet / CSV
FDL-Anomaly-Bench Labeled Vibration, GPS Drift & Sag Events Evaluating flight log anomaly detectors Parquet + JSON Labels