View a markdown version of this page

Dynamic data collection - Guidance for Connected Mobility on AWS

Dynamic data collection

The solution uses a three-layer data collection model that decouples what signals exist from how they are encoded on the wire from which signals to collect right now. This separation allows fleet operators to change collection behavior without modifying vehicle software or redeploying infrastructure.

Signal catalog

The signal catalog is the single source of truth for every telemetry signal the platform understands. It is stored in the cms-{stage}-signal-catalog DynamoDB table and seeded during deployment with a comprehensive catalog of 260 CAN bus signals across 55 CAN messages, plus 2 GPS signals and 6 metadata fields. The catalog covers a realistic 2026 model-year vehicle with ICE, EV, and hybrid support. It includes 48 actuatable commands for remote vehicle control. The default catalog is a starting point — it is fully customizable to any vehicle platform, OEM signal set, or industry standard.

Customizing the signal catalog

The signal catalog is not hardcoded. Customers replace or extend it to match their vehicle fleet:

  • OEM-specific signals — Add proprietary signals from your vehicle platform (for example, Toyota-specific hybrid signals, Tesla battery cell voltages, or commercial vehicle J1939 signals).

  • Subset deployments — Remove signals your fleet does not produce to reduce processing overhead.

  • Different units — Change from imperial (mph, °F, PSI) to metric (km/h, °C, kPa) by updating the unit and conversion factor.

  • New signal groups — Add entirely new groups (for example, autonomous driving signals, trailer telemetry, cargo monitoring).

When the signal catalog is updated, the SignalCatalogLoader detects the version change in Redis and hot-reloads across all Flink processors without restart. Downstream consumers (Fleet Manager UI, Commands Lambda, FWTelemetryProcessor) automatically pick up the new signals.

Signal entry structure

Each signal in the catalog contains:

Field Description

json_field

The key used in standard telemetry JSON (for example, speed)

signal_name

Human-readable name (for example, "Vehicle Speed")

vss_path

COVESA Vehicle Signal Specification fully qualified path (for example, Vehicle.Speed)

signal_group

Logical grouping: engine, transmission, body, tire, safety, battery, climate, maintenance

data_type

float, int, boolean, or string

unit

Physical unit (mph, PSI, °F, rpm, etc.)

min / max

Valid range for the signal

actuator

If present, this signal can be commanded remotely (see Remote commands)

Default signal catalog

The solution ships with a comprehensive fleet telemetry catalog covering ICE, EV, and hybrid vehicles. The 260 signals are organized into two tiers:

  • Original signals (IDs 1–75) — 15 CAN messages (0x100–0x190) covering core engine, transmission, body, tire, safety, battery, instrument, climate, maintenance, and lighting signals.

  • Expanded signals (IDs 101–287) — 40 CAN messages (0x1A0–0x1C7) covering ADAS, cabin climate, connectivity, doors, environment, EV charging, geofence/fleet management, lighting, maintenance, mirrors, powertrain, safety, security, TPMS, vehicle control, windows, and wipers.

All IDs are unique and aligned across the signal catalog, decoder manifest, protobuf encoding, and campaign definitions.

Original core signals (15 messages, 75 signals):

Message CAN ID Signals Coverage

ECM_Engine_1

0x100

4

Speed, RPM, engine temp, ignition

ECM_Engine_2

0x101

6

Oil pressure, throttle, coolant, intake air, engine hours

ECM_Engine_3

0x102

4

Acceleration, deceleration, odometer, fuel rate

TCM_Transmission

0x110

4

Transmission temp, gear, cruise, parking brake

BCM_Body_1

0x120

12

Seatbelt, phone, windows, trunk, lights, connectivity

TPMS_Pressure

0x130

5

Tire pressure (4 wheels) + max tire temp

TPMS_Tread

0x131

4

Tire tread depth (4 wheels)

ADAS_Safety_1

0x140

10

Braking, acceleration, turn, AEB, ABS, ESC, airbag

ADAS_Safety_2

0x141

4

Phone usage, seatbelt, lateral G, following distance

BMS_Battery

0x150

3

12V battery, alternator, fuel level

BMS_EV

0x151

4

EV: SOC, HV voltage, regen power

ICM_Instrument

0x160

2

Heading, DTC active

HVAC_Climate

0x170

4

HVAC, target temp, cabin temp, seat heat

MAINT_Indicators

0x180

6

Oil life, brake wear, filter life, idle hours, brake pressure

LIGHT_Systems

0x190

3

Headlights, hazard, turn signal

Expanded signals (40 messages, 185 signals):

Message Group CAN IDs Signals Coverage

ADAS / ADAS_1 / ADAS_2

0x1A0–0x1A2

18

Cruise control, adaptive cruise, lane keep, blind spot, AEB, forward collision, parking assist, traffic sign recognition, driver monitoring, auto park

CABIN_CLIMATE / _1 / _2

0x1A3–0x1A5

13

HVAC mode, dual-zone temp, fan speed, recirculation, defrost, seat heat/vent (driver + passenger), steering wheel heat, remote preconditioning

CONNECTIVITY / _1

0x1A6–0x1A7

8

Cellular signal/network, WiFi, Bluetooth, OTA updates, software version

CORE_TELEMETRY / _1 / _2

0x1A8–0x1AA

10

Extended core vehicle telemetry

DOORS

0x1AB

18

Per-door open/locked/child-lock (4 doors), all-doors lock, trunk, hood, fuel door, charge door

ENVIRONMENT / _1

0x1AC–0x1AD

5

Ambient temp, humidity, barometric pressure, rain intensity, light level

EV_CHARGING / _1–_4

0x1AE–0x1B2

19

SOC, SOH, HV voltage/current, battery temp, range, charge rate/type/limit, time-to-full, scheduled charging, regen level, motor torque/RPM/temp, energy consumed

EV_SPECIFIC / _1

0x1B3–0x1B4

5

Extended EV-specific signals

GEOFENCE / _1

0x1B5–0x1B6

14

Geofence active/center/radius/violation, speed limit, curfew, valet mode, immobilizer

LIGHTING

0x1B7

6

Headlight mode, high beam, fog lights (front/rear), hazard, interior lights, ambient color

MAINTENANCE / _1

0x1B8–0x1B9

8

Extended maintenance indicators

MIRRORS

0x1BA

5

Per-mirror fold/heat (left/right), fold-all

POWERTRAIN / _1 / _2

0x1BB–0x1BD

10

Transmission gear/mode/temp, turbo boost, throttle, intake/exhaust/catalyst temp, fuel pressure/type, remote start

SAFETY / _1

0x1BE–0x1BF

10

Extended safety signals

SECURITY

0x1C0

3

Horn, alarm armed/triggered, panic mode, find-my-vehicle

TPMS / _1

0x1C1–0x1C2

8

Per-wheel pressure + temperature (expanded)

VEHICLE_CONTROL / _1

0x1C3–0x1C4

15

Vehicle control signals

WINDOWS / _1

0x1C5–0x1C6

6

Per-window position (4 windows), sunroof position/shade

WIPERS

0x1C7

4

Front wiper mode/active, washer fluid level, rear wiper

Actuatable commands (48 commands across 6 categories):

Category Count Examples

Comfort

21

HVAC mode/temp/fan, seat heat/vent, steering wheel heat, window positions, sunroof, mirror fold/heat, remote preconditioning

Security

11

Per-door locks, all-doors lock, trunk lock, alarm arm/disarm, horn, panic mode, find-my-vehicle, immobilizer

Fleet

9

Geofence enable/center/radius, speed limit, curfew enable/start/end, valet mode

Lighting

5

Headlights, fog lights (front/rear), hazard flash, interior lights

Charging

4

Charge limit, charge schedule, start/stop charging, charge door

Engine

1

Remote start/stop

VSS path convention

The signal catalog follows the COVESA Vehicle Signal Specification (VSS) naming convention. VSS provides a standardized, tree-structured taxonomy for vehicle signals used across the automotive industry. The fully qualified path uses dot notation from the vehicle root:

Vehicle.Chassis.Axle.Row1.Wheel.Left.Tire.Pressure │ │ │ │ │ │ │ └── Signal name │ │ │ │ │ │ └── Component │ │ │ │ │ └── Position │ │ │ │ └── Assembly │ │ │ └── Location │ │ └── System │ └── Domain └── Root

This hierarchical naming enables:

  • Consistent signal identification across OEMs and platforms

  • Automatic grouping in the Fleet Manager UI

  • Compatibility with FleetWise signal catalogs and decoder manifests

  • Industry-standard data exchange with partners and third-party systems

DBC file format

The solution includes a DBC file (services/simulation/can/cms-fleet.dbc) that defines the CAN bus message layout. DBC (Database CAN) is the industry-standard format for describing CAN bus communication, used by every major automotive OEM and tool vendor.

How a DBC file works

A DBC file defines messages (CAN frames) and signals (data fields within frames):

BO_ 256 ECM_Engine_1: 8 Vector__XXX SG_ VehicleSpeed : 0|16@1+ (0.1,0) [0|655] "mph" Vector__XXX SG_ EngineRPM : 16|16@1+ (1,0) [0|8000] "rpm" Vector__XXX SG_ EngineTemp : 32|16@1+ (0.1,-40) [0|370] "degF" Vector__XXX SG_ IgnitionOn : 48|1@1+ (1,0) [0|1] "" Vector__XXX

Reading this:

  • BO_ 256 ECM_Engine_1: 8 — CAN message with ID 256 (0x100), named ECM_Engine_1, 8 bytes long

  • SG_ VehicleSpeed : 0|16@1+ — Signal starting at bit 0, 16 bits wide, little-endian, unsigned

  • (0.1,0) — Factor 0.1, offset 0. Physical value = raw × 0.1 + 0

  • [0|655] — Valid range 0 to 655 mph

  • "mph" — Unit

CAN message layout

The default DBC defines 55 CAN messages with different cycle times based on how frequently the data changes:

Original messages (0x100–0x190):

Message CAN ID Cycle Signals Purpose

ECM_Engine_1

0x100

100ms

4

Speed, RPM, engine temp, ignition

ECM_Engine_2

0x101

500ms

6

Oil pressure, throttle, coolant, intake air

ECM_Engine_3

0x102

100ms

4

Acceleration, deceleration, odometer, fuel rate

TCM_Transmission

0x110

200ms

4

Transmission temp, gear, cruise, parking brake

BCM_Body_1

0x120

500ms

12

Seatbelt, phone, windows, trunk, lights, connectivity

TPMS_Pressure

0x130

1000ms

5

Tire pressure (4 wheels) + max tire temp

TPMS_Tread

0x131

5000ms

4

Tire tread depth (4 wheels)

ADAS_Safety_1

0x140

100ms

10

Braking, acceleration, turn, AEB, ABS, ESC

ADAS_Safety_2

0x141

100ms

4

Phone usage, seatbelt, lateral G, following distance

BMS_Battery

0x150

1000ms

3

12V battery, alternator, fuel level

BMS_EV

0x151

1000ms

4

EV: SOC, HV voltage, regen power

ICM_Instrument

0x160

200ms

2

Heading, DTC active

HVAC_Climate

0x170

2000ms

4

HVAC, target temp, cabin temp, seat heat

MAINT_Indicators

0x180

5000ms

6

Oil life, brake wear, filter life, idle hours

LIGHT_Systems

0x190

500ms

3

Headlights, hazard, turn signal

Expanded messages (0x1A0–0x1C7):

Message Group CAN IDs Signals Purpose

ADAS (3 messages)

0x1A0–0x1A2

18

Cruise control, adaptive cruise, lane keep, blind spot, AEB, collision warning, parking assist, traffic sign recognition, driver monitoring

CABIN_CLIMATE (3 messages)

0x1A3–0x1A5

13

Dual-zone HVAC, fan speed, defrost, seat heat/vent, steering wheel heat, remote preconditioning

CONNECTIVITY (2 messages)

0x1A6–0x1A7

8

Cellular signal, WiFi, Bluetooth, OTA updates, software version

CORE_TELEMETRY (3 messages)

0x1A8–0x1AA

10

Extended core telemetry

DOORS (1 message)

0x1AB

18

Per-door open/locked/child-lock, trunk, hood, fuel door, charge door

ENVIRONMENT (2 messages)

0x1AC–0x1AD

5

Ambient temp, humidity, barometric pressure, rain, light level

EV_CHARGING (5 messages)

0x1AE–0x1B2

19

Battery SOC/SOH/voltage/current/temp, range, charge rate/type/limit, motor torque/RPM/temp

EV_SPECIFIC (2 messages)

0x1B3–0x1B4

5

Extended EV signals

GEOFENCE (2 messages)

0x1B5–0x1B6

14

Geofence, speed limit, curfew, valet mode, immobilizer

LIGHTING (1 message)

0x1B7

6

Headlights, high beam, fog lights, interior, ambient

MAINTENANCE (2 messages)

0x1B8–0x1B9

8

Extended maintenance indicators

MIRRORS (1 message)

0x1BA

5

Per-mirror fold/heat, fold-all

POWERTRAIN (3 messages)

0x1BB–0x1BD

10

Transmission, turbo, throttle, exhaust/catalyst temp, fuel pressure, remote start

SAFETY (2 messages)

0x1BE–0x1BF

10

Extended safety signals

SECURITY (1 message)

0x1C0

3

Horn, alarm, panic mode

TPMS (2 messages)

0x1C1–0x1C2

8

Per-wheel pressure + temperature (expanded)

VEHICLE_CONTROL (2 messages)

0x1C3–0x1C4

15

Vehicle control signals

WINDOWS (2 messages)

0x1C5–0x1C6

6

Per-window position, sunroof

WIPERS (1 message)

0x1C7

4

Front/rear wipers, washer fluid level

Safety-critical signals (speed, braking, acceleration) use 100ms cycle times for rapid detection. Slow-changing signals (tire tread, maintenance indicators) use 5000ms cycles to reduce bus load.

Customizing the DBC

To adapt the guidance for a different vehicle platform:

  1. Create or obtain the DBC file for your vehicle’s CAN bus.

  2. Update the signal catalog DynamoDB table to match the new signals.

  3. Update the decoder manifest to map the new CAN signal IDs to VSS paths.

  4. Create campaigns that reference the new signal IDs.

The Flink processors, Fleet Manager UI, and Redis LKS layer automatically adapt to the new signals through the signal catalog — no code changes required.

Decoder manifest

The decoder manifest bridges the gap between raw CAN bus data and human-readable signal names. It tells the FleetWise Edge Agent how to interpret the binary CAN frames coming off the vehicle bus. The default decoder manifest covers all 260 CAN signals plus 2 GPS custom decoding signals.

Each entry in the decoder manifest maps:

  • CAN message ID — The CAN frame identifier (for example, 0x100 for ECM_Engine_1)

  • Signal bit position and length — Where the signal sits within the 8-byte CAN frame

  • Factor and offset — Scaling values to convert raw integer values to physical units (for example, raw value 655 × factor 0.1 = 65.5 mph)

  • Signal ID — A numeric identifier used in the protobuf encoding

  • VSS signal name — The human-readable name (for example, Vehicle.Speed)

The decoder manifest is defined in a DBC file (services/simulation/can/cms-fleet.dbc) that follows the standard automotive DBC format. The CampaignSyncProcessor reads the manifest from DynamoDB and serializes it as a protobuf DecoderManifest message for delivery to the FWE agent.

Example — how a single signal flows through the decoder manifest:

CAN bus frame 0x100, bytes [0x02, 0x8F, ...] → DBC: signal "VehicleSpeed" at bit 0, length 16, factor 0.1, offset 0 → Raw value: 0x028F = 655 → Physical value: 655 × 0.1 = 65.5 → VSS name: Vehicle.Speed → standard JSON field: "speed": 65.5

The decoder manifest covers 15 CAN messages with cycle times ranging from 100ms (engine, safety) to 5000ms (tire tread, maintenance indicators). See Signal catalog reference for the complete mapping.

Campaigns

A campaign defines what to collect and from which vehicles. Campaigns are the control plane for FleetWise Edge data collection — they determine which signals the agent actively monitors and uploads.

Each campaign record in the cms-{stage}-campaigns DynamoDB table contains:

  • Campaign ID — Unique identifier

  • Campaign name — Human-readable name (for example, "Full Fleet Telemetry")

  • StatusRUNNING, SUSPENDED, or COMPLETED

  • Target vehicles — List of VINs or a target ARN pattern

  • Source fleet ID — If assigned via fleet-level campaign, the sourceFleetId links to the parent fleet campaign. Fleet-assigned campaigns are locked at the vehicle level.

  • Signals to collect — List of signal IDs from the decoder manifest (for example, all 66 signals, or a subset like only tire and engine signals)

  • Collection interval — How frequently the agent samples signals (for example, every 1000ms)

  • Condition — Optional trigger condition (for example, collect only when speed > 0)

Campaign lifecycle

  1. Create — A campaign is created in DynamoDB with status RUNNING and a list of target vehicles and signals.

  2. Push — On the next agent checkin, the CampaignSyncProcessor detects the new campaign, generates a CollectionScheme protobuf, and publishes it to the agent via IoT Core MQTT.

  3. Collect — The agent configures its CAN bus listeners based on the collection scheme and begins sampling the specified signals.

  4. Upload — The agent batches collected signals into VehicleData protobuf messages and uploads them to IoT Core.

  5. Suspend — Setting the campaign status to SUSPENDED causes the processor to push an empty collection scheme on the next checkin, which stops the agent from collecting those signals.

  6. Resume — Setting the status back to RUNNING re-enables collection on the next checkin.

  7. Complete — Setting the status to COMPLETED permanently ends the campaign.

Example campaigns

Full fleet telemetry — Collect all 66 signals from every vehicle at 1-second intervals. This is the default campaign created by the simulation service.

Tire health monitoring — Collect only tire pressure (4 signals) and tire tread (4 signals) from vehicles in a specific fleet at 5-second intervals. Useful for reducing data volume while monitoring a specific concern.

Safety investigation — Collect all ADAS safety signals (10 signals) from a single vehicle at 100ms intervals after a safety incident is reported. High-frequency collection for detailed analysis.

EV battery diagnostics — Collect state of charge, HV battery voltage, regenerative braking power, and motor temperature from EV vehicles only. Uses a condition to collect only when the vehicle is driving (speed > 0).

How the layers connect

Signal Catalog (what signals exist) ↓ defines Decoder Manifest (how to read signals from CAN bus) ↓ referenced by Campaign (which signals to collect, from which vehicles) ↓ pushed as protobuf to FWE Agent (collects and uploads) ↓ decoded by FWTelemetryProcessor (protobuf → standard JSON) ↓ flows into Standard processing pipeline (trips, safety, maintenance)

This layered design means:

  • Adding a new signal requires only a signal catalog update and a decoder manifest entry — no code changes.

  • Changing what a vehicle collects requires only a campaign update — no vehicle software update.

  • Multiple campaigns can target the same vehicle simultaneously — the agent merges the collection schemes.

  • The same downstream processors work regardless of whether data came from MQTT Direct or FleetWise Edge.