Technical Solution
Primary Innovation
arktrace is a Causal Inference Engine for Shadow Fleet Prediction. The primary technical contribution is the C3 Causal Sanction-Response model (src/score/causal_sanction.py) and the unknown-unknown detector (src/analysis/causal.py).
What this is not: real-time vessel monitoring, anomaly detection on raw AIS data, conventional sanctions screening, or off-the-shelf behavioural analytics.
What this is: A Difference-in-Differences (DiD) framework that tests, for each vessel, whether behavioural change was causally triggered by a specific sanction announcement — using HC3-robust OLS to distinguish genuine evasion responses from normal commercial variation. Vessels that pass the causal filter are then propagated through the ownership graph to surface connected unknown-unknown threats before any designation occurs.
| Model component | Implementation | Role |
|---|---|---|
| C3 DiD causal model | src/score/causal_sanction.py |
Tests causal response to sanction events; primary innovation |
| Unknown-unknown detector | src/analysis/causal.py |
Surfaces non-sanctioned vessels with evasion-consistent causal signatures |
| Backtracking propagation | scripts/run_backtracking.py |
Graph-walks ownership network from confirmed evaders to predict next designations |
| AIS behaviour signals | src/features/ |
Input substrate for the causal model; not the primary discriminator |
| Sanctions screening | src/data/sanctions.py |
Input substrate; frames the event timeline for DiD windows |
The 60–90 day pre-designation lead time (backtested — see docs/scoring-model.md) is a direct result of the causal model detecting evasion responses before those vessels accumulate enough evidence for a formal OFAC designation.
Challenge Alignment — Shadow Fleet Behaviours
Cap Vista Solicitation 5.0, Challenge 1 names three specific shadow fleet behaviours:
"sophisticated AIS spoofing, frequent name/flag changes, and illicit ship-to-ship (STS) transfers to bypass international sanctions"
arktrace maps directly to each:
| Challenge behaviour | arktrace feature(s) | Implementation |
|---|---|---|
| AIS spoofing | position_jump_count (implied speed > 50 kts indicates GPS broadcast spoofing) |
src/features/movement.py |
| AIS dark periods | ais_gap_count_30d, ais_gap_max_hours |
src/features/movement.py |
| Frequent name changes | name_changes_2y |
src/features/identity.py |
| Frequent flag changes | flag_changes_2y, high_risk_flag_ratio |
src/features/identity.py |
| Ship-to-ship (STS) transfers | sts_candidate_count, sts_hub_degree |
src/features/sts.py |
These five features are the evidentiary substrate that feeds the C3 DiD causal model — not the final output claim. The model tests whether AIS gap counts, identity churn, and STS event frequency increased causally in response to a specific sanction announcement, separating deliberate evasion from ordinary commercial variation.
Geographic Scope
The challenge specifies "major shipping lanes up to 1,600 nm from Singapore to water depth of 200 m below mean sea level." arktrace's default Singapore / Malacca Strait bounding box (−5°N 92°E → 22°N 122°E) covers this area. The 200 m bathymetric depth mask (GEBCO) is applied during STS candidate detection to exclude deep-ocean false positives — confirmed shallow-water STS sites are the operationally relevant targets. See docs/regional-playbooks.md for bbox details.
Tech Stack
| Layer | Tool | Version | Rationale |
|---|---|---|---|
| Analytical store | DuckDB | ≥ 1.1 | In-process columnar OLAP; queries Parquet natively; no server; edge-deployable |
| DataFrame / feature engineering | Polars | ≥ 1.0 | Lazy evaluation; fast AIS window operations; Arrow-native |
| Graph DB | Lance Graph | ≥ 0.5 | Embedded in-process graph engine; stores ownership graph as Lance columnar files; local path or S3-compatible (s3://) |
| Object store | MinIO | RELEASE.2025-09-07 | S3-compatible local object store; persists Parquet and Lance datasets; port 9000 (API) / 9001 (console) |
| ML / clustering | scikit-learn | ≥ 1.5 | HDBSCAN, Isolation Forest; no GPU required |
| Explainability | SHAP | ≥ 0.46 | TreeExplainer for Isolation Forest; per-vessel feature attribution |
| Dashboard | FastAPI + HTMX | ≥ 0.115 / — | Production-grade API layer + partial-page updates; SSE alerts; MapLibre GL JS |
| Local LLM | Ollama / MLX / LM Studio | — | Local inference for analyst briefs and chat; no cloud dependency; provider selected via LLM_PROVIDER env var |
| AIS streaming | websockets + httpx | — | aisstream.io WebSocket; Marine Cadastre HTTP download |
| Causal inference | numpy / scipy (built-in) | — | DiD OLS with HC3 robust SEs; no external causal library required |
| Language | Python 3.12 | — | Best ecosystem fit for all above |
| Packaging | uv | — | Fast lockfile-based dependency management |
Local-First Deployment
The full stack runs on a single machine — a field laptop, a shipboard server, or a detached tactical edge node — with no cloud dependency during operation. All data remains on-device. Cloud connectivity is optional and used only for upstream AIS streaming or report export.
┌─────────────────────────────────────────────────────────────────┐
│ TACTICAL EDGE NODE (patrol vessel / field laptop) │
│ │
│ AIS stream (aisstream.io WebSocket) │
│ SAR / EO imagery (offline batch or USB import) │
│ │ │
│ ▼ │
│ Scoring Engine (DuckDB + Polars + HDBSCAN + Isolation Forest) │
│ │ │ │
│ │ Lance Graph (ownership network) │
│ │ │ │
│ ▼ ▼ │
│ HTMX Dashboard ←───── Composite Score + SHAP signals │
│ │ │
│ ↕ (context window injection — no external calls) │
│ │ │
│ Ollama / MLX LLM → Analyst brief / chat response │
│ │ │
│ ▼ │
│ Duty Officer │
│ │
│ Persistence: MinIO (localhost:9000) — Parquet + Lance datasets │
└─────────────────────────────────────────────────────────────────┘
│ optional: report upload / AIS backfill
▼
Cloud / HQ network
| Component | Local runtime | Default persistence path | S3 path (when S3_BUCKET set) |
|---|---|---|---|
| OLAP store | DuckDB in-process | data/processed/mpol.duckdb |
— (DuckDB file stays local; Parquet outputs go to S3) |
| Parquet outputs | Polars → local or S3 | data/processed/*.parquet |
s3://arktrace/processed/*.parquet |
| Ownership graph | Lance embedded | data/processed/mpol_graph/ |
s3://arktrace/mpol_graph/ |
| Geopolitical index | Lance embedded | data/processed/gdelt.lance |
s3://arktrace/gdelt.lance |
| Object store | MinIO localhost:9000 |
minio_data Docker volume |
— (MinIO is the S3 backend) |
| Web app | FastAPI localhost:8000 |
— | — |
| LLM inference | Ollama localhost:11434 or MLX in-process |
— | — |
Storage backend selection is automatic: when S3_BUCKET is set in the environment, src/storage/config.py routes all Parquet and Lance I/O to s3://<bucket>/… via MinIO (or any S3-compatible store). When unset, everything writes to local data/processed/ paths. No code changes are required to switch between the two modes.
Data Sources
AIS Data
| Source | Coverage | Format | Cost |
|---|---|---|---|
| aisstream.io | Real-time global AIS WebSocket | JSON over WS | Free (API key) |
| Marine Cadastre | Historical US waters AIS, 2015–present | CSV / Parquet | Free download |
| AIS Hub | Near-real-time aggregated AIS | NMEA / JSON | Free tier available |
aisstream.io supports all regions via the --bbox lat_min lon_min lat_max lon_max flag. The default bbox is the Singapore / Malacca Strait (−5 92 22 122). For other regions, pass --bbox with the appropriate coordinates and --db to write to a region-specific DuckDB file. Marine Cadastre is used only for US coastal regions (Gulf of Mexico, US West Coast). For non-US historical backfill (Japan Sea, Europe, Middle East), use AISHub or MarineTraffic CSV exports loaded via load_csv_to_duckdb() with a custom bbox. See regional-playbooks.md for per-region configuration.
Sanctions & Registry Data
| Source | Data | Format | Cost |
|---|---|---|---|
| OFAC SDN | US sanctions: vessels, companies, individuals | XML | Free |
| EU Financial Sanctions | EU consolidated sanctions list | XML / CSV | Free |
| UN Consolidated List | UN Security Council sanctions | XML | Free |
| OpenSanctions | Merged sanctions + PEP dataset | JSON / Parquet | Free (CC0) |
| Equasis | Vessel ownership, flag, class history | Web (scraper) | Free (registration) |
| ITU MMSI database | MMSI → vessel mapping | CSV download | Free |
Trade Flow Data
| Source | Data | Format | Cost |
|---|---|---|---|
| UN Comtrade+ | Bilateral trade by HS code, port, period | REST API → JSON | Free (500 req/day) |
Geospatial Reference Data
| Source | Data | Format | Cost |
|---|---|---|---|
| GEBCO | Global bathymetric grid (water depth) | NetCDF / GeoTIFF | Free download |
GEBCO is used to build a 200m-depth boundary mask as an H3 hexagon set. STS candidate detection filters to events within this mask (shallow draught tankers cannot operate in deeper open ocean), reducing false positives from legitimate vessel interactions.
Geopolitical Event Data
| Source | Data | Format | Cost |
|---|---|---|---|
| GDELT Project | Global news events: sanctions, conflicts, corporate actions | CSV (daily) | Free |
GDELT event records (EventCode, Actor1, Actor2, GoldsteinScale) are ingested as a time-series alongside AIS data. The primary use is correlating sanction announcement dates with AIS gap spikes in the area of interest — providing geopolitical context for anomaly scoring rather than acting as a primary detection signal.
Key Algorithms
AIS Gap Detection (Polars)
# Identify gaps > 6h per MMSI, sorted by timestamp
df.sort(["mmsi", "timestamp"]) \
.with_columns(
pl.col("timestamp").diff().over("mmsi").alias("gap")
) \
.filter(pl.col("gap") > pl.duration(hours=6))
Gaps are then aggregated per MMSI over a rolling 30-day window.
Position Jump Detection (Polars)
Consecutive AIS positions are checked for implied speed:
implied_speed = haversine(pos_t, pos_{t+1}) / delta_t
Values > 50 knots between two non-gap positions indicate spoofed coordinates.
STS Candidate Detection
Two-vessel co-location is detected by:
1. Spatial join: pairs of vessels within 0.5nm at the same timestamp
2. Filter: both vessels have nav_status ∈ {drifting, at anchor} AND position is > 5nm from any port
3. Duration filter: co-location persists > 2 hours
Implemented as a DuckDB spatial query (using h3 or ST_Distance on lat/lon).
Ownership Graph (Lance Graph + Polars)
Vessel ownership chains are stored as Lance columnar datasets on disk (no external server). Graph features are computed by Polars joins over these datasets in src/features/ownership_graph.py and src/features/identity.py.
# BFS shortest path from vessel to nearest sanctioned entity
# 0 = directly sanctioned, 1 = 1-hop owner/manager, 2 = 2-hop via CONTROLLED_BY, 99 = none
tables = load_tables(db_path)
vessel_companies = pl.concat([OWNED_BY, MANAGED_BY]).unique()
one_hop = vessel_companies.filter(pl.col("dst_id").is_in(sanctioned_ids))["src_id"]
Cluster sanctions ratio is computed by self-joining the OWNED_BY dataset on company_id; cluster_sanctions_ratio is the fraction of co-owned vessels that are directly sanctioned.
# Hub vessel detection: STS contact degree
sts_hub = STS_CONTACT.group_by("src_id").agg(
pl.col("dst_id").n_unique().alias("sts_hub_degree")
)
# Shared-address clustering
vessel_address = vessel_company.join(REGISTERED_AT, on="company")
shared = vessel_address.join(vessel_address, on="address") \
.filter(pl.col("vessel") != pl.col("peer")) \
.group_by("vessel").agg(pl.col("peer").n_unique())
HDBSCAN Normal Behavior Baseline
HDBSCAN clusters vessels by their behavioral feature vector (gap frequency, speed variance, route entropy, loitering ratio), stratified by ship_type. Clusters with high internal consistency represent well-understood normal MPOL patterns (e.g. regular container feeders on fixed schedules). Vessels assigned to noise (cluster = -1) receive a baseline anomaly weight of 1.0.
Isolation Forest Scoring
Trained on the subset of vessels with sanctions_distance ≥ 3 (proxy for "clean"). The decision function is calibrated to [0,1] using a sigmoid fit against the OFAC-listed vessel validation set.
C3 · Causal Sanction-Response Model (DiD)
Implemented in src/score/causal_sanction.py. Quantifies the causal effect of sanction announcement events on AIS gap frequency for vessels connected within 2 hops in the Lance Graph ownership graph.
Model specification (for each regime × announcement date):
outcome_{it} = β₀ + β₁·treated_i + β₂·post_t + β₃·(treated_i × post_t)
+ γ_v (vessel-type fixed effects)
+ δ_r (route-corridor fixed effects)
+ ε_{it}
| Term | Meaning |
|---|---|
treated_i |
1 if vessel has sanctions_distance ≤ 2 |
post_t |
1 if observation is in the 30-day window after the announcement date |
| β₃ (ATT) | Average Treatment Effect on Treated: extra AIS gaps per 30 days attributable to the announcement |
vessel-type FEs |
One dummy per AIS ship_type bucket (tanker, cargo, passenger, other) |
route-corridor FEs |
One dummy per geographic corridor (Malacca, Persian Gulf, Red Sea, North Sea, …) |
OLS is estimated with HC3 heteroskedasticity-robust standard errors (implemented in pure numpy—no statsmodels dependency). Multiple announcement dates per regime are pooled via inverse-variance weighting.
Output: Per-regime ATT estimate + 95% CI. calibrate_graph_weight(effects) converts the fraction of positive-significant estimates into a w_graph value ∈ [0.20, 0.65] suitable for --w-graph in src/score/composite.py.
Supported regimes:
| Regime key | Label | Announcement dates |
|---|---|---|
OFAC_Iran |
OFAC Iran | 2012-03-15, 2019-05-08, 2020-01-10 |
OFAC_Russia |
OFAC Russia | 2022-02-24, 2022-09-15, 2023-02-24 |
UN_DPRK |
UN DPRK | 2017-08-05, 2017-09-11, 2017-12-22 |
Verifiable AI & Anti-Hallucination Grounding Pipeline
Defense and intelligence stakeholders require that AI-generated assessments be auditable, reproducible, and traceable to primary evidence — not black-box text. Arktrace addresses this through a strict two-phase architecture: all risk decisions are made by deterministic algorithms first; the LLM is only permitted to synthesise text from a pre-computed, structured context window.
Two-phase architecture
Phase 1 — Deterministic scoring (no LLM)
─────────────────────────────────────────────────────────────────────
AIS + registry + trade data
│
├─► HDBSCAN MPOL baseline → cluster_id, baseline_noise_score
├─► Isolation Forest → anomaly_score (sigmoid-calibrated)
├─► SHAP TreeExplainer → top_signals[] (feature, value, contribution)
├─► Lance Graph BFS → sanctions_distance (0 = direct, 99 = none)
└─► C3 Causal DiD (β₃ ATT) → causal_weight, att_estimate, p_value, 95% CI
│
▼
Composite score (weighted blend) → candidate_watchlist.parquet
│
▼ structured context injected into prompt
Phase 2 — LLM text synthesis (bounded)
─────────────────────────────────────────────────────────────────────
System prompt: vessel profile + SHAP signals + GDELT events + ATT evidence
│
▼
LLM role: one-paragraph brief, citing specific field values and event dates
│
▼
Analyst brief / chat response → cached in DuckDB (deterministic replay)
Grounding mechanisms
SHAP attribution prevents unsupported claims.
Every feature contribution is computed by shap.TreeExplainer against the calibrated Isolation Forest. The top_signals field written to the watchlist Parquet carries the raw feature value alongside its SHAP contribution score. The LLM system prompt receives these as structured triples (feature, value, contribution) — not prose — so every claim in the generated brief can be traced back to a specific observable AIS or registry measurement.
[
{"feature": "ais_gap_count_30d", "value": 14, "contribution": 0.34},
{"feature": "sanctions_distance", "value": 1, "contribution": 0.29},
{"feature": "flag_changes_2y", "value": 3, "contribution": 0.18}
]
C3 Causal ATT provides statistically verifiable claims.
The DiD model produces a point estimate (β₃ ATT) with 95% confidence intervals and a two-tailed p-value for each sanctions regime. These are injected verbatim into the brief system prompt as causal_evidence. The LLM can cite that "AIS gaps increased by X gaps/30-day window (ATT=X.X, 95% CI [X.X, X.X], p<0.05) following the OFAC Iran announcement of 2019-05-08" — a claim that is reproducible from the raw AIS data by any analyst running src/score/causal_sanction.py.
Lance Graph distance makes ownership chains auditable.
sanctions_distance is a BFS hop count through the Lance Graph ownership tables (OWNED_BY, MANAGED_BY, CONTROLLED_BY). A value of 1 means the vessel's direct registered owner appears in the OFAC SDN or EU/UN sanctions list. The path is materialized as rows in the Lance columnar store and can be queried directly — it is not an inference.
The LLM system prompt enforces citation and prohibits fabrication.
Both the brief endpoint (src/api/routes/briefs.py) and the analyst chat endpoint (src/api/routes/chat.py) inject the same structured context and instruct the LLM explicitly:
"Cite specific field values, GDELT event IDs/dates, or ownership chain hops to ground every claim."
The LLM has no access to external tools, no internet, and no retrieval beyond what is injected in the context window. It cannot add vessels to the watchlist, change scores, or assert risk signals not already in the structured inputs.
GDELT RAG anchors geopolitical context to dated, sourced news events.
Geopolitical context is retrieved from a Lance columnar GDELT index by flag country and vessel name (src/ingest/gdelt.py). Each event record carries event_date, actor1_name, actor2_name, action_geo, and source_url. The LLM is required to name the date and URL when citing geopolitical context — preventing the substitution of plausible-sounding but unfounded geopolitical narrative.
DuckDB cache enforces deterministic replay.
Completed briefs are written to the analyst_briefs table keyed on (mmsi, watchlist_version). Identical inputs always return the same cached output. This means a brief generated before a regulatory review can be reproduced verbatim from the same pipeline run, supporting chain-of-custody requirements.
What the LLM cannot do
| Prohibited action | Enforcement mechanism |
|---|---|
Assert a risk signal not in top_signals |
Structured prompt — only listed signals are present in context |
| Claim a causal relationship without statistical support | ATT + CI + p-value required in prompt; LLM instructed to cite them |
| Modify a vessel's confidence score | Score is computed deterministically before LLM is called; LLM receives it read-only |
| Retrieve information from the internet | No tool access; local-first LLM (llamacpp / Ollama / LM Studio) or API with no browsing |
| Produce a different answer for the same inputs | DuckDB cache enforces identical output for identical (mmsi, watchlist_version) |
Summary
Arktrace uses the LLM for one task only: converting a deterministic, structured risk assessment into readable English for the analyst. Every claim in a generated brief has a traceable origin — a SHAP contribution, a graph hop count, a DiD ATT estimate, or a dated GDELT source URL. The scoring pipeline can be re-run independently of the LLM and will produce identical numeric outputs, satisfying audit and chain-of-custody requirements for defence and regulatory applications.
Output Schema
data/processed/candidate_watchlist.parquet
| Column | Type | Description |
|---|---|---|
mmsi |
str |
MMSI number |
imo |
str |
IMO number (if known) |
vessel_name |
str |
Current name |
vessel_type |
str |
Ship type |
flag |
str |
Current flag state |
confidence |
f32 |
Composite score 0.0–1.0 |
anomaly_score |
f32 |
Isolation Forest score |
graph_risk_score |
f32 |
Normalised sanctions graph distance |
identity_score |
f32 |
Identity volatility score |
top_signals |
str (JSON) |
Top 3 SHAP-attributed features |
last_lat |
f64 |
Last known latitude |
last_lon |
f64 |
Last known longitude |
last_seen |
datetime |
Last AIS timestamp |
ais_gap_count_30d |
i32 |
AIS gaps > 6h in last 30 days |
ais_gap_max_hours |
f32 |
Longest gap in hours |
position_jump_count |
i32 |
Spoofing indicators |
sts_candidate_count |
i32 |
Co-location events |
flag_changes_2y |
i32 |
Flag changes in 2 years |
name_changes_2y |
i32 |
Name changes in 2 years |
owner_changes_2y |
i32 |
Ownership changes |
sanctions_distance |
i32 |
BFS hops to nearest sanctioned entity |
shared_address_centrality |
i32 |
Vessels sharing the same registered address in ownership chain |
sts_hub_degree |
i32 |
Distinct vessels contacted in STS co-location events |
data/processed/causal_effects.parquet (written by src/score/causal_sanction.py)
| Column | Type | Description |
|---|---|---|
regime |
str |
Regime key (OFAC_Iran, OFAC_Russia, UN_DPRK) |
label |
str |
Human-readable regime label |
n_treated |
i32 |
Treated vessel count |
n_control |
i32 |
Control vessel count |
att_estimate |
f64 |
Pooled ATT (extra AIS gaps / 30 days) |
att_ci_lower |
f64 |
95% CI lower bound |
att_ci_upper |
f64 |
95% CI upper bound |
p_value |
f64 |
Two-tailed p-value |
is_significant |
bool |
True if p < 0.05 |
calibrated_weight |
f64 |
Suggested w_graph for composite.py |
Example top_signals field
[
{"feature": "ais_gap_count_30d", "value": 14, "contribution": 0.34},
{"feature": "sanctions_distance", "value": 1, "contribution": 0.29},
{"feature": "flag_changes_2y", "value": 3, "contribution": 0.18}
]
Explainability Worked Example
Vessel: PACIFIC GHOST (MMSI 477123456) — Confidence: 0.87
SHAP TreeExplainer decomposes the composite score into per-feature contributions:
Feature Value SHAP contribution Meaning
──────────────────────── ─────── ─────────────────── ────────────────────────────────────────────────────
ais_gap_count_30d 14 +0.34 14 dark periods in 30 days (avg fleet: 1.2)
sanctions_distance 1 +0.31 direct owner on OFAC SDN list (1 BFS hop)
sts_hub_degree 6 +0.18 contacted 6 distinct vessels during co-location events
flag_changes_2y 3 +0.12 changed flag state 3 times in 2 years
position_jump_count 2 +0.09 2 AIS positions requiring implied speed > 50 kts
causal_weight 0.71 +0.06 statistically significant DiD response to
OFAC Iran 2024-10 announcement (ATT=3.1, p<0.01)
name_changes_2y 0 −0.03 no name changes — mild negative contribution
trade_flow_mismatch 0.12 +0.02 minor mismatch between declared and estimated volume
──────────────────────── ─────── ─────────────────── ────────────────────────────────────────────────────
Composite score 0.87
How to read this: the SHAP contribution column shows exactly how much each feature pushed the score up or down from the baseline. AIS dark periods (+0.34) and direct ownership proximity to a sanctioned entity (+0.31) are the dominant signals. The causal DiD weight (+0.06) confirms the evasion behaviour intensified specifically after the October 2024 OFAC announcement — distinguishing deliberate evasion from routine commercial rerouting.
The analyst sees this breakdown directly in the dashboard alongside the confidence badge. No black-box verdict: every flagging decision is traceable to specific, dated, observable events.
Dashboard rendering: the FastAPI + HTMX dashboard renders top_signals as a horizontal bar chart (one bar per feature, scaled by SHAP contribution) alongside the vessel map pin and confidence badge. Each bar links to the raw data source (AIS position table, Equasis ownership record, or GDELT event) for one-click audit.
Validation Against Ground Truth
Known OFAC-listed vessels (those already on the SDN list at time of analysis) are used as a positive label set for validation:
- Precision@50: fraction of top-50 candidates that are OFAC-listed
- Recall@200: fraction of all OFAC-listed vessels captured in top-200
- AUROC: area under ROC curve across all scored vessels
This validation is run in src/score/validate.py and reported in the FastAPI + HTMX dashboard.
Sensor Fusion and Electro-Optics
Phase A — open-source data fusion (screening layer)
The screening layer fuses four independent open-source data streams rather than relying on a single signal:
| Signal | Source | What it detects |
|---|---|---|
| AIS behaviour | aisstream.io / Marine Cadastre | Dark periods, spoofing, STS events, loitering |
| Ownership graph | Equasis + OpenSanctions | Proximity to sanctioned entities, shell-company layers |
| Trade flow | UN Comtrade+ | Route/cargo mismatch, declared vs. estimated volume |
| Geopolitical events | GDELT | Sanction announcements, flag-state risk changes |
No electro-optical (EO) or SAR satellite imagery is required at the screening layer. Open-source AIS and ownership data alone deliver a 6× lift over the base rate of sanctioned vessels (Precision@50 = 0.62 vs. base rate ≈ 0.10). Satellite EO imagery is expensive per-scene and adds the most value when a specific high-confidence target has already been identified — exactly what Phase A produces.
Phase B — EO sensors at close range (investigation layer)
Once Phase A identifies a high-confidence candidate, Phase B deploys tiered electro-optical sensors from a patrol vessel:
| Tier | EO sensor | Capability |
|---|---|---|
| Tier 1 | Hi-res camera (Sony RX100 / GoPro) | Vessel identity: IMO number, name, flag OCR |
| Tier 2 | LiDAR (Livox Mid-360 / Ouster OS0-32) | Hull shape deviation, waterline / draught, 3D point cloud |
| Tier 3 | FLIR Boson+ thermal + hyperspectral | Engine heat signature, night operation, cargo type proxy |
Phase B sensor output feeds the edgesentry-app evidence bundle — GPS-tagged, Ed25519-signed, BLAKE3 hash-chained — and is transmitted to the Port Operations Centre via VDES. See docs/field-investigation.md for full hardware specifications and cost breakdown.
Roadmap — satellite SAR / EO integration
Wide-area persistent EO coverage (satellite SAR for dark vessel detection; optical for identity confirmation at anchor) is tracked in issue #84. The intended integration point is Phase A feature engineering: SAR-derived vessel detections would add a sar_dark_period_count feature alongside the existing AIS gap features, feeding the same Isolation Forest scoring pipeline.
This is not required for the screening layer to meet the Precision@50 ≥ 0.60 target. It becomes valuable at global scale where AIS receiver coverage is sparse (open ocean, polar regions).
Computational Requirements
The full pipeline (historical AIS + scoring) runs on a standard laptop:
| Step | Runtime (est.) | Memory |
|---|---|---|
| AIS Parquet load (12 months) | ~5 min | ~4 GB |
| Feature engineering (Polars) | ~10 min | ~2 GB |
| Lance Graph build | ~15 min | ~1 GB |
| HDBSCAN + Isolation Forest | ~5 min | ~1 GB |
| C3 causal DiD model | ~1 min | ~0.5 GB |
| SHAP attribution | ~10 min | ~2 GB |
| Total | ~46 min | ~4 GB peak |
For live streaming (aisstream.io), the incremental update pipeline runs in under 60 seconds per batch.
Edge gateway benchmark (measured): Re-scoring 5,000 vessels — feature matrix (build_matrix.py) + composite scoring (HDBSCAN + Isolation Forest + SHAP) + watchlist output — completes in 5.75 seconds on a 14-core Apple M-series laptop. On a constrained 4-core / 4 GB edge gateway (Raspberry Pi 4 / NVIDIA Jetson Nano class), the same pipeline is well within the 30-second target given the pipeline is CPU-bound on the HDBSCAN and Isolation Forest steps which scale sub-linearly with vessel count. See scripts/benchmark_rescore.py and docs/deployment.md for the full benchmark command and reproduction instructions.