We built a unified trade logistics analytics pipeline over 14 weeks. (1) Universal carrier integration layer: standardised API adapters for all 45 carriers with normalised data models — shipment events (booking confirmed, departed, in transit, arrived, cleared customs) mapped to a canonical event schema regardless of carrier-specific terminology. Automated data matching: ML-based matching of shipment data across carriers (when a shipment transfers from ocean to road transport, different carriers report different identifiers — the pipeline automatically links these into a single shipment journey). Data quality scoring: each data point assigned a confidence score based on source reliability, freshness, and consistency with other data — enabling downstream applications to handle uncertain data appropriately. (2) AI-powered customs classification: ML classification model trained on 5 years of historical customs declarations (2.8 million classified shipments) — automatically classifying goods into HS codes with 97% accuracy (from 94% manual accuracy).
Confidence-based routing: high-confidence classifications (above 95% model confidence) processed automatically, medium-confidence routed for expedited human review, low-confidence routed for specialist review. Rules of origin engine: automated determination of preferential origin under EU trade agreements (EU-UK TCA, EU-Japan EPA, EU-Canada CETA, etc.) — calculating value-added, applying specific rules of origin, and generating origin declarations. (3) Real-time visibility pipeline: streaming data processing: carrier updates, port community system data, and AIS vessel tracking data processed in real-time through a Kafka-Flink pipeline — maintaining a current view of every shipment's status and position. Gap-filling intelligence: when carrier data was delayed or missing, the pipeline estimated shipment position based on: last known position, carrier schedule data, vessel tracking (AIS), and historical transit time data for the specific route-carrier combination. Visibility coverage: improved from 80-85% to 97% of active shipments having current status available at any time.
(4) Predictive ETA engine: feature store with transit time features: historical transit times by route, carrier, port, season, and commodity type — computed from 2.8 million historical shipments. Real-time features: current port congestion (from AIS vessel count and port community system data), weather conditions (from weather APIs), and carrier performance trends (computed from recent shipment data). ETA prediction accuracy: improved from +/- 3.5 days to +/- 1.1 days for ocean shipments — better than industry benchmarks. (5) CBAM data pipeline: supplier carbon data collection (automated questionnaire distribution and response processing), carbon calculation engine (applying EU CBAM methodology to compute embedded emissions), and reporting pipeline (generating CBAM declarations in the format required by the EU CBAM transitional registry).