Data & Analytics Reference Architecture
Purpose
This reference architecture defines the standard building blocks, technical capabilities and patterns for collecting, managing, governing and consuming data across the supply chain. It is the baseline that every data-related solution blueprint must follow or explicitly deviate from through an approved Architecture Decision Record (ADR).
It answers three questions:
- Which technical capabilities does the organization need to turn supply chain data into decisions?
- How do those capabilities fit together end to end, from operational systems to analytics and AI?
- Which patterns and standards must solutions use when they build on this platform?
Scope
In scope: operational and partner data integration, analytical storage and processing, master and reference data, data governance, analytics, machine learning and data sharing.
Out of scope: transactional application design (covered by the application reference architectures), network and infrastructure design (covered by the technology reference architectures).
Business context
Supply chain decisions depend on data that is spread across ERP, WMS, TMS, MES, supplier portals, carriers and IoT devices. The main business drivers for this architecture are:
- End-to-end visibility of inventory, orders and shipments across the network.
- Better planning through reliable demand, supply and inventory data for S&OP.
- Traceability of product and lot origin for quality and regulatory compliance.
- Resilience through early detection of supplier, logistics and demand risks.
- Sustainability reporting based on trustworthy emissions and logistics data.
Architecture overview
The architecture is organized in five layers. Data flows from left to right, while governance and platform operations apply across all layers.
Technical capabilities
Technical capabilities describe what the platform must be able to do, independently of the products used to implement them. Each solution blueprint should state which of these capabilities it consumes or extends.
TC-DI · Data Integration & Ingestion
| ID | Capability | Description | Example technologies |
|---|---|---|---|
| TC-DI-01 | Batch ingestion | Scheduled bulk extraction of data from source systems and files. | Azure Data Factory, Fabric Data Pipelines, Airflow |
| TC-DI-02 | Change Data Capture | Near real-time replication of changes from transactional databases without impacting source performance. | Debezium, Fivetran, Qlik Replicate |
| TC-DI-03 | Event streaming | Ingestion and distribution of business events such as order created or shipment departed. | Kafka, Azure Event Hubs, Confluent |
| TC-DI-04 | B2B partner integration | Exchange of data with suppliers, customers and carriers using EDI and APIs. | Azure Integration Services, SAP Integration Suite, Boomi |
| TC-DI-05 | IoT telemetry ingestion | Collection of sensor data from warehouses, fleets and production lines. | Azure IoT Hub, AWS IoT Core |
| TC-DI-06 | External data acquisition | Integration of third-party data such as weather, commodity prices and supplier risk scores. | Data marketplaces, REST connectors |
TC-DS · Data Storage & Processing
| ID | Capability | Description | Example technologies |
|---|---|---|---|
| TC-DS-01 | Raw data storage | Immutable, low-cost storage of source data in its original format (Bronze zone). | ADLS Gen2, OneLake, S3 |
| TC-DS-02 | Curated data processing | Cleansing, conforming and joining data into a trusted model (Silver zone). | Databricks, Fabric, Spark, dbt |
| TC-DS-03 | Analytical data serving | Business-ready, aggregated data organized as data products (Gold zone). | Lakehouse, Synapse, Snowflake |
| TC-DS-04 | Stream processing | Real-time transformation, enrichment and alerting on event streams. | Spark Structured Streaming, Flink, Stream Analytics |
| TC-DS-05 | Master data management | Single, governed version of product, supplier, customer and location data. | SAP MDG, Informatica MDM, Profisee |
| TC-DS-06 | Time-series storage | Efficient storage and querying of high-volume telemetry data. | Azure Data Explorer, TimescaleDB |
TC-DG · Data Governance & Management
| ID | Capability | Description | Example technologies |
|---|---|---|---|
| TC-DG-01 | Data catalog | Inventory of data assets with business definitions, owners and classification. | Microsoft Purview, Collibra, Unity Catalog |
| TC-DG-02 | Data lineage | End-to-end tracking of how data moves and transforms from source to report. | Purview, OpenLineage |
| TC-DG-03 | Data quality management | Definition, measurement and monitoring of quality rules with alerts to data owners. | Great Expectations, Soda, Purview Data Quality |
| TC-DG-04 | Metadata & business glossary | Shared vocabulary for supply chain terms such as OTIF, lead time and safety stock. | Purview, Collibra |
| TC-DG-05 | Data security & privacy | Access control, encryption, masking and classification of sensitive data. | Entra ID, Purview Information Protection |
| TC-DG-06 | Data lifecycle & retention | Archiving and deletion according to retention and regulatory requirements. | Storage lifecycle policies |
TC-DA · Data Consumption & Analytics
| ID | Capability | Description | Example technologies |
|---|---|---|---|
| TC-DA-01 | BI & reporting | Standard and self-service dashboards and reports for business users. | Power BI, Tableau |
| TC-DA-02 | Real-time operational analytics | Live visibility and alerting for the supply chain control tower. | Power BI Real-Time, Grafana, Fabric Real-Time Intelligence |
| TC-DA-03 | Advanced analytics & ML | Development, training and deployment of models such as demand forecasting and ETA prediction. | Azure ML, Databricks ML, MLflow |
| TC-DA-04 | AI & GenAI enablement | Grounding of AI assistants and agents on governed enterprise data. | Azure AI Search, vector stores, LLM APIs |
| TC-DA-05 | Data sharing & APIs | Secure exposure of data products to internal applications and external partners. | API Management, Delta Sharing |
TC-DP · Data Platform Operations
| ID | Capability | Description | Example technologies |
|---|---|---|---|
| TC-DP-01 | Orchestration | Scheduling and dependency management of data pipelines. | Data Factory, Airflow, Databricks Workflows |
| TC-DP-02 | DataOps / CI-CD | Version control, automated testing and deployment of data pipelines and models. | GitHub Actions, Azure DevOps, dbt |
| TC-DP-03 | Platform observability | Monitoring of pipeline health, freshness, cost and performance. | Azure Monitor, Monte Carlo |
| TC-DP-04 | Infrastructure as code | Reproducible provisioning of data platform resources. | Terraform, Bicep |
Capability mapping to the business
The table shows which technical capabilities are most critical for each SCOR domain of the business capability map.
| SCOR domain | Key data domains | Critical technical capabilities |
|---|---|---|
| Orchestrate | KPIs, risk, sustainability | TC-DA-01, TC-DA-02, TC-DI-06, TC-DG-02 |
| Plan | Demand, supply, inventory | TC-DS-03, TC-DA-03, TC-DS-05 |
| Order | Customer orders, ATP | TC-DI-02, TC-DI-03, TC-DA-05 |
| Source | Suppliers, purchase orders, contracts | TC-DI-04, TC-DS-05, TC-DI-06 |
| Transform | Production, quality, lots | TC-DI-05, TC-DS-06, TC-DG-03 |
| Fulfill | Warehouse, shipments, carriers | TC-DI-03, TC-DS-04, TC-DA-02 |
| Return | Returns, warranties, refurbishment | TC-DI-01, TC-DS-03 |
Mandatory patterns
- Medallion architecture. All analytical data follows the Bronze → Silver → Gold zones. No report or model reads directly from the Bronze zone.
- Change Data Capture over batch extraction for transactional sources where near real-time data is required.
- Event-driven integration for supply chain milestones (order, shipment, receipt, exception), using a canonical event schema.
- Data products as the unit of consumption in the Gold zone, each with a named owner, documented schema, quality SLAs and access policy.
- Master data from the master data hub only. Product, supplier, customer and location data must not be mastered in individual solutions.
Standards
- GS1 identifiers (GTIN, GLN, SSCC) for products, locations and logistics units.
- GS1 EPCIS 2.0 for traceability and track & trace events.
- EDI (EDIFACT / ANSI X12) or partner APIs for B2B exchanges.
- ISO 8601 for dates and time zones in all data products.
Non-functional requirements
| Attribute | Target |
|---|---|
| Data freshness (control tower) | Under 5 minutes from source event |
| Data freshness (planning) | Daily, before the S&OP cycle starts |
| Data quality (master data) | At least 98% completeness on mandatory attributes |
| Availability (consumption layer) | 99.5% during business hours |
| Lineage coverage | 100% of Gold-zone data products |
Principles applied
- DP-01 · Data is an asset — every data product has an accountable owner.
- DP-02 · Single source of truth — master data is governed centrally.
- DP-03 · Data is secure by design — classification and access control from ingestion onward.
- AP-02 · Integration through open standards — APIs, events and GS1 standards for partner exchange.
Related artifacts
- Business capability map — all SCOR domains
- RA-B2B · B2B Partner Integration Reference Architecture
- RA-CT · Supply Chain Control Tower Reference Architecture
- BP-CT-01 · Distribution Control Tower Blueprint
Architecture decisions
| ADR | Decision | Status |
|---|---|---|
| ADR-001 | Adopt a lakehouse with medallion zones as the analytical platform | Proposed |
| ADR-002 | Use GS1 EPCIS 2.0 as the canonical traceability event model | Proposed |