I built the Retail Demand Forecasting & Inventory Optimization repository as a Jupyter Notebook on my GitHub, starting in July 2026 and last updating in July 2026, using the Walmart M5 Forecasting Dataset.
Retail-Demand-Forecasting-Inventory-Optimization- A repo on @Sarathsriram23's GitHub. Jupyter Notebook. Started 2026-07, last updated 2026-07.
The Week 1 goal was to create a production‑ready, cloud‑native data engineering pipeline in Google BigQuery with a DuckDB fallback, define DDL schemas, build modular Python ETL with structured logging, enforce data‑quality checks, handle duplicates and missing values, and transform the wide daily sales columns into a normalized star‑schema fact table.
- Folder Architecture – enterprise‑grade modular layout
I chose the dual BigQuery/DuckDB engine to enable cloud deployment while keeping local development lightweight and credential‑free, and I built a star‑schema analytics table by unpivoting the 1,913 daily sales columns to support time‑series queries and downstream forecasting models.
The calendar table loaded successfully with 1,969 rows and 14 columns, and duplicate detection on the d key reported zero duplicates.
- Dual Engine Adapter (
db_adapter.py) – BigQuery production, DuckDB local - SQL Schemas – DDL for all 5 M5 tables plus an unpivoted
sales_facttable partitioned bydateand clustered bystore_id,item_id - Python ETL Engine – extractors, transformers, loaders, validators modules
- Data Quality Suite – duplicate detection on composite keys, missing‑value imputation, range check
sell_price > 0 - Structured Logging – timestamps, log levels, component tracking written to console and
logs/etl_pipeline.log - Documentation – setup guide, data dictionary, unit tests