FFG Data Pipeline Documentation¶

Welcome to the Fuel for Growth Data Pipeline¶

The FFG (Fuel for Growth) Data Pipeline is an enterprise-grade data processing system designed to transform and consolidate financial data across multiple business dimensions for Coty's luxury division.

Overview¶

The FFG Pipeline processes three main data streams:

• Sell-Out Data: Retail sales data from partners
• Sell-In Data: Internal sales data from SAP systems
• RSP Data: Recommended Selling Prices and floor prices

These data streams are processed, enriched, ventilated, and consolidated to provide comprehensive business insights and support price corridor analysis.

Key Features¶

• Multi-Source Integration: Combines data from Azure Blob Storage, SQL Data Warehouse, and MDM systems
• Advanced Ventilation Logic: Allocates financial metrics across house and product levels
• Intelligent Deduplication: Smart append mode prevents data duplication
• Price Corridor Analysis: Identifies pricing opportunities and risks
• Automated Processing: End-to-end automation with Power Automate integration

Quick Start¶

Prerequisites¶

• Databricks workspace with SQL Warehouse access
• Azure Storage account with configured containers
• Service principal authentication configured
• Python 3.8+ environment

Basic Usage¶

from main import run_pipeline

# Execute the complete pipeline
success = run_pipeline()

Architecture¶

The pipeline follows a medallion architecture:

Bronze Layer → Silver Processing → Gold Layer → Analytics

Processing Steps¶

1. Data Ingestion: Load source files from Azure Blob Storage
2. Mapping Application: Enrich data with business mappings
3. MDM Enrichment: Add master data attributes
4. Ventilation: Allocate metrics across dimensions
5. Consolidation: Combine all data streams
6. Corridor Analysis: Calculate price corridors and opportunities

Navigation¶

• Architecture Overview - System design and components
• Pipeline Steps - Detailed processing documentation
• Technical Reference - Configuration and API details
• Getting Started - Setup and deployment guide

Support¶

For questions or issues: - Slack: #ffg-pipeline - Email: data-engineering@coty.com - GitHub: coty/ffg-pipeline

Last updated: September 2025