Jatin C

Associate Data Engineer

American Express • Fulltime

Sep 2025 - Present • 8 mos

• Built and owned scalable data pipelines on Google Cloud Platform (GCP) using BigQuery and Apache Airflow to support enterprise-scale repeat customer rate (RCR) and incentive computation workflows, managing businesscritical datasets consumed by downstream analytics teams. • Automated data quality and validation workflows using Python and Apache Airflow for query orchestration, file generation, and secure GCP-to-SFTP data delivery pipelines. • Led validation and reconciliation efforts for large-scale on-premises Hive to GCP migration, ensuring schema consistency, data integrity, and reliable cross-platform validation. • Optimized SQL and data pipeline workflows by eliminating redundant processing steps and improving query efficiency, reducing compute cost and improving operational performance. • Supported and stabilized production Hive/SQL pipelines by troubleshooting failures, validating outputs, and ensuring timely delivery of business-critical reporting datasets.

Data Engineer

Not Found

Aug 2023 - Sep 2025 • 2 yrs 1 mo

• Built and optimized scalable data pipelines using Apache Spark, Delta Lake, and Databricks to support retail analytics, reducing redundant computations by 60% through reusable pre-aggregated datasets. • Improved pipeline performance by 30% by optimizing Spark SQL joins, partitioning, and Z-ordering, validated using Spark UI and job metrics. • Designed and implemented CI/CD-enabled, parameterized Azure Data Factory (ADF) pipelines supporting reliable multi-environment deployments with zero manual errors. • Developed a rule-based data quality framework with automated anomaly detection and alerting using Python and SQL, ensuring reliable downstream reporting and analytics. • Implemented data governance and access control standards using Unity Catalog, improving secure data sharing and metadata management across analytics teams