I will create scalable data pipelines using azure data factory and databricks

Sommige informatie wordt in het Engels weergegeven.

India

Ik spreek Kannada, Hindi, Engels

Big Data Engineer

Experienced Data Engineer with 7+ years of experience building scalable, cloud-based data pipelines and automating data workflows. I specialize in Python, SQL, Azure(ADLS Gen2, Azure Databricks, Azur...
Over deze dienst

I specialize in designing and developing end-to-end ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks (PySpark). I integrate data from multiple sourcesAPIs, SQL databases, cloud storage, files, and moreinto clean, structured, and production-ready pipelines.


With strong expertise in ADF orchestration, Databricks Delta pipelines, and PySpark transformations, I can help you build robust workflows that are easy to maintain and optimized for performance. Whether you need a simple ingestion pipeline or a complete multi-source data solution, I deliver scalable, secure, and well-documented results.


What I offer:

Data ingestion from multiple sources

ADF pipelines, triggers, orchestration

Databricks Delta ETL/ELT pipelines

PySpark transformations

Data quality checks and validation

Streaming or batch pipelines

Documentation and best practices


Lets build a pipeline that delivers accurate, reliable, and production-grade data for your business!

Bestemmingsplatform:

Google BigQuery

Azure Synapse Analytics

Tools & platforms:

AWS Glue DataBrew

Azure Data Factory

Mijn portfolio