Azure Data Engineer

Technology Stack : Azure Data Factory, ADLS Gen2, Azure Synapse, Azure Databricks, SQL / PySpark, Azure DevOps, Azure Monitor, Azure Key Vault Domain: Banking - Understanding of banking-specific data models About the Role We are seeking a motivated and technically strong Data Engineer to join our Data Platform team, with a primary focus on building and maintaining data pipelines for our Banking, Financial Services, and Insurance (BFSI) business units. You will work within an Azure-first environment, using Azure Data Factory as the core orchestration engine to move, transform, and deliver data that powers regulatory reporting, risk analytics, and customer intelligence. Key Responsibilities • Pipeline Development & Orchestration • Build and maintain data pipelines in Azure Data Factory including Linked Services, Datasets, Pipelines, and Mapping Data Flows. • Implement incremental/delta load patterns using watermarking, change data capture (CDC), and schedule and event-based triggers. • Develop parameterized, reusable pipeline templates to accelerate delivery and reduce duplication. Data Integration & Transformation • Integrate ADF with ADLS Gen2, Azure Synapse Analytics, Azure SQL, and Databricks to support the enterprise data lakehouse architecture. • Write transformation logic using SQL, PySpark, and ADF Mapping Data Flows to cleanse, conform, and enrich financial and insurance datasets. • Support ingestion from diverse BFSI source systems including core banking platforms, policy administration systems, claims databases, and market data feeds. Data Quality & Compliance • Implement data validation checks, null/duplicate handling, and anomaly alerts within pipeline logic. • Ensure pipelines comply with regulatory frameworks relevant to BFSI — including RBI guidelines, IRDAI data standards, GDPR, and SOC 2 controls. • Apply PII masking and data classification practices for sensitive financial and customer data, in line with data governance policies. Monitoring, Support & Operations • Monitor pipeline health via Azure Monitor and Log Analytics; set up alerts for failures, latency breaches, and data quality anomalies. • Investigate and resolve pipeline failures, data discrepancies, and performance bottlenecks in production environments. • Maintain clear documentation of pipeline designs, data flows, transformation logic, and runbooks. Required Qualifications, Experience • 12+ years of data engineering experience, with at least 3 years of production-grade work in Azure Data Factory. • Demonstrated exposure to BFSI data domains — banking transactions, insurance claims, policy data, or regulatory reporting. Azure Data Factory — Core Skills • Proficiency with ADF pipeline components: Linked Services, Datasets, Activities (Copy, Data Flow, Execute Pipeline, Web, Lookup, ForEach). • Experience configuring Integration Runtimes — Azure IR for cloud-to-cloud and Self-hosted IR for on-premises connectivity. • Hands-on with ADF Mapping Data Flows for schema-on-read transformations, joins, aggregations, and conditional splits. • Familiarity with ADF CI/CD using Azure DevOps, ARM template exports, and branch-based deployment strategies.