Azure Databricks Engineer

Hybrid Onsite in Irving, TX - Local Candidates Only 12 month contract, potential to go long term Position Overview We are seeking a highly skilled Senior Cloud Data Engineer to lead the strategic migration and modernization of our legacy on-premises SQL Server Integration Services (SSIS) packages into a scalable, high-performance cloud data platform. In this role, you will be the core technical driver responsible for translating traditional ETL workflows into distributed, parallel-processed Azure Databricks notebooks and workflows. You will also design modern orchestration layers using Azure Data Factory (ADF). Key Responsibilities • ETL Migration & Modernization: Analyze, refactor, and migrate complex legacy SSIS packages into optimized PySpark or Spark SQL notebooks within Azure Databricks. • Pipeline Orchestration: Build, monitor, and optimize end-to-end data ingestion and transformation pipelines using Azure Data Factory. • Database Optimization: Write and tune highly complex T-SQL queries, views, and stored procedures, and migrate relational logic efficiently to a Delta Lake environment. • Architecture & Performance Tuning: Optimize legacy single-threaded data loads for distributed, petabyte-scale cloud infrastructure. • Data Validation: Build parallel validation frameworks to test, audit, and compare historical SSIS data outputs against modern Databricks outputs for absolute accuracy.