Azure Data Engineer Lead with Databricks

Job Description Job Title : - Azure Data Engineering Lead with Databricks Location : - Philadelphia, PA (3 days Onsite/Week) Type of Employment : - Contract to hire (6 months contract then fulltime with the client) What’s in it for you :- This role offers the opportunity to lead and build enterprise‑scale Azure data platforms using a modern lake house architecture. Candidates gain deep exposure to Azure, Databricks, advanced data modeling, and governance while working on complex, high‑impact healthcare data. It strengthens long‑term career growth as a senior data engineering or architecture leader in a high‑demand domain Required Skills & Experience Core Technical Expertise • 10–15+ years of experience in data engineering and hands on technical delivery roles, preferably in Healthcare Payer / Medicaid environments • Extensive, Azure first cloud experience, including solution architecture, implementation, and operations • Strong hands-on expertise with Azure data platform services, including Azure Data Lake Storage (ADLS Gen2), Azure Databricks, Azure Data Factory (ADF), Azure Synapse Analytics, Azure SQL / SQL Server, Azure Key Vault, Azure DevOps (Repos, Pipelines, Boards. • Strong proficiency in: SQL (advanced querying, tuning, optimization) , Python and PySpark, ETL / ELT development • Strong data modeling experience, including Conceptual, logical, and physical modeling , Dimensional modeling (star/snowflake schemas), Experience handling SCDs and historized data • Deep understanding of: Cloud native Data Lake / Lakehouse architectures , Batch and streaming processing patterns , Incremental processing, and CDC • Proven experience reviewing and optimizing : Data models , Transformation logic, Schema evolution strategies • Experience implementing cloud and data governance controls across Dev/Test/Prod, Agentic IDE & AI Assisted SDLC Experience: • Proven experience architecting and delivering systems using agentic IDEs • Ability to: Define architectural and data modeling intent that agents can follow • Break features into agent executable tasks • Govern AI autonomy (guardrails, permissions, reviews) o Integrate agentic workflows into CI/CD pipelines • Strong understanding of Security implications of autonomous code execution • Compliance, auditability, and traceability o AI assisted SDLC operating models • Healthcare Domain Expertise • Strong experience in Healthcare Payer / Medicaid data environments • Familiarity with o Claims o Member o Provider o Eligibility data • Understanding of HIPAA, PHI, and healthcare compliance requirements Education: - • Engineering Degree – BE / ME / BTech / M Tech / B.Sc. / M.Sc. • Technical certification in multiple technologies is desirable.