Data Engineer

Databricks Data Engineer (Contract) Location: Dallas, TX (Remote) Industry: Healthcare Employment Type: Contract A leading Healthcare organization is seeking an experienced Databricks Data Engineer to support the design, development, and optimization of a modern enterprise data platform built primarily on Databricks. This is an exciting contract opportunity to play a key role in a large-scale healthcare data transformation initiative focused on data modernization, interoperability, advanced analytics, population health, and AI-driven innovation. The successful candidate will be responsible for developing scalable Databricks solutions that enable the ingestion, transformation, governance, and delivery of critical healthcare data across the enterprise. Working closely with Data Architects, Data Engineers, Clinical Analytics teams, Data Scientists, and business stakeholders, this individual will help establish Databricks as the organization's strategic data platform while driving best practices in engineering, security, and compliance. We are specifically seeking hands-on Databricks professionals with deep expertise in building enterprise-scale Lakehouse architectures using Databricks, Delta Lake, Apache Spark, and cloud technologies. Experience working with healthcare datasets, regulatory requirements, and healthcare interoperability standards is highly desirable. Key Responsibilities • Design, develop, and maintain scalable data pipelines using Databricks, PySpark, Spark SQL, and cloud-native services • Build and optimize enterprise Lakehouse architectures leveraging Databricks, Delta Lake, Unity Catalog, and Apache Spark • Develop robust ETL/ELT frameworks supporting healthcare analytics, clinical reporting, operational intelligence, and AI initiatives • Design and implement ingestion pipelines for healthcare data sources including EHR, EMR, claims, pharmacy, laboratory, provider, and patient data • Support integration and processing of healthcare interoperability standards such as HL7, FHIR, X12, and related healthcare data formats • Develop and maintain Delta Live Tables, Databricks Workflows, and streaming data pipelines where applicable • Implement data quality, lineage, observability, and monitoring frameworks across the Databricks platform • Optimize Databricks workloads for performance, reliability, scalability, and cost efficiency • Partner with business and clinical stakeholders to translate healthcare data requirements into scalable technical solutions • Implement governance and security controls utilizing Unity Catalog and cloud-native security capabilities • Ensure compliance with HIPAA, PHI, and healthcare data privacy requirements throughout the data lifecycle • Support advanced analytics, machine learning, predictive modeling, and AI use cases utilizing Databricks capabilities • Collaborate with DevOps and platform teams to establish CI/CD processes and Infrastructure-as-Code standards • Troubleshoot complex data integration, performance, and platform issues across the healthcare data ecosystem • Contribute to the organization's cloud modernization and healthcare data strategy initiatives • Promote engineering standards, documentation, and continuous improvement across the data platform Required Qualifications • 5+ years of experience in Data Engineering, Big Data Engineering, or Cloud Data Platform development • 3+ years of hands-on Databricks development experience in enterprise environments • Strong expertise with Databricks Lakehouse Platform, Delta Lake, Apache Spark, PySpark, and Spark SQL • Demonstrated experience building enterprise-scale data pipelines and cloud-native data platforms • Advanced proficiency in Python and SQL • Strong understanding of modern Lakehouse architecture principles and implementation best practices • Experience with Databricks Unity Catalog, Delta Live Tables, and Databricks Workflows • Experience utilizing Databricks for large-scale batch and streaming data processing workloads • Hands-on experience with Azure, AWS, or Google Cloud Platform • Strong knowledge of data modeling, data warehousing, and enterprise data architecture concepts • Experience implementing data governance, security, and data quality frameworks • Knowledge of healthcare compliance requirements including HIPAA and protected health information (PHI) • Experience with CI/CD practices, DevOps methodologies, and Infrastructure-as-Code tools • Excellent troubleshooting, analytical, and problem-solving skills • Strong communication skills with the ability to collaborate across technical and business teams Preferred Qualifications • Active Databricks certifications (Databricks Data Engineer Associate or Professional preferred) • Experience working within healthcare providers, payers, health systems, life sciences, or healthcare technology organizations • Experience integrating healthcare data sources including Epic, Cerner, Athena, Meditech, or other EHR/EMR platforms • Knowledge of healthcare interoperability standards including HL7, FHIR, CDA, and X12 • Experience supporting population health, clinical analytics, quality reporting, or value-based care initiatives • Experience with MLflow, Databricks Asset Bundles, Unity Catalog, and Databricks Data Intelligence Platform capabilities • Experience working with healthcare claims, patient, provider, clinical, laboratory, or operational datasets • Experience with real-time and streaming data architectures • Cloud certifications in Azure, AWS, or Google Cloud • Experience supporting AI, machine learning, and generative AI initiatives within regulated environments What We're Looking For We are seeking a highly skilled Databricks Data Engineer who is passionate about building modern healthcare data platforms and solving complex data challenges at scale. The ideal candidate combines deep Databricks expertise with a strong understanding of data engineering best practices, governance, and regulatory compliance. This individual will play a critical role in delivering a secure, scalable, and high-performing healthcare Lakehouse environment that empowers analytics, supports clinical and operational decision-making, and accelerates the organization's AI and digital transformation initiatives. If you are interested in learning more, please apply directly or contact us for additional details.