Data Engineer

About Us Zapcom is a global Product Engineering and Technology Services company, specializing in bespoke, customer-centric solutions across industries like BFSI, e-commerce, retail, travel, transportation, and hospitality. Headquartered in the US, with a presence in India, Europe, Canada, and MENA, we excel in transforming ideas into tangible outcomes using AI, ML, Cloud solutions, and full-stack development. At Zapcom, we value accountability, ownership, and equality, empowering you to excel. We listen to your aspirations and provide the support needed to achieve them. Our diverse, collaborative culture ensures every voice is heard, driving innovation and business value. With global opportunities and expansion plans, now is the perfect time to join our team. Work on impactful projects that shape the future. Apply today and be part of something extraordinary! • Design and deploy scalable ETL/ELT pipelines to ingest, transform, and load clinical data from diverse sources (EMRs, labs, IoT devices, data lakes, FHIR/HL7 APIs), including Azure and Snowflake. • Architect and optimize Microsoft Azure and Snowflake environments for clinical data storage, extraction, transformation, and loading, machine learning operations (MLOps) performance tuning, cost management, and secure data sharing. • Ensure compliance with healthcare regulations (HIPAA, GDPR) by implementing data anonymization, encryption, and audit trails. • Collaborate with clinical stakeholders to translate business requirements into technical solutions for analytics and reporting. • Develop and maintain data governance frameworks , including metadata management, data lineage, and quality checks (e.g., validation of lab results, patient demographics). • Automate data pipelines using orchestration tools (e.g., Apache Airflow, Prefect) and integrate real-time streaming solutions (e.g., Kafka) where applicable. • Build and maintain documentation for data models, pipelines, and processes to ensure reproducibility and transparency. • Advanced proficiency in Snowflake (Snowpipe, Time Travel, Zero-Copy Cloning) and SQL for complex transformations. • Hands-on experience with ETL/ELT tools (Apache Spark, AWS Glue, Azure Data Factory) and cloud platforms (AWS, Azure, GCP). • Strong programming skills in Python/Scala (Pandas, PySpark) for data scripting and automation. • Familiarity with healthcare data formats (OMOP, FHIR, HL7, DICOM) and clinical workflows. • Expertise in federated learning and running large jobs on high-performance computing servers is a plus. • Data Governance: Ability to implement data quality frameworks (e.g., Great Expectations) and metadata management tools. • Regulatory Compliance: Proven experience securing PHI/PII data and adhering to HIPAA/GDPR requirements. • Problem-Solving: Ability to troubleshoot pipeline failures, optimize query performance, and resolve data discrepancies. • Collaboration: Strong communication skills to work with cross-functional teams (clinicians, analysts, IT). Requirements • Snowflake SnowPro Core Certification (or higher). • AWS/Azure/GCP Data Engineering Certification (e.g., AWS Certified Data Analytics, Azure Data Engineer Associate). • Running jobs on high-performance computing servers • Healthcare-specific certifications (e.g., HL7 FHIR Certification, Certified Health Data Analyst (CHDA)). • Security certifications (e.g., CISSP, CIPP) for handling sensitive clinical data. • 3+ years of experience in data engineering, with 2+ years focused on healthcare/clinical data (e.g., hospitals, EMR systems, clinical trials). • 2+ years of hands-on experience with Snowflake in production environments. • Proven track record of building ETL pipelines for large-scale clinical datasets. • Experience with OMOP CDM, Epic/Cerner EHR systems, or clinical data lakes. • Exposure to DevOps practices (CI/CD, Terraform) and Agile methodologies. • Some front-end development is preferred.