Senior Data Engineer

Our client is seeking an experienced Data Engineer to join a growing data engineering team responsible for building modern, scalable data platforms that support real-time analytics, operational applications, and AI/ML initiatives. This is an exciting opportunity to work in a medical device IoT environment where your work will directly enable data-driven decisions using live device telemetry and streaming data. While the team supports traditional batch data processing, the primary focus of this role is designing and developing real-time streaming data solutions, robust data services, and high-performance data pipelines that power APIs, microservices, and machine learning applications. Location: Philadelphia, PA (Hybrid – 2-3 Days Onsite) Type: Contract-to-Hire Responsibilities • Design, develop, and maintain scalable real-time streaming data pipelines. • Build data ingestion and processing frameworks for IoT device telemetry and operational data. • Develop ETL/ELT solutions supporting both cloud and on-premises environments. • Create and optimize data services that support APIs, microservices, and downstream applications. • Build data pipelines that enable analytics, reporting, and AI/ML model development. • Utilize Databricks and Apache Spark technologies to process large-scale datasets efficiently. • Optimize Spark jobs using PySpark, Spark SQL, and Delta Lake. • Implement CI/CD pipelines and DevOps best practices for automated deployment and testing. • Collaborate with software engineers, data scientists, architects, and business stakeholders to deliver scalable data solutions. • Monitor, troubleshoot, and improve performance, reliability, and scalability of data platforms. • Ensure data quality, governance, and security across enterprise data environments. Required Qualifications • Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field. • 5+ years of experience in Data Engineering or related data platform development. • Strong experience building enterprise data pipelines in both cloud and on-premises environments. • Hands-on expertise with Databricks, including: PySpark, Spark SQL, Delta Lake • Strong knowledge of Apache Spark and distributed data processing. • Experience developing ETL/ELT data integration solutions. • Experience building data services supporting APIs and microservices. • Experience working with Azure Cloud Services. • Understanding of DevOps methodologies and CI/CD deployment pipelines. • Experience working with Big Data technologies and large-scale distributed systems. • Strong SQL development and data modeling skills. Preferred Qualifications • Experience with streaming technologies such as Spark Structured Streaming or similar platforms. • Experience working with IoT, connected devices, or telemetry data. • Familiarity with Azure Data Factory, Event Hubs, Azure Storage, or Azure Synapse. • Experience supporting machine learning or AI data pipelines. • Knowledge of containerization technologies such as Docker and Kubernetes. • Experience working in regulated industries.