Job Title: Azure Data Engineer Location: Blue Bell, PA (Hybrid – 3 Days Onsite) Interview Process: Final Round will be In-Person Job Summary We are seeking a skilled Azure Data Engineer to join our IT department. The role focuses on leveraging Databricks, Azure Data Factory, and related toolsets to enhance systems integration within the commercial insurance domain. The ideal candidate will have strong technical problem-solving skills, expertise in Databricks, and a strong understanding of data management and engineering principles. Key Responsibilities • Design, develop, and maintain scalable data pipelines using Databricks. • Ensure data quality, integrity, and consistency across all pipelines. • Integrate data from multiple sources, including Azure Data Lake, SQL databases, and APIs. • Leverage Databricks to integrate with different Large and Small Language Models. • Collaborate with data scientists, analysts, and stakeholders to gather and understand data requirements. • Develop and maintain ETL processes using Azure Databricks. • Implement Data Quality, Audit, Balance, and Control measures in workflows. • Optimize data processing performance and apply best practices for storage and retrieval. • Ensure security and compliance with industry standards and data governance policies. • Provide technical support, training, and guidance to team members and stakeholders. • Document data engineering processes, workflows, and best practices. • Generate reports and dashboards to provide insights into pipeline performance and data quality. • Stay updated with industry trends and identify opportunities for process improvements. • Demonstrate commitment to the Company’s Code of Business Conduct and Ethics. Required Qualifications • Bachelor’s degree in Computer Science, Engineering, or a related field. • Proven experience as a Data Engineer with a focus on Azure Databricks. • Strong knowledge of Azure Databricks and its ecosystem. • Experience with data pipeline and workflow management tools. • Proficiency in programming languages such as PySpark, Python, Scala, or SQL. • Familiarity with big data technologies such as Spark. • Experience with Azure services, including Azure Data Lake and Azure SQL Database. • Knowledge of data warehousing concepts and best practices. • Excellent problem-solving skills and strong attention to detail. • Strong communication and collaboration skills. Preferred Qualifications • Experience with integrating Databricks with Large and Small Language Models. • Experience implementing Data Quality, Audit, Balance, and Control measures. • Familiarity with the commercial insurance domain. • Experience in developing and optimizing data solutions in a fast-paced environment.