Lead Data Engineer

The ideal candidate will have a deep understanding of Microsoft data services, including Azure Fabric, Azure Data Factory (ADF), Azure Synapse, and ETL/ELT processes. This role focuses on designing, developing, and maintaining cloud-based data pipelines and solutions to drive our analytics and business intelligence capabilities. • Lead modernization of legacy data platforms, migrating ETL/ELT and ingestion frameworks to cloud (Azure/AWS). • Own end-to-end data solution architecture, ensuring scalability, security, and business alignment. • Provide technical leadership, mentoring teams and enforcing best practices. • Drive decision-making in ambiguous environments, proactively mitigating risks and constraints. • Implement data governance, security, and compliance frameworks (PII, regulatory standards). • Collaborate with stakeholders to translate business needs into scalable data solutions. • Architect and integrate data from multiple sources into high-performing cloud platforms. • Design and optimize ETL/ELT pipelines using ADF, Databricks, Spark, and SSIS. • Partner with analytics and BI teams to support reporting and advanced analytics use cases. • Ensure data quality through profiling, validation, and monitoring processes. • Establish CI/CD and DevOps practices for data engineering workflows. • Monitor and optimize pipeline performance, scalability, and cost efficiency. • Lead testing (unit, system, integration) and maintain comprehensive documentation. • Drive continuous improvement and innovation across tools, frameworks, and architecture. Required Skills and Qualifications: • 12+ years of experience in designing and implementing data warehouse and analytics solutions (on-premise and cloud). • Expertise in data warehousing concepts (ETL/ELT, data quality management, privacy/security, MDM) with hands-on experience using ADF, Data Factory, SSIS, and related tools. • Experience with cloud data and cloud-native data lakes/warehouses. Microsoft Azure services (Fabric Lakehouse, ADF, Data Factory, Synapse, etc.). • Experience in Python, Scala, or Java for use with distributed processing and analytics, such as Spark. • Familiarity with CI/CD practices and tools such as Azure DevOps, Git, or Jenkins. At ValueMomentum, we are a team of passionate engineers who thrive on tackling complex business challenges with innovative solutions while transforming the P&C insurance value chain. We achieve this through strong engineering foundation and continuously refining our processes, methodologies, tools, agile delivery teams, and core engineering archetypes. Our core expertise lies in six key areas: Cloud Engineering, Application Engineering, Data Engineering, Core Engineering, Quality Engineering, and Domain expertise.