Data Engineer

NO C2C / VENDOR PLEASE READ CAREFULLY BEFORE APPLYING Job Title: Data Engineer – Azure Location: Hybrid (Milwaukee, WI – multiple onsite days required each week) Role Overview: We are seeking an experienced Data Engineer with strong hands-on expertise in building scalable data solutions on Microsoft Azure. The role will focus on designing and implementing data ingestion, processing, governance, and machine learning integration pipelines using Azure Data Explorer (ADX), Azure Data Factory (ADF), and Azure ML Studio. This individual will play a key role in enabling predictive maintenance initiatives by ensuring high-quality, reliable, and governed data pipelines that feed analytics and ML models. Key Responsibilities: • Design and implement data ingestion strategies for both streaming (Event Hub, Stream Analytics) and batch flows into Azure Data Explorer (ADX) and Azure Data Lake. • Develop scalable data models, schemas, and transformations optimized for telemetry and time-series data using Kusto Query Language (KQL). • Implement robust data governance, anomaly detection, and quality frameworks across ADX and ADF pipelines. • Build, monitor, and optimize Azure Data Factory (ADF) pipelines, ensuring reliability, scalability, and cost efficiency with automated error handling and alerting mechanisms. • Design and maintain DevOps pipelines for Data Engineering (CI/CD for ADF pipelines, ADX ingestion, and ML pipelines). • Integrate curated datasets into machine learning models deployed in Azure ML Studio, enabling real-time and batch scoring workflows. • Automate feature delivery pipelines for ML training and inference, ensuring seamless integration with data engineering workflows. • Collaborate with business stakeholders, data scientists, and cross-functional teams to deliver high-impact, production-grade data solutions. Qualifications: • Bachelor’s degree in Computer Science, Data Science, Software Engineering, Information Systems, or a related field (Master’s degree preferred). • 7+ years of experience in data engineering, including pipeline development, data integration, and data governance. • Proven expertise with Azure Data Explorer (ADX), including schema design, KQL, and ingestion strategies. • Strong background in Azure Data Factory (ADF) with experience in performance optimization, monitoring, and CI/CD practices. • Hands-on experience integrating data pipelines with Azure ML Studio for ML model training, deployment, and inference. • Solid understanding of data quality, governance, anomaly detection, and monitoring best practices. • Experience working in hybrid environments with both cloud and on-prem data sources.