Data Engineer - ADX and ADF

Role Title: Data Engineer Employment Type: Contract / Contract-to-Hire Duration: 3–6 months with potential for conversion Location: Milwaukee, WI preferred, but open to remote with occasional travel to WI We are seeking a hands-on Data Engineer to design and implement scalable data ingestion, processing, governance, and machine learning integration solutions on Azure. This role will focus on Azure Data Explorer (ADX), Azure Data Factory (ADF), and integration of machine learning pipelines in Azure ML Studio. Key Responsibilities • Design and implement data ingestion strategies for streaming (Event Hub, Stream Analytics) and batch pipelines into ADX. • Build and optimize end-to-end pipelines in ADF, moving data from SQL databases to ADX and Azure Data Lake. • Develop robust monitoring, error handling, and governance frameworks for ADX and ADF. • Implement data modeling and schema design for telemetry and time-series data in ADX. • Integrate curated datasets into ML models deployed in Azure ML Studio, supporting real-time and batch scoring workflows. • Automate feature delivery pipelines for ML training and inference, with CI/CD for both data and ML pipelines. • Collaborate with cross-functional stakeholders to ensure data pipelines meet business needs. Requirements • 7+ years of experience in data engineering, including pipeline development, data modeling, and governance. • Proven expertise with Azure Data Explorer (ADX), including KQL, schema design, ingestion, and governance. • Strong experience with Azure Data Factory (ADF) pipeline design, optimization, and monitoring. • Knowledge of DevOps practices for data engineering (CI/CD for ADF, ADX ingestion). • Experience integrating data pipelines with Azure ML Studio, including real-time scoring and batch inference. • Bachelor’s degree in Computer Science, Data Science, or related field (Master’s preferred).