Azure Data Engineer

Job Title: Azure Data Engineer (ADF / PySpark / Python) Location: Cincinnati, OH Interview: Final round Face-to-Face in Ohio (submit only candidates ready for F2F) Experience Required: 10+ years Employment Type: Contract Job Summary: We are seeking an experienced Azure Data Engineer with strong expertise in Azure Data Factory (ADF), PySpark, and Python to design, build, and maintain scalable data pipelines. The role involves ownership of data engineering stories end-to-end, working in an Agile environment, and supporting both internal and external data applications. Key Responsibilities: • Take ownership of user stories and drive them through all phases of the SDLC. • Design and develop distributed data processing pipelines using PySpark. • Build and orchestrate multi-step data transformation pipelines using Azure Data Factory. • Develop transformation logic using object-oriented Python. • Perform unit, integration, and regression testing on data pipelines and packaged code. • Enhance and maintain CI/CD pipelines for production deployment. • Implement data quality checks for ingested and post-processed data. • Ensure data observability through monitoring, alerting, and logging. • Maintain and enhance existing data applications and pipelines. • Provision and manage cloud infrastructure using Infrastructure as Code (IaC). • Participate in sprint planning, estimations, and retrospective meetings. • Mentor junior team members and promote engineering best practices. • Continuously identify opportunities to improve processes, performance, and quality. Required Skills & Experience: Core Technical Skills: • Strong hands-on experience with Microsoft Azure, including: • Azure Data Factory (ADF) • Azure data services • Python (advanced) with object-oriented programming principles • PySpark (distributed data processing) • Strong SQL development experience • Experience with data pipeline orchestration • Experience with CI/CD pipelines • Experience with Git/GitHub (version control) • Experience with automated testing (PyTest or similar) • Experience with data observability, monitoring, and alerting • Experience implementing data quality frameworks and checks • Experience with Infrastructure as Code Additional Technical Knowledge: • Java and Spring Framework (working knowledge) • Cloud experience (Azure preferred; AWS/GCP acceptable) • Dependency management tools (Conda, venv, etc.) • Debugging and performance tuning of enterprise applications • Understanding of SOLID principles • Agile/Scrum development experience Education: • Bachelor’s degree in computer science, MIS, Mathematics, Business Analytics, or a related technical field. Soft Skills: • Strong ownership mindset and problem-solving skills • Ability to work in a fast-paced Agile environment • Strong collaboration and communication skills • Continuous learning and improvement mindset