Data Engineering Lead/Manager

Job Title: Data Engineering Lead/Manager Location: 100% Remote Employment Type: Long-Term Contract Job Overview We are seeking a Platform Engineer Lead to lead and manage the operational delivery of a modern enterprise data platform built on Azure and Databricks. This role combines technical leadership, platform operations, and stakeholder management, ensuring that the data and analytics platform remains reliable, scalable, and aligned with business needs. The ideal candidate will have strong experience managing cloud-based data platforms, leading support and engineering teams, and overseeing infrastructure components such as AKS, Databricks, DataIQ, Airflow, and Terraform. This individual will serve as the operational leader responsible for platform stability, service delivery, and continuous improvement of the data ecosystem. Key Responsibilities Platform Operations & Service Delivery • Lead the day-to-day operations of the enterprise Data & Analytics platform built on Azure and Databricks. • Manage platform intake, service requests, support tickets, and incident management processes. • Ensure high availability, performance, and reliability of platform services across data engineering and analytics workloads. • Establish operational best practices for monitoring, troubleshooting, and performance optimization. Team Leadership & Management • Manage and mentor a team of data engineers, platform engineers, and support specialists. • Provide technical direction and ensure the team follows platform engineering and DevOps best practices. • Coordinate cross-functional collaboration with data engineering, analytics, and infrastructure teams. Azure & Platform Infrastructure Oversight Oversee infrastructure components supporting the platform including: • Azure Kubernetes Service (AKS) • Databricks • DataIQ • Apache Airflow • Terraform-based infrastructure automation • Ensure infrastructure is secure, scalable, and cost optimized. • Implement governance policies and best practices for cloud infrastructure and data platforms. Automation & DevOps • Promote Infrastructure as Code (IaC) using Terraform to standardize platform provisioning. • Implement automation for deployment, monitoring, and platform maintenance. • Collaborate with DevOps teams to improve CI/CD pipelines for data platform components. Stakeholder & Business Engagement • Serve as the primary operational contact for business and technical stakeholders. • Communicate platform updates, incidents, and roadmap initiatives to leadership and business teams. • Manage expectations and ensure alignment between platform capabilities and business requirements. Governance, Security & Compliance • Implement and enforce data platform governance standards. • Ensure platform security across infrastructure, data pipelines, and access management. • Work with security teams to maintain compliance with enterprise and regulatory policies. Required Qualifications • 8+ years of experience in Data Engineering, Data Platform Engineering, or Cloud Infrastructure. • 3+ years of experience in team leadership or managerial roles. • Hands-on experience with Databricks-based data platforms. • Strong experience with Microsoft Azure cloud infrastructure. • Experience managing or supporting the following technologies: • Azure Kubernetes Service (AKS) • Databricks • DataIQ • Apache Airflow • Terraform • Experience with data pipeline orchestration and workflow automation. • Strong understanding of cloud platform operations, monitoring, and incident management. • Excellent communication and stakeholder management skills.