Technical Lead - Data Engineering

About the Company The RoleTechnical Lead - Data Engineering and play a critical role in driving our enterprise data platform strategy. You will lead the design, development, and delivery of modern data solutions using Snowflake, DBT, Azure/AWS, Python, Airflow, and CI/CD technologies. This role combines deep technical expertise with leadership responsibilities, guiding engineering teams, defining best practices, and ensuring the successful delivery of scalable, secure, and high-performing data platforms. Responsibilities • Lead the design and implementation of scalable and secure data platforms using Snowflake, DBT, Airflow, Python, and Azure/AWS services. • Define technical architecture, coding standards, engineering best practices, and development frameworks for the data engineering team. • Drive the adoption of CI/CD pipelines using GitHub Actions, Azure DevOps, Jenkins, or similar tools to automate build, test, deployment, and release management processes. • Lead the development, optimization, and maintenance of complex ETL/ELT pipelines for large-scale data processing. • Establish DevOps and DataOps practices to improve deployment efficiency, reliability, and operational excellence. • Mentor and coach data engineers through technical guidance, code reviews, architecture reviews, and knowledge-sharing sessions. • Collaborate with enterprise architects and business stakeholders to translate business requirements into scalable technical solutions. • Own platform reliability, monitoring, performance tuning, and troubleshooting of production data pipelines. • Implement Infrastructure as Code (IaC) using Terraform/Terragrunt to automate cloud resource provisioning. • Drive data quality, governance, security, and compliance standards across the data ecosystem. • Lead technical discussions, solution design workshops, and project planning activities. • Evaluate emerging technologies and recommend innovative approaches to improve data engineering capabilities and delivery processes. • Support Agile delivery and provide technical leadership throughout the project lifecycle. Qualifications • Proven experience as a Technical Lead, Lead Data Engineer, or similar leadership role. • Experience leading distributed development teams and delivering large-scale data engineering projects. • Strong stakeholder management and technical decision-making capabilities. Required Skills • Python & Data Engineering • Expert-level proficiency in Python for data engineering, automation, orchestration, and application development. • Strong experience developing scalable ETL/ELT frameworks using Python and SQL. • Hands-on experience with DBT, Airflow, Snowflake, and cloud-native data services. • CI/CD & DevOps • Strong experience designing and implementing CI/CD pipelines using GitHub Actions, Azure DevOps, Jenkins, GitLab CI/CD, or similar platforms. • Experience implementing automated testing, code quality checks, release management, and deployment automation. • Strong understanding of DevOps, DataOps, CI/CD best practices, and release governance. • Cloud & Platform Engineering • Extensive experience designing cloud-based data solutions on Azure and/or AWS. • Strong knowledge of cloud security, networking, monitoring, and operational best practices. • Experience with Infrastructure as Code using Terraform and Terragrunt. • Data Architecture • Expertise in Data Vault, dimensional modelling, data warehousing, and modern data platform architectures. • Advanced SQL development and performance optimization skills. • Experience building enterprise-grade data products and analytics platforms • Version Control & Engineering Practices • Strong Git/GitHub experience, including branching strategies, pull requests, code reviews, and release processes. • Experience implementing engineering standards, quality gates, and development best practices. • Communication & Stakeholder Engagement • Excellent communication and presentation skills. • Ability to engage with business and technical stakeholders at all levels. • Strong problem-solving, analytical thinking, and decision-making capabilities. Preferred Skills • Experience with Generative AI and AI-powered data engineering solutions. • Experience with Power BI, MicroStrategy, or other BI tools. • Knowledge of Kubernetes, Docker, and containerized deployments. • Experience with Databricks and modern lakehouse architectures. • Azure Data Factory, Synapse Analytics, or AWS Glue experience. • Experience implementing DataOps frameworks and observability platforms. • Exposure to enterprise architecture and governance frameworks.