Lead Data Engineer – AWS & Big Data

Job Summary We are seeking a highly experienced Lead Data Engineer to design, develop, and manage scalable data platforms and pipelines on AWS. The ideal candidate will have strong expertise in big data technologies, cloud architecture, data modeling, and ETL/ELT development, along with exposure to AI/ML integration and BI tools. You will play a key role in building robust data infrastructure that supports analytics, reporting, and advanced data-driven decision-making across the organization. Key Responsibilities • Design, build, and maintain scalable batch and real-time data pipelines using AWS services • Develop and optimize data lakes and data warehouses using Amazon S3, Redshift, and Glue • Build efficient ETL/ELT pipelines using Python, PySpark, and SQL • Implement data modeling techniques such as Star Schema, Snowflake Schema, and Dimensional Modeling • Work with distributed computing frameworks like Hadoop and Apache Spark • Automate infrastructure provisioning using Terraform (Infrastructure as Code) • Ensure data quality, governance, security, and compliance across pipelines • Integrate AI/ML models into data pipelines to enable advanced analytics use cases • Collaborate with data scientists, analysts, and business stakeholders to deliver data solutions • Develop dashboards and reports using Power BI • Monitor, troubleshoot, and optimize performance of data pipelines and cloud resources Required Skills & Experience • 12–15 years of experience in Data Engineering / Big Data / Cloud Data Platforms Strong hands-on experience with AWS services: • Amazon S3 • Amazon Redshift • AWS Glue • AWS Lambda • EMR • Athena • Strong proficiency in Python, PySpark, and SQL • Experience with Hadoop and Apache Spark ecosystems • Strong knowledge of data warehousing and dimensional data modeling • Hands-on experience with Terraform (IaC) • Understanding of AI/ML concepts and integration into data pipelines • Experience with Power BI or similar BI tools