Data Engineer

Job Title: Data Engineer Experience: 5–7 Years Location: US – Hybrid/Remote Job Description We are seeking an experienced Data Engineer with 5–7 years of hands-on experience building scalable data pipelines and modern data platforms. The ideal candidate will have strong expertise in Databricks (AWS preferred), PySpark, Python, SQL, and Apache Airflow. Key Responsibilities • Develop and optimise ETL/ELT pipelines using Databricks, PySpark, Python, and SQL. • Build and manage Airflow DAGs for workflow orchestration. • Work with Delta Lake, Unity Catalog, Databricks Workflows, and lakehouse architectures. • Integrate data from APIs, databases, and other sources. • Optimise Spark jobs, SQL queries, and cloud data platforms for performance and cost. • Implement data quality, governance, monitoring, and CI/CD practices. • Troubleshoot production pipelines and collaborate with Data Scientists, Analysts, and stakeholders. Required Skills • 5–7 years of Data Engineering experience. • Strong Databricks experience; AWS preferred, Azure Databricks acceptable. • Strong PySpark, Python, SQL, and Apache Airflow skills. • Experience with AWS data services such as S3, Glue, Lambda, EMR, and Redshift. • Knowledge of Delta Lake, data lakes, data warehouses, and lakehouse architecture. • Experience with Git and CI/CD. • Strong data pipeline troubleshooting and performance tuning skills. Nice to Have Kafka/Kinesis, Spark Streaming, Unity Catalog, Terraform, Docker/Kubernetes, Power BI/Tableau/Looker, and MLOps experience.