Big Data Engineer

Big Data Engineer Position Overview We are seeking a Big Data Engineer with strong experience building and supporting scalable cloud-based data platforms on AWS. In this role, you will develop, maintain, and optimize batch and real-time data pipelines, troubleshoot production issues, and collaborate with cross-functional teams to deliver reliable, high-performance data solutions. Key Responsibilities • Design, build, and support large-scale data pipelines using Spark, Kafka, and Apache NiFi. • Develop and maintain Big Data applications on AWS, leveraging services such as EMR, EKS, S3, Aurora PostgreSQL, and DocumentDB. • Monitor, troubleshoot, and optimize batch and streaming data workflows to ensure high availability and performance. • Perform root cause analysis and implement long-term solutions for production issues. • Support CI/CD deployments, automation, and infrastructure improvements to enhance platform reliability. • Collaborate with Engineering, Infrastructure, SRE, and Product teams to deliver scalable data solutions and improve operational efficiency. Qualifications • 6+ years of IT experience, including 4+ years in Big Data or Data Engineering. • Hands-on experience with Spark (Scala or Python), Kafka, Apache NiFi, and AWS. • Experience with Kubernetes/EKS, distributed systems, and ETL/data ingestion pipelines. • Strong knowledge of relational and NoSQL databases, CI/CD, and cloud-native architectures. • Experience with production support, performance tuning, and troubleshooting in enterprise environments. • AWS certification and experience with Infrastructure as Code (Terraform or CloudFormation) are a plus.