Big Data Engineer (Spark/Hadoop)

We are seeking a Big Data Engineer with experience building and supporting scalable data platforms in AWS. The ideal candidate will have strong expertise in Spark, Kafka, and cloud-native technologies, with a passion for developing reliable data pipelines and optimizing large-scale distributed systems. Responsibilities • Design, develop, and maintain scalable batch and real-time data pipelines on AWS. • Build and optimize data processing applications using Spark (Scala/Python), Kafka, and Apache NiFi. • Develop cloud-native solutions using Kubernetes (EKS), EMR, and AWS services. • Monitor, troubleshoot, and optimize data platforms to ensure high availability and performance. • Support production deployments, investigate system issues, and implement long-term solutions. • Collaborate with engineering, infrastructure, and DevOps teams to improve platform reliability and automation. • Work with relational and NoSQL databases, including Aurora PostgreSQL, DocumentDB, and Couchbase. • Contribute to CI/CD pipelines and infrastructure automation using modern DevOps practices. Qualifications • 6+ years of software or data engineering experience, including 4+ years working with Big Data technologies. • Strong hands-on experience with Spark (Scala or Python), Kafka, Hadoop, and distributed data processing. • Experience building data pipelines using Apache NiFi and working with data lake architectures. • Proficiency with AWS services including EMR, EKS, S3, and cloud-native application development. • Experience with Kubernetes, CI/CD pipelines, and production troubleshooting. • Familiarity with SQL and NoSQL databases, performance tuning, and system optimization. • Experience with Infrastructure as Code (Terraform or CloudFormation) and AWS certifications is a plus.