Data Engineer

Job Title: Senior Data Engineer Duration: 6 months contract to Perm Location: New York, NY 10001, (onsite) Tech Stack: Python, Java, Spark, AWS, Kubernetes, Airflow, Glue, EMR, Athena, Kinesis, Parquet, Iceberg, CI/CD. About the Role We are seeking a Lead Software Engineer – Data Engineering to design and build scalable, cloud-native data platforms and pipelines. You will lead the development of robust data solutions, drive architectural decisions, and mentor engineers while ensuring reliability, security, and operational excellence. Responsibilities • Design and develop large-scale data pipelines and data platforms. • Build cloud-native solutions using AWS services and Spark. • Lead data modeling, warehousing, migration, and analytics initiatives. • Improve system performance, resilience, monitoring, and automation. • Collaborate with cross-functional teams and mentor junior engineers. Required Qualifications • Strong proficiency in Python and/or Java with Spark/PySpark. • Experience with Data Pipelines, Data Modeling, Data Warehousing, and Data Migration. • Hands-on experience with AWS (Glue, EMR, S3, Athena, Kinesis/MSK, Airflow, Lambda, Step Functions, Lake Formation). • Knowledge of Kubernetes, CI/CD, Agile methodologies, and software design principles. • Experience with data formats such as Parquet, Iceberg, Avro, or JSON. • Understanding of AWS security concepts including IAM, KMS, Encryption, and Secrets Manager. Preferred Qualifications • Experience building REST APIs using Java. • Knowledge of Hybrid Cloud architectures. • Exposure to Jenkins, Git, Linux Scripting, JIRA, and TDD. • Strong leadership, problem-solving, and communication skills.