Data Engineer Lead

Title: Data Engineer Lead Location: New York City, NY (Onsite) Type: Contract Description • An AWS Data Engineer role involves designing, building, and maintaining scalable data pipelines, architectures, and solutions on the Amazon Web Services (AWS) cloud platform, with additional focus on HR Data Warehouse (HR DWH). • The role includes developing secure, compliant, and high-performance data platforms to support enterprise analytics across HR, Finance, and Business domains. Key responsibilities include data integration, building ETL/ELT processes using services like AWS Glue and Redshift, data modeling, and ensuring data quality, governance, and security-particularly for sensitive HR and employee data. • This role requires strong proficiency in programming languages such as Python and Scala, as well as experience with SQL, Apache Spark, and serverless architectures. Key Responsibilities • Design, build, and maintain scalable data pipelines and ETL/ELT processes using AWS Glue, EMR, Lambda, and Redshift to support analytics and reporting. • Develop and manage HR Data Warehouse (HR DWH) solutions, integrating data from HR systems such as Workday, SAP HCM, Oracle HCM, ADP, payroll, benefits, recruiting, and learning platforms. • Create and maintain HR data models (employee, position, compensation, headcount, attrition, performance, time & attendance) optimized for reporting and analytics. • Integrate data from multiple structured and semi-structured sources across enterprise systems. • Ensure data quality, data lineage, security, and compliance, including handling PII and sensitive HR data in accordance with regulatory requirements (GDPR, SOC, internal controls). • Implement data validation, reconciliation, and audit checks for HR and enterprise datasets. • Monitor, optimize, and tune data pipelines and Redshift/Snowflake performance. • Collaborate with HR stakeholders, business analysts, and reporting teams to understand People Analytics and workforce reporting requirements. • Maintain, support, and operationalize existing data solutions in production environments. Minimum Skills Required • 8+ years of experience in design, development, and end-to-end implementation of enterprise-wide big data solutions. • Strong experience designing and developing big data solutions using Spark, Scala, AWS Glue, Lambda, SNS/SQS, CloudWatch. • Strong application development experience in Scala and Python. • Strong SQL development experience, preferably with Amazon Redshift. • Hands-on experience with HR Data Warehousing (HR DWH) and workforce analytics. • Experience integrating and modeling data from HR systems such as Workday, SAP HCM, Oracle HCM, payroll, benefits, recruiting, and learning systems. • Experience with ETL/ELT frameworks and best practices. • Strong background in AWS cloud services: Lambda, Glue, S3, EMR, SNS, SQS, CloudWatch, Redshift. • Expertise in SQL and relational databases such as Oracle, MySQL, PostgreSQL. • Experience with Snowflake is an added advantage. • Proficiency in Python for data engineering, automation, and orchestration. • Experience with shell scripting in Linux/Unix environments. • Experience with Big Data technologies: Hadoop, Spark. • Financial Services experience required. • Nice to have: Knowledge of Machine Learning models, regression, and validation techniques. • Nice to have: Experience with People Analytics, HR reporting, and workforce metrics