Senior Data Engineer - Data Bricks

Job Title: Senior Data Engineer – Python / Databricks / AWS Location: New Jersey, USA / Boston Work Type: W2 Contract, 5 days/week, Onsite Experience: 10+ Years Candidate Requirements: Local candidates only (no relocation assistance) Job Description: We are seeking a highly experienced Senior Data Engineer to design, develop, and maintain scalable data pipelines and ETL processes. The ideal candidate will have 10+ years of hands-on experience with Python, Databricks, and AWS Glue, and a strong background in data architecture, data lakes, and data warehouses. This is an onsite W2 contract role based in New Jersey, USA. Key Responsibilities: • Design, develop, and maintain scalable data pipelines and ETL workflows using Python, Databricks, and AWS Glue. • Build and optimize data warehouses and data lakes leveraging PySpark and AWS cloud platforms. • Implement data ingestion, transformation, and orchestration workflows using AWS Step Functions, Lambda, and Glue Jobs. • Monitor, troubleshoot, and optimize pipeline performance using AWS CloudWatch and other monitoring tools. • Collaborate with data analysts, data scientists, and business stakeholders to define data requirements and deliver reliable datasets. • Ensure data quality, governance, and security across all systems and workflows. • Automate repetitive data engineering tasks and contribute to building reusable frameworks and templates. • Support continuous improvement by adopting best practices for CI/CD, version control, and infrastructure-as-code (IaC). Qualifications: • 10+ years of experience in Data Engineering or related roles. • Strong expertise in Python, PySpark, Databricks, and AWS services (Glue, Lambda, Step Functions, CloudWatch). • Proven experience building and managing data warehouses and data lakes. • Solid understanding of ETL pipelines, data modeling, and workflow orchestration. • Familiarity with CI/CD, version control (Git), and infrastructure-as-code tools. • Excellent problem-solving skills and ability to collaborate with both technical and non-technical stakeholders. Preferred Skills: • Knowledge of data governance, security best practices, and compliance standards. • Experience in monitoring and optimizing large-scale data pipelines. • Strong communication skills for interacting with business and technical teams. Candidate Notes: • Local profiles only – candidates must already reside in New Jersey or nearby; no relocation assistance will be provided. • Work schedule: 5 days/week onsite at the NJ location.