Cloud Data Engineer - Boston, MA (Onsite)

Cloud Data Engineer with Databricks and PySpark Expertise Location: Austin, TX (Onsite) Duration: 6 months+ Contract Contract - C2C/W2/1099 Rate: $55/hr to $60/hr (depending on experience and engagement type) Job Summary: We are seeking a highly skilled Data Engineer with strong expertise in Databricks and PySpark to design, develop, and optimize large-scale data integration and transformation solutions. The ideal candidate will have proven experience in architecting Lakehouse-based solutions, integrating data from multiple enterprise systems, and implementing best practices in ETL/ELT pipelines, data modeling, and orchestration frameworks. Key Responsibilities: • Architect, design, and implement scalable data processing solutions using Databricks, PySpark, and Delta Lake. • Develop and manage ETL/ELT pipelines integrating data from various sources including Salesforce, MDM systems, and other enterprise applications. • Design, configure, and optimize Databricks clusters, jobs, and workflows for performance and cost efficiency. • Integrate Databricks with data governance tools (e.g., Collibra) and ensure compliance with data management policies. • Implement and maintain Apache Airflow DAGs for orchestration, monitoring, and automation of data pipelines. • Work with AWS services including S3, EC2, Lambda, and IAM for data storage, compute, and access management. • Perform complex data transformations and ensure data quality using SQL and Python. • Design and maintain Lakehouse architectures following data warehousing best practices. • Collaborate with cross-functional teams (data scientists, analysts, business users) to deliver data-driven insights. • Ensure system reliability, security, and performance tuning for data platforms. Required Skills & Experience: • 5–10 years of hands-on experience in data engineering, with at least 3+ years in Databricks and PySpark. • Strong understanding of Delta Lake, Lakehouse architecture, and data modeling principles. • Experience integrating Databricks with Salesforce, MDM platforms, and other enterprise applications. • Proficiency in SQL (complex queries, optimization) and Python for data processing and automation. • Strong experience with Apache Airflow (DAG creation, scheduling, and monitoring). • Hands-on experience with AWS services (S3, EC2, Lambda, IAM). • Familiarity with data catalog and governance tools such as Collibra. • Excellent problem-solving skills, attention to detail, and ability to work in a collaborative environment.