Data Engineer

We are seeking a skilled and experienced Databricks Data Engineer to join our team. As a Databricks Data Engineer, you will be responsible for designing, developing, and implementing data engineering solutions using the Databricks platform to support data ingestion, transformation, performance reporting, analytics, sales incentive compensation processing, and enterprise data pipeline processes. Responsibilities: • Collect data from different systems and bring it into Databricks. • Clean, transform, and organize data so it's accurate and usable. • Build automated data pipelines that move data from source systems to reports and analytics tools. • Support Sales Incentive Compensation (SIC) processes such as: • Tracking sales performance • Managing quotas • Calculating commissions and bonuses • Creating payout reports for sales reps • Work closely with Sales Operations and Finance teams to understand compensation rules and translate them into data logic. • Troubleshoot data issues, improve performance, and ensure reports are reliable. Design and maintain the overall Databricks architecture and data models. Must Have Skills: • 7 years of experience in data engineering, with a minimum of 3 years hands-on experience building and managing Databricks solutions • Extensive experience with Databricks platform components: Delta Lake, Delta Live Tables, Unity Catalog and Databricks Workflows. • Strong proficiency in Python and/or Scala for data engineering; SQL expertise required. • Strong experience with core data engineering practices including data ingestion, transformation (ETL/ELT), data modeling, and pipeline orchestration. • Functional and technical experience with Sales Incentive Compensation (SIC) — including incentive plan design, quota allocation, attainment tracking, commission calculations, and payout processing. • Ability to understand complex incentive compensation rules and model them accurately in data pipelines and transformation logic. • Experience integrating Databricks with cloud platforms (Azure, AWS, or GCP) and data sources such as Azure Data Factory, Event Hubs, Kafka, or equivalent. • Passion and ability to create holistic, end-to-end, integrated data solutions.