About Company We are seeking a skilled and detail-oriented Azure Databricks Data Engineer with hands-on experience in building and optimizing large-scale data pipelines. The ideal candidate will be proficient in Databricks, Azure Data Factory, Python/PySpark, and SAP HANA integrations, with a strong focus on performance optimization and scalable architecture. Mandatory Skills & Responsibilities 🔹 Databricks & Delta Lake • Develop scalable data pipelines using Databricks with PySpark or Spark SQL • Efficiently implement Delta Lake for large-scale data processing • Manage data lineage, versioning, and schema evolution within Databricks 🔹 Azure Data Factory (ADF) • Design and manage ETL pipelines using ADF • Orchestrate data flows between Databricks, Blob Storage, and SAP sources • Implement monitoring, error handling, and pipeline performance tuning 🔹 Performance Optimization • Identify and resolve performance bottlenecks in Databricks jobs • Tune Spark configurations and optimize resource usage 🔹 Python & PySpark • Write robust, reusable, and efficient data transformation scripts • Implement custom logic using Python and Spark for data integration 🔹 SAP HANA Expertise • In-depth understanding of SAP HANA architecture and data modeling • Perform large-scale data extraction, transformation, and migration • Integrate SAP data with Azure services and other modern platforms Good To Have Skills 🔸 ETL Tools – SAP Data Services (BODS) • Create and optimize data jobs in SAP BODS • Handle complex SAP-specific mappings and CDC scenarios 🔸 Data Profiling & Validation • Execute data profiling and validation tasks • Reconcile large datasets during data migrations 🔸 Azure Cloud & Networking • Hands-on with Azure compute, storage, and network security services • Troubleshoot VNet, VPN, firewall, and credential issues • Familiarity with IAM, RBAC, and secure secret management Key Attributes • Strong analytical and problem-solving skills • Excellent communication and collaboration across global teams • Detail-oriented with a focus on data quality and efficiency Preferred Certifications (optional) • Microsoft Certified: Azure Data Engineer Associate • Databricks Certified Developer – Data Engineering Skills: data,databricks,data profiling,hana,data validation,pyspark,etl tools,networking,pipelines,sap hana,sparql,sap,spark,azure cloud,azure,sql,python,azure data factory,delta lake,architecture