Data Architect

We are seeking a highly skilled Databricks Architect to lead the design and implementation of next-generation data platforms built on the Lakehouse paradigm. This role goes beyond pipeline development—you will own the Databricks platform architecture end-to-end, driving scalability, governance, performance, and cost optimization across enterprise data ecosystems. Databricks Platform Architecture • Architect and implement enterprise-scale Databricks environments (dev/test/prod) • Define workspace strategy, cluster policies, and job orchestration frameworks • Design secure, scalable Lakehouse architecture using Delta Lake Data Engineering & Processing • Build and optimize high-performance data pipelines using Apache Spark (PySpark / Scala) • Implement Medallion Architecture (Bronze, Silver, Gold) • Develop batch and real-time streaming pipelines using Structured Streaming Governance, Security & Compliance • Implement fine-grained access control using Unity Catalog • Define enterprise-wide data governance, lineage, and auditing frameworks • Ensure compliance with security and regulatory standards Performance & Cost Optimization • Optimize workloads using partitioning, caching, and Photon engine • Design cost-efficient cluster strategies (autoscaling, spot instances, DBU optimization) • Monitor and improve query and pipeline performance at scale DevOps & Automation • Implement CI/CD pipelines for Databricks using Azure DevOps, Jenkins, or GitHub Actions • Enable Infrastructure as Code using Terraform or equivalent • Standardize reusable frameworks and engineering best practices Cloud Integration • Architect Databricks solutions on AWS / Azure / GCP • Integrate with cloud-native services (AWS S3/Glue, Azure ADLS/ADF, etc.) Must-Have Skills • Advanced proficiency in SQL and Python • Hands-on experience with Delta Lake, Medallion Architecture, and Unity Catalog • Strong experience with at least one cloud platform (AWS / Azure / GCP) • Experience with CI/CD tools (Azure DevOps, Jenkins, GitHub Actions) • Knowledge of workflow orchestration tools (Airflow or equivalent) Good to Have • Databricks certifications (Associate / Professional) • Experience with Databricks SQL & dashboards • Exposure to ML pipelines (MLflow) • Experience with Kafka or event-driven architectures • Domain experience in Finance / Retail / Healthcare What Makes You a Great Fit • Strong architectural mindset with ability to design scalable data platforms • Deep understanding of performance tuning and cost optimization • Ability to balance technical depth with business impact • Excellent communication and stakeholder management skills