Data Architect

Role: Data Architect Location: Milpitas, CA Job Type: Hybrid Job Description: Position Overview We are seeking a hands-on Data Architect with deep technical expertise in Azure and Databricks to design and build enterprise data platforms and products. This role requires a unique blend of architectural vision and hands-on implementation skills - you'll design solutions AND write code to bring them to life. Key Responsibilities Architecture & Design • Design scalable, secure, and cost-effective data architectures on Azure and Databricks • Define data platform standards, patterns, and best practices • Create reference architectures for data lakehouse implementations • Design end-to-end data product architectures from ingestion to consumption Hands-On Implementation • Build proof-of-concepts and reference implementations personally • Write production-grade Spark code (PySpark, Scala, SQL) • Implement complex data pipelines and transformations • Configure and optimize Databricks environments and Azure services • Lead by example through hands-on technical contributions Data Platform Development • Design and implement data lakehouse architectures using Delta Lake • Architect streaming and batch processing solutions at scale • Build data governance frameworks using Unity Catalog and Azure Purview • Design data quality and observability solutions Technical Leadership • Provide technical guidance to data engineering teams • Conduct architecture reviews and code reviews • Mentor engineers on Databricks and Azure best practices • Evaluate and recommend new technologies and tools Stakeholder Engagement • Translate business requirements into technical architectures • Present architectural designs to technical and business stakeholders • Collaborate with security, infrastructure, and platform teams • Drive architectural decisions and alignment across teams Required Qualifications Technical Expertise • 8+ years of data engineering/architecture experience • Deep hands-on expertise in Databricks (Delta Lake, Unity Catalog, Medallion architecture, cluster optimization) • Extensive Azure experience (Data Lake Storage, Data Factory, Synapse Analytics, Event Hubs, Purview) • Expert-level Apache Spark skills (performance tuning, optimization, architecture) • Strong programming skills in PySpark, Scala, and/or Python • Advanced SQL knowledge and query optimization Architecture Experience • Proven track record designing and implementing large-scale data platforms • Experience with data lakehouse, data mesh, or modern data stack architectures • Deep understanding of data modeling techniques (dimensional, normalized, data vault) • Knowledge of real-time streaming architectures (Kafka, Event Hubs, Structured Streaming) • Experience with data governance, security, and compliance frameworks Technical Depth • THIS IS NOT A PURE ARCHITECTURE ROLE - Daily coding and hands-on work required • Ability to debug complex Spark jobs and optimize performance • Experience with infrastructure as code (Terraform, ARM templates) • Understanding of DevOps practices and CI/CD for data pipelines • Knowledge of data quality, testing, and observability frameworks Preferred Qualifications • Azure Solutions Architect Expert certification • Databricks Certified Data Engineer Professional or Solutions Architect • Experience with machine learning platforms and MLOps • Background in cloud migration and modernization projects • Knowledge of FinOps and cloud cost optimization • Experience with multiple cloud platforms (AWS, GCP) • Previous consulting or advisory experience Technical Skills Summary Must Have: • Azure (comprehensive knowledge across data services) • Databricks (expert-level implementation and optimization) • Apache Spark (architecture, tuning, advanced optimization) • PySpark and/or Scala (production-level coding) • SQL (complex queries, performance tuning) • Data architecture patterns and best practices What Makes This Role Unique This is NOT a traditional "ivory tower" architect role. We need someone who: • Designs architectures in the morning and implements them in the afternoon • Gets into the weeds with Spark optimization and cluster tuning • Can architect a solution AND build a working prototype • Debugs production issues alongside the engineering team • Writes code daily, not just PowerPoint slides Ideal Candidate Profile: • Senior engineer who's ready to take on architecture responsibilities • Or architect who refuses to stop coding and building • Believes the best way to validate an architecture is to implement it • Values pragmatic solutions over theoretical perfection Key Differentiators • 70% hands-on technical work, 30% architecture/design • Direct contribution to codebase and data products • Technical leadership through example, not delegation • Ownership of both design and implementation Education • Bachelor's degree in Computer Science, Engineering, Mathematics, or related field • Master's degree or equivalent experience preferred Best Regards, Nitin (Nishant) Email: Nishant.s@erostechnologies.com