AI Engineer with Databricks

AI Engineer with Databricks Location: King of Prussia PA, (onsite) Note- (Only Locals No relocation) We are seeking a highly skilled AI Engineer with proven experience in developing, deploying, and optimizing AI models using the Databricks AI platform. The ideal candidate will have a strong background in machine learning, distributed data processing, and will be a champion of Databricks platform. Responsibilities • Generative AI: Experience of LLM-based solutions (LlamaIndex, Lang Chain, RAG pipelines, or similar frameworks). Ability to integrate GenAI and Agentic AI into business workflows. • Design and implement end-to-end Generative AI solutions on Databricks, leveraging Unity Catalog, MLflow, Delta Lake, and Vector Search • Design, build, and deploy large-scale AI/ML models using the Databricks environment. • Implement data validation, lineage, and monitoring using Delta Live Tables and Unity Catalog. • Leverage Databricks’ data engineering workflows for feature engineering, model training, and evaluation. • Optimize training pipelines for efficiency, scalability, and accuracy. • Integrate AI models into production systems using APIs and microservices. • Build reusable ML pipelines using Databricks Repos, MLflow, and Feature Store. • Implement robust testing, monitoring, and retraining protocols for deployed models. • Ensure adherence to compliance, security, and performance standards. • Stay updated on advancements in AI frameworks, distributed computing, and Databricks platform updates. Required Skills and Qualifications • Bachelor’s or Master’s degree in Computer Science, Data Science, or related field. • Certified Databricks Certified Generative AI Engineer • Proven experience developing AI solutions on Databricks. • Strong knowledge of Python, PySpark, MLflow, Spark and Databricks Notebooks. • Strong understanding and knowlege of Databricks platform features such as Unity Catalog, DLT, MosaicAI, Data Assets Bundles, etc. • Experience with Transformer-based models, generative AI, and Databricks pipelines. • Proficiency in integrating AI models with cloud-native architectures (AWS, Azure, or GCP). • Solid understanding of MLOps practices, Data Assets bundles (CI/CD), and containerization (Docker, Kubernetes) on Databricks Platform. • Familiarity with vector databases, embeddings, and retrieval-augmented generation (RAG). • Strong problem-solving, analytical thinking, and communication skills.