Data Engineer - IV

Seeking an experienced Data Analytics/AI Engineer to join our team to: • Translating complex business problems into data analytics solutions • Create data products in Databricks while leveraging AI-powered tools to enhance workflow automation, perform pattern recognition, analyze and generate content • Collaborate with business users and engineers to align data analytics solutions with Business Required Technical Skills • Databricks Proficiency: Hands-on experience with AWS Databricks platform; knowledge of Spark performance and cost-optimization technics (partitioning, clustering, caching) • Advanced SQL and Spark Skills: Proficiency in writing complex SQL queries and Spark code (Python/PySpark) for data manipulation, transformation, aggregation, and analysis tasks within Databricks notebooks • GenAI and LLMs: Prompt engineering, RAG, experience leveraging Large Language Models (LLMs); familiarity with tools such as Amazon SageMaker and Bedrock • Data Engineering & Processing: Experience building ETL pipelines and working with big data frameworks • System Integration and Deployment: Deploy AI models and integrate them with existing systems via APIs • Data Visualization: Using data visualization tools (Tableau, Power BI) to create interactive dashboards and communicate insights • Cloud Computing and MLOps: Proficiency in deploying models using AWS; familiarity with Domino Data Lab, GitLab CI/CD • Data Governance and Security: Understanding of data governance principles and implementing security measures to ensure data integrity, confidentiality, and compliance within the centralized data lake environmentgoals Quals-- Top Skills • High proficiency in Python and SQL • ETL experience with Databricks using PySpark • Data processing and analytics/AI in Databricks • Strong understanding of algorithms, data structures, statistics, and mathematics • GenAI, LLM deployment and monitoring