Senior Data Engineer – GenAI Engineering

Position Overview We are seeking a Senior Data Engineer – GenAI Engineering to design, build, and optimize scalable data platforms that power Generative AI, Machine Learning, and Advanced Analytics solutions. This role will be responsible for developing modern data pipelines, managing large-scale structured and unstructured datasets, enabling LLM applications, and supporting AI model development through robust data engineering practices. The ideal candidate combines strong expertise in Data Engineering, Cloud Platforms, Big Data Technologies, and AI/ML data workflows with the ability to collaborate across Data Science, MLOps, Software Engineering, and Product teams. Key Responsibilities Data Engineering & Platform Development • Design, develop, and maintain scalable batch and real-time data pipelines. • Build data ingestion frameworks for structured, semi-structured, and unstructured datasets. • Develop and optimize data lakes, data warehouses, and lakehouse architectures. • Implement data quality, lineage, governance, and monitoring solutions. • Create reusable data services supporting AI/ML and GenAI workloads. GenAI & AI Data Enablement • Build data pipelines supporting LLM training, fine-tuning, RAG, and vector search applications. • Prepare, clean, transform, and validate large datasets for AI model consumption. • Develop frameworks for document processing, embedding generation, metadata management, and semantic search. • Support retrieval systems using vector databases and knowledge repositories. • Collaborate with AI/ML teams to improve model performance through high-quality data engineering practices. Cloud & Big Data Engineering • Develop cloud-native data solutions using AWS, Azure, or GCP. • Optimize distributed data processing workloads using Spark and related technologies. • Build scalable ETL/ELT frameworks and workflow orchestration pipelines. • Implement data security, access controls, and compliance standards. Collaboration & Technical Leadership • Partner with Data Scientists, ML Engineers, Software Engineers, and Product teams. • Drive architecture discussions and recommend scalable data solutions. • Mentor junior engineers and establish engineering best practices. • Participate in code reviews, design reviews, and technical planning sessions. Required Qualifications Experience • 7+ years of experience in Data Engineering or Big Data Engineering. • 2+ years of experience supporting AI/ML or Generative AI initiatives. • Experience building enterprise-scale data platforms and pipelines. Technical Skills • Strong proficiency in Python and SQL. • Experience with Apache Spark, Databricks, PySpark, Kafka, Airflow, or similar technologies. • Expertise in data modeling, ETL/ELT development, and data warehousing concepts. • Experience with cloud platforms (AWS, Azure, or GCP). • Familiarity with REST APIs, microservices, and distributed systems. • Experience with Git-based development workflows and CI/CD practices. GenAI Skills • Understanding of LLMs, RAG architectures, embeddings, and vector databases. • Experience working with OpenAI, Azure OpenAI, Anthropic, or similar GenAI platforms. • Knowledge of vector databases such as Pinecone, Weaviate, Chroma, FAISS, or Elasticsearch/OpenSearch. • Experience processing unstructured data including documents, PDFs, text, and knowledge repositories. Preferred Qualifications • Experience with MLOps tools such as MLflow, Kubeflow, or SageMaker. • Knowledge of LangChain, LlamaIndex, Semantic Kernel, or similar frameworks. • Experience with Kubernetes and containerized deployments. • Exposure to data governance and enterprise security frameworks. • Experience building enterprise GenAI applications in production environments.