Data Engineer

About the Role We are seeking an experienced Data Engineer with strong expertise in modern data engineering practices and hands-on experience working with AI tools, machine learning pipelines, and large-scale data platforms. The ideal candidate will have 5+ years of experience designing, building, and optimizing data infrastructure that supports analytics, AI, and machine learning initiatives. This role requires someone who can bridge the gap between data engineering and AI operations by creating scalable data solutions, managing data pipelines, and enabling AI teams with high-quality, reliable datasets. Key Responsibilities • Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data. • Build and optimize data warehouses, data lakes, and cloud-based data platforms. • Develop data architectures that support AI, machine learning, and advanced analytics use cases. • Collaborate with Data Scientists, AI Engineers, and business stakeholders to understand data requirements. • Create and maintain data ingestion frameworks from multiple sources including APIs, databases, cloud storage, and third-party systems. • Ensure data quality, integrity, security, and governance across platforms. • Monitor and optimize data pipeline performance, reliability, and scalability. • Support AI model training and deployment by preparing and managing large datasets. • Implement automation and orchestration workflows using modern data engineering tools. • Build and maintain metadata management, data cataloging, and monitoring solutions. • Troubleshoot production data issues and implement preventive solutions. • Stay updated with emerging AI technologies, data engineering trends, and industry best practices. Required Qualifications • Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field. • 5+ years of professional experience in Data Engineering. • Strong experience with Python and SQL. • Experience building and maintaining ETL/ELT pipelines. • Hands-on experience with cloud platforms such as AWS, Azure, or Google Cloud Platform. • Strong knowledge of data warehousing concepts and modern data architectures. • Experience working with large-scale datasets and distributed processing frameworks. • Strong understanding of database technologies including SQL and NoSQL databases. • Experience with workflow orchestration tools such as Airflow, Prefect, or similar platforms. • Knowledge of version control systems such as Git. • Excellent problem-solving and analytical skills. AI s Machine Learning Requirements (Mandatory) • Hands-on experience supporting AI and Machine Learning projects. • Experience working with AI-powered tools and platforms such as: • OpenAI • Claude • Gemini • LangChain • LlamaIndex • Hugging Face • Vector Databases (Pinecone, Weaviate, ChromaDB, Milvus) • Experience building data pipelines for AI model training and inference workflows. • Understanding of Retrieval-Augmented Generation (RAG) architectures. • Experience preparing, transforming, and managing datasets for LLM-based applications. • Familiarity with AI model monitoring, evaluation, and performance optimization. • Experience integrating AI services and APIs into enterprise applications. Preferred Qualifications • Experience with Spark, Databricks, Kafka, or Snowflake. • Experience supporting MLOps and AI deployment workflows. • Knowledge of containerization technologies such as Docker and Kubernetes. • Experience with real-time streaming data pipelines. • Relevant cloud certifications or data engineering certifications. • Exposure to Generative AI, LLM applications, and AI agent frameworks.