Data Engineer

Job Title: Data Engineer Experience: 4–10 Years Location: Remote Job Poster: Sai Vardhan Bolla Job Overview We are looking for a highly skilled Data Engineer to design, build, and optimize scalable data pipelines and platforms. The ideal candidate will work closely with analytics, data science, and product teams to enable reliable, high-performance data solutions across the organization. Key Responsibilities • Design, develop, and maintain end-to-end data pipelines for batch and real-time processing • Build and optimize data lakes and data warehouses for structured and unstructured data • Implement streaming data solutions using event-driven architectures • Ensure data quality, reliability, and observability across pipelines • Optimize query performance and storage strategies for large-scale datasets • Collaborate with data scientists to support feature engineering and model deployment • Implement data governance, lineage, and access controls • Automate data workflows and monitoring using orchestration tools Required Skills & Experience • Strong experience with Apache Spark (including Spark Structured Streaming) • Hands-on expertise with Kafka / Pulsar for real-time data ingestion • Advanced SQL skills and experience with distributed query engines (Trino / Presto / Druid) • Experience building data lakehouse architectures using Delta Lake / Apache Iceberg / Hudi • Proficiency in Python and Scala for data engineering workloads • Experience with columnar storage formats (Parquet, ORC, Avro) • Strong understanding of data modeling for analytics and OLAP systems • Experience with workflow orchestration (Apache Airflow / Dagster) • Knowledge of cloud-native data platforms (AWS, Azure, or GCP) Preferred / Added Advantage • Experience with Change Data Capture (CDC) tools such as Debezium • Exposure to stream-table duality and real-time analytics • Experience implementing data mesh or domain-oriented data platforms • Knowledge of metadata management and data lineage tools Please keep in mind: • All applications are reviewed thoroughly by our Recruitment team. • You will receive a response only if your profile is shortlisted for the next steps. • Response times may be delayed during periods of bulk or high-volume applications. • Make sure your resume is updated and includes your latest experience and contact details. • If not shortlisted, your profile will be stored in our database for future opportunities.