Data Engineer

Job Title: Data Engineer (GCP) Company: Caspex Location: Sunnyvale, CA (Hybrid – Local to CA required) Employment Type: Contract Duration: 6–12+ Months Salary: $60–70/hour Education Level: Bachelor’s About Caspex Caspex is a leading international consulting and technology services firm. With over 15 years of experience, our multinational team delivers transformative, client-focused technology solutions to businesses worldwide. We specialize in AI-driven solutions, big data analytics, open-source development, and cloud and mobile applications, helping organizations streamline operations and drive long-term growth. Role Overview We are seeking a Data Engineer with expertise in Google Cloud Platform (GCP) to join our client’s team. The ideal candidate will have strong proficiency in managing large-scale datasets and building optimized, fault-tolerant data pipelines. Key Responsibilities • Manage and manipulate huge datasets (terabytes in scale). • Build batch data pipelines using big data technologies such as Hadoop, Apache Spark (Scala preferred), Apache Hive, or similar cloud frameworks (GCP preferred; AWS/Azure experience also considered). • Focus on pipeline optimization, SLA adherence, and fault tolerance. • Build idempotent workflows using orchestrators such as Automic, Airflow, or Luigi. • Write and optimize SQL for data analysis and profiling, preferably in BigQuery or Spark SQL. • Design data schemas that accommodate data source evolution and facilitate seamless data joins. • Collaborate directly with stakeholders to understand data requirements and translate them into pipeline development and data solutions. • Identify and resolve issues related to data integration and schema evolution using strong analytical and problem-solving skills. • Deliver high-quality work with minimal ramp-up time in a fast-paced environment. • Communicate effectively and collaborate with team members and stakeholders. Required Skills & Qualifications • Proficiency in GCP. • Experience with Hadoop, Apache Spark, Apache Hive, or similar big data frameworks. • Experience with workflow orchestrators (Automic, Airflow, Luigi, etc.). • Strong SQL skills (BigQuery or Spark SQL preferred). • Strong data modeling skills. • Ability to work with stakeholders and translate requirements into technical solutions. • Strong analytical and problem-solving abilities. • Excellent communication and collaboration skills. Nice to Have • Experience building complex near real-time (NRT) streaming data pipelines using Apache Kafka, Spark Streaming, or Kafka Connect. • Understanding of REST APIs and technologies such as Apache Druid, Redis, Elastic Search, or GraphQL. • Knowledge of API contracts, telemetry building, and stress testing. • Experience developing reports/dashboards using Looker or Tableau. • Experience in the eCommerce domain. Tech Stack • Google Cloud Platform (GCP) • HDFS • Spark • Scala • Python (optional) • Automic / Airflow • BigQuery • Kafka • APIs • Druid How to Apply If interested, please send the following details: 1. A copy of your resume 1. Your contact details 1. Your availability 1. A good time to connect Company Size: 201–1000 employees Industry: Information Technology and Services