Data Engineer

About the job MeeBoss is sharing this active opportunity on behalf of the hiring company. This is a contract Data Engineer role focused on building and optimizing data pipelines using Google Cloud Platform (GCP) technologies. Job title Data Engineer Company Caspex is a leading international consulting and technology services firm with over 15 years of experience. The company specializes in delivering transformative technology solutions, including AI-driven solutions, big data analytics, open-source development, and cloud applications. Caspex has 201-1000 employees and operates within the Information Technology and Services industry. Location Sunnyvale, CA (Hybrid) Compensation $60-70/hour Why this role This is a 6-12+ month contract position requiring immediate availability and the ability to deliver with minimal ramp-up time. The role involves working directly with stakeholders to translate data requirements into pipeline development and data solutions. What you will do • Manage and manipulate huge datasets in the order of terabytes (TB). • Build batch data pipelines using big data technologies such as Hadoop, Apache Spark (Scala preferred), and Apache Hive on cloud platforms, with a preference for GCP. • Ensure pipeline optimization, SLA adherence, and fault tolerance. • Build idempotent workflows using orchestrators like Automic, Airflow, or Luigi. • Write SQL to analyze, optimize, and profile data, preferably in BigQuery or SPARK SQL. • Design data schemas that accommodate the evolution of data sources and facilitate seamless data joins. • Collaborate with stakeholders to understand data requirements and translate them into pipeline development. • Identify and resolve issues during data integration and schema evolution processes. • Work effectively in a team environment, coordinating efforts between different stakeholders. What we are looking for • Proficiency in managing and manipulating large datasets (terabytes). • Expertise in big data technologies (Hadoop, Apache Spark, Apache Hive) and cloud platforms (GCP preferred, AWS, Azure). • Experience building idempotent workflows using orchestrators (Automic, Airflow, Luigi). • Strong SQL skills for data analysis and optimization (BigQuery or SPARK SQL). • Strong data modeling skills for designing evolving data schemas. • Ability to work directly with stakeholders to define and implement data solutions. • Strong analytical and problem-solving skills. • Ability to move at a rapid pace with quality and minimal ramp-up time. • Effective communication and collaboration skills. • Bachelor’s degree. • Proficiency in English. • Technical stack experience: Google Cloud, HDFS, SPARK, Scala, Python (optional), Automic/Airflow, BigQuery, Kafka, API, Druid. About MeeBoss MeeBoss helps job seekers and hiring teams make direct, relevant career connections. How to apply Apply through the MeeBoss link on this LinkedIn post.