Data Engineer

About the job MeeBoss is sharing this active opportunity on behalf of the hiring company. This is a contract Data Engineer role based in Sunnyvale, CA, with a hybrid work arrangement. The position focuses on building and optimizing batch data pipelines using GCP and big data technologies, managing terabyte-scale datasets, and collaborating with stakeholders to deliver high-quality data solutions. Job title Data Engineer Company MeeBoss Location Sunnyvale, CA (Hybrid) Compensation $60-70/hour Why this role • Opportunity to work with a leading international consulting and technology services firm with over 15 years of experience. • Engage in transformative technology solutions involving AI-driven solutions, big data analytics, and cloud applications. • Collaborate directly with stakeholders to understand data requirements and drive long-term growth initiatives. What you will do • Manage and manipulate huge datasets in the order of terabytes (TB). • Build batch data pipelines using big data technologies like Hadoop, Apache Spark (Scala preferred), Apache Hive, or similar frameworks on the cloud (GCP preferred, AWS, Azure etc.), with a focus on optimization, SLA adherence, and fault tolerance. • Build idempotent workflows using orchestrators like Automic, Airflow, Luigi etc. • Write SQL to analyze, optimize, and profile data, preferably in BigQuery or SPARK SQL. • Design data schemas that accommodate the evolution of data sources and facilitate seamless data joins across various datasets. • Work directly with stakeholders to understand data requirements and translate them into pipeline development or data solution work. • Identify and resolve issues that may arise during the data integration and schema evolution process. • Move at a rapid pace with quality and deliver with minimal ramp-up time. • Communicate and collaborate effectively in a team environment, coordinating efforts between different stakeholders. What we are looking for • Proficiency in managing and manipulating huge datasets (terabytes). • Expertise in big data technologies (Hadoop, Apache Spark, Apache Hive) and cloud platforms (GCP preferred). • Experience building idempotent workflows using orchestrators (Automic, Airflow, Luigi). • Strong SQL skills for data analysis and optimization (BigQuery or SPARK SQL). • Strong data modeling skills for designing evolving schemas. • Ability to work directly with stakeholders to translate requirements into solutions. • Strong analytical and problem-solving skills. • Ability to work at a rapid pace with minimal ramp-up time. • Effective communication and collaboration skills. Benefits • Exposure to AI-driven solutions, big data analytics, open-source development, and cloud/mobile applications. • Opportunity to work with a multinational team of experts in a purpose-driven innovation environment. About MeeBoss MeeBoss helps job seekers and hiring teams make direct, relevant career connections.