GCP Data Engineer
⭐ - Featured Role | Apply direct with Data Freelance Hub
This role is for a GCP Data Engineer on a long-term contract, offering remote work. Key skills include recent GCP experience, Apache Hudi, Hadoop, Hive, Spark, Airflow, and proficiency in Python and Scala. Data engineering experience is essential.
🌎 - Country
United States
💱 - Currency
$ USD
-
💰 - Day rate
-
🗓️ - Date discovered
June 5, 2025
🕒 - Project duration
Unknown
-
🏝️ - Location type
Remote
-
📄 - Contract type
Unknown
-
🔒 - Security clearance
Unknown
-
📍 - Location detailed
United States
-
🧠 - Skills detailed
#Spark (Apache Spark) #Data Processing #Python #"ETL (Extract #Transform #Load)" #BigQuery #Cloud #Schema Design #Monitoring #Hadoop #Big Data #Storage #Apache Airflow #Scala #Perl #Programming #Data Storage #Dataflow #Data Engineering #Datasets #GCP (Google Cloud Platform) #Airflow #Scripting #Data Pipeline #Java #Data Quality
Role description
GCP Data Engineer
Location: Remote
Duration: Long Term Contract
Mandatory Areas
Must have skills.
• Overall Experience level:
• Recent GCP experience
• Apache Hudi for big data processing and storage
• Hands-on experience Hadoop, Hive or Spark, Airflow or a workflow orchestration solution
• Experience with programming languages: Python, Java, Scala, etc.
• Experience with scripting languages: Perl, Shell, etc.
Responsibilities
• Design, develop, and maintain robust and scalable ETL workflows and data pipelines using tools like Hive, Spark, and Airflow.
• Implement and manage data storage and processing solutions using Apache Hudi and BigQuery.
• Develop and optimize data pipelines for structured and unstructured data in GCP environments, leveraging GCS for data storage.
• Write clean, maintainable, and efficient code in Scala and Python to process and transform data.
• Ensure data quality, integrity, and consistency by implementing appropriate data validation and monitoring techniques.
• Work with cross-functional teams to understand business requirements and deliver data solutions that drive insights and decision-making.
• Troubleshoot and resolve performance and scalability issues in data processing and pipelines.
• Stay updated with the latest developments in big data technologies and tools and incorporate them into the workflow as appropriate.
Required Skills and Qualifications
• Proven experience as a Data Engineer, preferably in a big data environment.
• Expertise in Hive, Spark, and Apache Hudi for big data processing and storage.
• Hands-on experience with BigQuery and Google Cloud Platform (GCP) services such as GCS, Dataflow, and Pub/Sub.
• Strong programming skills in Scala and Python, with experience in building data pipelines and ETL processes.
• Proficiency with workflow orchestration tools like Apache Airflow.
• Solid understanding of data warehousing concepts, data modelling, and schema design.
• Knowledge of distributed systems and parallel processing.
• Strong problem-solving skills and ability to work with large datasets in a fast-paced environment
GCP Data Engineer
Location: Remote
Duration: Long Term Contract
Mandatory Areas
Must have skills.
• Overall Experience level:
• Recent GCP experience
• Apache Hudi for big data processing and storage
• Hands-on experience Hadoop, Hive or Spark, Airflow or a workflow orchestration solution
• Experience with programming languages: Python, Java, Scala, etc.
• Experience with scripting languages: Perl, Shell, etc.
Responsibilities
• Design, develop, and maintain robust and scalable ETL workflows and data pipelines using tools like Hive, Spark, and Airflow.
• Implement and manage data storage and processing solutions using Apache Hudi and BigQuery.
• Develop and optimize data pipelines for structured and unstructured data in GCP environments, leveraging GCS for data storage.
• Write clean, maintainable, and efficient code in Scala and Python to process and transform data.
• Ensure data quality, integrity, and consistency by implementing appropriate data validation and monitoring techniques.
• Work with cross-functional teams to understand business requirements and deliver data solutions that drive insights and decision-making.
• Troubleshoot and resolve performance and scalability issues in data processing and pipelines.
• Stay updated with the latest developments in big data technologies and tools and incorporate them into the workflow as appropriate.
Required Skills and Qualifications
• Proven experience as a Data Engineer, preferably in a big data environment.
• Expertise in Hive, Spark, and Apache Hudi for big data processing and storage.
• Hands-on experience with BigQuery and Google Cloud Platform (GCP) services such as GCS, Dataflow, and Pub/Sub.
• Strong programming skills in Scala and Python, with experience in building data pipelines and ETL processes.
• Proficiency with workflow orchestration tools like Apache Airflow.
• Solid understanding of data warehousing concepts, data modelling, and schema design.
• Knowledge of distributed systems and parallel processing.
• Strong problem-solving skills and ability to work with large datasets in a fast-paced environment
