Big Data Engineer
⭐ - Featured Role | Apply direct with Data Freelance Hub
This role is for a Big Data Engineer with a contract length of "unknown" and a pay rate of "unknown." Key skills include Java, Apache Flink, Kafka, Docker, Kubernetes, and Terraform. Experience in streaming data pipelines and distributed systems is essential.
🌎 - Country
United States
💱 - Currency
$ USD
-
💰 - Day rate
-
🗓️ - Date discovered
September 5, 2025
🕒 - Project duration
Unknown
-
🏝️ - Location type
Unknown
-
📄 - Contract type
Unknown
-
🔒 - Security clearance
Unknown
-
📍 - Location detailed
Texas, United States
-
🧠 - Skills detailed
#Version Control #AWS (Amazon Web Services) #Data Pipeline #GIT #API (Application Programming Interface) #Deployment #Data Engineering #Kafka (Apache Kafka) #Docker #Big Data #Metadata #Programming #Terraform #Java #Kubernetes #Debugging #Automation
Role description
Responsibilities:
• Excellent expertise in Object oriented programming Java and API's.
• Experience in Streaming data pipelines using Apache Flink.
• Experience in distributed systems specially Kafka and its eco systems like Gateways, Publisher, Consumer.
• Good understanding of containerization using Docker, Kubernetes and EKS.
• Good experience of deployment tools like terraform.
• Understanding of version control like Git and CI/CD workflow.
• Understanding of File formats like Parquet and Avro.
• Debugging and trouble shooting.
• Good Communication and excellent collaboration skills to interact with internal customers for collecting requirements.
Deliverables:
• Automation of cluster setup in Machinery for PFA customer. Service deployment (Kafka, Schema Registry, - Metadata.
• Service, Data Platform Gateway, and Kafka Admin Operator) in new AWS accounts.
• Integrate PFA with Kafka Automation, Schema Automation and Journal automation.
• Integrate PFA with Apple Ingest Control Plane.
• Implement the required enhancements to existing streaming pipelines developed using Apache Flink and Java.
• Onboarding topics, Flinks jobs for journaling into iceberg format.
• Monitor and address user requests and issues as they arise.
• Support Kafka infrastructure and Flink streaming pipelines to meet defined SLA’s and SLO’s.
Responsibilities:
• Excellent expertise in Object oriented programming Java and API's.
• Experience in Streaming data pipelines using Apache Flink.
• Experience in distributed systems specially Kafka and its eco systems like Gateways, Publisher, Consumer.
• Good understanding of containerization using Docker, Kubernetes and EKS.
• Good experience of deployment tools like terraform.
• Understanding of version control like Git and CI/CD workflow.
• Understanding of File formats like Parquet and Avro.
• Debugging and trouble shooting.
• Good Communication and excellent collaboration skills to interact with internal customers for collecting requirements.
Deliverables:
• Automation of cluster setup in Machinery for PFA customer. Service deployment (Kafka, Schema Registry, - Metadata.
• Service, Data Platform Gateway, and Kafka Admin Operator) in new AWS accounts.
• Integrate PFA with Kafka Automation, Schema Automation and Journal automation.
• Integrate PFA with Apple Ingest Control Plane.
• Implement the required enhancements to existing streaming pipelines developed using Apache Flink and Java.
• Onboarding topics, Flinks jobs for journaling into iceberg format.
• Monitor and address user requests and issues as they arise.
• Support Kafka infrastructure and Flink streaming pipelines to meet defined SLA’s and SLO’s.
