Thrive IT Systems Ltd
AWS Data Engineer
⭐ - Featured Role | Apply direct with Data Freelance Hub
This role is for an AWS Data Engineer with 8–10 years of experience, focused on ETL pipelines and telecom data. Contract length is unspecified, with a competitive pay rate. Key skills include AWS S3, Redshift, and Apache Iceberg.
🌎 - Country
United Kingdom
💱 - Currency
£ GBP
-
💰 - Day rate
Unknown
-
🗓️ - Date
February 10, 2026
🕒 - Duration
Unknown
-
🏝️ - Location
Unknown
-
📄 - Contract
Fixed Term
-
🔒 - Security
Unknown
-
📍 - Location detailed
London Area, United Kingdom
-
🧠 - Skills detailed
#Deployment #Amazon Redshift #Data Catalog #SQL (Structured Query Language) #Data Pipeline #Python #Data Processing #AWS (Amazon Web Services) #Data Engineering #"ACID (Atomicity #Consistency #Isolation #Durability)" #Airflow #Lambda (AWS Lambda) #AWS S3 (Amazon Simple Storage Service) #Monitoring #"ETL (Extract #Transform #Load)" #Data Lifecycle #Kafka (Apache Kafka) #Security #ML (Machine Learning) #IAM (Identity and Access Management) #Metadata #Data Quality #Business Analysis #S3 (Amazon Simple Storage Service) #GIT #CRM (Customer Relationship Management) #Cloud #Debugging #Redshift #Athena #Data Lake #Terraform #Spark (Apache Spark) #Storage #Apache Iceberg #DevOps #Scala #Datasets #Data Architecture #Data Governance #Data Ingestion
Role description
We are seeking an experienced AWS Data Engineer with strong expertise in ETL pipelines, Redshift, Iceberg, Athena, and S3 to support large-scale data processing and analytics initiatives in the telecom domain. The candidate will work closely with data architects, business analysts, and cross-functional teams to build scalable and efficient data solutions supporting network analytics, customer insights, billing systems, and telecom OSS/BSS workflows.
Key Responsibilities
1. Data Engineering & ETL Development
• Design, develop, and maintain ETL/ELT pipelines using AWS-native services (Glue, Lambda, EMR, Step Functions).
• Implement data ingestion from telecom systems like OSS/BSS, CDRs, mediation systems, CRM, billing, network logs.
• Optimize ETL workflows for large-scale telecom datasets (high volume, high velocity).
1. Data Warehousing (Redshift)
• Build and manage scalable Amazon Redshift clusters for reporting and analytics.
• Create and optimize schemas, tables, distribution keys, sort keys, and workload management.
• Implement Redshift Spectrum to query data in S3 using external tables.
1. Data Lake & Iceberg
• Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations.
• Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark.
• Ensure high performance for petabyte-scale telecom datasets (CDRs, tower logs, subscriber activity).
1. Querying & Analytics (Athena)
• Develop and optimize Athena queries for operational and analytical reporting.
• Integrate Athena with S3/Iceberg for low-cost, serverless analytics.
• Manage Glue Data Catalog integrations and table schema management.
1. Storage (S3) & Data Lake Architecture
• Design secure, cost-efficient S3 data lake structures (bronze/silver/gold zones).
• Implement data lifecycle policies, versioning, and partitioning strategies.
• Ensure data governance, metadata quality, and security (IAM, Lake Formation).
1. Telecom Domain Expertise
• Understand telecom-specific datasets such as:
o CDR, xDR, subscriber data
o Network KPIs (4G/5G tower logs)
o Customer lifecycle & churn data
o Billing & revenue assurance
• Build models and pipelines to support network analytics, customer 360, churn prediction, fraud detection, etc.
1. Performance Optimization & Monitoring
• Tune Spark/Glue jobs for performance and cost.
• Monitor Redshift/Athena/S3 efficiency and implement best practices.
• Perform data quality checks and validation across pipelines.
1. DevOps & CI/CD (Preferred)
• Use Git, CodePipeline, Terraform/CloudFormation for infrastructure and deployments.
• Automate pipeline deployment and monitoring.
Required Skills
• 8–10 years’ experience in data engineering.
• Strong hands-on experience with:
o AWS S3, Athena, Glue, Redshift, EMR/Spark
o Apache Iceberg
o Python/SQL
• Experience in telecom data pipelines and handling large-scale structured/semi-structured data.
• Strong problem-solving, optimization, and debugging skills.
Good to Have Skills
• Knowledge of AWS Lake Formation, Kafka/Kinesis, Airflow, or Delta/Apache Hudi.
• Experience with ML workflows in telecom (churn, network prediction).
• Exposure to 5G network data models.
Recruiter's Email: shikharsharma@thriveitsystems.com
We are seeking an experienced AWS Data Engineer with strong expertise in ETL pipelines, Redshift, Iceberg, Athena, and S3 to support large-scale data processing and analytics initiatives in the telecom domain. The candidate will work closely with data architects, business analysts, and cross-functional teams to build scalable and efficient data solutions supporting network analytics, customer insights, billing systems, and telecom OSS/BSS workflows.
Key Responsibilities
1. Data Engineering & ETL Development
• Design, develop, and maintain ETL/ELT pipelines using AWS-native services (Glue, Lambda, EMR, Step Functions).
• Implement data ingestion from telecom systems like OSS/BSS, CDRs, mediation systems, CRM, billing, network logs.
• Optimize ETL workflows for large-scale telecom datasets (high volume, high velocity).
1. Data Warehousing (Redshift)
• Build and manage scalable Amazon Redshift clusters for reporting and analytics.
• Create and optimize schemas, tables, distribution keys, sort keys, and workload management.
• Implement Redshift Spectrum to query data in S3 using external tables.
1. Data Lake & Iceberg
• Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations.
• Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark.
• Ensure high performance for petabyte-scale telecom datasets (CDRs, tower logs, subscriber activity).
1. Querying & Analytics (Athena)
• Develop and optimize Athena queries for operational and analytical reporting.
• Integrate Athena with S3/Iceberg for low-cost, serverless analytics.
• Manage Glue Data Catalog integrations and table schema management.
1. Storage (S3) & Data Lake Architecture
• Design secure, cost-efficient S3 data lake structures (bronze/silver/gold zones).
• Implement data lifecycle policies, versioning, and partitioning strategies.
• Ensure data governance, metadata quality, and security (IAM, Lake Formation).
1. Telecom Domain Expertise
• Understand telecom-specific datasets such as:
o CDR, xDR, subscriber data
o Network KPIs (4G/5G tower logs)
o Customer lifecycle & churn data
o Billing & revenue assurance
• Build models and pipelines to support network analytics, customer 360, churn prediction, fraud detection, etc.
1. Performance Optimization & Monitoring
• Tune Spark/Glue jobs for performance and cost.
• Monitor Redshift/Athena/S3 efficiency and implement best practices.
• Perform data quality checks and validation across pipelines.
1. DevOps & CI/CD (Preferred)
• Use Git, CodePipeline, Terraform/CloudFormation for infrastructure and deployments.
• Automate pipeline deployment and monitoring.
Required Skills
• 8–10 years’ experience in data engineering.
• Strong hands-on experience with:
o AWS S3, Athena, Glue, Redshift, EMR/Spark
o Apache Iceberg
o Python/SQL
• Experience in telecom data pipelines and handling large-scale structured/semi-structured data.
• Strong problem-solving, optimization, and debugging skills.
Good to Have Skills
• Knowledge of AWS Lake Formation, Kafka/Kinesis, Airflow, or Delta/Apache Hudi.
• Experience with ML workflows in telecom (churn, network prediction).
• Exposure to 5G network data models.
Recruiter's Email: shikharsharma@thriveitsystems.com
