AWS Data Platform

AWS Data Platform / Platform Engineering Lead Location: Irvine, CA Work Arrangement: Onsite – 5 Days/Week Job Summary We are seeking a highly experienced AWS Data Platform / Platform Engineering Lead to design, build, automate, and support enterprise-scale cloud data platforms and DevOps infrastructure. The ideal candidate will have deep hands-on experience with AWS, Data Platform Engineering, Platform Engineering, Jenkins, CI/CD, Terraform, Infrastructure as Code (IaC), GitHub, SonarQube, ROC/D, and cloud-based DevOps. This role will be responsible for building scalable and secure AWS data-platform infrastructure, developing automated CI/CD pipelines, implementing infrastructure automation, establishing code-quality controls, and improving platform reliability and operational excellence. The candidate should be comfortable working across cloud infrastructure, data platforms, DevOps automation, CI/CD pipelines, infrastructure provisioning, deployment automation, and platform operations. Key Responsibilities AWS Cloud & Data Platform • Architect, build, and manage enterprise AWS data platforms supporting data engineering, analytics, reporting, and business-critical workloads. • Design scalable and highly available cloud-native data platform architectures. • Build and manage AWS infrastructure supporting data ingestion, processing, storage, transformation, and analytics. • Work extensively with AWS services such as S3, Glue, Redshift, Athena, Lambda, EMR, EKS, EC2, IAM, VPC, CloudWatch, KMS, and Secrets Manager. • Develop secure and reusable cloud infrastructure patterns for data engineering and analytics teams. • Implement AWS security, IAM, networking, encryption, access control, and governance. • Support data-platform scalability, reliability, performance, availability, and cost optimization. • Implement monitoring, logging, alerting, and observability for AWS data-platform environments. • Collaborate with Data Engineering, Architecture, Security, Application, and Infrastructure teams. Data Platform Engineering • Build and support modern enterprise data-platform infrastructure. • Support Data Lake / Data Lakehouse architectures and cloud-based data workloads. • Enable data ingestion, batch processing, streaming, transformation, and analytical workloads. • Support technologies such as Databricks, Spark/PySpark, dbt, Airflow, AWS Glue, and Redshift. • Develop standardized infrastructure and deployment patterns for data engineering teams. • Implement data-platform security, governance, monitoring, availability, and operational standards. • Troubleshoot infrastructure and platform issues impacting data pipelines and analytics workloads. Jenkins / CI/CD • Design, develop, and maintain enterprise CI/CD workflows using Jenkins Pipelines. • Build automated pipelines for source control, build, testing, quality validation, infrastructure provisioning, deployment, and release management. • Install, configure, and maintain Jenkins plugins required for Git/GitHub repositories and enterprise CI/CD workflows. • Configure SCM Polling, Git Webhooks, automated triggers, and pipeline orchestration. • Develop reusable Jenkins pipeline frameworks and Shared Libraries. • Integrate Jenkins with GitHub, Terraform, SonarQube, AWS, and other DevOps tools. • Troubleshoot Jenkins pipeline failures, deployment issues, build failures, and integration problems. SonarQube / Quality Gates • Integrate SonarQube into Jenkins CI/CD pipelines. • Configure and enforce SonarQube Quality Gates. • Automate code-quality and security validation within CI/CD workflows. • Ensure code and deployments meet defined enterprise quality standards. • Monitor and resolve quality-gate failures before application or platform deployment. Terraform / Infrastructure as Code • Design and implement AWS infrastructure using Terraform. • Develop reusable and standardized Terraform modules. • Automate provisioning and configuration of AWS infrastructure and data-platform components. • Integrate Terraform with Jenkins CI/CD pipelines. • Implement automated Terraform plan, validation, approval, and deployment workflows. • Manage infrastructure changes through Git-based version control. • Establish enterprise standards for Infrastructure as Code, automation, security, and governance. ROC/D & DevOps Automation • Implement and support ROC/D within enterprise DevOps and CI/CD workflows. • Integrate ROC/D with applicable source-control, pipeline, deployment, and infrastructure automation processes. • Support automated release, deployment, and operational workflows involving ROC/D. • Troubleshoot ROC/D-related deployment, pipeline, and platform issues. • Work with engineering teams to standardize and automate release and operational processes. Platform Engineering • Establish enterprise Platform Engineering standards, automation frameworks, and reusable platform capabilities. • Build self-service infrastructure and deployment capabilities for engineering and data teams. • Standardize development, testing, and production deployment processes. • Promote automation, GitOps, CI/CD, IaC, observability, security, and platform reliability. • Reduce manual infrastructure and deployment activities through automation. • Establish operational readiness, monitoring, incident management, and reliability practices. • Continuously improve platform scalability, availability, security, and engineering productivity. Required Skills Mandatory Technical Skills • AWS Cloud – Deep hands-on experience • AWS Data Platform Engineering – Deep experience • Platform Engineering • Jenkins / Jenkins Pipelines • CI/CD • Terraform • Infrastructure as Code (IaC) • Git / GitHub • SonarQube / Quality Gates • ROC/D • Cloud DevOps • Infrastructure Automation • Monitoring & Observability • Python, Bash, or Shell scripting AWS Skills Strong hands-on experience with multiple AWS services: S3 | AWS Glue | Redshift | Athena | Lambda | EMR | EKS | EC2 | IAM | VPC | CloudWatch | KMS | Secrets Manager Candidate must understand how AWS services are integrated to create and operate enterprise data platforms. Data Platform Skills Strong understanding of: • Data Lake / Data Lakehouse • Enterprise Data Platforms • Data ingestion and integration • Batch and streaming data processing • Data pipelines • Data transformation • Data storage and analytics • Data platform security • Data governance • Data quality • Platform monitoring and observability • High availability and disaster recovery • Data-platform performance and scalability Databricks, Spark/PySpark, dbt, Airflow, AWS Glue, and Redshift experience is highly preferred. Preferred Skills • Databricks • Apache Spark / PySpark • dbt • Apache Airflow • AWS Glue • Amazon Redshift • Amazon EMR • Kubernetes / Amazon EKS • Docker • GitOps • Jenkins Shared Libraries • Terraform Enterprise / Terraform Cloud • Python • Bash/Shell scripting • Cloud security and governance • Observability and monitoring • Financial Services / Investment Management experience Ideal Candidate Profile The ideal candidate should be a hands-on Platform/Data Platform Engineer or Lead, not simply a traditional DevOps Engineer. The candidate should demonstrate strong experience across: AWS Cloud + Data Platform + Platform Engineering + Terraform/IaC + Jenkins CI/CD + GitHub + SonarQube + ROC/D + DevOps