DevOps Engineer

• Lead the design, development, and implementation of innovative platform automation solutions for the Databricks data lake on AWS. • Automate the provisioning, configuration, and management of core AWS services (S3, IAM, EC2, VPC, Lambda, CloudWatch, SNS, SQS, ELB, Route53) using Infrastructure as Code (Terraform). • Develop robust automation scripts and tools using Python for Databricks workspaces, clusters, jobs, and security configurations, leveraging Databricks APIs and CLI. • Design, implement, and maintain CI/CD pipelines (e.g., GitHub Actions) for continuous delivery of infrastructure, platform, and automation code. • Implement and enforce security best practices, compliance, and cost optimization strategies across the AWS and Databricks data platform. • Integrate and automate the management of AWS data services (Redshift, DynamoDB, Glue, Athena) within the Databricks ecosystem. • Drive continuous improvement and innovation in platform operations, monitoring, and performance. • Collaborate with data engineers, data scientists, governance, and operations teams to deliver automated solutions that enhance productivity, platform stability and improve customer experience. Required Skills & Qualifications: • 5-7 years of professional experience in cloud platform engineering, DevOps, or data platform automation. • AWS (Mandatory): Extensive hands-on experience with S3, IAM, EC2, API Gateway, Step Functions, VPC, Lambda, CloudWatch, SNS, SQS, ELB, and Route53. • DevOps and Automation (Mandatory) • Proven expertise in Infrastructure as Code (IaC) with Terraform. • Advanced Python scripting for platform automation, API integration, and data pipeline orchestration. • A passion for identifying and implementing innovative automation solutions. • Solid understanding of CI/CD principles and practical experience with tools like GitHub Actions. • Proficiency with Git for version control.