Senior DataOps Engineer

Akkodis is proud to partner with an innovative automotive client who is seeking a Senior DataOps Engineer for a contractual opportunity in the Charlotte, NC area. Pay Range: $55/hr. - $65/hr. (W2) - The rate may be negotiable based on experience, education, geographic location, and other factors. The Senior DataOps Engineer will lead the design, implementation, automation, and operational support of enterprise-scale data platforms primarily leveraging Databricks and AWS cloud services. The role will focus on building highly scalable, reliable, and governed data pipelines, optimizing Databricks platform operations, and implementing Infrastructure as Code (IaC) and DataOps best practices across the enterprise data ecosystem. The engineer will serve as a senior technical resource responsible for Databricks platform engineering, operational excellence, workload optimization, governance implementation, automation, and CI/CD enablement supporting analytics, AI/ML, and enterprise data initiatives. Responsibilities Databricks Platform Engineering & DataOps • Lead the design, development, optimization, and operational management of enterprise-scale ETL/ELT pipelines within Databricks. • Build and maintain scalable batch and streaming data pipelines using PySpark, Spark SQL, Delta Lake, and Databricks Workflows. • Configure, optimize, and manage Databricks clusters for performance, scalability, reliability, and cost efficiency. • Implement and enforce Delta Lake best practices including partitioning, schema evolution, compaction, optimization, and performance tuning. • Administer and manage Unity Catalog, including governance policies, access controls, lineage, auditing, and data security standards. • Design and support medallion/lakehouse architecture patterns across Bronze, Silver, and Gold data layers. • Implement operational monitoring, observability, alerting, and troubleshooting processes for Databricks jobs, workflows, clusters, and platform services. • Support enterprise AI/ML and analytics workloads running on Databricks. Cloud Data Engineering & Integration • Develop and maintain scalable data ingestion and transformation pipelines using Python, PySpark, SQL, AWS Glue, and cloud-native AWS services. • Integrate structured, semi-structured, unstructured, and streaming data from enterprise and cloud-based data sources. • Implement real-time and event-driven data processing using AWS Kinesis, Firehose, and related streaming technologies. • Collaborate with architecture, analytics, AI/ML, and platform teams to deliver enterprise-grade data solutions. Infrastructure Automation & DevOps • Lead Infrastructure as Code (IaC) implementation using Terraform for provisioning and managing Databricks workspaces, clusters, jobs, permissions, and related cloud infrastructure. • Automate environment provisioning, deployment processes, configuration management, and operational workflows. • Implement and maintain CI/CD pipelines supporting Databricks code deployments, infrastructure automation, and platform operations. • Maintain version-controlled repositories and DevOps processes supporting enterprise DataOps practices. • Drive platform standardization, operational governance, and deployment consistency across environments. Governance, Security & Operational Excellence • Ensure compliance with enterprise data governance, privacy, security, and regulatory standards. • Implement data quality validation, lineage tracking, auditability, and operational controls. • Establish operational best practices, platform standards, monitoring frameworks, and support procedures. • Provide technical leadership, mentorship, and guidance for DataOps and Databricks engineering practices. Deliverables • Production-ready Databricks ETL/ELT pipelines and workflows. • Optimized and governed Databricks platform environments. • Terraform modules and Infrastructure as Code automation templates. • Monitoring, observability, and operational dashboards for Databricks workloads and pipelines. • Enterprise data models, lineage documentation, and operational runbooks. • CI/CD pipelines and deployment automation frameworks. • Weekly status reports and participation in Agile sprint ceremonies. Qualifications • 8+ years of experience in Data Engineering, Platform Engineering, or DataOps roles. • 5+ years of hands-on experience with Databricks in enterprise-scale environments. • Strong expertise in PySpark, Spark SQL, Python, SQL, and distributed data processing. • Deep hands-on experience with Delta Lake, Databricks Workflows, Unity Catalog, cluster optimization, and platform administration. • Strong experience implementing medallion/lakehouse architectures in Databricks. • Proven expertise with Terraform and Infrastructure as Code (IaC) automation. • Experience implementing CI/CD pipelines and DevOps/DataOps operational practices. • Strong knowledge of AWS cloud services including AWS Glue, Kinesis, Firehose, S3, and IAM. • Strong understanding of data governance, security, observability, and operational monitoring frameworks. • Excellent communication, leadership, troubleshooting, and collaboration skills. If you are interested in this Senior DataOps Engineer job located in Charlotte, NC please apply. For other opportunities available at Akkodis go to www.akkodis.com. No C2C Equal Opportunity Employer/Veterans/Disabled Benefit offerings available for our associates include medical, dental, vision, life insurance, short-term disability, additional voluntary benefits, an EAP program, commuter benefits, and a 401K plan. Our benefit offerings provide employees with the flexibility to choose the type of coverage that meets their individual needs. In addition, our associates may be eligible for paid leave including Paid Sick Leave or any other paid leave required by Federal, State, or local law, as well as Holiday pay where applicable. Disclaimer: These benefit offerings do not apply to client-recruited jobs and jobs that are direct hires to a client. To read our Candidate Privacy Information Statement, which explains how we will use your information, please visit https://www.akkodis.com/en/privacy-policy. The Company will consider qualified applicants with arrest and conviction records by federal, state, and local laws and/or security clearance requirements, including, as applicable: · The California Fair Chance Act · Los Angeles City Fair Chance Ordinance · Los Angeles County Fair Chance Ordinance for Employers · San Francisco Fair Chance Ordinance