Senior Data Engineer

Data Engineer III Location: Charlotte, NC (Hybrid) Overview We are seeking a Data Engineer III to join our team and help build a robust data platform for portfolio health assessment. This role will focus on creating data products and pipelines that feed into a Power BI dashboard for project managers, enabling insights into application reliability, cost, and sustainability. The initial phase involves deploying a Model-View-Presenter (MVP) dashboard in January, followed by the development of a web application to manage reliability scoring and related data. Key Responsibilities • Design and implement data ingestion pipelines to consolidate data from multiple sources into AWS Redshift and Glue • Build and maintain a data lake and ensure efficient data modeling for analytics and reporting • Collaborate with cross-functional teams to identify required data sources and define integration strategies • Develop processes to calculate application reliability scores and portfolio health metrics • Support visualization efforts in Power BI for MVP dashboards and future enhancements • Ensure scalability and sustainability of data products for long-term use • Work with GitHub data, deployment data, and AWS services to support portfolio health assessments • Contribute to the transition from MVP dashboards to a web application for managing reliability scores Required Skills Programming: Python (advanced), SQL Data Processing: Pandas, PySpark AWS Services: Lambda, Redshift, Glue Data Modeling: Strong experience designing schemas and optimizing for analytics Pipeline Development: Proven ability to build and maintain data ingestion pipelines Analytics: Ability to analyze and extract actionable insights from large datasets Collaboration: Excellent communication skills for working with multiple teams Nice-to-Have Skills Visualization: Power BI (especially dashboard development) DevOps: GitHub Actions, Terraform Cost Analysis: Familiarity with AWS Cost Explorer Project Timeline Phase 1: Deploy MVP dashboard in January Phase 2: Add data sources, refine reliability scoring, and begin web app development (runs through 2026) Ideal Candidate You are a hands-on data engineer with strong technical expertise in Python, SQL, and AWS data services. You thrive in collaborative environments, can communicate effectively with stakeholders, and have a passion for building scalable data products that drive business decisions. Eight Eleven Group provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, national origin, age, sex, citizenship, disability, genetic information, gender, sexual orientation, gender identity, marital status, amnesty or status as a covered veteran in accordance with applicable federal, state, and local laws.