Lead Data Engineer

Role: Lead Data Engineer Type: 6 Month Contract To Hire Location: Atlanta Metro Area (In-office Monday through Friday) About the Opportunity Insure Technologies is partnering with one of it's key strategic partners, a leading enterprise financial payments organization, to identify a hands-on Lead/Senior Data Engineer in Atlanta to play a key role in designing and optimizing large-scale data solutions in a modern Azure + Databricks environment. This individual will lead development of robust data pipelines, drive automation, and ensure delivery excellence across a globally distributed data team. You’ll serve as both a technical expert and mentor—working closely with offshore engineers, guiding best practices, and ensuring the business can make sense of data through effective storytelling and visualization. Key Responsibilities • Design, build, and optimize scalable data pipelines and ETL frameworks using Python / PySpark within Databricks. • Automate performance monitoring, reporting, and pipeline management across large data ecosystems. • Leverage Azure Data Lake and related MS data technologies to support enterprise analytics and data science initiatives. • Analyze and interpret datasets to identify trends, provide insights, and communicate findings effectively to business stakeholders. • Collaborate with cross-functional teams to align data strategy with business objectives. • Manage, coach, and mentor a small offshore data engineering team, promoting collaboration and quality execution. • Drive continuous improvement through automation, code reviews, and adoption of modern tools (GitHub, CI/CD, etc.). What We’re Looking For • 8+ years of experience in data engineering, including recent hands-on development in Python and PySpark. • Strong experience with Databricks, including notebook development, pipeline setup, and automation. • Solid understanding of Azure cloud data services and Data Lake architectures. • Proven ability to interpret and visualize data—comfortable using spreadsheets, dashboards, or BI tools to communicate insights. • Self-directed and delivery-oriented with the ability to thrive in fast-moving, dynamic environments. • Excellent communication skills with a collaborative and coaching mindset. • Exposure to machine learning / AI concepts or experience integrating GitHub with AI tools is a strong plus. • Experience mentoring or managing offshore teams preferred. Additional Details • This position requires on-site presence Monday through Friday. • Applicants must be authorized to work in the U.S. without current or future sponsorship. • This role will be an initial 6 month contract engagement with a strong likelihood of conversion to FTE status