Sr. Data Engineer (Lead) - Contract-to-Hire

Senior Data Engineer Contract-to-Hire Hybrid Local Candidates only! Richmond, VA YOUR ROLE The Life, Annuity, and Long Term Care Data Solutions team —responsible for endtoend data architecture, data governance, and data engineering from source systems to data consumers—is seeking a Principal Data Engineer with strong Databricks expertise (3–5 years required) and proven experience and subject matter expertise in building analytical pipelines from ingestion, data standardization, and curation that includes raw, entity relationship model in a relational model, and dimensional models. In this role, you will lead and implement with guidance on using the current architecture and design patterns with no mentoring expected in using current DataBricks features. You will perform all roles in the software development lifecycle including business requirements definition, design, build, test, and run. You will lead the definition of best practices, establish common code frameworks, elevate engineering maturity, prove out and determine when new DataBricks features should be implemented, and ensure consistent delivery of high quality data products and analytical solutions across build teams that will enhance the current architecture. You will be part of a high performing, mission driven organization of 20 individuals across four agile teams. Collaboration will be central as you work closely with the solution architects, agile team tech leads, business stakeholders, and other partners across security, cloud, testing, and data modeling. The platform leverages a modern streaming architecture supporting both transactional and analytical pipelines, grounded in Medallion and OneLake design principles. A primary outcome for this role and others on the team is overseeing the implementation of DataBricks that will replace many legacy analytical platforms that result in new and better business outcomes. WHAT YOU WILL BE DOING  Perform hands on designing, building, testing, and supporting analytical pipelines.  Establishing and enforcing design, engineering, and coding standards, including code reviews.  Leading CI/CD and design of reusable data tests for future releases.  Expanding your expertise into AI, ML, Lakebase, and other capabilities available on the DataBricks platform to these features are implemented as needed.  Driving Spark performance optimization and leading platform cost optimization opportunities.  Managing production monitoring, troubleshooting, and defect resolution. WHAT YOU BRING  3–5 years of hands-on Databricks experience and with other complementary technologies including data governance, reporting and dashboard, and source system technologies.  Proven experience in leading and performing all SDLC roles not just building programming. Role includes gathering requirements, decomposing work into stories with work effort estimates, and performing the work independently.  Strong communication skills with the ability to mentor, influence, and align cross functional stakeholders.  Ability to engage proactively with business leaders to recommend solutions aligned to strategy and support data self service capabilities.  Solid grounding in general analytics, data warehousing, and modern data engineering practices.  Experience working in Agile frameworks and writing high-quality user stories.  Ability to navigate ambiguity and provide clarity for your team and partners. NICE TO HAVE  Azure cloud experience  Insurance industry business process and data knowledge.  Medical data experience.  Familiarity with data governance and data quality needs for business consumers and how they integrate with pipeline design.  Data modeling skills and knowledge on how to build SCD2, surrogate keys, dimensions, and other aspects of data models in an analytical pipeline.