Data Engineer

Data Engineer, SQL Server to Databricks Migration The Role This is a hands-on migration role on an enterprise energy client's lakehouse rebuild. Databricks Lake Bridge auto-translates the first 80% of the SQL Server workload. Your job is the gnarly 20%. Type-mapping edge cases, transpile tuning, stored procedures that don't port cleanly, and the architectural calls that decide whether cutover is clean or painful. You'll be paired with a senior engineer who owns strategy on client calls. You'll be the one in the codebase every day, making the translation layer actually work. What You'll Do • Migrate SQL Server stored procedures and ETL jobs to Databricks. • Run Databricks Lake Bridge for initial translation, then resolve the type mismatches, transpile gaps, and logic it can't handle. • Design Delta tables, notebooks, and Unity Catalog structures that scale in production. • Troubleshoot JDBC, networking, and VPC issues in AWS when data doesn't move the way it should. • Sit in on strategy calls as the technical voice on execution. What You'll Bring • A migration project you actually shipped. We'll ask for stored-proc counts, row volumes, and how you handled cutover. "Five years of SQL Server" isn't the signal we're looking for. • Real Databricks experience. Notebooks, Delta tables, Unity Catalog in practice, not just a certification. • AI coding tools woven into your daily workflow. Claude Code, GitHub Copilot, or Codex. Be ready to walk us through a task last week where the tool changed how you approached a problem. • AWS troubleshooting comfort, especially around JDBC and networking. Baseline Tech Python, SQL, Databricks, PySpark, ETL, AWS, JDBC, Git, DBeaver Not a Fit If • Your background is pure SQL Server DBA with no migration work. • You've heard of AI coding tools but don't use them daily. Why Join You'll work on a migration with real scale, paired with senior engineers who know the terrain. You'll be measured on what you ship, not hours logged. And you'll use the AI tools you already love without having to justify them in a meeting. If you've done this work before and want to do it again with a team that respects the craft, we'd like to meet you.