Principal Data Engineer - W2 (Remote)

contract-to-hire, longer-term, around 1 year We are seeking a highly skilled Principal Data Engineer / AI Solutions Architect to support a large-scale payment systems modernization initiative involving migration from IBM i (AS/400) to Microsoft Azure. This role is focused on ensuring migration quality, operational sustainability, automation, reconciliation accuracy, and production stability for high-volume transactional systems processing millions of daily transactions. The ideal candidate will bring deep expertise in enterprise data engineering, AI-driven automation, observability, operational intelligence, and large-scale financial systems. Key Responsibilities • Design and implement AI-driven operational frameworks to support payment platform migration and long-term production sustainability. • Develop automated reconciliation processes between legacy and modernized systems using Python and Databricks. • Build certification and validation frameworks to ensure migration integrity, operational accuracy, and production readiness. • Create intelligent monitoring, telemetry, alerting, and issue-triaging solutions with minimal manual intervention. • Develop dashboards and operational controls to support system health, reconciliation tracking, and production support activities. • Architect scalable automation solutions enabling supportability with a lean engineering footprint. • Collaborate with engineering, infrastructure, DevOps, and migration teams to validate deliverables and production controls. • Support CI/CD integration and cloud-native deployment strategies within Azure environments. • Drive operational excellence for high-volume transaction processing platforms handling millions of daily transactions. Required Qualifications • Strong experience with large-scale transactional or operational systems in enterprise environments. • Proven expertise in Data Engineering and AI solution architecture/building. • Hands-on experience with Databricks and Python development. • Experience building automation frameworks, reconciliation engines, and operational dashboards. • Strong understanding of telemetry, monitoring, observability, and issue triaging. • Experience supporting modernization or migration initiatives from legacy platforms to cloud-native architectures. • Solid knowledge of enterprise production support and operational sustainability practices. Preferred Qualifications • Experience with payment systems or financial services platforms. • Experience migrating applications from IBM i / AS400 or other legacy platforms to Azure. • Knowledge of CI/CD pipelines and DevOps practices. • Experience designing AI-assisted operational support frameworks. • Exposure to Azure cloud services and enterprise-scale data platforms. Technical Skills • Python • Databricks • Azure • CI/CD • AI/ML Automation • Monitoring & Telemetry • Dashboard Development • Data Reconciliation • Enterprise Transaction Processing • Cloud Migration • Operational Intelligence • Production Support Automation