Azure Data Factory Architect

Key Responsibilities • Lead consulting and advisory on Azure Data Factory (ADF) pipeline design and implementation, ensuring alignment with business-as-usual (BAU) operations. • Architect and implement robust ADF pipelines with a focus on API integration, authentication, and end-to-end security best practices. • Collaborate with stakeholders to translate business requirements into scalable ADF solutions, optimizing for maintainability and operational efficiency. • Provide expert guidance on ADF features, troubleshooting, and continuous improvement of data workflows in production environments. • Mentor teams on secure ADF pipeline development, API management, and compliance with enterprise security standards. Required Qualifications • Technical Expertise: • Strong expertise in Azure Data Factory, including pipeline design, data flow, and integration runtime configuration. • Proficiency in Azure cloud services such as Azure Data Lake, Azure Synapse Analytics, Azure SQL Database, and Azure Key Vault. • Hands-on experience with ETL/ELT processes, data transformation, and data integration patterns. • Programming and Scripting: • Proficiency in SQL, Python, or other scripting languages for data manipulation and automation. • Experience with REST APIs and integrating ADF with external systems. • Architectural Knowledge: • Deep understanding of cloud-based data architecture, including data lakes, data warehouses, and real-time data processing. • Familiarity with DevOps practices, CI/CD pipelines, and Infrastructure as Code (IaC) using tools like Azure DevOps, ARM templates, or Terraform. • Soft Skills: • Strong communication and interpersonal skills to engage with both technical and non-technical stakeholders. • Problem-solving mindset with the ability to provide innovative solutions to complex challenges. • Leadership and mentoring abilities to guide teams and ensure project success. Preferred Qualifications • Azure certifications such as Microsoft Certified: Azure Data Engineer Associate or Microsoft Certified: Azure Solutions Architect Expert. • Experience with other data integration tools like SSIS, Informatica, or Talend (optional but beneficial). • Knowledge of big data technologies such as Databricks.