Azure Data Lake Engineering Lead

Job Summary: We are seeking an experienced and highly skilled Software Engineering Lead to spearhead our data initiatives, with a primary focus on Azure Data Lake and its associated ecosystem. The ideal candidate will possess deep architectural knowledge of Azure Data Lake and Databricks, a solid understanding of security concepts within Azure, and a proven track record of leading teams in delivering scalable and secure data solutions. Major Responsibilities: Architectural Design and Implementation: • Design and implement robust, scalable, and efficient data architectures leveraging Azure Data Lake and Databricks. • Define and enforce best practices for data ingestion, storage, processing, and retrieval. • Optimize data workflows to ensure high performance and cost efficiency. Data Governance and Security: • Develop and implement security measures for Azure Data Lake, ensuring compliance with organizational and regulatory standards. • Manage role-based access control (RBAC), encryption, and other security protocols within Azure Subscriptions. • Collaborate with security teams to perform regular audits and vulnerability assessments. Team Leadership and Collaboration: • Lead and mentor a team of data engineers, providing technical guidance and fostering professional development. • Collaborate with cross-functional teams, including data scientists, business analysts, and IT teams, to deliver data-driven solutions. • Drive agile practices and ensure timely delivery of projects. Platform Optimization and Monitoring: • Oversee the deployment and management of Azure Data Lake and Databricks environments. • Implement monitoring and alerting systems to ensure system reliability and performance. • Evaluate and incorporate new Azure services and technologies to enhance the data platform. Strategic Planning and Roadmap Development: • Develop and execute a roadmap for data engineering aligned with business objectives. • Stay abreast of industry trends and advancements in data engineering and Azure technologies. • Provide recommendations for long-term data strategy, including data lakehouse adoption and cloud optimization. Education and Experience Requirements: • Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. • 8+ years of experience in data engineering, with at least 3 years in a leadership role. • Extensive hands-on experience with Azure Data Lake, Databricks, and other Azure services. • Proven expertise in architecting and implementing large-scale data solutions. Required Knowledge and Skills: • Proficiency in SQL. • Deep understanding of Azure security concepts, including subscription management, RBAC, and data encryption. • Experience with data modeling, ETL pipelines, and big data technologies. • Familiarity with CI/CD pipelines and DevOps practices in a data engineering context. Preferred Qualifications: • Azure certifications, such as Azure Data Engineer Associate or Azure Solutions Architect Expert. • Experience with implementing data lakehouse architectures. • Familiarity with data governance frameworks like GDPR, CCPA, or HIPAA.