Sr. Data Engineer

We are seeking an experienced Sr. Engineer, Data – Privacy to design, develop, and optimize scalable data architectures and enterprise-grade data pipelines across cloud, on-premises, and hybrid environments. The ideal candidate will possess deep expertise in Azure Data Services, Databricks, Snowflake, Python, and SQL, along with hands-on experience in implementing privacy-focused data engineering solutions and leveraging AI-powered development tools. This role will partner closely with data engineers, architects, and business stakeholders to deliver high-performing, secure, and compliant data platforms that enable data-driven decision-making at scale. Key Responsibilities • Design and develop scalable data pipelines using Azure Data Factory (ADF) for integrating data from multiple sources. • Build and optimize data ingestion, transformation, and processing frameworks using Snowflake, including: • Stored Procedures • Streams • Tasks • Snowpipe • Dynamic Tables • Iceberg Tables • Storage Integrations and Views • Design and manage enterprise data solutions using Azure Databricks, Apache Spark, and Unity Catalog. • Develop and optimize solutions on Azure Data Lake Storage (ADLS) for secure and high-performance data storage. • Create efficient and scalable SQL queries, stored procedures, and data models. • Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and cost efficiency. • Implement data quality frameworks, validation mechanisms, and monitoring processes. • Ensure data governance and compliance with privacy and regulatory requirements. • Collaborate with cross-functional teams to deliver scalable and innovative data solutions. • Mentor team members and contribute to engineering best practices and technical excellence. AI & Automation Responsibilities • Utilize AI productivity tools such as Claude, Cursor, or similar IDEs for: • Code generation • SQL development • Debugging • Documentation • Code reviews • Testing • Design and implement AI agents for privacy data engineering solutions. • Apply Foundation Models, Prompt Engineering, and Retrieval-Augmented Generation (RAG) techniques for data engineering use cases. • Implement observability, audit logging, and human-in-the-loop controls for AI-assisted workflows. Required Qualifications • Bachelor's degree in Computer Science, Computer Engineering, or a related field. • 5–7 years of hands-on experience in Data Engineering and ETL development. • Strong experience with: • Azure Data Factory (ADF) • Azure Databricks • Azure Data Lake Storage (ADLS) • Snowflake • Oracle • Python • SQL • Experience building and migrating enterprise data solutions on Microsoft Azure. • Strong expertise in SQL, NoSQL, and relational database design. • Experience with Azure Flink, dbt, and open-source data frameworks. • Hands-on experience with CI/CD implementation using Azure DevOps or GitLab. • Working knowledge of Kafka and stream processing technologies. • Excellent analytical, problem-solving, communication, and stakeholder management skills. • Ability to manage multiple priorities in a fast-paced environment. Preferred Qualifications • Experience with Foundation Models, Prompt Engineering, RAG, and AI Agent Development. • Microsoft Azure certifications such as: • DP-203: Azure Data Engineer Associate • Snowflake Certification • Databricks Certification • Experience with Power BI reporting and visualization. ABOUT BRICKRED SYSTEMS BrickRed Systems is a global leader in next-generation technology consulting and workforce solutions, specializing in delivering high-quality talent across digital,engineering, marketing, analytics, finance, operations, and business transformation domain With a strong emphasis on innovation, scalability, and client success, BrickRed Systems helps organizations solve complex business challenges by providing skilled professionals across strategy, technology, creative, and operational functions. BrickRed fosters a culture of continuous learning, collaboration, and excellence, enabling professionals to contribute to high-impact global initiatives while advancing their careers.