Sr. Data Engineer

We are seeking a highly skilled Sr. Engineer, Data – Privacy responsible for designing, developing, and optimizing scalable data engineering solutions that support enterprise analytics, privacy compliance, and business intelligence initiatives. This role will lead the development of modern data architectures and cloud-based data pipelines across Azure and Snowflake environments while collaborating closely with cross-functional teams to deliver high-quality, secure, and scalable data solutions. The ideal candidate will possess strong expertise in Azure Data Factory, Databricks, Snowflake, Python, SQL, ADLS, ETL development, and cloud-native data platforms, along with experience leveraging AI-powered productivity tools and modern data engineering practices. Success in this role will be measured by the ability to deliver reliable data solutions, drive technical excellence, improve data quality, and support enterprise privacy initiatives at scale. Key Responsibilities Data Engineering & Pipeline Development • Design and develop scalable data pipelines using Azure Data Factory (ADF) for seamless data integration. • Build and optimize ETL/ELT workflows across cloud and hybrid environments. • Develop large-scale data processing solutions using Azure Databricks, Apache Spark, and Unity Catalog. • Implement data ingestion, transformation, and orchestration processes using Snowflake, including: • Stored Procedures • Streams • Tasks • Snowpipe • Dynamic Tables • Iceberg Tables • Storage Integrations • Views • Manage and optimize Azure Data Lake Storage (ADLS) environments for secure and high-performance data storage. Data Architecture & Analytics • Design scalable data architectures supporting enterprise analytics and reporting. • Develop robust data models and optimize data structures for performance and scalability. • Analyze complex datasets to identify trends, anomalies, and actionable business insights. • Ensure efficient data delivery through performance tuning and optimization of pipelines and SQL workloads. Data Quality, Governance & Privacy • Implement data validation, cleansing, monitoring, and quality assurance processes. • Ensure compliance with enterprise data governance standards and privacy regulations. • Build privacy-focused data engineering solutions supporting compliance and audit requirements. • Maintain secure and governed access controls across enterprise data environments. AI & Automation • Utilize AI productivity tools for data engineering activities, including development, debugging, testing, documentation, and code reviews. • Design, develop, and operate AI-powered automation solutions for privacy data engineering. • Apply Prompt Engineering, Foundation Models, Retrieval-Augmented Generation (RAG), and AI Agent frameworks to enterprise data use cases. • Implement observability, audit logging, and human-in-the-loop controls for AI-assisted workflows. Cross-Functional Collaboration • Collaborate with Data Engineering, Analytics, Architecture, and Business teams to deliver enterprise data solutions. • Provide technical guidance and mentorship to team members. • Participate in solution design discussions, code reviews, and architecture reviews. • Support Agile development and continuous improvement initiatives. Project & Delivery Management • Manage multiple workstreams and priorities in a fast-paced environment. • Support project estimation, planning, execution, testing, and release activities. • Ensure adherence to engineering standards, coding best practices, and delivery timelines. • Drive continuous improvements in performance, quality, and operational efficiency. Required Qualifications Education • Bachelor’s degree in Computer Science, Computer Engineering, Information Technology, or related field. • Equivalent practical experience will also be considered. Experience • 5–7 years of hands-on experience in Data Engineering, ETL Development, and Data Integration. • Strong experience designing and supporting enterprise-scale data solutions. • Proven experience building cloud-native data platforms and modern data pipelines. Technical Skills Must Have • Azure Data Factory (ADF) • Azure Databricks • Snowflake • Python • SQL • Azure Data Lake Storage (ADLS) • Azure Cloud Platform • ETL/ELT Development • Data Modeling • Oracle Databases • CI/CD using Azure DevOps or GitLab • Apache Spark • Kafka / Stream Processing • Cloud-Native Data Platforms • AI Productivity Tools (Claude, Cursor or similar) Core Competencies • Strong analytical and problem-solving skills. • Excellent communication and collaboration abilities. • Strong organizational and prioritization skills. • Ability to manage multiple projects simultaneously. • Passion for learning emerging technologies and modern data engineering patterns. • Ability to thrive in fast-paced, evolving environments. Preferred Qualifications • Experience with Azure Flink and dbt. • Knowledge of vendor-agnostic data engineering frameworks. • Experience with Foundation Models, Prompt Engineering, RAG, and AI Agent Development. • Microsoft Azure Data Engineer (DP-203), Snowflake, or Databricks certifications. • Experience with Power BI reporting and analytics. • Experience designing privacy and compliance-focused data solutions. ABOUT BRICKRED SYSTEMS BrickRed Systems is a global leader in next-generation technology consulting and workforce solutions, specializing in delivering high-quality talent across digital, engineering, data & analytics, cloud, AI/ML, finance, operations, and business transformation domains. With a strong emphasis on innovation, scalability, and client success, BrickRed Systems helps organizations solve complex business challenges by providing skilled professionals across strategy, technology, data engineering, cloud modernization, analytics, and operational functions. BrickRed fosters a culture of continuous learning, collaboration, and excellence, enabling professionals to contribute to high-impact enterprise initiatives while advancing their careers.