Data Engineer Architect

Job Summary We are seeking an experienced Data Engineer/Architect with strong expertise in Cloudera NiFi, data pipeline optimization, and ETL/ELT processes. This role involves assessing, enhancing, and supporting scalable data integration solutions. Key Responsibilities NiFi Platform Assessment & Optimization • Evaluate existing Cloudera NiFi flows for performance, scalability, and maintainability • Identify improvements to simplify architecture and reduce operational overhead • Recommend best practices, reusable patterns, and standards Data Integration & Pipeline Support • Develop and enhance ETL/ELT pipelines using NiFi • Integrate data from databases, APIs, filesystems, streaming platforms, and cloud storage • Troubleshoot pipeline failures, latency, and performance issues • Implement monitoring, error handling, and recovery mechanisms Python & Custom Development • Use Python for data transformation, automation, and scripting • Build integrations between NiFi and external systems Governance, Security & Operations • Ensure secure data movement with authentication, authorization, and encryption • Support deployment, versioning, and operational processes • Align platform changes with compliance and operational standards Advisory & Collaboration • Act as a technical advisor to data and platform teams • Document findings, recommendations, and improvements • Provide knowledge transfer and long-term support guidance Required Skills • Hands-on experience with Cloudera NiFi in production • Strong understanding of ETL/ELT and data pipeline architecture • Proficiency in Python for scripting and automation • Experience with Linux and distributed systems • Ability to analyze and optimize existing data systems Preferred Skills • Experience with Cloudera ecosystem (Kafka, HDFS, Hive, Iceberg) • Familiarity with CI/CD for data pipelines • Experience in monitoring and performance tuning • Exposure to cloud or hybrid data environments Engagement Expectations • Hands-on, deliverables-focused role • Ability to work independently and collaborate with teams • Strong communication and documentation skills