Hadoop Admin

Job Title: Hadoop Administrator Responsibilities • Responsible for implementation and ongoing administration of Hadoop infrastructure. • Responsible for cluster maintenance, troubleshooting, monitoring, and following proper backup & recovery strategies. • Provisioning and managing the life cycle of multiple clusters like EMR & EKS. • Infrastructure monitoring, logging & alerting with Prometheus, Grafana, and Splunk. • Performance tuning of Hadoop clusters and Hadoop workloads, including capacity planning at the application/queue level. • Responsible for memory management, queue allocation, and distribution experience in Hadoop/Cloud era environments. • Should be able to scale clusters in production and have experience with 18/5 or 24/5 production environments. • Monitor Hadoop cluster connectivity and security, including file system (HDFS) management and monitoring. • Investigate and analyze new technical possibilities, tools, and techniques that reduce complexity, create more efficient and productive delivery processes, or deliver better technical solutions that increase business value. • Involved in fixing issues, performing RCA, and suggesting solutions for infrastructure/service components. • Responsible for meeting Service Level Agreement (SLA) targets and collaboratively ensuring team targets are met. • Ensure all changes to production systems are planned and approved in accordance with the Change Management process. • Collaborate with application teams to install operating system and Hadoop updates, patches, and version upgrades when required. • Maintain central dashboards for all system, data, utilization, and availability metrics. Ideal Candidate Profile • Experience: 6–12 years of total experience, with at least 3 years of hands-on work in developing, maintaining, optimizing, and resolving issues in Hadoop clusters supporting business users. • Operating Systems: Experience in Linux/Unix OS services, administration, shell, and awk scripting. • Programming: Strong knowledge of at least one programming language — Python, Scala, Java, or R — with debugging skills. • Hadoop Ecosystem: Experience in Hadoop components — MapReduce, Hive, Pig, Spark, Kafka, HBase, HDFS, H-Catalog, Zookeeper, and Oozie/Airflow. • Security: Experience in Hadoop security including Kerberos, Knox, and TLS. • Databases: Hands-on experience with SQL and NoSQL databases (HBase), including performance optimization. • Tools & Automation: Experience in tool integration, automation, and configuration management using GIT and Jira platforms. • Soft Skills: Excellent oral and written communication, presentation skills, and strong analytical and problem-solving abilities.