Splunk Administrator / Developer

Splunk Administrator/Developer Overview We are seeking a Splunk Administrator/Developer to take full ownership of a large-scale Splunk environment supporting mission-critical operational and engineering systems. This role will be responsible for managing the entire Splunk ecosystem from the infrastructure layer (bare metal) through application-level development, in direct support of software engineering teams responsible for complex physical systems and real-time operations. This is a hands-on role requiring deep technical expertise across infrastructure, data ingestion, and observability, with a strong emphasis on reliability, scalability, and performance. \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ Key Responsibilities • Own and manage the full Splunk stack o End-to-end responsibility from bare metal infrastructure OS Splunk architecture, dashboards, and analytics • Design, deploy, and maintain Splunk infrastructure o Install, configure, and optimize Splunk components (indexers, search heads, forwarders, clustering) o Manage both on-premise/physical server environments and hybrid integrations • Support real-time operational systems o Enable monitoring and observability for high-availability, real-time engineering systems o Build dashboards and alerts used by engineering and operations teams • Develop data pipelines and ingestion strategies o Integrate data from diverse sources (logs, sensors, applications, control systems) o Optimize parsing, indexing, and search performance • Partner with software engineering leadership o Work closely with Software Engineering Managers and system teams o Translate operational requirements into scalable Splunk solutions • Performance tuning and system reliability o Identify bottlenecks across infrastructure and data pipelines o Ensure high availability, redundancy, and system resilience • Security, governance, and access control o Implement role-based access and data governance best practices o Maintain compliance with enterprise security standards • Continuous improvement o Recommend architecture improvements and modernization strategies o Evaluate tools within the Splunk ecosystem (e.g., ITSI, ES, Cribl, etc.) \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ Required Qualifications • 5–10+ years of experience with Splunk administration and development • Proven experience owning a Splunk environment end-to-end • Strong hands-on experience with: o Splunk Enterprise architecture (indexers, search heads, clustering) o Data ingestion, parsing, and optimization o Dashboard and alert development • Experience managing on-prem / bare-metal infrastructure o Linux administration (preferred) o Server provisioning, performance tuning, and system monitoring • Experience supporting real-time, high-availability systems • Strong scripting skills (Python, Bash, or similar) \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ Preferred Qualifications • Experience supporting engineering, industrial, or operational technology (OT) environments • Familiarity with observability and logging tools beyond Splunk (e.g., Cribl, ELK, Prometheus) • Experience with event-driven systems, telemetry data, or IoT/sensor integrations • Exposure to high-throughput environments with large-scale data ingestion • Splunk certifications (Admin, Architect, or Developer) \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ What Success Looks Like • Full ownership of a complex Splunk environment with minimal oversight • Reliable, scalable monitoring of mission-critical systems • Strong partnership with engineering leadership and operational stakeholders • Continuous improvements in performance, visibility, and system uptime \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ Why This Role Is Unique • Direct impact on real-world systems with physical and operational outcomes • Highly visible role supporting critical engineering and operational teams • Opportunity to own both infrastructure and application layers within a single ecosystem