Splunk Observability Engineer

Job Title: Splunk Observability Engineer

Location: Chicago, IL - Hybrid, 3 days onsite

Experience: 4 6 years Job Summary

STAFFXPERT LLC is seeking a Splunk Observability Engineer on behalf of our client in Chicago, IL to design, implement, and optimize enterprise observability solutions using Splunk Observability Cloud and Splunk Enterprise/Cloud. The ideal candidate will have strong hands-on experience with metrics, logs, traces, OpenTelemetry, SPL, SignalFlow, anomaly detection, and dashboard development.

Key Responsibilities

• Design and implement observability solutions covering metrics, logs, and distributed traces .
• Configure and manage OpenTelemetry (OTel) collectors , including receivers, processors, and exporters.
• Develop SPL and SignalFlow analytics for monitoring, pattern detection, and anomaly identification.
• Implement logging-in-context and correlate logs with trace IDs for efficient troubleshooting.
• Build dashboards and visualizations using Splunk Dashboard Studio , Glass Tables, and related observability tools.
• Develop monitoring strategies for high-cardinality environments and large-scale containerized systems.
• Configure anomaly detection using statistical methods and Splunk monitoring capabilities.
• Implement data filtering, masking, and aggregation using Splunk Ingest Actions and Edge Processors .
• Use Splunk machine-learning capabilities to identify patterns and support faster root-cause analysis.
• Create contextual drill-downs that correlate metrics, logs, and traces.
• Collaborate with engineering and DevOps teams to improve proactive incident detection and system reliability.

Required Qualifications

• 4 6 years of hands-on experience with Splunk development, observability, or monitoring platforms.
• Strong experience with Splunk Observability Cloud and Splunk Enterprise/Cloud .
• Proficiency in SPL and SignalFlow .
• Hands-on experience with OpenTelemetry and OTel Collector configuration .
• Strong understanding of metrics, logs, and distributed tracing .
• Experience implementing trace instrumentation for applications using Java, Python, Go, or similar technologies .
• Experience with logging-in-context and trace/log correlation.
• Knowledge of anomaly detection, statistical analysis, and pattern identification.
• Experience building dashboards using Splunk Dashboard Studio and JSON-based configurations.
• Strong understanding of DevOps, monitoring, troubleshooting, and production environments.
• Excellent analytical, communication, and problem-solving skills.

Preferred Qualifications

• Splunk certifications such as Splunk Cloud Certified Metrics User or Splunk Core Certified Power User .
• Experience with Splunk ITSI and Glass Tables .
• Knowledge of Infrastructure as Code (IaC) and GitOps practices.
• Experience with containerized and microservices-based environments.
• Familiarity with Splunk Edge Processors and data ingestion optimization.
• Strong understanding of observability best practices and proactive incident prevention.

Work Arrangement

Hybrid - 3 days onsite in Chicago, IL.

Note: Candidates should be prepared to provide a LinkedIn profile as part of the screening process.