Observability Engineer (Grafana SME)

Job-ID29079312Reference26-23106Remote50% RemoteHybrid onsite at Jersey City, NJ, 07310 / New York, NY, 10041 / Dallas, TX, 75019 / Tampa, FL, 33647 / Boston, MA, 02210 CTH Looking for Grafana and Obserability EngineerDescription: We are seeking a highly skilled Senior Observability Engineer to join our Observability Engineering team. This role is responsible for designing, implementing, administering, and automating enterprise observability solutions with a primary focus on Grafana, OpenTelemetry, monitoring, alerting, and telemetry management. The ideal candidate has strong experience building and operating observability platforms at scale, automating infrastructure through Terraform, and enabling application and infrastructure teams to adopt standardized observability practices. This role will play a key part in modernizing observability capabilities and driving migration from legacy monitoring tools to Grafana-based solutions.Key Responsibilities Observability Platform Engineering Administer and support Grafana Cloud and on-premises Grafana deployments. Design and implement enterprise observability solutions for metrics, logs, traces, synthetic monitoring, and alerting. Establish and maintain observability standards, best practices, and governance processes. Configure and manage Grafana data sources, alerting, RBAC, folders, teams, and integrations. Ensure platform scalability, reliability, resiliency, and operational excellence.Automation & Infrastructure as Code Develop and maintain Terraform modules for Grafana infrastructure and configuration management. Automate onboarding of applications, infrastructure, dashboards, alerts, and data sources. Build self-service capabilities that reduce manual operational effort and improve adoption. Integrate observability capabilities into CI/CD and infrastructure provisioning workflows.Monitoring, Alerting & Incident Management Design meaningful monitoring and alerting strategies based on service health and business-critical workflows. Implement and optimize alerting standards to reduce noise and improve signal quality. Support incident response, troubleshooting, root cause analysis, and post-incident reviews. Drive continuous improvement of operational visibility and platform health.OpenTelemetry & Telemetry Engineering Implement and support OpenTelemetry instrumentation across applications and infrastructure. Establish standards for logs, metrics, traces, and telemetry collection. Support telemetry pipelines, agent deployments, and data collection strategies. Assist teams with instrumentation design and observability adoption.Migration & Modernization Support migration initiatives from legacy observability platforms to Grafana. Analyze existing monitoring, alerting, logging, and tracing implementations and recommend modernization approaches. Develop reusable migration patterns, automation, and engineering standards. Partner with application teams to accelerate adoption of enterprise observability capabilities.Collaboration & Leadership Work closely with application development, infrastructure, cloud, and SRE teams. Provide technical leadership and mentoring to engineers across the organization. Contribute to observability architecture, strategy, and roadmap development. Promote observability as a core engineering practice across the enterprise.Required Qualifications Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.5+ years of experience in observability, monitoring, operations, or platform engineering. Hands-on experience administering Grafana in large-scale enterprise environments. Strong experience with Terraform and Infrastructure as Code practices. Experience implementing monitoring, alerting, logging, and distributed tracing solutions. Experience with OpenTelemetry concepts, instrumentation, and telemetry pipelines. Strong Linux and cloud platform administration skills. Experience with scripting and automation using Python, PowerShell, Bash, or similar languages. Knowledge of operational excellence, reliability engineering, and incident management practices.Preferred Qualifications Experience migrating from tools such as Splunk, Dynatrace, AppDynamics, New Relic, OpenText OBM, or similar platforms. Experience with Grafana Alloy, Tempo, Loki, Mimir, or Prometheus. Experience operating observability platforms in AWS environments. Knowledge of Kubernetes, containers, and cloud-native observability. Experience designing enterprise observability strategies and governance models. Familiarity with CI/CD platforms and DevOps practices.Desired Skills Grafana Administration Terraform OpenTelemetry (OTEL) Monitoring & Alerting Observability Engineering Platform Engineering Linux Administration AWS Cloud Services Automation & Scripting Incident Management Infrastructure as Code Telemetry Pipelines Reliability Engineering Root Cause Analysis Enterprise Monitoring ArchitectureEEO: “Mindlance is an Equal Opportunity Employer and does not discriminate in employment on the basis of – Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans.”