Principal Metrics & Observability Engineer

Job Description

Job Description

Job Description

ACTIVE SECURITY CLEARANCE AT THE TS/SCI POLYGRAPH LEVEL IS REQUIRED

We are seeking a high-performing Software Engineer with deep expertise in Splunk to join an agile engineering team responsible for designing, developing, and maintaining a comprehensive distributed Metrics and Monitoring solution for High-Performance Computing (HPC) systems. In this role, you will be instrumental in ensuring the reliability, performance, and scalability of mission-critical HPC infrastructure by turning complex system telemetry into meaningful dashboards, correlation models, and automated alerting pipelines.

The annual base salary range for this role is $217,000-$257,000 (USD) , which does not include discretionary bonus compensation or our comprehensive benefits package. Actual compensation offered to the successful candidate may vary from posted hiring range based upon geographic location, work experience, education, and/or skill level, among other things.

, Required Skills

Splunk Expertise: Deep, hands-on experience using Splunk SPL, creating dynamic dashboards, building reports, data onboarding, defining field extractions, data modeling, and performance tuning.

Alerting Event Management: Proven experience developing Splunk alerting mechanisms, event correlation models, and incident management workflows.

Software Development Scripting: Recent software development experience using Python , along with practical proficiency developing Shell or Perl scripts for process automation.

Log Ingestion Messaging: Operational familiarity interfacing with log management utilities ( Syslog-ng , rsyslog ) and message brokers ( Apache Kafka ).

Infrastructure as Code (IaC): Experience applying IaC principles using automation tools like Ansible .

Linux Environment: Strong command of the Linux CLI and system administration tools.

CI/CD Version Control: Practical experience with CI/CD methodologies ( GitLab CI ) and source control with Git .

Education:  Bachelor’s Degree in Data Analytics, Intelligence Studies, Computer Science, Finance/Economics, or a related discipline, PLUS 12+ years of specialized data analytics or financial intelligence experience OR  Master’s Degree in a related field, PLUS 10+ years of specialized analytical experience OR  High School Diploma / GED , PLUS 14+ years of hands-on intelligence analysis, financial tracking, and data discovery experience in lieu of a degree.

, Desired Skills

Agile Tooling: Practical experience using the Atlassian Tool Suite ( Jira , Confluence ) for workflow tracking and technical documentation.

, About Nyla Technology Solutions

Nyla Technology Solutions delivers exceptional Artificial Intelligence (AI), Data Science, and Software Engineering services for the U.S. Government. Nyla embraces a forward-thinking and bold approach at every turn, earning us a solid reputation of technical trendsetters within the industry. We have a passion for developing solutions that have a quick and immediate impact on mission. Headquartered in Columbia, Maryland, our customers love how we tackle their most challenging problems and get things done.

If you have the unique experience and expertise we are seeking, along with the desire and determination to invest your time and energy as a part of Nyla’s team, 

Taking Care of All of You

Nyla provides a top-of-market compensation and benefits package. And through our unique Nyla FLEX program, we custom tailor these benefits to best fit your lifestyle. 

The Nyla FLEX benefit program is designed to offer you flexibility in the 3 biggest areas of your life: your pay, your leave, and your schedule. 

PAY - Nyla starts with 4 weeks of Annual Leave plus 11 holidays and an additional day of Annual Leave for each year you’re at the company. You have the flexibility to cash out your annual leave hours, opt out of other Nyla benefits, and/or arrange for additional hours on contract* (over 40 hrs/week). There’s even an option to earn 1.3 times your hourly rate once you work over 1880 hours on contract!

LEAVE - Want to spend more time with the family? Want more time to travel the world? You can BUY additional annual leave for a total of 6 weeks of annual leave. That’s up to 240 hours of leave plus 11 holidays! That’s not even including paid anniversary leave!

SCHEDULE - Does the traditional 40-hour workweek no longer fit your lifestyle? With Nyla FLEX, you have the freedom to scale down to 30-32 hours while still enjoying the top-notch Nyla benefits you know and love. It's flexibility that works for you without compromising the perks!

WHAT ABOUT OTHER BENEFITS? Nyla’s health care (medical, dental, and vision) is 100% covered by the company. We provide 10% 401k matching - with full vesting day 1! Our Professional Development offers $5,000 per year to be used towards fees, tuition, or time off for your continued growth. We even have a student loan repayment program and we provide 8 hours of volunteering annually so you can support your community, making your world a better place.

To learn more about Nyla's culture and our exceptional benefit packages click here.

Nyla is an equal opportunity employer.