Senior Elastic Stack Data Integration Engineer

Senior Elastic Stack Data Integration Engineer

HX5 is an award-winning provider of engineering, research and development, and technical services to clients such as NASA and the Department of Defense. Founded in 2004, HX5 is a fast-growing veteran- and woman-owned company with locations nationwide.

HX5 is currently seeking a qualified Senior Elastic Stack Data Integration Engineer to support the Missile Defense Agency (MDA) on the Integrated Research and Development for Enterprise Solutions (IRES) contract at Redstone Arsenal in Huntsville, AL. The Senior Elastic Stack Data Integration Engineer will serve as the primary technical authority for designing, building, and maintaining data ingestion pipelines supporting Elastic SIEM. This role focuses on creating scalable, resilient Logstash architectures; developing advanced pipeline logic; normalizing, enriching, and transforming security telemetry; and ensuring reliable delivery of high-fidelity data to Elasticsearch.

Essential Duties and Responsibilities:

• Architect, build, and maintain Logstash pipelines to ingest and transform logs from diverse systems, including network devices, servers, cloud services, and security platforms.
• Implement parsing, grok patterns, JSON transformations, conditional routing, enrichment logic, and ECS mapping.
• Optimize pipeline performance, resiliency, and scalability (e.g., persistent queues, pipeline workers, memory tuning, load balancing).
• Ensure all ingested data aligns to ECS (Elastic Common Schema) or internal schema requirements.
• Implement data enrichment workflows (GeoIP, threat intel lookups, metadata injection).
• Validate data completeness, integrity, and fidelity across ingestion flows.
• Maintain and optimize Logstash clusters, including version management, scaling, tuning, and high-availability configurations.
• Manage integrations with Beats, Elastic Agent, Kafka, syslog endpoints, and custom data collectors.
• Monitor ingestion throughput, latency, and error rates; implement proactive alerting and troubleshooting processes.
• Create and maintain technical documentation, including pipeline diagrams, data flow maps, runbooks, and schema references.
• Establish enterprise standards for parsing, enrichment, normalization, and ingestion patterns.
• Support internal and external audits by documenting data handling flows and pipeline logic.
• Work closely with SIEM integration engineers to align pipelines with customer environments and logging requirements.
• Partner with detection engineering teams to ensure data supports analytic coverage and rule development.
• Collaborate with infrastructure and platform operations for deployment, scaling, and reliability engineering.

The successful candidate will have:

• Demonstrated ability to work independently, drive pipeline architecture decisions, and mentor junior engineers.
• Strong documentation, workflow diagramming, and communication skills.
• Deep command of Logstash architecture, patterns, and performance optimization.
• Mastery of parsing, enrichment, normalization, and ECS alignment.
• Strong understanding of network protocols, logging patterns, and telemetry generation from enterprise systems.
• Advanced troubleshooting skills across data ingestion, pipeline logic, and Elastic Stack processing layers.
• Capability to design scalable, HA ingestion workflows with clear operational boundaries.
• Ability to conduct data modeling, schema design, and transformation mapping.
• Effective at interfacing with multiple teams, gathering requirements, and aligning pipeline designs with SIEM analytics needs.
• Focused on reliability, maintainability, and observability across all pipeline components.
• Strong attention to detail and a disciplined approach to documentation, versioning, and configuration management.

Education and Experience:

• Must have 10, or more, years of general (full-time) work experience
• May be reduced with completion of advanced education
• Must have 5, or more, years of experience in log ingestion, data engineering, or SIEM pipeline development.
• Must have 2, or more, years of experience in a lead or senior role, mentoring and guiding other team members.
• Must have a strong background in Elastic Stack components (Elasticsearch, Kibana, Beats, Elastic Agent).
• Must have experience with data ingestion, processing, and enrichment techniques.
• Must have hands-on experience ingesting, processing, and normalizing diverse log types (Windows events, syslog, firewall logs, cloud telemetry, security tooling).
• Must be proficient with Linux administration, system-level debugging, and CLI-based operations.
• Must have a DoD 8570.01-M IAT Level II certification with Continuing Education (CE) - (CCNA-Security, CySA+, GICSP, GSEC, Security+ CE, CND, SSCP)
• Must have an active DoD Secret Security Clearance
• Must be able to obtain an active DoD Top Secret Security Clearance

Desired Requirements:

• Be an Elastic Certified Engineer or have relevant Elastic Stack certifications.
• Have strong experience integrating Kafka, Redis, or other message bus technologies into ingestion workflows.
• Be proficient with scripting in Python, Bash, or PowerShell for automation and data validation.
• Have experience designing geo-distributed or multi-cluster ingestion architectures.
• Have knowledge of threat intelligence ingestion, correlation data enrichment, and advanced ECS mapping.
• Have experience with CI/CD pipelines, GitOps workflows, or Infrastructure-as-Code (Terraform, Ansible).
• Be familiar with data quality assurance frameworks and pipeline testing methodologies.
• Have knowledge of cloud-native logging architectures (AWS Firehose, Azure Event Hub, GCP Logging).

Position Type/Expected Hours of Work:

This is a full-time position requiring 40 hours per week and offers a flexible work schedule Monday through Friday during core business hours.

Other Position Requirements:

• Proof of U.S. Citizenship or US Permanent Residency is a requirement for this position.
• Must be able to complete a U.S. government background investigation.
• Must have an active DoD Secret Security Clearance
• Must be able to obtain an active DoD Top Secret Security Clearance
• Must be able to travel, including air travel up to 10%.
• Must have a valid Driver's License.

HX5 offers a competitive salary and benefits package to include:

• Medical/Dental/Vision Insurance
• 401(k) plan with Company Match
• Paid Holidays
• Paid Time Off
• Parental Leave
• Life Insurance
• Tuition Reimbursement
• Medical and Dependent Care Flexible Spending Accounts
• Pet Insurance

HX5, LLC is an Equal Opportunity Employer that recruits and hires qualified candidates without regard to race, religion, sex, age, national origin, ancestry, citizenship, disability, or veteran status.

HX5, LLC is a Drug Free Workplace Employer.

ACCESSIBILITY NOTICE: If you need a reasonable accommodation for any part of the employment process due to a physical or mental disability, please call View phone number on click.appcast.io.