Data Engineer

A rapidly expanding company supporting federal and military programs is seeking a fast-learning and self-motivated Data Engineer with Apache Kafka/NiFi experience to design, build, and maintain systems and infrastructure that collect, store, and transform data for a mission-critical federal program. This effort is primarily centered around Tactical Network data engineering support. This is a hybrid role in North Charleston, SC. You Will:

• Support designing and building ETL (Extract, Transform, Load) processes and pipelines to move data from disparate sources into centralized repositories.
• Develop automated deployment pipelines and Infrastructure-as-Code templates using scripts and orchestration tools to automate repetitive data tasks, reducing manual intervention and error.
• Design and optimize data architectures to improve performance, cost-efficiency, security, and scalability.
• Design and implement large-scale processing systems and database architecture solutions (e.g., Data Lakes, Data Warehouses).
• Provide recommendations for data engineering best practices and process improvements.
• Contribute to team objectives and support data engineering initiatives.
• Prepare technical documentation and present architecture proposals to engineering teams.
• Partner with other engineering groups to understand their data requirements and with leadership to align technical infrastructure with mission objectives.

• Possession of and the ability to maintain a Secret Security Clearance.
• 3–5 years of experience supporting data engineering efforts and initiatives.
• Experience with containerization technologies (Docker, Kubernetes) is highly valued.
• Experience with Helm charts and deploying, managing, and scaling applications.
• Working knowledge of Agile methodology and DevOps practices.
• Experience with open-source, distributed event streaming platforms used to collect, process, store, and analyze data feeds (Apache Kafka).
• Experience with data orchestration tools (Apache NiFi).
• Experience with open-source monitoring and observability architectures including Grafana LGTM+ Stack and Elastic (ELK) Stack.
• Experience with high-performance, open-source networking infrastructure tools (Nginx, Apache APISIX).
• Experience with open-source, distributed event streaming platforms used to collect, process, store, and analyze data feeds (Apache Kafka, Debezium).
• Experience with data orchestration tools (Apache NiFi).
• Experience with open-source data storage and caching services (Postgres, Redis, MinIO).
• Strong documentation and communication skills to work effectively across technical teams.
• Bachelor's degree preferred or equivalent experience.

• Bachelor's degree in Computer Science, Engineering, or related field.
• 5+ years of experience in data engineering roles.
• Experience with CI/CD tools (Jenkins, GitLab CI, Azure DevOps, GitHub Actions).
• Proficiency with version control systems and collaborative development tools
• Experience with configuration management and automation platforms (Ansible, Chef, Puppet).
• Programming/scripting experience (Python, PowerShell, Bash, Go).
• Experience with data engineering design and visualization tools (Swagger).
• Knowledge of microservices architecture and serverless computing