Senior Data Engineer

Job Description

Job Description

NIRA is seeking a highly experienced Senior Data Engineer to support the U.S. Department of Justice Digital Evidence Review Platform program. The Data Engineer will design, develop, test, operate, and optimize high-volume data ingestion and processing pipelines for digital evidence and associated metadata. The platform must process large and diverse datasets while preserving integrity, provenance, auditability, access controls, and chain-of-custody information throughout the evidence lifecycle.Key ResponsibilitiesDesign, develop, test, operate, and optimize high-volume data ingestion, transformation, integration, and processing pipelines.Ingest, normalize, enrich, validate, store, search, export, archive, and restore structured and unstructured data.Process sources such as forensic extraction packages, documents, messages, multimedia, call records, geolocation data, cloud-provider returns, metadata, and outputs from Government-furnished parsers and external systems.Implement validation, hashing, error handling, retry, reconciliation, reprocessing, and data-quality controls.Preserve source identifiers, processing history, lineage, provenance, and chain-of-custody events.Support OCR, transcription, translation, deduplication, metadata extraction, entity extraction, and enrichment workflows.Develop schemas, canonical data models, metadata structures, and open, portable export formats.Support search, analytics, and AI/ML-enabled processing while maintaining human oversight and auditability.Automate deployment and testing through CI/CD and configuration-management practices.Tune pipeline performance and troubleshoot failures across development, test, UAT, production, and disaster-recovery environments.Maintain technical documentation and collaborate with architects, software engineers, cloud integrators, cybersecurity personnel, and product specialists.Minimum QualificationsBachelor 's degree in engineering, mathematics, or science.At least 15 years of relevant data engineering, data integration, data architecture, software engineering, or large-scale data-processing experience.A master's degree may substitute for two years of required experience; a PhD may substitute for four years.Extensive experience designing, developing, and maintaining enterprise data ingestion, transformation, integration, and processing pipelines.Experience processing structured and unstructured data from multiple sources and formats.Experience with data modeling, metadata, schema design, data lineage, data quality, error handling, reconciliation, and audit logging.Experience developing in Python, Java, Scala, SQL, or comparable data-engineering languages.Experience with cloud-based data platforms, distributed processing, object storage, databases, search platforms, or analytics environments.Experience supporting CI/CD, automated testing, configuration management, performance tuning, and production troubleshooting.S. citizenship and ability to meet DOJ residency and personnel-security requirements.Ability to obtain and maintain a High-Risk Public Trust investigation and PIV credential.Preferred QualificationsExperience with digital evidence, forensic data, eDiscovery, law-enforcement data, case-management data, or other high-integrity federal datasets.Experience with AWS GovCloud, Azure Government, FedRAMP-authorized SaaS, or hybrid federal cloud environments.Experience processing PDFs, text, messages, multimedia, call records, geolocation data, forensic extraction files, or large document collections.Experience with OCR, natural-language processing, entity extraction, transcription, translation, deduplication, AI/ML enrichment, or LLM/RAG pipelines.Experience with Python-based data frameworks, Spark, Databricks, Kafka, Airflow, cloud-native data services, or comparable technologies.Experience implementing hashing, immutable audit records, provenance, lineage, retention, archival, and data-restoration controls.Experience designing open-format exports, APIs, canonical schemas, or data-exchange interfaces that reduce vendor lock-in.