Data Engineer: End-to-End Pipelines & DB Optimization
Koch Industries
Your Job i360 is seeking a Data Engineer to join its Data Engineering team. With your technical expertise, you will architect, implement, and improve end to end pipelines, processes, procedures, and automation for all database-centric processes. You will maintain our relational and NoSQL systems for performance and reliability. You are responsible for tuning and configuring our databases and data platforms, as well as building tools and scripts to monitor, troubleshoot and automate our systems. Our Team The Data Engineering team plays a central role in our organization, given our focus on data. We collaborate closely with both our internal departments and external clients. Our primary responsibility involves designing, building, and upkeeping data pipelines and APIs to provide data to other teams, enabling them to create their products on that data. What You Will Do Manage multiple projects simultaneously, balancing priorities and maintaining clear communication with stakeholders and team members. Design, architect, and build end to end SQL and NoSQL database solutions. This includes determining business requirements, recommending technologies, architecting, developing code, mentoring team members, and overseeing QA and deployment processes. Load and process disparate data sets using technologies including but not limited to PostgreSQL, Elasticsearch, Snowflake, Spark, Java, and Python. Work with business users to translate requirements into system flows, data flows, data mappings etc., and develop solutions to complex business problems. Code applications that adhere to enterprise design patterns. Resolve technical issues through debugging, research, and investigation. Who You Are (Basic Qualifications) BA/BS or Master's degree in Computer Science, Computer Engineering, Information Science or Data Science/Statistics At least 3 years of hands-on experience in data engineering, business intelligence, data modeling, building ETL pipelines and multi-dimensional data warehouses. Experience managing multiple projects simultaneously with competing priorities Hands-on experience with and strong understanding of RDBMS concepts and query optimization. Strong coding experience in developing enterprise applications using Java or Python Understanding of DevOps/DataOps and CI/CD toolset such as git, GitLab CI, GitHub Actions. Experience building, scaling, and maintaining high volume systems. Experience analyzing large complex data sets to resolve data quality and performance issues. What Will Put You Ahead Experience with Cloud system architecture and design, large scale streaming data processing Experience with Spring framework, AOP, JPA and REST Experience with building RESTful web services Experience with libraries like scikit-learn, keras and functional programming concepts. At Koch companies, we are entrepreneurs. This means we openly challenge the status quo, find new ways to create value and get rewarded for our individual contributions. Any compensation range provided for a role is an estimate determined by available market data. The actual amount may be higher or lower than the range provided considering each candidate's knowledge, skills, abilities, and geographic location. If you have questions, please speak to your recruiter about the flexibility and detail of our compensation philosophy. Hiring Philosophy All Koch companies value diversity of thought, perspectives, aptitudes, experiences, and backgrounds. We are Military Ready and Second Chance employers. Learn more about our hiring philosophy here. Who We Are As a Koch company, i360 works across industry lines to bring unique data, technology and analytics solutions to help our customers win. Our analysts and data scientists build and refine proven, sophisticated models that help customers understand and reach their target audience, visualize data and expand their reach. At Koch, employees are empowered to do what they do best to make life better. Learn how our business philosophy helps employees unleash their potential while creating value for themselves and the company. Our Benefits Our goal is for each employee, and their families, to live fulfilling and healthy lives. We provide essential resources and support to build and maintain physical, financial, and emotional strength - focusing on overall wellbeing so you can focus on what matters most. Our benefits plan includes - medical, dental, vision, flexible spending and health savings accounts, life insurance, ADD, disability, retirement, paid vacation/time off, educational assistance, and may also include infertility assistance, paid parental leave and adoption assistance. Specific eligibility criteria is set by the applicable Summary Plan Description, policy or guideline and benefits may vary by geographic region. If you have questions on what benefits apply to you, please speak to your recruiter. Additionally, everyone has individual work and personal needs. We seek to enable the best work environment that helps you and the business work together to produce superior results. Equal Opportunities Equal Opportunity Employer, including disability and protected veteran status. Except where prohibited by state law, some offers of employment are conditioned upon successfully passing a drug test. This employer uses E-Verify. Please click here for additional information. (For Illinois E-Verify information click here, aqui, or tu). #J-18808-Ljbffr Koch Business Solutions, LP
- ...AWS Data Engineer The AWS Data Engineer designs and develops... ...and testing of data pipelines and solutions. Additionally... ...IICS, Spark, Dynamo DB etc. in the AWS... ...Design, develop, optimize, and maintain data architecture... ...experience is a plus. Front-end software development...PipelineContract work
- Eliassen Group seeks a Palantir Foundry Engineer to establish Foundry as the central hub for enterprise inventory data. In this remote role, you will design and implement data ingestion and transformation processes, ensuring reliable inventory data. The ideal candidate...PipelineRemote job
- ...Front-End Developer About the Role: We are looking... ..., managing real-time data visualization, and... ...Collaborate with AI/ML engineers and backend developers... ...Apply performance optimization techniques for handling... ...code reviews, and CI/CD pipelines. Use Git for source...Pipeline
- ...Azure Data Engineer As an Azure Data Engineer you should... ...data processing pipelines by using different tools... ...implements, monitors, and optimizes data platforms to meet... ...other stakeholders/ end users in the software... ...Databricks, Azure Cosmos DB, Azure Data Factory,...Pipeline
$135k - $200k
...Application Developer - Defense (Front-End Eng) Title of Role: Web... ...innovative solutions for data-driven decisions and... ...design, and ship new features. Optimize applications for maximum speed... ...best practices in Git and CI/CD pipelines. Troubleshoot and debug applications...PipelineWork at office- ...End-To-End Solution Architect GDIT is seeking an End-To... ...Solution Architect with a strong engineering background, customer-facing... ...for sustainment, O&M cost optimization, and lifecycle cost... ...readiness, and enterprise-scale data ingestion pipelines Experience integrating...PipelineContract work
- ...Job Description Lead front-end development using Angular (v10+), ensuring scalable... ...systems. Ensure front-end applications are optimized for performance, accessibility, and SEO.... .... Familiarity with Git, CI/CD pipelines, Jenkins, and Agile/Scrum environments....PipelineFull timeShift work
- ...Senior Data Engineer Polymarket is looking for a Senior Data Engineer... ...can drive entire workloads end-to-end, not just execute on... ...catalog, query planning, and cost optimization. Partner on the OLTP... ...strategy, indexing, trigger pipelines, autovacuum tuning, and...Pipeline
- ...Master Data Management Architect Lead the design, development... ...selection, comparison, and optimization. Architect and present end-to-end MDM architectures, ensuring... ...Implement data integration pipelines leveraging modern data engineering tools and practices....PipelineFlexible hours
- ...Data Bricks Data Engineer Location: Seattle, WA / Bellevue, WA / Everett, WA / Renton, WA... ...• Proven experience building and optimizing complex data pipelines in Azure. • Hands-on experience... ...• Design, develop, and maintain end-to-end data pipelines and ETL/ELT...PipelineFull time
- ...seeking a skilled and highly motivated Data Engineer to join an exciting and growing... ...Responsibilities Design, develop, and implement end-to-end data pipelines, utilizing ETL processes and... ...Jupyter Notebooks. Create and optimize data pipelines from scratch,...PipelineWork from homeFlexible hours
- ...highly skilled and motivated Data Engineer to join our Data... ...secure, and efficient data pipelines that transform raw data into... ...will do Design and develop end-to-end data pipelines that... ...schema design, indexing, and optimization ~ Experience with API development...Pipeline
- ...Data Engineer With Azure And Data Bricks Location: Arlington, VA (... ...building infrastructure and CI/CD pipelines ~ Experience working... ...disparate sources. Enable end-to-end automation for deployment... ...data processing, and optimizing data solutions (Azure) ~ Expertise...PipelineFull timeLocal area
- ...Medicaid/Medicare data structures (MMIS, EDI 83... ...assets. Support and optimize SSRS report queries and... ...source control and CI/CD pipelines from TFS to Azure DevOps... ...projects. Full system engineering lifecycle. ITIL practices. End-to-end project delivery...PipelineLocal area
- ...Data Engineer As a Data Engineer, you will be required to interpret... ...experts to plan and deliver optimal data solutions. You will create... ...maintain, and optimize data pipelines as workloads move from development... ...data pipelines to support an end-to-end solution. Develop...PipelineWork experience placementWeekend workAfternoon shift
- ...Team. Job Title: EDP Data Engineer. Location: Washington,... ...batch and streaming data pipelines and curated data products on... ...tables) including compaction/optimization and lifecycle management.... ...following: Build and maintain end-to-end pipelines in...PipelineFor contractors
$130k - $145k
...Data Engineer (ADVANA) - Hybrid Washington, DC 20024 JTEC Consulting LLC focuses... ...Design, develop, and implement end-to-end data pipelines utilizing ETL processes and technologies... ...Jupyter Notebooks Build and optimize data pipelines from scratch, ensuring...PipelineFor contractorsLocal areaRelocation package2 days per week3 days per week- ...Data Engineer The Data Engineer role is a member of our team of analytics experts... ...role is responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing... ...data exploration capabilities to end users. Collaborate across multiple...PipelineFor contractors
- ...Data Engineer The Data Engineer will be developing data integration... ...and translate this into data pipeline designs and solutions. The solutions... ...reviews, Data validation & end to end testing of ETL Objects... ...Create and maintain optimal data pipeline architecture...PipelineWork experience placement
$105k - $125k
...model evaluations, and intelligence collection enable engineering, safety, and security teams to stay ahead of evolving threats... ...In this role, you will: Design, implement, and optimize end-to-end data pipelines for scraping and processing structured and unstructured...PipelineWork experience placementRemote work- ...Enterprise Data Architecture & Data Modeling Expertise... .... Advanced Data Engineering & ETL/ELT Development ~ Build, optimize, and automate ETL/ELT processes... ..., data integration pipelines, and large-scale data... ...Period of Performance: end of the year with likely...PipelineContract work
$170k - $230k
...passionate about leveraging data to deliver actionable... ...looking for a Sr. Data Engineer to join the Strategic... ...be responsible for end-to-end development of data... ...data is clean, fresh, and optimized for downstream... ..., and documentation of pipelines and data sources Leverage...PipelineFlexible hours$160k - $200k
...information Job Name: Data Engineer Location:... ...responsible for creating robust data pipelines from various data vendors to... ...objectives. Implement and optimize high-performance data... ...software development projects end to end involving large language...PipelineWork at office$152k - $205.6k
...Science team is looking for a Data Engineer. This individual will join a... ...testing new features, developing end-to-end data engineering... ...responsibilities include: • Data Pipeline Development: Design and... ...handling for data workflows; optimize performance, reliability, and...PipelineLocal areaFlexible hours- ...Data Engineer At Leidos, we deliver innovative solutions through the... ...to prepare, automate, and optimize data workflows for real-time... ...simultaneous projects and take open-ended or high-level guidance,... ...and maintaining data pipelines using modern programming languages...Pipeline
- ...Infinitive Infinitive is a data and AI consultancy that... ...a Senior / Lead Data Engineer to join our team. In... ...scalable data pipelines, while actively leading... ...Engineering: Design, build, and optimize scalable data pipelines... ...Deployment: Own the end-to-end proficiency of data...Pipeline
- ...Senior Data Engineer Position: Senior Data Engineer Location: Alexandria, VA Work... ...will lead the design, development, and optimization of data pipelines, architectures, and analytics... ...capabilities for approximately 2000 end users and over 60 locations around the...PipelineContract workWork at office
$128.25k - $266.88k
...Yahoo Consumer Data Team Position Yahoo serves as a trusted... ...with some of the brightest engineers you have known to improve... ...to design, build, and optimize scalable data pipelines and infrastructure that power... ...transform large datasets, support end-to-end model development...PipelineWork at officeFlexible hours$92.3k - $166.85k
...Department of War enterprise data and analytics program... ...Senior Data Engineer to support the delivery... ...build, and maintain data pipelines and architectures to support... ...prepare, automate, and optimize data workflows for real... ...for the fiscal year ended January 3, 2025. For more...PipelineLocal areaImmediate start- ...Data Engineer Elder Research Inc., a wholly owned subsidiary of MANTECH... ...scalable, and automated data pipelines that power advanced analytics... ...scalable data pipelines and end-to-end systems, including... ...implementation plans ~ Modernize and optimize data and ML workflows by...PipelineWork at officeLocal areaRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer: End-to-End Pipelines & DB Optimization. Be the first to apply!
- bi data engineer Arlington, VA
- data visualization developer Arlington, VA
- data science developer Arlington, VA
- senior data center engineer Arlington, VA
- sr information security engineer Arlington, VA
- junior big data engineer Arlington, VA
- entry level big data engineer Arlington, VA
- data engineer contract Arlington, VA
- aws data engineer Arlington, VA
- data engineer manager Arlington, VA

