Sr. Big Data Engineer
Samprasoft
Big Data Engineer
Location: Day 1 onsite in McLean, VA on Tuesdays, and Wednesdays.
Duration: 06+ months Contract (Possibility of Extension)
Job Description
Must Haves: Python, PySpark, SQL. Hadoop Platform.
Responsibilities include:
- Cleanse, manipulate and analyze large datasets (Structured and Unstructured data – XMLs, JSONs, PDFs) using Hadoop platform.
- Develop Python, PySpark, Spark scripts to filter/cleanse/map/aggregate data.
- Manage and implement data processes (Data Quality reports)
- Develop data profiling, deduping logic, matching logic for analysis
- Programming Languages experience in Python, PySpark and Spark for data ingestion
- Programming experience in Bigdata platform using Hadoop platform
- Present ideas and recommendations on Hadoop and other technologies best use to management
Qualifications:
- 5+ years of experience in processing large volumes and variety of data (Structured and unstructured data, writing code for parallel processing, XMLS, JSONs, PDFs) - Mandatory
- 3+ years of programming experience in Python, Spark for data processing and analysis. - Mandatory
- Strong SQL experience is a must - Mandatory
- 3+ years of experience – using Hadoop platform and performing analysis. Familiarity with Hadoop cluster environment and configurations for resource management for analysis work
- 2+ years of experience with containerization and orchestration. Hands on experience with AWS, Kubernetes, Kubeflow, Docker etc. - Optional
- Detail oriented.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Sr. Big Data Engineer in McLean, VA vacancy
$229.9k - $262.4k
...Sr. Lead Data Engineer (Enterprise Platforms Technology) Do you love building and pioneering in the technology space? Do you enjoy solving... ...iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve...SeniorBig dataFull timePart timeInternshipLocal areaCapital One
McLean, VA3 days ago- ...Analyze system requirements and design responsive algorithms and solutions Use big data and cloud technologies to produce production quality code Engage in performance tuning and scalability engineering Work with team, peers and management to identify objectives and set...SeniorBig data
Compugain
Rockville, MD4 days ago - ...with a premier financial regulatory organization, is seeking a Data Engineer for their Tysons Corner, VA or Rockville, MD location! This position... ...right candidate 5+ years of experience in Data Engineering, Big Data Engineering, or similar software engineering disciplines....SeniorBig data
Optomi
Falls Church, VA3 days ago $229.9k - $262.4k
Sr. Lead Data Engineer (Enterprise Platforms Technology) Do you love building and pioneering in the technology space? Do you enjoy solving complex... ...delivery environment? At Capital One, you’ll be part of a big group of makers, breakers, doers and disruptors, who solve...SeniorBig dataFull timePart timeInternshipLocal area$77.6k - $176k
...learning, and artificial intelligence means that there’s more structured and unstructured data available today than ever before. As a data engineer, you know that organizing big data can yield pivotal insights when it’s gathered from disparate sources. We need an experienced...SeniorBig dataFull timeContract workPart timeLocal areaRemote workPhase2 Technology
McLean, VA3 days ago- ...Data Engineer As part of the Data Management area, the Data Engineer will provide leadership in the conceptualization and design of innovative... ...of message queuing, stream processing, and highly scalable ‘big data’ data stores. Experience supporting and working with cross...SeniorBig data
Software Technology Inc
Washington DC10 hours ago $125k - $165k
...evolving technology landscape, an organization’s data has never been a more important aspect in... ...We are looking for seasoned Senior Data Engineer to work with our team and our clients to... ...with experience in tools such as: ~ Big data tools: Hadoop, Spark, Kafka, etc. ~...SeniorBig dataSteampunk.com
McLean, VA10 hours ago- ...sponsorship. Design, develop, and optimize data pipelines and systems for the collection,... ...storage Develop and enforce data engineering policies and best practices to ensure... ...of data warehousing, ETL processes, and big data technologies (SQL Server, SSIS, Azure...SeniorBig dataInternshipMonday to Friday
Navy Federal Credit Union
Vienna, VA4 days ago - ...Senior Data Engineer BasisPath is seeking a highly motivated Senior Data Engineer to support mission-critical data acquisition, processing... ...AWS. Experience with distributed computing frameworks and big data technologies. Experience with Linux administration and...SeniorBig dataRemote workRelocation package
BasisPath Inc
McLean, VA1 day ago $180k - $200k
Job DescriptionEverforth ECS is seeking a Sr. Data and AI Engineer to work Hybrid.(Typically 1-2x per week in Arlington, VA office for team planning... ...and Data Architecture concepts, Data Warehousing and Big Data tools, Data Analytics and applicable knowledge of Data...SeniorBig dataWork at office$125k - $165k
Overview We are looking for seasoned Senior Data Engineer to work with our team and our clients to develop enterprise grade data platforms,... ...experience in Data Engineering with experience in tools such as: Big data tools: Hadoop, Spark, Kafka, etc. Relational SQL and...SeniorBig data$130k - $140k
Senior Data Engineer Job Number: 912 This is a remote position. Ad Hoc is a technology company that empowers organizations to deliver... ...data processing frameworks such as Apache Spark or equivalent big data technologies. ~ Strong understanding of data modeling,...SeniorBig dataWork at officeRemote workFlexible hoursAd Hoc LLC
McLean, VA5 days ago- ...nation's vital interests. Requisition #: 1355 Job Title: Senior Data Scientist/Engineer Location: Chantilly, VA Clearance Level: Active DoD - TS/... ...-related datasets. Preferred Skills * Experience with big-data platforms (Spark, Hadoop) and stream processing (Kafka)...SeniorBig dataFull time
$106.13k
Overview Pyramid Systems is looking for a Data Engineer (Senior) who is passionate about bringing creative architect solutions to end customers... ...Experience with large-scale, high-performance enterprise big data application deployment and solution Understanding how to...SeniorBig dataContract workWork experience placement- ...Senior Data Engineer We are seeking a seasoned Senior Data Engineer to join a dynamic team in Washington, DC. The ideal candidate will have... ...data across various platforms, leveraging cloud services and big data technologies. Responsibilities Design, develop, and...SeniorBig data
Samprasoft
Washington DC2 days ago - ...Booz Allen Hamilton is seeking an AWS Streaming Data Engineer to design and implement end-to-end streaming solutions, including real-time data ingestion, processing, and geopatial data components. You will work across distributed environments to deliver production-ready...Big data
Booz Allen Hamilton
McLean, VA1 day ago - ...from Ageatia Global Solutions... We are looking for a Senior Data Engineer @ Reston, VA for the below mentioned job description. Kindly... ...maintaining and scaling cloud OR on-premise infrastructure targeting big data and platform data management (Relational and NoSQL,...SeniorBig dataWork experience placementRemote work
Ageatia Global Solutions
Reston, VA2 days ago - ...MicroStrategy And Big Data Sr. Engineer Position The Senior Data Engineer is responsible for design, change code and modernize MicroStrategy to integrate with Big Data in the Cloud AWS. Responsibilities: Modernize existing code to integrate MicroStrategy and...SeniorBig data
Samprasoft
Reston, VA2 days ago $50k - $290k
...Reston, VA. Our capabilities include Software Development, Engineering & IT, Data Science, Cyber Enablement, Logistics, and Training. Founded... ...teams Desired Experience Experience working with data lake and big data technologies such as Spark, Accumulo, or Elasticsearch...SeniorBig dataContract workWork experience placementThe Swift Group, LLC
Reston, VA3 days ago- ...Consulting Inc. is seeking a highly skilled Full Stack Developer with Big Data expertise to support mission-critical programs within the... .... The ideal candidate will bring a strong mix of full stack engineering capabilities and data-intensive application experience , with...SeniorBig dataFull time
Kda Consulting Inc
McLean, VA1 day ago - ...Data Engineering Lead Spear AI is a growing defense contracting company dedicated to delivering cutting-edge solutions that support our nation... ...~ Expertise in distributed computing frameworks and big data technologies ~ Advanced knowledge of database systems,...SeniorBig dataFull timeRemote workHome officeRelocation packageFlexible hours
Spear AI
Washington DC10 hours ago - ...Enterprise Data Architecture & Data Modeling Expertise ~ Design and maintain... ...analytics and operations. Advanced Data Engineering & ETL/ELT Development ~ Build,... ...unstructured data. Strong SQL, Python, and Big Data Experience ~7+ years of...SeniorBig dataContract work
3B Staffing LLC
Washington DC1 day ago - ...Ex Parte provides our customers with the data and insight to make smart and informed decisions on the most important... .... We are looking for talented, enthusiastic senior data engineers who share our passion for big data, AI, and machine learning and are excited by seemingly...SeniorBig data
Ex Parte
Bethesda, MD4 days ago $115k - $150k
...Senior Data Engineer Insomniac Design is a global digital agency headquartered in Washington D.C., with offices in London, Bucharest and... ...Factory) GCP (BigQuery, Dataflow) ~ Hands-on experience with big data tools: Apache Spark, Kafka, Airflow, dbt, Snowflake, Databricks...SeniorBig dataVisa sponsorshipInsomniac Design, Inc.
Washington DC4 days ago- ...motivated candidate to join our talented Team. Job Title: Senior IT Data Engineer Locations: NW Washington, DC Job Description: At least seven... ...: Ability to implement and manage cloud-based data and big data systems including: Amazon Aurora, Snowflake, Databricks, Amazon...SeniorBig data
- ...candidate to join our talented Team. Job Title: Data Engineer. Location: McLean, VA. Overview We are seeking a highly skilled Sr. Data Engineer to join our team in McLean, VA.... ...in data engineering, cloud services, and big data technologies, with a strong focus on AWS...Big dataLocal areaRemote work
Ampcus
McLean, VA4 days ago - ...Big Data Engineer We are looking for a skilled and motivated Big Data Engineer to join our team and help build scalable, high-performance data systems. In this role, you will design, develop, and maintain data pipelines, ensure efficient data processing, and optimize...Big dataTemporary workFor contractorsImmediate startFlexible hours
Cymertek
McLean, VA4 days ago - ...Data Engineer II Seeking an experienced Data Engineer II to join the Analytics Solutions Team. This team partners with stakeholders to extract... ...delivery of business-critical data. Develop and optimize big data workflows using Apache Spark and Hadoop technologies....Big dataMonday to Friday3 days per week
Delan Associates Inc
Arlington, VA3 days ago - ...KDA Consulting Inc. is seeking a highly skilled Data Engineer to design, build, and maintain scalable data pipelines and architectures supporting... ...with AWS cloud services, data engineering best practices, and big data technologies , with the ability to operate in a fast-...Big data
KDA CONSULTING INC
McLean, VA1 day ago - ...Overview We are seeking a skilled Data Engineer with at least 5 years of experience designing, building, and maintaining scalable data pipelines... ...(e.g., PostgreSQL, MySQL, SQL Server). Experience with big data processing frameworks (e.g., Spark, Hadoop). Hands‑on experience...Big data
VTG Defense
McLean, VA4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Sr. Big Data Engineer. Be the first to apply!
Related searches
- software data engineer McLean, VA
- entry level big data engineer McLean, VA
- big data developer McLean, VA
- senior data quality engineer McLean, VA
- sr data engineer McLean, VA
- junior big data engineer McLean, VA
- big data cloud engineer McLean, VA
- senior cloud data engineer McLean, VA
- data developer McLean, VA
- data science developer McLean, VA