Developer - Data Engineer

Developer - Data Engineer Location: Must be onsite in McLean, VA Extension/Conversion: This is a 12 month contract position, there are no plans for conversion. Technical Must Haves: Python, Pyspark, Hadoop, UNIX, and containerization experience are the MUST HAVES. Job Description • Cleanse, manipulate and analyze large datasets (Structured and Unstructured data – XMLs, JSONs, PDFs) using Hadoop platform

• Develop Python, PySpark, Spark scripts to filter/cleanse/map/aggregate data.
• Be able to build Dashboards in R/Shiny for end user consumption
• Manage and implement data processes (Data Quality reports)
• Be able to handle source code management (GIT, Bitbucket)
• Develop data profiling, deduping logic, matching logic for analysis
• Improve the Query performance, understand and implement required Containerization
• Manage integrated development environments (VSCode. IntelliJ. PyCharm etc)
• Programming Languages experience in Python, PySpark and Spark (Execution, Configuration, Containerization, etc.) for data ingestion.
• Strong knowledge of the Python ecosystem, the Jupyter ecosystem (Lab and Notebook) and their libraries, norms and tooling
• Programming experience in BigData platform using Hadoop platform (HDFS, Hive, Oozie, Sqoop, Parquet, ORC, etc.)
• Familiar with Beeline HQL, Unix Linux, Unix scripting
• Present ideas and recommendations on Hadoop and other technologies best use to management.

Qualifications • Bachelor’s degree in Computer Science, Statistics, Data science or a related quantitative field.

• 4+ years of experience in processing large volumes and variety of data (Structured and unstructured data, writing code for parallel processing, XMLS, JSONs, PDFs)
• 4+ years of programming experience in Hadoop, Spark, Python for data processing and analysis.
• Strong SQL experience is a must.
• 4+ years of experience using Hadoop platform and performing analysis. Familiarity with Hadoop cluster environment and configurations for resource management for analysis work.
• 4+ years of experience Manage integrated development environments, source code management.
• 2+ Prior experience working in Cloud platforms AWS, GCP or Azure. Kubernetes experience is highly desirable.
• Coursework or work experience applying predictive modeling techniques from data science, statistics, machine learning, and econometrics to large data sets.
• Coursework or work experience writing computer programs to implement data science pipelines and predictive algorithms. Programming languages may include—but are not limited to—Python, R, SQL, Java, SAS, and MATLAB.
• Coursework or work experience using technologies for manipulating structured and unstructured big data. Big data technologies may include—but are not limited to—Hadoop, Hive, Pig, Spark, relational databases, and NoSQL. Software Technology Inc