Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Big Data Engineer (Java or Python, ETL)

Samprasoft

Senior Software Engineer Location: Remote/MA/NH/RI/NJ/NC/TX/UT/KY Will be doing API Development, stream processing and model deployments. API Development experience (will be deploying API's into AWS) and understanding of Kinesis and stream processing. MUST HAVE: Java OR Python, AWS and ETL experience.

MANAGER NOTES:

Project Information : Will be on the Content Recommendation Engineer Squad and building out processes, recommending content to customer's. Will be doing API Development, stream processing and model deployments. Top must have skills: AWS, Java (middle tier and back end 30/70) and python programming & experience with data movement technologies (ETL/ELT) Preferred skills: API Development experience (will be deploying API's into AWS) and understanding of Kinesis and stream processing. Selling point: Will work with modern cutting edge technologies and work to develop content the customer's see. Also, this role has potential to convert to a FTE down the road and also participate in Learning Day Tuesday. Job Description: The Role: The Customer Data Technology group within CLIENT’s Personal Investing (PI) organization is seeking a Senior Software Engineer to build and maintain large scale data processing systems. In this role, you will apply variety of technologies to develop innovative Big data solutions. This position is a critical element to delivering CLIENT’s promise of creating the best customer experiences in financial services. If you are an experienced Software engineer with a passion for designing and delivering big data solutions using groundbreaking technologies, want to be a part of exciting data simplification journey, looking for a collaborative team environment where you will have wealth of opportunities to innovate and have intellectual curiosity to learn, a career in Customer Data Technologies in PI may be right for you! The Expertise and Skills You Bring Bachelor’s or Master’s degree in a technology related field (e.g. Engineering, Computer Science, etc.) required. 6+ years of experience in software engineering and development 4+ years of experience in implementing Big data solutions in data analytics space Extensive experience in Object Oriented Programming (Java, Scala, Python) Experience with building data pipelines in getting the data required to build and evaluate ML models, using tools like Apache Spark or other distributed data processing frameworks. Experience Data movement technologies (ETL/ELT), Messaging/Streaming Technologies (AWS SQS, Kinesis/Kafka), Relational and NoSQL databases (Cassandra, Elastic search, Graph database), API and in-memory technologies. Solid understanding of developing highly scalable distributed systems using Open-source technologies Experience with DevOps, Continuous Integration and Continuous Delivery (Maven, Jenkins, Stash, Ansible, Docker) Solid experience in Agile methodologies (Kanban and SCRUM) Experience in developing applications in Cloud is a plus Strong technical design and analysis skill Solid understanding and technology trends in implementing of Big data ecosystem Solid understating of data architecture patterns such as Lambda, Kappa, Event driven Architecture, Data as a Service, Microservice etc. Designing, Building and supporting mission critical applications to provide the best customer experience Delivering system automation by setting up continuous integration/continuous delivery pipelines The Team: Customer Data Technology in PI supports the platforms that enable business users to collect and analyze the customer data needed to provide the best customer experience. Samprasoft

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Big Data Engineer (Java or Python, ETL) in New York, NY vacancy
  •  ...talented and driven Senior AWS Data Engineer to join our growing team in...  ...develop, and maintain scalable ETL pipelines using AWS Glue, EMR,...  ...SQL skills and experience with big data technologies (Spark, Hadoop, Hive) ~ Experience with Python or Scala for data engineering... 
    Python
    Big data

    Ensisinfo Inc

    New York, NY
    22 days ago
  • Big Data Software Engineer (SQL, Java/Scala) Position: Contract to Hire Summary of the project/initiative: The candidate will be supporting big data application...  ...hands on in SQL. Knowledge of Spring Framework, Kafka, Python is preferable. Healthcare industry background is good to... 
    Python
    Big data
    Contract work
    Remote work

    Samprasoft

    New York, NY
    1 day ago
  •  ...supporting a world-class data platform for our portfolio...  ...an experienced and eager engineer to join our team in supporting...  ...Petabyte-level ingestion, ETL, data quality, storage,...  ...especially timeseries and big data stores.• Strong skills in Python/Rust/C++ and at least one... 
    Python
    Big data
    Work experience placement

    Balyasny Asset Management

    New York, NY
    4 days ago
  • $10k

     ...About the Role:   As a Senior Data Engineer at Hinge, you will create...  ...will be working on an actual big data architecture while concentrating...  ...data modeling, developing our ETL pipelines, and more. Use...  ...experience ~ Proficient in Python, SQL, DevOps, and databases.... 
    Python
    Big data
    Work experience placement

    Hinge

    New York, NY
    a month ago
  • Job Description: Senior ML Data Engineer Feature Engineering ETL Qualifications 7 years in data engineering and at least 4...  ...advanced skills in Apache Spark PySpark for big data processing and analytics Proficient in Python for data manipulation and SQL for query optimization... 
    Python
    Big data
    Remote job

    TriOptus LLC

    New York, NY
    1 day ago
  • $100 - $130 per hour

     ...financial firm is looking for a Data Engineer - Databricks to join their...  ...more scripting languages (e.g., Python, bash, etc.)Embrace data platform...  ..., Cassandra, or Solr other Big Data query and transformation...  ...is preferredExperience using ETL/ELT tools and technologies such... 
    Python
    Big data
    Remote work

    Open Systems Technologies

    New York, NY
    3 days ago
  • $150k - $175k

    Data Engineer Title of Role: Data Engineer Location: -, hybrid Company Stage...  ...in SQL and experience with ETL tools and data warehousing...  ...programming languages such as Python or Java for data manipulation and analysis...  .... Experience working with big data technologies such as... 
    Python
    Big data
    Work at office

    Recruiting from Scratch

    New York, NY
    1 day ago
  • Job Title: Data Engineer Location: NJ/NYC or Chicago - 3-4 days a week...  ...Science with a focus on Big Data, you will play a critical...  ...programming skills in Python, Scala, R, or Java, with experience in libraries...  ...large datasets, performing ETL tasks and optimizing data pipelines... 
    Python
    Big data
    3 days per week

    3B Staffing LLC

    New York, NY
    1 day ago
  • Job Title:Lead Data Engineer - Big Data & Cloud (AWS)(Python) Location: New York, NY (Onsite) Contract: 6 Months Visa Status: ONLY GC & USC Strong Python and...  ...systems . The role focuses on building event-driven ETL/ELT pipelines, real-time data ingestion frameworks, and... 
    Python
    Big data
    Contract work

    Argyll Infotech, Inc.

    New York, NY
    1 day ago
  • $150k - $200k

     ...verticals, supplemented with Data & AI, Cybersecurity, Risk...  ...an experienced Senior Java Developer – Credit Risk & Data Engineering to design, develop, and...  ...integration components using Java, Python, Scala, and/or comparable...  .... Experience with Big Data ecosystems and... 
    Python
    Big data
    Full time
    Temporary work
    Remote work
    Worldwide

    BIP US

    New York, NY
    1 day ago
  • $70 - $75 per hour

     ...a versatile Full Stack Data Engineer to spearhead the modernization...  ...at the intersection of big data processing, robust...  ...Data Architecture & ETL: Build and optimize...  ...using Kafka, Spark, and Python ecosystems. Systems...  ...Django, SQLAlchemy), Core Java, Node.js (ExpressJS,... 
    Python
    Big data
    Hourly pay
    Contract work
    Temporary work
    Work experience placement

    Randstad Digital

    New York, NY
    2 days ago
  • Position We are looking for an experienced AWS Data Engineer to join the team that develop a data-...  ...from a wide variety of data sources using Python, SQL, scripting, and AWS big data technologies. Design and implement complex ETL pipelines and other BI solutions. Work... 
    Python
    Big data
    Remote work
    Home office
    Flexible hours

    Grinteq

    New York, NY
    1 day ago
  •  ...BDG) is looking for a talented Data Engineer to join our growing data team...  ...Build scalable and efficient ETL pipelines to support analytics...  .... Proficient in SQL & Python: Strong knowledge of SQL for...  ...scripting and automation. Cloud & Big Data Savvy: Hands-on... 
    Python
    Big data
    Work at office
    Remote work
    Flexible hours

    BDG

    New York, NY
    more than 2 months ago
  • Principal Data Engineer As a Principal Data Engineer, you will be expected...  ...Previous experience with Big data technologies, ETL, ELT, statistics and causal...  ...programming skills in Python Experience in handling large...  ...analytics languages (Python, Java), in addition to open... 
    Python
    Big data
    Remote job

    Software Technology Inc

    New York, NY
    1 day ago
  • Data Engineer Location: Remote. Duration: Full-Time. NEED IRS MBI Clearance...  ...pipelines using modern big data technologies. The ideal...  ...experience Strong proficiency in Python, Java or Scala programming languages...  ...of data modeling and ETL / ELT best practices Familiarity... 
    Python
    Big data
    Full time
    Remote work

    3M Consultancy

    New York, NY
    1 day ago
  • NAVA Software is looking for a Senior Data Engineer Details: Senior Data Engineer...  ...below tech skills Strong experience in Python Scripting and PySpark for data processing...  ...Proficiency in SQL , dealing with big data over Informatica ETL Proven experience in Data quality... 
    Python
    Big data
    Remote job
    Work experience placement

    Nava Software Solutions

    New York, NY
    1 day ago
  •  ...Principal Software Engineering position at Associate...  ...PySpark‑based ETL frameworks with error...  ...environments. Perform data enquiry, data...  ...in Relational and Big Data database environments...  ...in all: Pyspark, Python, SQL Strong knowledge...  ...Experience in Java Development Experience... 
    Python
    Big data
    Temporary work

    Morgan Stanley

    New York, NY
    15 hours ago
  •  ...Echelon. Senior Investment Data Engineer Position Summary We are seeking...  ..., build, and maintain robust ETL/ELT processes for ingesting,...  ...solutions. Proficiency in Python and modern data engineering tools...  .... Experience with big data platforms and technologies... 
    Python
    Big data
    Work at office

    Blake Smith Staffing, LLC

    New York, NY
    3 days ago
  • Senior AWS Data Engineer Location: New Jersey - Onsite Duration: Fulltime Job Description: Key...  ...Ability to proficiently write code in Python, Node.js, Bash (shell), PowerShell, or...  ...Keys, VPC NACLs, and SCPs Familiar with ETL and big data tool-chains such as those provided... 
    Python
    Big data
    Full time

    Zortech Solutions

    New York, NY
    1 day ago
  • $125k - $160k

     ...are looking for seasoned Data Engineer to work with our team...  ...Assess and understand ETL jobs, workflows, data marts...  ..., SQL, PySpark/Python, AWS Support an Agile software...  ...in tools such as: Big data tools: Databricks,...  ...languages: PySpark/Python, Java, C++, Scala, etc.... 
    Python
    Big data

    Steampunk.com

    New York, NY
    1 day ago
  • $30 - $80 per hour

    About the job Remote | Big Data Engineer - $30-$80/hour We are sharing a specialised part-time consulting...  ...reliability. The work requires strong Python skills, deep knowledge of database...  ...integration solutions using Python Implement ETL and data-transformation workflows across... 
    Python
    Big data
    Remote job
    Part time
    For contractors

    24-MAG LLC

    New York, NY
    1 day ago
  • Role Data Engineer + Gen AI Location NYC (Hybrid) Duration:...  ...Cloud Platform, Flask, Python, SQL, Docker & Kubernetes...  ...Platform services (Big Query, Dataflow, Dataproc...  ...expertise in coding in Java/Python/Scala. 3+ years...  ...creating low code/no code ETL tool for setting up... 
    Python
    Big data
    Worldwide

    Nucleusteq

    New York, NY
    1 day ago
  •  ...Role We're looking for a Senior Data Engineer that is innovative, curious,...  ...Build, optimize, and maintain ETL/ELT processes for both batch...  ...Lakehouse paradigms Advanced SQL, Python, and PySpark skills and...  ...visualize, and sometimes update big data stores in a hybrid OLAP and... 
    Python
    Big data
    Work experience placement

    datafuelX Inc.

    New York, NY
    1 day ago
  •  ...Description EXL is seeking a Senior Data Engineer to join our Data & Analytics...  .... \n Build and optimize ETL/ELT frameworks for large-scale...  ...in Data Engineering, Big Data Engineering, or Cloud Data...  ...Strong hands-on expertise in SQL, Python, and PySpark. \n Extensive experience... 
    Python
    Big data
    Local area
    Relocation

    Energy Jobline ZR

    New York, NY
    1 day ago
  • $149.57k - $184.92k

    Sr Staff Data Engineer - GE07DE We're determined to make a difference and...  ...experiment, and utilize leading big data methodologies (AWS,...  ...including SQL and NoSQL; and ETL tools; AND 3 years of experience...  ...Technologies, including AWS and Azure; Python or Spark; Datamesh; Datalake;... 
    Python
    Big data
    Remote work

    The Hartford

    New York, NY
    1 day ago
  • $145.3k - $196.6k

     ...Technology, & Enablement organization (SITE). We are a big-data focused engineering team that provides unique, pan-Amazon datasets...  ...lake infrastructure using AWS big data stack, Python, Redshift.• Develop and manage ETLs to source data from various data lakes and create... 
    Python
    Big data
    Flexible hours

    Amazon

    New York, NY
    3 days ago
  • Data Engineer Transform technology into opportunity as a Data Engineer at...  ...routines and functions ETL. Implement data governance and...  ...highly preferred Experience with big data technologies such as Apache...  ...programming languages like Python, Java, Scala, or Go. Excellent... 
    Python
    Big data
    Remote work

    Gdit

    New York, NY
    1 day ago
  • $90k - $135.29k

     ...50 E 42nd Street, New York, NYThe Data Engineer II focuses on data collection, movement...  ...works with both current ETL/Data Warehousing and future Big Data/Streaming/Pipeline architectures...  ...programming languages among Scala/Python/ Java. Must be flexible and fast to pick... 
    Python
    Big data
    Traineeship
    Local area
    Remote work
    Flexible hours

    Mount Sinai Health System

    New York, NY
    3 days ago
  • Title: Lead Data Engineer - Databricks Location: New York, NY (3 days/week...  ...use cases. Build and optimize ETL/ELT frameworks for large-scale...  ...experience in Data Engineering, Big Data Engineering, or Cloud...  ...Strong hands-on expertise in SQL, Python, and PySpark. Extensive... 
    Python
    Big data
    Work at office
    Remote work
    3 days per week

    Wise Skulls

    New York, NY
    1 day ago
  • Lead Data Engineer Join us as we embark on a journey of collaboration and...  ...expectations for this role include: Python programming for data...  ...data modeling, data integration/ETL processes, and interoperability...  ...operational management Familiar with big data and analytics engines/... 
    Python
    Big data

    Hackajob

    New York, NY
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Big Data Engineer (Java or Python, ETL). Be the first to apply!