Data Engineer
Prophecy Technologies
Job Summary: We are seeking an experienced Data Engineer to join our team. The ideal candidate will have strong expertise in designing and implementing efficient and scalable data pipelines using Apache Spark with Java, integrating data from diverse sources into data lakes (e.g., S3) and data warehouses (e.g., Redshift). This role offers an exciting opportunity to work on enterprise-level data engineering projects. Key Responsibilities:
- Design and develop efficient and scalable data pipelines using Apache Spark with Java, integrating data from diverse sources into data lakes (e.g., S3) and data warehouses (e.g., Redshift).
- Leverage a range of AWS services for data storage, processing, and analytics, including but not limited to S3, Redshift, Glue, EMR, Lambda, Kinesis, and DynamoDB.
- Optimize Spark applications and data pipelines for performance, cost-efficiency, and reliability, including tuning Spark configurations and utilizing appropriate AWS resources.
- Design and implement data models for structured and unstructured data, and contribute to the overall data architecture strategy within an AWS environment.
- Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.
- Implement monitoring solutions for data pipelines and infrastructure, and troubleshoot issues to ensure data quality and system stability.
- Implement data security measures and adhere to data governance policies within the AWS ecosystem.
- Create and maintain comprehensive documentation for data engineering processes, designs, and deployments.
- Strong proficiency in Java, with experience in developing Spark applications.
- In-depth knowledge and hands-on experience with Apache Spark, including Spark SQL, DataFrames, and RDDs.
- Extensive experience with AWS services, particularly those related to data engineering (S3, Redshift, Glue, EMR, Kinesis, Lambda).
- Experience with data warehousing concepts and technologies (e.g., Redshift, Snowflake), and building data lakes on S3.
- Proficiency in SQL and experience with both relational and NoSQL databases.
- Strong understanding and practical experience in designing and implementing ETLELT processes.
- Excellent analytical and problem-solving skills, with the ability to troubleshoot complex data issues.
- 8-10 years of experience in data engineering.
- Relevant certification in data engineering or related field.
- Bachelor's degree in Computer Science or related field.
- Experience with PySpark.
- Strong communication and collaboration skills to work effectively within a team and with various stakeholders.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Data Engineer in Wilmington, DE vacancy
$179.4k - $204.7k
...Lead Data Engineer The Bank Tech Data, Decisioning, & Market Services Domain serves as the analytical engine of the bank, building the infrastructure that turns data into smart, fast decisions. We integrate data and AI capabilities into a single, cohesive platform that...SuggestedFull timePart timeInternshipH1bLocal areaCapital One
Wilmington, DE17 days ago$106.7k - $177.9k
...secure source code at an advanced level in a minimum of one programming language to be verified by a lead software engineer and apply knowledge of appropriate data structure and algorithms to solve business problems. Utilize source code management tools to manage and...SuggestedWork experience placementWork from home1 day per weekM&T Bank
Wilmington, DE1 day ago- ..., we combine global reach with deep expertise. Our Digital & Data Analytics practice drives growth and innovation for some of the... ...in the right place. JOB DETAILS: Job Title: Data Engineer Location: Wilmington DE - Local preferred Duration: 9+ Months...SuggestedContract workLocal area
Staffworxs Inc
Wilmington, DE1 day ago - ...A client of Innova Solutions is immediately hiring a Data Engineer Position type: Contract Duration: 7+ months with possible extension Location: Wilmington, DE 19801 Hybrid As a Data Engineer you will: This is for a Data engineer focused on building...SuggestedHourly payContract workTemporary workWork experience placementImmediate startWorldwideFlexible hours
Innova Solutions
Wilmington, DE2 days ago - ...Job Title: Data Engineer (Ab Initio → Databricks Modernization) - Location: Wilmington, DE - 5 Days onsite role Overview We are seeking a Data Engineer to lead the modernization of legacy ETL systems by migrating Ab Initio workflows to scalable, modular PySpark...Suggested
ApTask
Wilmington, DE5 days ago - ...Tiger Analytics Data Engineer Opportunity Tiger Analytics is a global leader in AI and advanced analytics consulting, empowering Fortune 1000 companies to solve their toughest business challenges. We are on a mission to push the boundaries of what AI can do, providing...
Tiger Analytics
Wilmington, DE13 hours ago - Job Description A large semiconductor manufacturing company is seeking a Data Engineer to join their team for a large ongoing digital transformation project. This person will be responsible for designing, building and maintaining cloud native data pipelines on Azure to...Remote work
- ...of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers who are passionate about marrying data with emerging technologies. As a Capital One Lead Data Engineer, you’ll have the opportunity...Full timePart timeInternshipLocal area
$106.7k - $177.9k
## Snowflake Data EngineerApplyremote type: Hybrid Positionlocations: Wilmington, DEtime type: Full timeposted on: Posted Todayjob requisition... ...of one programming language to be verified by a lead software engineer and apply knowledge of appropriate data structure and algorithms...Work experience placementLocal areaWork from home1 day per week$80 - $85 per hour
Overview Location: Wilmington, DE Salary: $80.00 USD Hourly - $85.00 USD Hourly Description: Role: Senior Data Engineer • Duration: 6-12+ months RTH • Location: Wilmington, Delaware • Interview Process: 2 rounds Responsibilities Day to day is primarily setting up data...Hourly pay- BDO USA is looking for a Lead Data Engineer in Wilmington, Delaware, who will lead the team in managing and optimizing data assets for the firm. This role requires strong collaboration with various teams to ensure accurate data management and governance. The ideal candidate...Remote job
- Capital One is seeking a Lead Data Engineer in Wilmington, DE, to lead innovative solutions that transform data into actionable insights. This role combines collaboration with Agile teams, advanced programming, and the use of cloud technologies to facilitate financial empowerment...
$193.4k - $220.7k
...Python and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Redshift and Snowflake Share your... ...technology communities, and mentoring other members of the engineering community Collaborate with digital product managers, and deliver...Full timePart timeInternshipH1bLocal area- ...request applicants to apply who are authorized to work without sponsorship* Employment TYPE: Contract W2 - NO C2C Job Title: Big Data Engineer About BCforward BCforward is a leading global IT consulting and workforce solutions firm providing services and support to...Contract work
- M&T Bank is looking for an advanced-level developer responsible for writing code and gathering technical requirements. This position requires at least 5 years of systems development experience and proficiency in programming languages, with opportunities for autonomous work...Flexible hours
$269.1k - $307.2k
...Distinguished Data Engineer- Bank Tech Distinguished Data Engineers are individual contributors who strive to be diverse in thought so we visualize the problem space. At Capital One, we believe diversity of thought strengthens our ability to influence, collaborate and...Full timePart timeLocal areaCapital One
Wilmington, DE10 days ago- Job Summary We are seeking a Data/Platform Engineer to accelerate a biomarker data lake modernization within Computational Discovery. The project replaces a fragmented ETL landscape with a unified AWS-native platform: Dagster orchestration, dbt transformations, Iceberg...For contractorsLocal area
- A technology firm is seeking an experienced Data Engineer for a contract position located in Yorklyn, DE. The role requires 8-10+ years of experience, with strong competencies in Databricks, EDI, and Supply Chain knowledge. The candidate should also possess hands-on experience...Contract work
- DATA ENGINEER RETAIL _ SUPPLY CHAIN Location: Yorklyn, DE Hire Type: Contract Competencies: 8-10+ years experience required Digital : Databricks EDI - Electronic Data Interchange Azure Databricks and Data Pipelines knowledge Good Retail experience with Supply Chain knowledge...Permanent employmentContract workLocal area
$80 - $85 per hour
Jobs via Dice is looking for a Senior Data Engineer in Wilmington, DE. The role involves setting up data pipelines daily and requires expertise in Java or Python, Spark, AWS or Databricks, and Airflow. This position offers a salary range of $80.00 - $85.00 USD per hour...Hourly pay$239.9k - $273.8k
A leading financial institution in Wilmington, DE is seeking a Distinguished Data Engineer to drive modern data architecture and strategy. The role requires extensive experience in data engineering, architecture, and a strong technical foundation in AWS. Candidates should...- M&T Bank Corporation is looking for a Snowflake Data Engineer in Wilmington, DE. This hybrid role requires strong programming skills and experience in data management to develop applications that meet banking needs and ensure compliance with standards. The ideal candidate...Remote job
- ...We are seeking a Principal Data Engineer to drive scalable, business-focused data solutions that power insight-driven decision-making across the enterprise. This role is ideal for someone who combines deep technical expertise in modern data platforms with the ability...Local areaRemote work
Cornerstone OnDemand
Wilmington, DE2 days ago - ...practical LLM use case development experience using OpenAI Azure. Experience - Formal training or certification in software engineering concepts with over 7+ years of applied experience. Degree in Computer Science, Computer Engineering, Mathematics, or a...
Procyon TS
Wilmington, DE5 days ago - A leading financial services company in Wilmington, DE is seeking a Senior Lead Data Engineer to drive transformative projects. This role involves collaborating with Agile teams, utilizing emerging technologies, and leading development efforts in big data and cloud computing...
- Symbotic is seeking an AI Analytics Engineer in Wilmington, Delaware to develop an intelligent analytics ecosystem. This role involves data integration, AI-driven analytics and dashboard creation, enabling real-time visibility into business performance. The ideal candidate...
$120k - $165k
...speed and flexibility. What we need We are looking for an AI Analytics Engineer to join Symbotic. Your job will be to design and build an intelligent analytics ecosystem that aggregates operations data across multiple systems and leverages agentic AI to generate insights...- A leading IT consulting firm is looking for a Big Data Engineer in Wilmington, Delaware to design and maintain scalable data processing pipelines. The ideal candidate will have strong experience in cloud data services and programming languages such as Python. Responsibilities...Contract work
- A leading financial services consulting firm is seeking an experienced IBM TM1 Developer for a long-term project in Wilmington, Delaware. The role involves hands-on development in IBM Planning Analytics and requires strong financial planning and analysis expertise, particularly...
- JPMorgan Chase & Co. is looking for an Analytical Solutions Senior Associate to enhance financial data transformation and reporting throughout the organization. This role focuses on developing and implementing analytical solutions to convert data into actionable insights...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!
Related searches
- data engineering intern summer Wilmington, DE
- senior data integration developer Wilmington, DE
- data science developer Wilmington, DE
- senior data center engineer Wilmington, DE
- software data engineer Wilmington, DE
- data developer Wilmington, DE
- sr data engineer Wilmington, DE
- big data cloud engineer Wilmington, DE
- junior data engineer remote Wilmington, DE
- aws data engineer Wilmington, DE