Big Data Developer
Techvilla Solutions
We are looking for an experienced Big Data Developer with strong expertise in big data technologies, distributed data processing, and large-scale data analysis. The ideal candidate should have hands-on experience with Kafka, Spark, Scala, SQL, Hadoop, Hive, and Impala, along with performance tuning and troubleshooting in the Cloudera Big Data ecosystem.
Key Responsibilities
- Develop, maintain, and optimize big data applications and data processing pipelines.
- Work with Kafka and Apache Spark for real-time and batch data processing.
- Perform data analysis, data wrangling, and processing of large volumes of structured and unstructured data.
- Write and optimize complex SQL queries using Hive and Impala.
- Monitor, troubleshoot, and resolve performance issues in the Cloudera Big Data ecosystem.
- Use Splunk for log analysis, monitoring, and troubleshooting.
- Manage and monitor batch workflows and job scheduling using AutoSys.
- Work in Linux-based environments for development, deployment, and operational support.
- Collaborate with business users and cross-functional teams to deliver high-quality data solutions.
- Ensure data quality, reliability, scalability, and performance of big data applications.
Required Technical Skills
- Strong hands-on experience with Kafka, Spark, Scala, and SQL.
- Experience with Hive, Impala, Hadoop, and the Cloudera ecosystem.
- Proficiency in data analysis and data wrangling for large datasets.
- Experience in performance analysis, tuning, troubleshooting, and issue resolution.
- Hands-on experience with Splunk and AutoSys.
- Strong knowledge of Linux-based infrastructures.
- Experience in distributed data processing and big data technologies.
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Big Data Developer in Brighton, NY vacancy
- ...Design, develop, and maintain scalable data pipelines supporting Financial Crime & Compliance (FCC), AML, KYC, and Trade Surveillance initiatives. Work with FCC products and platforms such as NICE Actimize, ActOne, AIS, and Trading Hub to integrate transaction...Suggested
Longfinch Technologies
Brighton, NY1 day ago $90k - $140k
...Data EngineerMindex is looking for a Data Engineer to build the data pipelines, models, and warehouse structures behind SchoolTool's... ...warehouse structures for SchoolTool's dashboards and analytics.Develop analytics solutions with Python, Apache Spark, Snowflake, SQL and...SuggestedWork at officeMindex Technologies
Rochester, NY1 day ago$80 - $93 per hour
...SigmaFlow SME – Data Engineer 6-month contract Location: Remote Pay: $80 - $93/hr (compensation may vary based on experience... ...data inconsistencies. Data Management & Database Development Develop and maintain connections between SigmaFlow and enterprise...SuggestedContract workRemote workCypress HCM
Rochester, NY3 days ago- ...Who We’re Looking For Megazone Cloud is looking for a Data Engineer Co-op who is a "builder" at heart and eager to apply technical skills to real-world customer problems. We need students who are curious, driven, and comfortable asking "what if?" to solve complex...SuggestedImmediate start
Megazone Cloud US
Rochester, NY3 days ago - ...Job Title: Senior Data Engineer Location: Remote (US) Overview We are seeking a hands-on Senior Data Engineer to build,... ...models across Bronze, Silver, and Gold layers in Databricks. Develop ingestion patterns and Bronze-to-Silver transformations that produce...SuggestedContract workRemote work
NexTech Solutions
Rochester, NY3 days ago - ...Data is essential to how the firm manages risk and delivers outcomes. This role leads a high-visibility data lineage delivery effort by... ...solutions. Technical knowledge of data management or governance, big data platforms, or data architecture. Master of Business...Big dataWork at office
$105.4k - $207.8k
...accelerate new digital ventures, and fuel growth through innovation.Work you'll do As a Data Engineer III on the AI & Data team, you will be responsible for… Designing, developing, testing, and maintaining batch and near-real-time data pipelines using modern data engineering...Local areaDeloitte
Rochester, NY4 days ago- ...internal and external IT tickets, and assisting in server and switch upgrades to improve production outputs. You will also help the data team implement new tools and ensure production drift detection is enhanced and tools are deployed effectively with operator training...
SAFRAN AEROSPACE COMPOSITES LLC.
Rochester, NY3 days ago - ...Clear Channel Outdoor Job Summary: Clear Channel Outdoor (CCO) has developed an industry-leading platform (RADAR) that provides data-driven audience insights and measurement for out-of-home (OOH) campaigns. We are looking for a detail-oriented and business-minded Analytics...Temporary workWork experience placementWork at officeFlexible hours
Clear Channel Communications, Inc.
Rochester, NY4 days ago $105.79k - $141.05k
...the trusted network for the AI‑powered world, connecting people, data, and applications through our expansive fiber network and... ...defensive, business, and executive stakeholders. ~ Experience developing security tools, exploits, payloads, automations, and new Red Team...Full timeTemporary workRemote workLumen
Rochester, NY2 days ago- ...Investments (and more), working with major brokerages and financial data providers. What Were Building InvestOS — an open standard for... ...generation, AI reasoning, analytics, and investment products. Develop reliable, scalable pipelines that ingest, transform, and enrich...Remote workFlexible hours
Tilt.com
Rochester, NY5 days ago $110.7k - $218.3k
...marketing, commerce, sales, and service. We are a team of strategists, data scientists, operators, creatives, designers, engineers, and... ...at ****@*****.***. Recruiting tips From developing a stand out resume to putting your best foot forward in the...Local areaDeloitte
Rochester, NY4 days ago$75k - $100k
...actionable opportunities and driving customer satisfaction through data-driven analysis for prioritization of corrective measures and... ...charts, histograms, pareto charts, and scatter diagrams. Ability to develop statistically appropriate sampling plans. Ability to use and...Temporary workFlexible hoursAmerican Packaging Corp.
Rochester, NY4 days ago- ...Senior Mechanical Engineer — Data Centers & Mission-Critical Facilities (Remote) As a Senior Mechanical Design Engineer at Critical Facility Group, youll lead mechanical design for data centers and other mission-critical facilities — from engineering calculations and...Work at officeRemote work
Critical Facility Group
Rochester, NY1 day ago - ...adMarketplace in New York City is hiring an Analytics Engineer to build the trusted analytical data foundation that powers product decisions, experimentation, measurement, and self-service analytics across the company. As the founding hire for Analytics Engineering...
adMarketplace
Rochester, NY1 day ago $27.16 - $38.03 per hour
...determined by considering factors including, but not limited to, market data, education, experience, qualifications, expertise of the... ...'s responsibilities. Keeps abreast of current application developments through continuing education, professional reading, online forums...Hourly payDaily paidFor contractorsLocal areaShift workUniversity of Rochester
Rochester, NY5 days ago- ...Role: Senior Azure Data Migration & Dataverse Integration Engineer Company: Arch Systems Location: Remote Type: Full-... ...Azure Data Migration & Dataverse Integration Engineer to design, develop, implement, and support enterprise data migration and...Full timeTemporary workRemote work
Arch Systems
Rochester, NY4 days ago $130k - $175k
...exploitation of sensitive personal and business data, demand for national security‑focused risk... ...Maturity Model Certification (CMMC). Develop comprehensive project plans, establish key... ...security controls for distributed, big data systems with emphasis on least‑privilege...Big dataPart timeFlexible hoursAlvarez & Marsal
Rochester, NY1 day ago- ...employing an innovative, disciplined and business-focused approach to develop a wide variety of high-quality products and solutions. Across 1... ...deploy everything from enterprise technology initiatives to big data and mobile solutions, as well as innovations in Agentic/...Big dataFull timeWorldwide
$300k - $350k
...in the world. Agile teams of researchers, engineers, and traders develop robust systems that allow us to operate performant at scale and... ...variety of areas, including but not limited to: using sophisticated data analysis skills to build predictive models, driving construction...Citadel
Rochester, NY5 days ago$300k - $350k
...Global Quantitative Strategies (GQS) in the United States seeks Quantitative Researchers to develop predictive models and multi-asset portfolio construction using large-scale data. You will collaborate with researchers, engineers and traders to apply advanced statistics...Citadel
Rochester, NY4 days ago$4,500 - $5,800 per week
...world. Quantitative Researchers play a key role in this mission by developing next-generation models and trading approaches for a range of... ...techniques to financial markets, some of the most complex data sets in the world. As an intern, you'll dive into research through...InternshipFlexible hoursCitadel LLC
Rochester, NY55 minutes ago$4,500 - $5,800 per week
...brings trading strategies to life every day. Specifically, this team develops and tests automated quant trading strategies using sophisticated... ...techniques to financial markets, some of the most complex data sets in the world. As an intern, you’ll dive into research through...InternshipFlexible hoursCitadel Securities
Rochester, NY1 day ago$225k - $300k
...world. Quantitative Researchers play a key role in this mission by developing next-generation models and trading approaches for a range of... ...techniques to financial markets, some of the most complex data sets in the world. Your Objectives Conceptualize valuation...Citadel LLC
Rochester, NY55 minutes ago- Position Summary Data Engineer-Consultant AI&EAre you an experienced, passionate pioneer in technology who wants to work in a... ...stakeholder collaboration.Technical lead, responsible for Design, develop, and operationalize scalable healthcare data platformsDesign,...
Deloitte
Rochester, NY3 days ago - ...Megazone Cloud US is seeking a Data Engineer Co-op who is a builder at heart and eager to apply technical skills to real-world customer problems. You will work as a real member of the technical team, helping build and maintain data pipelines powering our business and...
Megazone Cloud US
Rochester, NY3 days ago $107.66k - $161.7k
...and our engineers are encouraged to think big and experiment with new ideas. Using... ...product. You will also play a key role in developing tools and abstractions that our other developers... ...Learning systems -- from prototyping, data pipelines and training, to realtime LLM...Remote jobFull timeWork experience placementInternshipQuora
Rochester, NY5 days ago- ...engineers and product leaders as part of your team. Together, you'll develop and enhance Instacart's marketplace systems. You will use... ...foundational problems executing on a high throughput system with dynamic data. About the Job: Design, develop, and deploy machine...Permanent employmentFull timeWork at officeRemote workWork from homeFlexible hours
Instacart
Rochester, NY5 days ago $18.89 - $31.49 per hour
...troubleshooting production and NPI fixtures. Develop documentation to enable fixture... ...the role At MKS, it's all about courage, big ideas, and a serious passion for innovation... ...artificial intelligence, cloud technology, and big data. We're the behind-the-scenes support for...Big dataHourly payPermanent employmentWork at officeRelocation package$71.03k - $118.38k
...partner with engineering, IT, and operations to develop highly-automated test and production... ...the role At MKS, it's all about courage, big ideas, and a serious passion for innovation... ...artificial intelligence, cloud technology, and big data. We're the behind-the-scenes support for...Big dataPermanent employmentWork experience placementSecond jobWork at officeRelocation package
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Big Data Developer. Be the first to apply!
