Big Data Engineer (Cloud - Azure & GCP)
Prophecy Technologies
Job Summary: Responsible for designing, building, and optimizing large-scale data pipelines and ETL processes in cloud environments (Azure & GCP). Develop high-performance data ingestion, transformation, and analytics solutions while ensuring data quality, scalability, and reliability. Collaborate with global teams to implement cloud-based data warehouses and analytics platforms. Key Responsibilities:
- Build and maintain cloud-based data pipelines using Azure Databricks, Azure Data Factory (ADF), Azure Data Lake, Synapse, BigQuery, and GCP data services.
- Develop ETL frameworks and reusable solutions to process large volumes of structured and unstructured data.
- Perform performance tuning on Spark (Scala/Python), Hive, and SQL queries to ensure efficient data processing.
- Design and implement Star/Snowflake schemas for analytical data warehouses.
- Ensure data quality, governance, and compliance across cloud environments.
- Collaborate with global teams to deliver high-quality, production-ready solutions and optimize system performance.
- Troubleshoot and resolve data processing and pipeline issues.
- Strong expertise in Azure (Databricks, ADF, Data Lake, Synapse) and GCP (BigQuery, Dataproc, Cloud Storage, Cloud Composer).
- Hands-on experience with Spark (Scala/Python), Hive, SQL, and performance tuning.
- Solid knowledge of relational data modeling and data warehouse concepts (Star/Snowflake schema).
- Experience with Kafka or other streaming technologies.
- Proficiency in Python, Scala, and SQL for data engineering tasks.
- Strong analytical, problem-solving, and communication skills.
- Cloud Data Engineering (Azure & GCP)
- Big Data Processing and Analytics
- ETL Development and Optimization
- Data Warehousing & Modeling
- Performance Tuning & Data Quality
- Snowflake, MySQL, NoSQL databases
- Streaming data pipelines and real-time analytics
- Experience with CI/CD for data pipelines
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Big Data Engineer (Cloud - Azure & GCP) in Cary, NC vacancy
- Mandatory Skills: Data Engineer with experience on Palantir Palantir Data Engineer We are seeking a highly skilled... ...design, and implementing data governance. Cloud & Big Data Ecosystems: Familiarity with AWS, Azure, or GCP and Big Data tools (e.g., Spark, Hadoop)....CloudBig data
- ...leading financial services company in North Carolina seeks a Lead Data Engineer to design, architect, and optimize data solutions. This role requires expertise in big data technologies, particularly Azure and Databricks, and involves mentoring a team while ensuring...Big data
- ..., and optimizing complex data solutions. This role leads... ...- Mentor junior data engineers and provide technical guidance... ...Spark, Airflow, BODS), Big data technologies (e.g.,... ...Spark) (Expert, Required), Cloud data platforms (e.g., AWS, Azure, GCP) , Data streaming and...CloudBig data
First Citizens Bank
Raleigh, NC2 days ago $125.5k - $230.2k
...working world. Technology – Data and Decision Science – Data Engineering – Manager We are... ...and implementing complex cloud analytics solutions with... ...cloud platforms (e.g., AWS, Azure, GCP). Oversee the... ...experience with Spark for big data processing. Strong...CloudBig dataSummer holidayFlexible hoursErnst & Young Oman
Raleigh, NC4 days ago$99k - $232k
...Specialty/Competency: Data, Analytics & AI... ...data and analytics engineering focus on leveraging... ...- Certification in Cloud Platforms [e.g., AWS... ...Professional Cloud Architect, GCP Data Engineer Microsoft Azure Solutions Architect,... ...- Familiarity with big data technologies...CloudBig dataFull timeH1bPwC
Raleigh, NC16 days ago$106.9k - $176.5k
...working world. Technology – Data and Decision Science – Data Engineering – Senior We are... ...Engineer with expertise in cloud data engineering,... ...cloud platforms (e.g., AWS, Azure, GCP). Collaborate with clients... ...experience with Spark for big data processing. ~...CloudBig dataSummer holidayFlexible hoursEY
Raleigh, NC5 days ago$142.3k - $195.7k
...caring community The Lead Data Engineer oversees data processing... ...external sources using cloud native platforms and software... ..., machine learning, and big data to turn data into insights... ...such as Snowflake, Azure, Databricks, AWS, or GCP. Travel: While this is...CloudBig dataFull timeTemporary workApprenticeshipRemote workCenterWell
Raleigh, NC3 days ago$120k - $175.6k
...the newly consolidated Data and Analytics (D&A) organization... ...data governance, data engineering, data modeling, data... ...Engineer serves as a big data development expert... ...‑edge big data and cloud technologies. This role... ...utilizing Big Data and Azure/Databricks for Eligibility...CloudBig dataTemporary workWork at officeLocal area3 days per week$83.43k - $185.4k
9025 CVS Shared Services Resources LLC is searching for a data engineer to design and develop scalable, cloud-native data pipelines on Google Cloud Platform (GCP). The ideal candidate will work with financial datasets and collaborate with finance stakeholders for data accuracy...Cloud9025 CVS Shared Services Resources LLC
Raleigh, NC5 days ago- ...Senior Data Engineer We are seeking a Senior Data Engineer to join our... ...who specializes in preparing big data infrastructure for analytical... ...and transformation flows in Azure Data Factory Manages storage... ...in designing and deploying cloud-based data platforms (preferably...CloudBig data
PLANIT Group
Raleigh, NC2 days ago - A leading technology firm in Raleigh, North Carolina is seeking a GCP Data Engineer to design and implement scalable data pipelines using Google Cloud services. You will oversee deployments with Kubernetes, create CI/CD pipelines with Jenkins, and optimize data workflows...Cloud
- ...TEKsystems is helping the Public Cloud transformation at major... ...have a mixed environment within Azure and GCP and have just started their 1... .... The Senior Cloud Security Engineer, will be a spark to the groundwork... ...-Experience with protecting Data at Rest, Data in Motion,...CloudRemote work
6AM City
Raleigh, NC4 days ago - Mandatory: Big Query, Big Table, Google Cloud Storage, PubSub, Data Fusion, Dataflow, Dataproc Skills Google Cloud Platform (GCP) Tools (Preferred): BigQuery, Cloud Storage, Dataflow, Cloud... ...operational efficiency. Data Engineering: Expertise in ETL/ELT pipelines, data...Cloud
$97.9k - $133.5k
...of our caring community The Data Engineer 2 will develop scalable data... ...internal and external sources using cloud-native platforms and applies... ..., machine learning, and big data techniques to transform data... ...Cloud. Experience with the Azure ecosystem. Familiarity with...CloudBig dataFull timeTemporary workApprenticeshipWork at officeRemote workCenterWell
Raleigh, NC1 day ago- ...Data Platform Engineer II, F/T at Truist Bank (Raleigh, NC) Responsible for dvlpg, building, & maintaining... ...on on-premise infrastructure & cloud (SaaS) instance, where appropriate. Dsgns... ...architecture; Data integration concepts; Big data concept; & Data, BI, content, &...CloudBig dataRemote work
Truist Bank
Raleigh, NC5 days ago - ...advice, visit The Team You’ll Join The Lead Data Engineer will play a critical implementation role on the Data... ...platform being developed with leading edge big data technologies in a highly secure cloud infrastructure. The Data Engineer will serve as a liaison...CloudBig dataWork experience placementRemote work
Envestnet
Raleigh, NC3 days ago - ...The Team The AV Safety Engineering Analytics team supports safety... ...vehicle, simulation, and external data sources. The team supports both... ...for scalable, automated cloud execution. Your Skills & Abilities... ...: Python, SQL ~ Cloud & Big Data : cloud-based large-scale...CloudBig dataH1bLocal areaRemote workWork from homeRelocationRelocation package
General Motors
Raleigh, NC5 days ago - ...collaboration abilities. Nice-to-Have Experience with Control-M Automation API or Control-M Self Service. Exposure to cloud environments (AWS, Azure, GCP) and SaaS job scheduling. Knowledge of enterprise data warehouses, including Netezza, is a plus....Cloud
IT Avalon
Apex, NC2 days ago - ...focuses on structuring enterprise data to enable seamless integration, governance... ...systems. Collaborate with data engineers, analysts, and business teams to... ...management, and security. Understanding of cloud platforms (AWS, Azure, GCP) and big data ecosystems. Excellent problem-...CloudBig data
TechDigital Group
Cary, NC5 days ago - ...compliance capabilities to meet global data privacy standards. Learn more:... ...we do nor sponsor) Job Title Data Engineer Focus Areas Data ingestion, data... ...observability, or monitoring and alerting • AWS, GCP, or Azure cloud platform experience • Docker, CI/CD,...CloudFull timeRemote work
CWILL
Cary, NC1 day ago - ...developing, building, and maintaining data, BI, or content management,... ...on‑premise infrastructure and cloud (SaaS) instances, where... ...information. Leads small data engineering projects with manageable risks... ...architecture; Data integration concepts; Big data concept; and Data, BI,...CloudBig dataRemote workShift workDay shift
Habitat for Humanity of Durham
Raleigh, NC4 days ago - ...Cloud Security Engineer/Architect Tier One Technologies has an immediate need for a Cloud... ...network micro-segmentation, and data encryption across AWS, Azure, or GCP. Security Posture & Compliance... ...at massive scale. ~ Big Data Analytics for Security: Ability...CloudBig dataPermanent employmentContract workImmediate start
A.C.Coy Company
Morrisville, NC5 days ago - ...implement, and optimize data-driven solutions using... ...The role involves data engineering, pipeline development, ontology modeling, and cloud integration to enable scalable... ...Foundry with AWS, Azure, GCP, and enterprise data... ...Engineering, Cloud, and Big Data; 3+ years in Palantir...CloudBig data
- Join a forward-thinking company where your expertise in Azure Data Factory and Python scripting will drive impactful data migration projects... ...play a crucial role in migrating data from various sources to cloud databases, ensuring data integrity through effective strategies...Cloud
- ...Azure Data Engineer Location: Raleigh, NC (5 Days Work from office) Duration: 12+ months Rate: DOE Expert level skills writing and optimizing... ...stream and process datasets at low latencies Fluent with Big Data technologies like Spark, Kafka and Hive Expert level...Big dataWork at office
Georgia IT Inc
Raleigh, NC12 days ago $117.6k - $161.7k
Humana Inc is seeking a Senior CyberSecurity Engineer for managing HashiCorp Vault across cloud environments. You will oversee secure secrets management and... ...compliance with Humana's standards while working with Azure and Google Cloud Platform. This role offers opportunities...CloudRemote job$120k - $140k
A financial services company in Cary, NC is seeking an Associate to analyze user requirements and design big data solutions. The ideal candidate will hold a Master's degree and possess experience in developing microservices, optimizing them for scalability, and implementing...Big data$100k - $110k
...seamless configuration, accurate data integration, and reliable... ...science, Software Engineering, or a related technical field... .... Familiarity with cloud environments (AWS, Azure, GCP). Experience with Docker... ...orchestration tools. Knowledge of Big Data ecosystems and data...CloudBig dataXylem
Morrisville, NC1 day ago- ...Mandatory Skills Azure Data Factory Azure Data Lake Storage REST API Synapse Analytics Databricks Informatica Intelligent Cloud Services ITIL Operating Model Python Data Frames Shell... ...Responsibilities Design and Development of Data Engineering Pipelines - ELT/ETL Framework on...Cloud
TechDigital Group
Cary, NC5 days ago $100k - $120k
...AI ML Data Scientist Must Have Technical/Functional Skills -Machine... ..., matplotlib, seaborn. -Big Data Technologies • Spark (Pyspark... ..., MS Visio, TFS, JIRA -Cloud ,Web frameworks & Virtualization • Azure, Flask, Docker & Kubernets, Kafka...CloudBig dataTata Consultancy Services
Cary, NC2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Big Data Engineer (Cloud - Azure & GCP). Be the first to apply!
Related searches
- data science developer Cary, NC
- senior data center engineer Cary, NC
- data developer Cary, NC
- etl data engineer Cary, NC
- data engineer Cary, NC
- finance data engineer Cary, NC
- sr information security engineer Cary, NC
- data center engineer Cary, NC
- senior cloud data engineer Cary, NC
- data engineer machine learning Cary, NC