Senior Backend Engineer (Spark, Scala, Python, AWS/GCP)
Cloud Analytics Technologies LLC
About the Role:
We are seeking a skilled and passionate Back-End Engineer with a strong background in data engineering to join our growing team. In this role, you will be responsible for designing, developing, and maintaining scalable and reliable data pipelines and back-end systems. You will work with large datasets, leverage cloud technologies, and collaborate with data scientists and other engineers to deliver impactful data-driven solutions. This position is available in both the United States and India.
Responsibilities:
* Design, develop, and maintain data pipelines using Spark, Python, Scala, and Java.
* Write efficient and optimized SQL queries for data extraction, transformation, and loading (ETL) processes.
* Work with DataFrames to manipulate and analyze large datasets.
* Implement data storage and processing solutions using cloud technologies (preferably AWS or GCP).
* Build and maintain real-time data streaming pipelines using MSK/Kafka.
* Utilize S3 for data storage and retrieval.
* Work with data lake technologies like Iceberg.
* Ensure data quality, integrity, and security.
* Collaborate with data scientists and other engineers to understand data requirements and deliver solutions.
* Participate in code reviews and contribute to improving our development processes.
* Troubleshoot and resolve issues in data pipelines and back-end systems.
* Bachelor’s degree in computer science or a related field (or equivalent experience).
* 5+ years of experience in back-end development with a focus on data engineering.
Required Skills: * Strong proficiency in Spark, Python, Scala, and Java.
* Expertise in SQL and working with relational databases.
* Experience with DataFrames for data manipulation and analysis.
* Experience with cloud technologies (preferably AWS or GCP).
* Experience with message streaming platforms like MSK/Kafka.
* Experience with S3 or similar object storage.
* Experience with data lake technologies like Iceberg.
* Solid understanding of data warehousing concepts.
* Excellent communication and collaboration skills.
Bonus Points:
* Experience with data modelling and schema design.
* Experience with data governance and data quality management.
* Experience with DevOps practices and CI/CD pipelines.
* Relevant certifications (e.g., AWS Certified Data Engineer, Google Cloud Certified Professional Data Engineer).
Vacancy posted 13 days ago
Similar jobs that could be interesting for youBased on the Senior Backend Engineer (Spark, Scala, Python, AWS/GCP) in Elk Grove, CA vacancy
- ...seeking a skilled and passionate Back-End Engineer with a strong background in data... ...develop, and maintain data pipelines using Spark, Python, Scala, and Java. Write efficient and... ...using cloud technologies (preferably AWS or GCP). Build and maintain real-time data streaming...Amazon Web ServiceSeniorPythonHourly payPermanent employmentContract workLocal area
- ...consulting firm is seeking a skilled Back-End Engineer to design and maintain data pipelines and back... .... Candidates should have strong experience in Spark, Python, and SQL, with a focus on cloud technologies like AWS and GCP. This position requires collaboration with data...Amazon Web ServiceSeniorPythonHourly pay
- A technology company is seeking a skilled Back-End Engineer to design and maintain scalable data pipelines and back-end systems. You'll utilize tools such as Spark, Python, and cloud technologies to develop impactful data-driven solutions, collaborating with data scientists...Amazon Web ServiceSeniorPython
- A tech company is seeking a skilled Back-End Engineer to design and maintain data pipelines and back-end systems. The ideal candidate will have over 5 years of experience, deep expertise in Spark, Python, and SQL, and will work closely with data scientists to create effective...Amazon Web ServiceSeniorPythonHourly payContract work
- ...technology firm is seeking a skilled Back-End Engineer with a strong background in data... ...systems. Responsibilities include using Spark, Python, and SQL for ETL processes,... ...proficiency in cloud technologies like AWS or GCP. This position is based in Elk Grove, CA...Amazon Web ServiceSeniorPythonHourly pay
- A technology firm is seeking a skilled Back-End Engineer with expertise in data engineering. In this role, you... ...scalable data solutions, working with technologies like Spark, Python, and cloud platforms such as AWS or GCP. The ideal candidate has over 5 years of experience...Amazon Web ServiceSeniorPythonRemote work
$106.9k - $176.5k
...and Decision Science - Data Engineering - Senior We are seeking a highly... ...on cloud platforms (e.g., AWS, Azure, GCP). Collaborate with clients... ...Databricks and experience with Spark for big data processing.... ...in languages such as Python, Scala, or SQL. Experience with...Amazon Web ServiceSeniorPythonSummer holidayFlexible hours$160k - $200k
...oriented and impact-driven Senior Data Engineer to strengthen our capabilities... ...data workflows using Python, SQL, dbt, Airflow, Terraform... ...cloud-based data platforms in AWS, Azure, or GCP, including Infrastructure-... ...Fivetran, Airflow, Kafka, Spark, Databricks, or streaming/event...Amazon Web ServiceSeniorPythonDatavant
Sacramento, CA4 days ago- Position: Senior Solution Data Warehouse Architect - on-site required (Ref: 5704) Duration... ...architecture experience (such as Azure, AWS, or GCP), including deployment strategies and... ...years of hands‑on experience with C# / Python, SQL databases, and advanced SQL queries...Amazon Web ServiceSeniorPython
- ...is a three-person engineering firm building production... ...We are hiring a Senior Data Engineer to... ...-not-yet-deployed AWS data platform from... ...pipelines and backend services, validating... ...backend services in Python on AWS Design and... ...frameworks (e.g., Spark, Databricks)...Amazon Web ServiceSeniorPythonRemote workShift work
$120k - $135k
Cognizant is seeking a Senior Data Engineer / Technical Lead with expertise in AWS Glue and Spark Optimization to join their global team. This fully remote role offers a salary range of $120,000 - $135,000 per year, alongside comprehensive benefits and a flexible work...Amazon Web ServiceSeniorRemote workFlexible hours$135k - $145k
...is looking for an experienced Senior Solution Architecture Lead!... ...technologies, cloud platforms, data engineering, machine learning, and... ...architecture experience (such as Azure, AWS, or GCP), including deployment... ...‑on experience with C# / Python, SQL databases, and advanced...Amazon Web ServiceSeniorPythonFull timeFlexible hours$155k - $235k
...and ensuring data works for engineers, analysts, and business... ...Role We're looking for a Senior AI Data Engineer to lead the... ...Scribd. Tech Stack We use Python, SQL, Databricks (Unity... ...Spaces), Apache Airflow, Spark, Terraform/Spacelift, and AWS. You Have 5+ years of data...Amazon Web ServiceSeniorPythonHome officeFlexible hours$200k
...has an exciting opportunity for a Senior Director, AI Systems Engineering to join the Maximus AI... ...and methodologies Expertise in AWS, Azure, and/or GCP US Citizen, Clearable with a Public... ...languages, including but not limited to Python/Java and frontend frameworks for...Amazon Web ServiceSeniorPythonImmediate startRemote workFlexible hours$167k - $196.5k
...operational support. We are looking for a Senior Site Reliability Engineer who is excited about establishing... ..., Containers and public clouds (GCP or AWS) ~ Experience with deployment and monitoring... ...clusters. ~ Experience in Python or Go programming language. ~ Experience...Amazon Web ServiceSeniorPythonWork at officeWork from homeFlexible hoursNight shiftLiveRamp
Sacramento, CA2 days ago- ...Title: Senior Architect Location: Sacramento, CA Job Description... ...certifications : a. AWS Certified Solutions Architect... ...cloud functions using Python, Ruby, Go or something similar... ...performing IT infrastructure engineering, administration, and operations...Amazon Web ServiceSeniorPython2 days per week3 days per week
Apex Informatics
Sacramento, CA4 days ago - Modelyst is looking for a Senior Data Engineer to lead the development of an AWS data platform. You will transform the designed system into a working MVP deployed... ...candidates will have substantial experience with Python and AWS, along with a willingness to own systems over...Amazon Web ServiceSeniorPythonRemote work
$70.35k - $205.8k
...investigations Leverage EDR solutions, cloud platforms (AWS, Azure, GCP), and threat intelligence to identify... ...in scripting and programming languages (Python, PowerShell, Bash) You have experience with reverse engineering and sandboxing technologies You have advanced...Amazon Web ServiceSeniorPythonWork experience placementLive inWork at officeLocal area- ...of cloud architecture experience with platforms such as Azure, AWS, or GCP, including deployment strategies and infrastructure-as-code... ...and data integration. 5+ years of hands‑on experience with C#, Python, SQL databases, and advanced SQL queries. 5+ years of experience...Amazon Web ServiceSeniorPythonWork at office
$209k - $238.5k
...Senior Manager, Premium Support Engineer (Databolt) - Capital One Software (Remote) Summary As a Premium Support... ...At least 5 years of experience with AWS (Multi-AZ, EKS, KMS, Secrets Manager... ...experience programming with Java, Python, Go, or PySpark SDKs. At least 3...Amazon Web ServiceSeniorPythonFull timePart timeLocal areaRemote workCapital One
Sacramento, CA3 days ago- ...Big Data Engineer CA – Los Angeles, CA - Los Alamitos... ...direction set by senior management as it relates... ...solutions utilizing Hadoop and Python. • Experience... ...MapReduce, Pig, Hive o Scala, Spark o knowledge on Jenkins,... ...Solutions in the cloud with AWS or Azure • Ability to...Amazon Web ServicePythonWork experience placement
ClifyX
Sacramento, CA20 hours ago - ...The purpose of the Senior Data Engineer at SkySlope is to develop, test, and deploy new and improved... ...with cloud database services (AWS RDS, Azure SQL, or Google Cloud SQL) Experience... ...Talend, Pentaho, or custom solutions) Python or scripting experience for database automation...Amazon Web ServiceSeniorPythonTemporary workFlexible hours
CloudDevs
Sacramento, CA4 days ago $82.8k - $175k
...people where they live, learn and work. The Senior Software Engineering Manager accomplishes departmental and organizational... ...in management role. # Proficiency in Python & Java required. # Experience with AWS deployments required. # Experience with auto scaling...Amazon Web ServiceSeniorPythonTemporary workLocal areaNelnet
Sacramento, CA13 hours ago$84.63k - $112.84k
...Cybersecurity Incident Response Team (CIRT) Engineers at Lumen are on the front lines of... ...audiences Experience with cloud security (AWS, Azure, GCP) Broad knowledge of current and... ...data exposure ~ Scripting experience (Python or Perl) ~ Experience in large...Amazon Web ServiceSeniorPythonTemporary workRemote workShift workLumen Inc
Sacramento, CA4 days ago- ...Position Summary We are seeking a Mid to Senior ETL Data Engineer with strong experience in Informatica... ...Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform.... ...cloud data warehouses. Experience with Python or Shell scripting for ETL automation....Amazon Web ServiceSeniorPythonRemote work2 days per week
$124k - $280k
...people in data and analytics engineering focus on leveraging advanced technologies... ...system and health plans. As a Senior Manager, you will drive use... ...modeling, prompt engineering, Python-based development, testing,... ...Azure Health Data Services), AWS (SageMaker, Bedrock), and/or...Amazon Web ServiceSeniorPythonFull timeH1bPwC
Sacramento, CA4 days ago- ...are seeking a proactive Site Reliability Engineer to champion the evolution of our... ...repetitive tasks and complex workflows using Python or Go Experience configuring and managing... ...Experience managing cloud environments (AWS, GCP, or Azure) using Terraform, Ansible, or...Amazon Web ServicePythonShift work
$77k - $202k
...testing and validation processes. As a Senior Associate you will analyze complex... ...Data Management Professional (CDMP), AWS Certified Data Engineer - Associate, SnowPro Core, ISTQB Test... ...skills for data validation Utilizing Python for test automation and data validation...Amazon Web ServiceSeniorPythonFull timeH1b$150k - $190k
...internal users, customers and partners. The Senior Data Engineer will be responsible for the analysis,... ...‑on experience and knowledge of the Python programming language Experience... ...expertise and/or certifications - Azure, AWS, GCP Hands‑on experience with DevOps...Amazon Web ServiceSeniorPythonWork experience placementWorldwideFlexible hoursShift work$209k - $238.5k
Sr Lead Software Engineer, Analytics - Shopping (Remote-Eligible) What... ..., HTML/CSS, TypeScript, SQL, Python, and Go, Open Source RDBMS and... ...Kubernetes, and a variety of AWS tools and services Basic Qualifications... ...years of experience with AWS, GCP, Microsoft Azure, or another...Amazon Web ServiceSeniorPythonRemote jobFull timePart timeInternshipLocal area
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Backend Engineer (Spark, Scala, Python, AWS/GCP). Be the first to apply!
Related searches
- senior application security Elk Grove, CA
- senior vice president of operations Elk Grove, CA
- senior performance tester Elk Grove, CA
- senior sas developer Elk Grove, CA
- senior magento developer Elk Grove, CA
- senior mainframe developer Elk Grove, CA
- senior manager legal Elk Grove, CA
- senior tech Elk Grove, CA
- senior manager diversity & inclusion Elk Grove, CA
- senior manager tax Elk Grove, CA