Data Engineer with DevOps
System One
Job Description
Job Description
Job Title: Senior Data Engineer with DevOps
Duration : Full Time Location : Pittsburgh, PA, Cleveland, OH, or Dallas, TX
Work Mode : 5 Days Onsite Years Of Exp : 8+ Yrs Job Description: We are seeking a Data Engineer with 5 years of experience to design and maintain scalable data pipeline supporting analytics, reporting, and operational needs. The role involves collaborating with cross functional teams to ensure data alignment with business requirements and enterprise standards. Duties and Responsibilities:
- Design and build scalable data pipelines aligned with business needs
- Process large dataset (batch + sometimes near Realtime)
- Ensure data quality, consistency, and governance standards across systems
- Support data integration and transformation efforts for analytics and reporting platforms
- Maintain data dictionaries, metadata, and documentation
- Participate in data architecture reviews and model validation processes
- Support analytics reporting and risk platforms.
- 5+ years of experience in data engineering and big data processing
- Strong expertise in Apache Spark (Spark Core, Spark SQL) and PySpark for large scale batch processing
- Experience working with structured and semi structured data, including complex transformations and performance tuning
- Proficiency in data ingestion and integration from sources like Oracle, SQL Server, Hive, HDFS, and S3; transform data into ‘curated data models'
- Experience writing data to Hive tables, Data Lakes (Iceberg), and downstream reporting systems
- Strong knowledge of SQL and data modeling concepts
- Hands on experience with Apache Airflow for workflow orchestration (DAG design, scheduling expectations, monitoring)
- Proficiency in shell scripting for job automation, file validation, dependency handling, and logging. Trigger Spark Jobs, perform file checks and validation; Archive & purge data; mange job dependency, logging & error handling
- Strong understanding of batch processing and batch job scheduling frameworks
- Experience migrating from CA7/Control M Airflow (daily, hourly, weekly schedules) CI/CD for data pipelines
- Fundamentals in Linux and Networking
- Docker, OCP containerization / Kubernetes
- Knowledge of CI/CD pipeline tools: Tools commonly include Jenkins, GitHub Actions, Azure DevOps, GitLab Cl, Maven, and Gradle
- Automate operational tasks using Python, Bash/Shell, and PowerShell
- Implement monitoring and alerting, Application Insights. Enable centralized logging with tools such as ELK.
- Experience ensuring data quality, reliability, and compliance in regulated environments
- Good communication and documentation skillsGood communication and collaboration skills in cross-functional teams
- Agile/Safe methodologies
- Airflow
- Containerization
- DevOps
- Elastic Stack & Elasticsearch
- GitHub
- Hadoop Hive
- Jenkins
- JSON Web Token (JWT)
- Kubernetes
- OpenShift
- Oracle
- Python
- Shell Script
- SQLite
Vacancy posted 9 days ago
Similar jobs that could be interesting for youBased on the Data Engineer with DevOps in Dallas, TX vacancy
- • Analyze and understand data sources & APIs• Design and Develop... ...from the store• Work closely with data Scientists to ensure the... ...pipelines and work flows• Work with DevOps to build automated data... ...Exposure to machine learning engineering• Exposure to NLP and text processing...Devops
- ...Company: Addison GroupTitle: Senior Data EngineerLocation: Coppell,... ...to bring on a Senior Data Engineer to our team due to growth. What... ...but will be primarily tasked with designing, building, maintaining... ...environment (Azure and/or AWS), Azure DevOps (ADO), SQL (ETL, SSIS packages...Devops
- Overview We are looking for a Senior Data Engineer with strong 7-10 year hands-on experience in Azure and Microsoft Fabric to help modernize... ...Spark-based implementationsStrong experience with Git and Azure DevOps, including pull requests, CI/CD, and deployment...DevopsFull time
- Overview We are looking for a SeniorDenodo Data Engineer with hands-on experience designing, developing, and supporting data virtualization and... ...using source control and CI/CD tools such as Git and Azure DevOps.Preferred / Nice-to-Have Skills Experience in Metadata Management...DevopsFull time
$76.73k - $127.88k
...collaborative environment? As an experienced Data Engineer, you will have the ability to share new... ...travel. If so, consider an opportunity with Deloitte under our Project Talent Model.... ...and ELT patterns.Knowledge of DevOps principles: CI/CD pipelines, version control...DevopsLocal areaVisa sponsorship$119.77k - $140.9k
...is unique in their potential. A career with U.S. Bank gives you a wide, ever-growing... ...Job DescriptionWe're looking for a Senior Data Engineer to design, build, and support scalable... ...causes, and implement improvementsChampion DevOps, DataOps, and software reliability...DevopsFull timeLocal area3 days per week$116.2k - $229.1k
...Summary Join Deloitte’s AI & Engineering practice and help... ...technology platforms, modernize data environments, and unlock value... ...transformation. You will work with business and technology leaders... ...leads, and technology teams. • DevOps and Automation: Oversee the implementation...DevopsLocal areaVisa sponsorship$137.4k - $161.7k
...impact?West Monroe is seeking a Senior Data Engineer to join our Data & AI practice. In this... ...AI, and business intelligence. Working with technologies such as Databricks, Snowflake... ...monitoring, performance optimization, and DevOps.Mentor junior consultants, review...DevopsLocal areaImmediate startFlexible hours$119.77k - $140.9k
...in their potential. A career with U.S. Bank gives you a wide, ever... ...S. Bank is seeking a Software Engineer who will contribute toward the... ...journey. The Senior Data Engineer will be responsible for... ...software development lifecycle and DevOps principles.The Senior Data Engineer...DevopsFull timeLocal area3 days per week- ...SRI Tech SolutionsJob Title: Associate Data EngineerLocation - Dallas, TX ( DAY 1... ...:Support the development of data engineering solutions on Azure with foundational skills in SQL and cloud... ...analytics using SQL.Exposure to Agile and DevOps practices.Required Skills:Basic...DevopsFull timeLocal area
- ...Big Data EngineerLocation: Irving, TX (Day 1 Onsite-Hybrid)HexawarePrimary Skills: Hadoop... ...Spark FrameworkResponsibilities:Experience with big data processing and distributed... ...standards.Knowledge of CI/CD pipelines and DevOps practices.Strong problem-solving and analytical...Devops
InterSources
Irving, TX22 hours ago $128k - $249k
...smart, imaginative and hard-working people with diverse backgrounds, experiences and... ...the same place.We are seeking a Senior Data Engineer to join the firm. The Senior Data Engineer... ...architectures while collaborating within an Agile DevOps model. Supporting CI/CD pipelines,...DevopsFull timeTemporary workLocal areaRemote workFlexible hoursAfternoon shift- ...someone who can work independently with minimal oversight while... ...APIs or GraphQL endpointsManage Data Integrate data storage solutions... ...Alchemy or Django ORM Tools DevOps Familiarity with Git version control... ...degree in Computer Science Engineering or equivalent practical...Devops
- ...Data EngineerLocation: Dallas, TexasType: ContractRate: DOEStrong Python, bash, Linux shell or similarExperience integrating with native AWS services (CodePipeline, CodeCommit, CodeBuild, CodeDeploy... ...security tools into the DevOps environment (such as Zap or Burp)...Devops
Staffing the Universe
Dallas, TX22 hours ago - ...Bachelor's degree in Computer Science, Computer Engineering, Technology, Information Systems (CIS/MIS... ...solution development using agile, DevOps, operating in a product model that includes... ...implementing large-scale applications or data engineering solutions3 years data...Devops
Syntricate Technologies
Dallas, TX22 hours ago - ...A technology solutions provider is seeking a Big Data Infrastructure DevOps / Support Administrator to lead the development and support of a data... ...on-call rotation. A Bachelor's degree in Computer Science is required, with a Master's degree preferred. #J-18808-Ljbffr...Devops
Right Talent Right Now
Garland, TX4 days ago - ...Skills:Strong in SQL and Python, 8+ years' experience.Experience with big data frameworks (i.e. Hadoop and Spark), 5+ years' experience.... ...Spark Streaming)Experience with API build. Exposure/understanding DevOps best practice and CICD (i.e. Jenkins)Exposure/understanding of...Devops
Syntricate Technologies
Richardson, TX1 day ago - ...Data Bricks Data EngineerLocation: Seattle, WA / Dallas, TX / St. Louis, MODuration: FulltimeSkills... ...pipelines in Azure.Hands-on experience with Azure Databricks: cluster management, job... ....Proficient with GitLab and Azure DevOps for CI/CD and source control workflows.Strong...Devops
JConnect Infotech
Dallas, TX22 hours ago - ...Data Engineer Location:- Dallas,TX Duration:- Full Time Job Description Role Summary We are seeking... ...a highly skilled Senior Data Engineer with expertise in Fivetran, Snowflake, Dbt,... ...Automation Performance Optimization DevOps / DataOps Git CI/CD Pipelines Agile Delivery...DevopsFull time
Siri InfoSolutions
Dallas, TX3 days ago - ...DriveWe are seeking an experienced Big Data Engineer to design, build, and optimize large-scale... ...cloud and on-prem data platforms.Work with cloud technologies (AWS, Azure, or GCP)... ...control (Git) and CI/CD tools (Jenkins, Azure DevOps, GitHub Actions).Hands-on experience...Devops
Purple Drive
Dallas, TX1 day ago $95k - $154k
...Screens? and No Offers? Let's Change That with SynergisticITSince 2010, we've helped... ...developers, Python/Java developers, Data analysts/Data Engineers/Data Scientists, Machine Learning... ...with clients.We focus on Java/Full stack/Devops and Data Science/Data Engineers/Data analysts...DevopsFull timeH1bRemote workSynergisticIT
Irving, TX22 hours ago- ...Job TitleKey Skills:Data Factory, Azure Databricks, Azure DevOps, Azure Blob Storage, Azure Data Lake, Azure Power Apps and Power BI.5-7 years data analytics... ...:Bachelor's degree in Computer Science, Computer Engineering, Technology, Information Systems (CIS/MIS),...Devops
Syntricate Technologies
Dallas, TX4 days ago - ...Senior Data Engineer - IIIMiramar, FL or Dallas, TX (Hybrid Onsite)12+ Months (Contract to Hire... ...initiatives. Working closely with Data Scientists, analysts, and business... ...alongside Data Science teamsKnowledge of DevOps practices including CI/CD and version control...DevopsContract workFlexible hours
Artech
Dallas, TX2 days ago - ...degree in Computer Science, Computer Engineering, Technology, Information Systems, Engineering... ...: Demonstrated experience automating data analysis workflows with Excel, PowerBI, and Python.Expert-... ...Agile tools such as Rally or Azure DevOps (ADO).Experience with IT asset...Devops
Kaav Inc.
Dallas, TX22 hours ago - ...production support• 5-7 years data analytics experience using SQL... ...Factory, Azure Databricks, Azure DevOps, Azure Blob Storage, Azure... ...in Computer Science, Computer Engineering, Technology, Information Systems... ...& Certifications• Expertise with the Azure Technology stack for...Devops
Software Technology Inc
Dallas, TX22 hours ago - Work Flexibility: RemoteAs a Senior Lead, Data Engineering, you will serve as a technical leader... ...influence technical strategy, and partner with teams across the organization to build... ..., Delta Lake, Azure Data Factory, Azure DevOps, CI/CD pipelines, and infrastructure automation...DevopsFull timeFor contractorsRemote work
- ...Principal Data EngineerDallas or Chicago – hybrid WFHThis role will... ...both the architecture and engineering for a new enterprise-grade analytics... ...customers. You'll work with key stakeholders to understand... ...other talented Data, Software and DevOps engineers. To summarize our...Devops
1872 Consulting
Dallas, TX22 hours ago $105.8k - $180.2k
...Senior Data EngineerRealPage is hiring a Senior Data Engineer to join the Front Office - Screening BI and Data Warehouse... ...the technical SME and will work with the product owner for changes that... ...tiers.Review pull requests in Azure DevOps; enforce SQL/SSIS standards and...DevopsRealPage
Richardson, TX22 hours ago- ...Experience: 15+ Years PETADATA is seeking a seasoned Senior Data Lake Engineer with over 15 years of experience in data engineering and a... ...cross-functional teams including data science, analytics, DevOps, and product teams. Provide mentorship and technical leadership...DevopsRemote work
Petadata
Dallas, TX4 days ago - ...Senior Data EngineerExperience Required: 10+ Years Location: RichardsonKey... ...across BI tools.Collaborate with data analysts, BI developers,... ...(Git, GitHub Actions/Azure DevOps/Jenkins).Optimize Snowflake... ...knowledge transfer.Mentor junior data engineers and enforce engineering best...Devops
Infovision
Richardson, TX4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer with DevOps. Be the first to apply!
Related searches
- data engineer analytics Dallas, TX
- principal data engineer Dallas, TX
- senior cloud data engineer Dallas, TX
- data center engineer Dallas, TX
- director data engineering Dallas, TX
- data engineer graduate Dallas, TX
- big data devops engineer Dallas, TX
- azure data engineer Dallas, TX
- data engineer manager Dallas, TX
- sr data engineer Dallas, TX