Data Engineer with DevOps
System One
Job Description
Job Description
Job Title: Senior Data Engineer with DevOps
Duration : Full Time Location : Pittsburgh, PA, Cleveland, OH, or Dallas, TX
Work Mode : 5 Days Onsite Years Of Exp : 8+ Yrs Job Description: We are seeking a Data Engineer with 5 years of experience to design and maintain scalable data pipeline supporting analytics, reporting, and operational needs. The role involves collaborating with cross functional teams to ensure data alignment with business requirements and enterprise standards. Duties and Responsibilities:
- Design and build scalable data pipelines aligned with business needs
- Process large dataset (batch + sometimes near Realtime)
- Ensure data quality, consistency, and governance standards across systems
- Support data integration and transformation efforts for analytics and reporting platforms
- Maintain data dictionaries, metadata, and documentation
- Participate in data architecture reviews and model validation processes
- Support analytics reporting and risk platforms.
- 5+ years of experience in data engineering and big data processing
- Strong expertise in Apache Spark (Spark Core, Spark SQL) and PySpark for large scale batch processing
- Experience working with structured and semi structured data, including complex transformations and performance tuning
- Proficiency in data ingestion and integration from sources like Oracle, SQL Server, Hive, HDFS, and S3; transform data into ‘curated data models'
- Experience writing data to Hive tables, Data Lakes (Iceberg), and downstream reporting systems
- Strong knowledge of SQL and data modeling concepts
- Hands on experience with Apache Airflow for workflow orchestration (DAG design, scheduling expectations, monitoring)
- Proficiency in shell scripting for job automation, file validation, dependency handling, and logging. Trigger Spark Jobs, perform file checks and validation; Archive & purge data; mange job dependency, logging & error handling
- Strong understanding of batch processing and batch job scheduling frameworks
- Experience migrating from CA7/Control M Airflow (daily, hourly, weekly schedules) CI/CD for data pipelines
- Fundamentals in Linux and Networking
- Docker, OCP containerization / Kubernetes
- Knowledge of CI/CD pipeline tools: Tools commonly include Jenkins, GitHub Actions, Azure DevOps, GitLab Cl, Maven, and Gradle
- Automate operational tasks using Python, Bash/Shell, and PowerShell
- Implement monitoring and alerting, Application Insights. Enable centralized logging with tools such as ELK.
- Experience ensuring data quality, reliability, and compliance in regulated environments
- Good communication and documentation skillsGood communication and collaboration skills in cross-functional teams
- Agile/Safe methodologies
- Airflow
- Containerization
- DevOps
- Elastic Stack & Elasticsearch
- GitHub
- Hadoop Hive
- Jenkins
- JSON Web Token (JWT)
- Kubernetes
- OpenShift
- Oracle
- Python
- Shell Script
- SQLite
Vacancy posted 19 days ago
Similar jobs that could be interesting for youBased on the Data Engineer with DevOps in Dallas, TX vacancy
$125.5k - $147.6k
...impact? West Monroe is seeking a Senior Data Engineer to join our Data & AI practice. In this... ...AI, and business intelligence. Working with technologies such as Databricks, Snowflake... ..., performance optimization, and DevOps. Mentor junior consultants, review technical...DevopsFull timeWork at officeLocal areaImmediate startFlexible hours- ...Experience: 15+ Years PETADATA is seeking a seasoned Senior Data Lake Engineer with over 15 years of experience in data engineering and a... ...cross-functional teams including data science, analytics, DevOps, and product teams. Provide mentorship and technical leadership...DevopsRemote work
Petadata
Dallas, TX5 days ago - ...and that trust will be extended to you. The Data Engineer will be responsible for scaling Highway’s platform in AWS using the best DevOps practices. This role reports directly to... ...and secure data handling practices aligned with financial and regulatory requirements....Devops
Highway
Dallas, TX3 days ago $114.28k - $140k
...Irving, TX 75038, USA Description The Senior Data Engineer leads the design, modernization, and... ...streamline the data ecosystem. Partners with leadership to provide visibility into... ...Automate Power Query Visual Studio Azure DevOps Advanced SQL development and tuning Microsoft...DevopsLERETA
Irving, TX4 days ago- ...Position Overview We are seeking a Senior Data Engineer to design, build, and maintain scalable... ...combines hands‑on engineering expertise with a strategic mindset to optimize the... ...compatibility, cleanliness, and compliance. Utilize DevOps tools (e.G., Git, CI/CD) to deploy and...Devops
Peyton Resource Group
Irving, TX5 days ago $130k - $155k
...We are seeking an experienced Senior Data Engineer to join our growing IT team. This role will... ...integration, processing, and storage. Collaborate with analysts and other stakeholders to... ...support data warehousing. Utilize Azure DevOps for continuous integration, continuous...DevopsWork at officeFlexible hoursHealth-E Commerce
Dallas, TX4 days ago$90k - $150k
...Overview Your role at GEI: As a Data Engineer, you will design, build, and maintain data pipelines... ...capabilities can be deployed and scaled with confidence. Responsibilities Design,... ...pipeline deployment (e.g., Terraform, Azure DevOps). Knowledge of data governance...DevopsGEI Consultants
Dallas, TX5 days ago- ...We’re looking for a Lead Data Engineer to help design, build, and scale a modern enterprise data... ...through production. Partner closely with architects, analytics teams, and business... ...tools (Airflow, Step Functions) and modern DevOps practices. Strong communicator who can...Devops
Harnham
Dallas, TX5 days ago $92.7k - $167.5k
Job Summary An individual with the ability to design and develop data integrations to deliver Enterprise BI Data Marts and BI Semantic Data layers. This... ...with Azure, ADLS, ADF, Azure SQL etc Knowledge of DevOps / continuous deployment / integration process and tools...DevopsRelocationFlexible hoursHCSC
Richardson, TX16 hours ago$120k - $140k
Title: Senior Data Engineer Location: Coppell, TX Salary Range: $120K-$140K We are looking to... ...Reporting" teams but will be primarily tasked with designing, building, maintaining, and... ...environment (Azure and/or AWS), Azure DevOps (ADO), SQL (ETL, SSIS packages), and PySpark...Devops$105.8k - $180.2k
Overview RealPage is hiring a Senior Data Engineer to join the Front Office - Screening BI and... ...will be the technical SME and will work with the product owner for changes that flow... ...Prod tiers. Review pull requests in Azure DevOps; enforce SQL/SSIS standards and protect...Devops- We are seeking a skilled Data Engineer to design, develop, and maintain scalable data pipelines... ...candidate will have experience working with large datasets, cloud platforms, ETL/ELT... ...Apache Airflow. Familiarity with CI/CD, DevOps practices, and version control systems....Devops
- The Lead Data Solutions Engineer is responsible for designing, developing, implementing, and supporting... ...individual will collaborate closely with architects, engineers, analytics teams,... ...platforms. Infrastructure Automation & DevOps Implement Infrastructure as Code (IaC)...Devops
- Hilltop Holdings is looking for a Senior Data Engineer! The Senior Data Engineer is a key... ...combines traditional DBA responsibilities with advanced engineering work in Microsoft Azure... ...in PowerShell, Python, or Azure DevOps. Excellent problem-solving, communication...DevopsFull timeWork experience placement
$128k - $249k
...smart, imaginative and hard‑working people with diverse backgrounds, experiences and... ...place. Job Summary We are seeking a Senior Data Engineer to join the Enterprise Applications,... ...scalable data architectures within an Agile DevOps model, supporting CI/CD pipelines and...DevopsLocal areaFlexible hoursAfternoon shift- Overview We are seeking a Data Engineer to join our growing Data & Analytics team. This role is... .... This individual will partner closely with business stakeholders to understand requirements... ...and semantic models. Utilize Azure DevOps and Git for source control, deployments,...DevopsFull timeWork at officeFlexible hours
$99k - $164.8k
...We are seeking a highly skilled Senior Data Engineer to guide the design, development, and delivery... ...role requires deep technical expertise with Snowflake, Core AWS Services, cloud-... ...Function Experience with modern DataOps, DevOps, and automation frameworks: CI/CD (...DevopsCasual workWork at officeLocal area- ...A technology solutions provider is seeking a Big Data Infrastructure DevOps / Support Administrator to lead the development and support of a data... ...on-call rotation. A Bachelor's degree in Computer Science is required, with a Master's degree preferred. #J-18808-Ljbffr...Devops
Right Talent Right Now
Garland, TX5 days ago - A technology solutions firm in Dallas is seeking a Big Data Infrastructure DevOps/Support Administrator. This role involves supporting and developing the Enterprise Infrastructure Big Data Service using Java, Python, and Hadoop. Ideal candidates will have a strong background...Devops
$112.2k - $202.6k
...Job Summary An individual with the ability to design and develop data integrations to deliver Enterprise BI Data... ...end data integration and analytics engineering pipelines. The role includes... ...ADF, Azure SQL etc. Knowledge of DevOps / continuous deployment / integration...DevopsFlexible hoursHCSC
Richardson, TX3 days ago- ...Jconnect INC . Below is the requirement with my client. Please let me know if you are... ...for this role. Title: Cloud Data Engineer Location: Seattle, WA / Dallas,... ...in Human Resources • Experience with DevOps/DevSecOps and CI/CD practices, methodologies...DevopsFull timeImmediate startRelocation
3B Staffing LLC
Dallas, TX1 day ago - ...Experience: 8+ years GCP Platform Engineer Responsibilities: Develop,... ...Platform services. Work closely with other developers, architects,... ...deployment pipelines using DevOps methodologies (Terraform, Ansible... ...as needed. Implement data protection and identity access...DevopsFor contractors
- ...Information Security Engineer Location: Charlotte, NC/Dallas, TX /Phoenix, AZ Position... ...ideal candidate has hands-on experience with Cloud Native Application Protection Platforms... ...will partner with Infrastructure, DevOps, Security Operations, and Application Development...DevopsWork at office
Leading Utilities Organization
Dallas, TX5 days ago - ...for an Analytics / Reporting Engineer to join the Enterprise Reporting... ...You'll build and maintain the data pipelines, dashboards, and... ...design reports, and work directly with stakeholders who depend on... ...processing Experience with Azure DevOps for version control and...DevopsTemporary work
$79.6k - $139.3k
Senior Data Engineer with DevOps Position Description We are seeking a Data Engineer with 5 years of experience to design and maintain scalable data pipeline supporting analytics, reporting, and operational needs. The role involves collaborating with cross functional...DevopsHourly payPermanent employmentLocal area$90.32 per hour
...Job Title: AWS/Snowflake Data Engineer Location: Richardson, OK Type: Contract Compensation... ...Web Services (AWS) cloud platform, with a strong emphasis on native AWS services... ...patterns); Familiarity with DevOps practices, CI/CD pipelines, and infrastructure...DevopsLong term contractContract workLocal areaRemote work- ...systems. BSI is changing that - and we need engineers who can move fast. As a Software... ...Claude Code, Cursor, GitHub Copilot), build with modern data stacks, and ship real features to... ...equivalent ~ Dev Tools: GitHub, Azure DevOps, Jira, Claude Code, Cursor Education...DevopsFull timeInternshipSummer internshipWork at officeMonday to Friday
BSI Financial
Irving, TX3 days ago - ...Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion...For contractors
NTT DATA
Garland, TX4 days ago $110k - $169k
...Mcdermott Will & Schulte Build your big career with the firm that does Big Law, Better. McDermott Will & Schulte is a leading... ...hard work and excellence. Job Description: The Pricing Data Engineer builds and maintains the data infrastructure and tools that enable...Full timeWork at officeMcDermott Will & Schulte
Dallas, TX4 days ago- ...skills in Python Programming Strong Experience in SQL Experience with data pipeline and workflow management tool: GCP Data Proc, Composer... ...SQL BigQuery Experience (may not be expert) To work on data engineering solutions to leverage best practices and industry-leading...
programmers.io
Dallas, TX3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer with DevOps. Be the first to apply!
Related searches
- junior data engineer remote Dallas, TX
- data engineering intern summer Dallas, TX
- senior data quality engineer Dallas, TX
- aws data engineer Dallas, TX
- data engineer Dallas, TX
- finance data engineer Dallas, TX
- senior data center engineer Dallas, TX
- data engineer manager Dallas, TX
- big data devops engineer Dallas, TX
- entry level big data engineer Dallas, TX