Data Engineer

Data Engineer

Location: Kansas City, MO - onsite (LOCAL ONLY!!)

Contract

Kelly

Rate - $65/hr C2C

Strong experience in AWS Data Services:

S3, AWS Glue, Athena, Lambda, Redshift, EMR

Proficiency in Python and SQL

Experience with PySpark / Spark-based processing Expertise in ETL/ELT pipeline design and implementation

Data Engineering & Architecture

Strong understanding of data lake and lakehouse architectures Knowledge of data modeling and warehousing concepts Experience with file formats (Parquet, ORC, Avro)

DevOps & CI/CD

Experience with version control systems (Git) Knowledge of CI/CD tools and deployment automation Familiarity with infrastructure-as-code (Terraform/CloudFormation is a plus)

Production Support

Experience in incident management and RCA Exposure to monitoring/observability tools (CloudWatch, etc.)

Agile & Leadership Skills

Hands-on experience in Agile/Scrum environments Ability to act as a team anchor/technical lead Strong communication and coordination skill

Key Responsibilities

1. Data Engineering & Development

Design, build, and maintain scalable data pipelines using AWS services.

Develop and optimize ETL/ELT workflows using tools such as AWS Glue, PySpark, and Python.

Work on data ingestion, transformation, and processing for structured and semi-structured data.

Implement data lake/lakehouse solutions on AWS (S3-based architecture).

2. AWS Data Platform Implementation

Build solutions leveraging AWS services like S3, Glue, Athena, Lambda, Redshift, EMR.

Ensure cost-efficient, scalable, and secure architecture design.

Implement best practices for partitioning, file formats (Parquet/ORC), and performance tuning.

3. Agile Delivery & Team Anchor

Act as the Agile anchor for the team, ensuring sprint commitments are met.

Support backlog grooming, sprint planning, and tracking delivery progress.

Proactively identify and resolve blockers, ensuring smooth execution.

Collaborate closely with Product Owners and Scrum Masters.

4. Data Pipeline Design & Optimization

Design efficient batch and (if applicable) streaming pipelines.

Optimize pipelines for performance, reliability, and cost.

Apply best practices in data partitioning, indexing, and query optimization.

5. Production Support & Incident Handling Provide L2/L3 production support for data pipelines and workflows.

Troubleshoot and resolve data pipeline failures and performance issues.

Perform root cause analysis (RCA) and implement preventive fixes.

Participate in on-call rotations and incident management.

6. Code Quality & Peer Review

Conduct and participate in code reviews and design reviews.

Ensure adherence to coding standards, security, and best practices.

Maintain proper documentation and version control.

7. Data Quality & Governance

Implement data validation, quality checks, and monitoring frameworks.

Ensure data integrity, consistency, and lineage tracking.

Align with data governance and compliance standards.

8. Stakeholder Collaboration

Work with business analysts, data architects, and stakeholders to understand requirements.

Translate functional requirements into technical data solutions.

Provide regular updates on progress, risks, and dependencies.

9. CI/CD & Automation

Implement CI/CD pipelines for data workflows.

Automate deployment, monitoring, and alerting mechanisms.

Use tools such as Git, Jenkins/AWS CodePipeline, Terraform (optional).

10. Mentorship & Team Contribution

Guide and mentor junior engineers.

Promote best practices and knowledge sharing within the team.

Contribute to continuous improvement initiative

Essential skills of the position:

• Design and build AWS-based data pipelines

• Develop and optimize ETL workflows (Python, PySpark)

• Implement data lake / lakehouse solutions

• Ensure scalable, secure, and cost-efficient architecture

• Anchor Agile delivery (sprint planning, tracking, execution)

• Design and optimize batch data pipelines

• Handle production support, incident resolution, RCA

• Perform code reviews and enforce coding standards

• Implement data quality checks and validation

• Collaborate with stakeholders to deliver data solutions

• Experience managing large, distributed onshore/offshore teams.

• Strong understanding of ITIL-based service management and delivery governance.

• Excellent stakeholder management

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.