Data Engineer

Overview

Experience: 15+ Years

PETADATA is seeking a seasoned Senior Data Lake Engineer with over 15 years of experience in data engineering and a strong focus on building and managing AWS-native Data Lake solutions. The ideal candidate will have deep expertise with AWS Lake Formation, serverless data processing using Lambda and Python, and experience with AI-assisted development tools such as Amazon Q.

Responsibilities

• Data Lake Architecture: Design, build, and optimize scalable, secure data lakes using AWS Lake Formation and best practices for data governance, cataloging, and access control.
• Serverless Development: Build and deploy AWS Lambda functions using Python for real-time data processing, automation, and event-driven workflows.
• ETL / ELT Pipelines: Develop and maintain robust data pipelines using AWS Glue, integrating data from various structured and unstructured sources.
• AI Tools for Development: Leverage AI-powered coding tools (such as Amazon Q, GitHub Copilot, or similar) to increase development speed, code quality, and automation.
• Database Integration: Design and implement integrations between the Data Lake and DynamoDB, optimizing for performance, scale, and consistency.
• Security & Compliance: Implement fine-grained access control using Lake Formation, IAM policies, encryption, and data masking techniques to meet enterprise and compliance standards (e.g., GDPR, HIPAA).
• Monitoring & Optimization: Implement logging, monitoring, and performance tuning for Glue jobs, Lambda functions, and data workflows.
• Collaboration & Leadership: Collaborate with cross-functional teams including data science, analytics, DevOps, and product teams. Provide mentorship and technical leadership to junior engineers.

Position Details

Position: Senior Data Lake Engineer

Location: Dallas, TX (Remote)

Work Type: C2C

Experience: 15+ Years

Note: This role requires strong hands-on skills in AWS Glue, DynamoDB, and building secure, scalable, and automated data platforms that support advanced analytics and machine learning use cases.

Roles & Responsibilities (duplicate sections consolidated)

• Data Lake Architecture: Design, build, and optimize scalable, secure data lakes using AWS Lake Formation and best practices for data governance, cataloging, and access control.
• Serverless Development: Build and deploy AWS Lambda functions using Python for real-time data processing, automation, and event-driven workflows.
• ETL / ELT Pipelines: Develop and maintain robust data pipelines using AWS Glue, integrating data from various structured and unstructured sources.
• AI Tools for Development: Leverage AI-powered coding tools (such as Amazon Q, GitHub Copilot, or similar) to increase development speed, code quality, and automation.
• Database Integration: Design and implement integrations between the Data Lake and DynamoDB, optimizing for performance, scale, and consistency.
• Security & Compliance: Implement fine-grained access control using Lake Formation, IAM policies, encryption, and data masking techniques to meet enterprise and compliance standards (e.g., GDPR, HIPAA).
• Monitoring & Optimization: Implement logging, monitoring, and performance tuning for Glue jobs, Lambda functions, and data workflows.
• Collaboration & Leadership: Collaborate with cross-functional teams including data science, analytics, DevOps, and product teams. Provide mentorship and technical leadership to junior engineers.

Required Skills

• Experience: 15+ years in data engineering roles, with 5+ years focused on AWS-native data lake development.
• Cloud Expertise: Deep, hands-on expertise in AWS Lake Formation, Glue, Lambda, and DynamoDB.
• Programming: Proficient in Python, especially for serverless and data processing applications.
• AI Coding Tools: Experience using AI-assisted development tools (e.g., Amazon Q, GitHub Copilot, AWS CodeWhisperer).
• Security: Strong knowledge of data security practices in AWS, including IAM, encryption, and compliance standards.
• Orchestration & Automation: Experience with workflow orchestration tools such as Step Functions, Airflow, or custom AWS-based solutions.
• Soft Skills: Strong communication, problem-solving, and collaboration skills. Able to lead discussions on architecture and best practices.

Preferred Skills

• AWS Certifications (e.g., AWS Certified Data Analytics, AWS Certified Solutions Architect)
• Experience with Athena, Redshift, or other analytics services in the AWS ecosystem
• Exposure to DevOps practices and tools like Terraform, CloudFormation, or CDK
• Familiarity with data cataloging and metadata management tools

Education

Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

We offer a professional work environment and provide every opportunity for growth in the Information technology world.

Note

Candidates are required to attend Phone/video calls and in-person interviews. After the Selection, the candidate (He/She) should undergo all background checks on Education and Experience.

Application

Please email your resume to View email address on click.appcast.io. After carefully reviewing your experience and skills, one of our HR team members will contact you on the next steps.