Cloud Engineer

Job Title: Cloud Engineer
Location: Boston, MA or Atlanta, GA - 2-3 days a week onsite
Length: 6 months

We are seeking an experienced Software Engineer/Cloud Developer to design, develop, and implement an end-to-end event-driven data processing pipeline on AWS. The ideal candidate will be responsible for building a scalable, resilient, and production-ready solution that can process incoming configuration-based messages, orchestrate data workflows, and integrate with downstream services deployed on AWS infrastructure.

 

Key Responsibilities:

• Design and develop an event-driven pipeline capable of receiving and parsing incoming messages containing configuration details.

• Implement message queuing mechanisms to reliably capture and process requests using AWS SQS or similar services.

• Develop logic to identify and retrieve the appropriate data files from Amazon S3 based on the information contained in the queued messages.

• Build robust data parsing and transformation components to process files according to business and configuration requirements.

• Integrate with containerized services running on Amazon EKS, ensuring the correct parameters and payloads are passed to downstream applications.

• Process and validate service responses and publish output messages to the appropriate destination queues for downstream consumption.

• Implement monitoring, logging, error handling, retry mechanisms, and operational alerting to ensure high availability and reliability.

• Create reusable Infrastructure-as-Code templates using Terraform for automated provisioning and deployment.

• Build and validate the solution in the Development environment and support infrastructure provisioning and deployment across UAT and Production environments.

• Collaborate with architects, DevOps engineers, and business stakeholders to ensure solution scalability, security, and maintainability.

• Participate in code reviews, testing, deployment planning, and production support activities.

 

Required Technical Skills:

• Strong programming experience in Python.

• Hands-on experience with AWS services, including:

- Amazon EKS

- AWS Lambda

- Amazon API Gateway

- Amazon SQS

- Amazon S3

- AWS Event-Driven Architectures and Event Triggers

 

• Experience with Docker containerization and container orchestration concepts.

• Strong knowledge of Terraform and Infrastructure as Code practices.

• Experience with Kubernetes and kubectl for cluster management and troubleshooting.

• Understanding of CI/CD pipelines, deployment automation, and cloud-native application development.

 

Preferred Skills:

 

• Experience working with Snowflake and cloud-based data platforms.

• Knowledge of data engineering, ETL processes, and distributed systems.

• Familiarity with monitoring and observability tools in AWS environments.

• Experience working in Agile development environments and cross-functional teams.

 

The successful candidate will play a key role in building and operationalizing a modern, event-driven AWS platform that supports scalable data processing and seamless integration across multiple environments.