Data Engineer (AWS / Databricks)

Requirements

• 8 or more years of experience in enterprise data engineering and production pipeline architecture.

• Proficiency in Databricks and Amazon S3-backed data lakehouse storage environments.

• Experience with AWS compute infrastructure including Amazon EC2 and cloud-native services.

• Experience in Apache streaming technologies including Apache Kafka and distributed computing frameworks.

• Experience with high-throughput time-series data pipelines and device telemetry ingestion.

• Experience in automated data quality validation frameworks, data observability, and modern table formats.

• Excellent verbal and written communication skills.

Responsibilities

• Architect and build greenfield data pipelines from scratch to ingest complex hardware telemetry.

• Develop scalable ETL and streaming data transformation workflows using Databricks and PySpark.

• Design secure Amazon S3 data lakehouse storage layers optimized for high-performance querying.

• Integrate streaming message queues using Apache Kafka to capture real-time time-series datasets.

• Implement automated data quality validation checks and end-to-end pipeline observability tools.

• Establish data engineering standards and governance to ensure data integrity for downstream AI models.

• Partner with testing and engineering stakeholders to streamline data migration and external integrations.

Founded in 1999, Resourcesoft, Inc. is a leading Technology Consulting and Professional Services organization. Headquartered in Marlborough, MA, the company serves the technology needs of its clients nationwide. Resourcesoft has often been recognized by prominent rating agencies for its exemplary growth and stability. With over two decades of industry experience, the Company has evolved as a front runner in enabling project optimization. We partner with leading organizations to provide technology solutions within the financial, insurance, education, government, publishing, healthcare and pharmaceutical domains. We take pride in mentoring a workforce that is well positioned to respond to the emerging IT trends and needs. With the employees at the crux of every business endeavor, our success is driven by our expertise in pairing the right talent with the best jobs in the technology sector. We forge long term, personalized relationships with our employees to advance their career to the next level. We engage them in technology centric client projects that provide opportunities for them to evolve, innovate and deliver world class products and services. Our career opportunities offer challenging assignments and exposure to emerging and cutting edge technologies. We are committed to providing our employees with the tools necessary to accelerate their career progression, while maintaining a healthy, work-life balance. We are deeply committed to providing a workplace ambience that is both challenging and fulfilling.

Resourcesoft is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status.

For more information about Resourcesoft and our services, please visit our website.