Senior Data Engineer

Hello, Greetings!!! This is Harsh from Jconnect INC . Below is the requirement with my client. Please share the updated resume and below details. Title: Data Engineer Location: New York, NY (Onsite) Duration: Full-time only

JOB DESCRIPTION

Must Have Technical/Functional Skills Hands-on experience in building ETL using Databricks SaaS infrastructure. Experience in developing data pipeline solutions to ingest and exploit new and existing datasources Expertise in leveraging SQL, programming language like Python and ETL tools like Databricks Perform code reviews to ensure requirements, optimal execution patterns and adherence to established standards. Expertise in AWS Compute (EC2, EMR), AWS Storage (S3, EBS), AWS Databases (RDS, DynamoDB), AWS Data Integration (Glue). Advanced understanding of Container Orchestration services including Docker and Kubernetes, and a variety of AWS tools and services. Good understanding of AWS Identify and Access management, AWS Networking and AWS Monitoring tools. Proficiency in CI/CD and deployment automation using GITLAB pipeline. Proficiency in Cloud infrastructure provisioning tools e.g., Terraform. Proficiency in one or more programming languages e.g., Python, Scala. Experience in Starburst, Trino and building SQL queries in federated architecture. Good knowledge of Lake house architecture. Design, develop, and optimize scalable ETL/ELT pipelines using Databricks and Apache Spark (PySpark and Scala). Build data ingestion workflows from various sources (structured, semi-structured, Develop reusable components and frameworks for efficient data processing. • Implement best practices for data quality, validation, and governance. • Collaborate with data architects, analysts, and business stakeholders to understand data requirements. • Tune Spark jobs for performance and scalability in a cloud-based environment. • Maintain robust data lake or Lakehouse architecture. • Ensure high availability, security, and integrity of data pipelines and platforms. • Support troubleshooting, debugging, and performance optimization in production workloads. Roles & Responsibilities Work on migrating applications from an on-premises location to the cloud service providers. Develop products and services on the latest technologies through contributions in development, enhancements, testing and implementation. Develop, modify, extend code for building cloud infrastructure, and automate using CI/CD pipeline. Partners with business and peers in the pursuit of solutions that achieve business goals through an agile software development methodology. Perform problem analysis, data analysis, reporting, and communication. Work with peers across the system to define and implement best practices and standards. Assess applications and help determine the appropriate application infrastructure patterns. Use the best practices and knowledge of internal or external drivers to improve products or services. If you are interested, please send me your updated resume ASAP with below details : Full Name: Alternate mail ID: Current Location/Zip: Expected salary: Willingness to travel/relocate to job location: Visa/Work Permit Status: Notice Period/Availability to Start: Preferred Interview timings (Specify Time zone): LinkedIn URL: Looking forward for your response.. Thanks and Regards, Harsh Srivastava Jconnect Infotech Inc . Cherry Hill, NJ 08034 Email: View email address on click.appcast.io [We are an E-Verify and Equal Employment Opportunity Employer with adherence to EEO policy] The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited. 3B Staffing LLC