Principal Data Engineer

Principal Data EngineerThis position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Principal Data Engineer based in United States. This role offers the opportunity to lead the design and evolution of scalable, reliable data platforms within a complex enterprise environment. You will shape data architecture, lead technical decisions, and build robust pipelines and ETL solutions that support critical business and reporting needs. The position combines hands-on engineering with technical leadership, mentoring, and cross-functional collaboration. You will work across cloud, data engineering, application, infrastructure, and network teams to deliver secure and high-performing solutions. The role also involves guiding projects through the full software development lifecycle, from requirements and design through testing and production deployment. This is a senior technical position suited to an experienced data engineer who enjoys solving complex data challenges and improving enterprise-scale platforms.Accountabilities:Design scalable and reliable data architectures, leading complex technical decisions related to data pipelines, ETL processes, data migration, performance, scalability, and platform reliability.Review technical implementations delivered by engineering team members and external resources, providing guidance on engineering standards and best practices.Analyze existing systems and identify opportunities to improve data processing, architecture, performance, and reliability.Create high-level technical designs covering ETL processes, data dictionaries, metadata, file layouts, data flows, validation, transformation, and loading requirements.Gather and analyze business and application requirements, identify gaps or inconsistencies, and translate them into effective technical solutions.Plan and execute Agile sprints while coordinating development, testing, deployment, and other phases of the software development lifecycle.Develop technical documentation for parallel jobs, batch processing, sequencers, and data migration activities.Participate in architecture, code, and design reviews with network, infrastructure, and solutions engineering teams to ensure alignment with enterprise standards.Lead and mentor junior engineers, promoting strong engineering practices and supporting technical growth across the team.Requirements:Bachelor's degree or foreign equivalent in Computer Science, Electronic Engineering, or a related field, followed by at least 7 years of progressive experience as a Senior Data Engineer, IT Consultant, or in a related role.At least 7 years of experience building data pipelines, data warehouse solutions, and frameworks supporting end-user reporting requirements.Strong experience gathering requirements, creating software designs, and developing efficient solutions within an Agile software development lifecycle.Proven ability to build data pipelines using diverse sources, including flat files, databases, APIs, and streaming applications.Hands-on experience building ETL and data warehousing solutions with IBM DataStage, Talend, and ODI.Strong experience designing data enrichment frameworks and processing large datasets using technologies such as Spark, Python, Talend, DataStage, ODI, Unix, PL/SQL, Teradata, AWS, and Java.Experience designing, developing, and automating ETL processes for data validation and reconciliation across multiple source systems.Proficiency with programming and scripting languages including Python, Unix, Java, JavaScript, C, and C++, including experience retrieving data from REST APIs and working with security keys.Extensive experience with AWS services, including S3, Glue, Athena, Lake Formation, Lambda, EC2, RDS, EMR, Redshift, and IAM, particularly for data migration and cloud-based processing.Experience working with cloud environments including AWS, Azure, and GCP and integrating cloud-based data with downstream systems.Experience developing and integrating APIs using REST web services.Familiarity with CI/CD tools such as Bitbucket, GitHub, and Jenkins.Strong technical leadership, communication, problem-solving, mentoring, and cross-functional collaboration skills.Ability to work remotely from any location within the United States.Benefits:100% remote work permitted from any US location.Full-time schedule of 40 hours per week.Additional compensation and benefits details were not specified in the source job description.