Senior Python Developer with Spark

Job order - J0526-0018 - Permanent Full Time Title Senior Python Developer with Spark Category Software Development / Engineering Location Reston, Virginia, United States Job Description We are seeking a Senior Python Developer with Spark skills to design, build, and optimize large-scale data processing systems in a cloud-native AWS environment. This role focuses heavily on developing high-performance data pipelines using Spark and Python, as well as working with complex relational datasets. We partner with 15 of the top 20 banks globally, and our top 10 banking clients have worked with us for an average of 26 years. Work location is a client site in Reston, VA. A hybrid working model is acceptable. Responsibilities Design, build, and optimize large-scale data processing systems in a cloud-native AWS environment. Develop high-performance data pipelines using Spark and Python. Write advanced SQL to support hierarchical and analytical use cases. Diagnose and improve inefficient data workflows, particularly Spark jobs. Partner with cross-functional teams to deliver scalable, reliable, and well-architected data solutions that support critical business functions in a financial services setting. Required Qualifications 8+ years of hands-on experience with Python for data engineering, including building and maintaining data pipelines. Deep expertise in Apache Spark, including performance tuning (partitioning, caching, broadcast joins, shuffle optimization), understanding of execution plans (DAGs, stages, tasks), memory and resource management. Solid experience with big-data ecosystems such as Hadoop, Hive, and EMR. Advanced proficiency in SQL, including writing recursive CTEs for hierarchical data, query optimisation, indexing strategies, and execution plan analysis. Strong experience with AWS services including EMR, Lambda, Step Functions, EventBridge, Redshift, S3, and Glue. Experience building and consuming APIs, along with data transformation and ingestion workflows. Proven ability to work with large-scale datasets, performing data analysis and extracting actionable insights. Familiarity with data modeling concepts (normalised/denormalised structures, handling hierarchical data). Hands-on experience with CI/CD pipelines and tools such as GitLab and Terraform. Strong understanding of performance troubleshooting, including identifying bottlenecks in distributed systems. Ability to clearly explain technical decisions, especially around Spark optimisation and SQL logic. Strong analytical thinking, problem-solving skills, and attention to detail. Effective collaboration skills in cross-functional, matrixed environments. Desired Skills Experience in financial services or regulated environments. Exposure to data visualisation tools. Familiarity with event-driven architectures on AWS. Education Bachelor’s degree in Computer Science, Information Systems, or a related field. Benefits Competitive compensation. Comprehensive insurance options. Matching contributions through the 401(k) plan and the share purchase plan. Paid time off for vacation, holidays, and sick time. Paid parental leave. Learning opportunities and tuition assistance. Wellness and well-being program. Employment Equity Qualified applicants will receive consideration for employment without regard to their race, ethnicity, ancestry, color, sex, religion, creed, age, national origin, citizenship status, disability, pregnancy, medical condition, military and veteran status, marital status, sexual orientation or perceived sexual orientation, gender, gender identity, and gender expression, familial status or responsibilities, reproductive health decisions, political affiliation, genetic information, height, weight or any other legally protected status or characteristics, to the extent required by applicable federal, state, and/or local laws where we do business. CGI provides reasonable accommodations to qualified individuals with disabilities. If you need an accommodation to apply for a job in the U.S., please email the CGI U.S. Employment Compliance mailbox at View email address on click.appcast.io. Refer the Position ID when applying. The CGI U.S. Employment Compliance team will assist you. All CGI offers of employment in the U.S. are contingent upon successful completion of a background investigation. Background investigation components vary depending on the assignment and/or level of U.S. government security clearance held. Some investigations may include a credit check. CGI will consider qualified applicants with arrests and conviction records in accordance with local regulations and ordinances. CGI will not discriminate against employees or applicants because they have inquired about or disclosed their own pay or the pay of others, except where required by law. #J-18808-Ljbffr