Data Engineer

Data Engineer/PySpark Developer

Software Guidance & Assistance, Inc., (SGA), is searching for a Data Engineer/PySpark Developer for a right to hire assignment with one of our premier financial services clients in Tampa, FL. Responsibilities include:

• Design, develop, and implement robust, scalable, and high-performance data pipelines and applications using Python, PySpark, and Big Data technologies.
• Work autonomously to analyze requirements, propose technical solutions, and deliver high-quality code and data products, ensuring alignment with architectural standards and business objectives.
• Utilize expertise in various Big Data platforms (e.g., Hadoop, Hive, Kafka, Spark) to process, transform, and manage large datasets efficiently.
• Write complex SQL queries, stored procedures, and optimize database performance for large-scale data warehousing and analytics solutions.
• Develop and enhance ETL (Extract, Transform, Load) processes, ensuring data quality, integrity, and timely delivery. Experience with various ETL tools and methodologies is a plus.
• Proactively research, evaluate, and integrate new and emerging technologies, frameworks, and tools to improve development processes and solution capabilities.
• Ensure adherence to coding standards, conduct thorough code reviews, and implement best practices for software development, data governance, and security.
• Diagnose and resolve complex technical issues related to data pipelines, performance bottlenecks, and system integrations in a fast-paced environment.
• Collaborate effectively with cross-functional teams including architects, data scientists, business analysts, and QA engineers. Provide technical guidance and mentorship to junior team members.
• Identify opportunities to use AI tools to speed up development, code reviews, unit testing, and deployment.

Required Skills:

• Bachelor's degree/University degree or equivalent experience
• 10+ years of experience in Applications Development, Systems Analysis, or equivalent senior engineering roles.
• Extensive hands-on experience delivering enterprise-scale, database-driven platforms in a regulated environment.
• Expert-level proficiency in Python programming, including object-oriented design, data structures, algorithms, and extensive experience with various Python libraries.
• Deep expertise in developing, optimizing, and deploying PySpark applications for large-scale data processing, ETL, and real-time analytics on distributed systems (e.g., Spark SQL, Spark Streaming, DataFrames).
• Strong understanding of Apache Spark architecture, Hadoop ecosystem, and experience with distributed computing concepts. Familiarity with big data storage formats (e.g., Parquet, ORC).
• Solid experience with both relational databases (e.g., Oracle) and NoSQL databases (e.g., MongoDB). Strong SQL writing and optimization skills.
• Experience in designing, developing, and consuming RESTful APIs using Python frameworks (e.g., Flask, FastAPI, Django REST Framework).
• Strong understanding and practical experience with CI/CD tools (e.g., Jenkins) and containerization technologies (Docker, Kubernetes).
• Expert-level proficiency with Git.
• Experience with unit testing (e.g., Pytest), integration testing, and performance testing frameworks for Python and PySpark applications.
• Exposure to or direct experience with Artificial Intelligence (AI) and Machine Learning (ML) concepts, frameworks (e.g., TensorFlow, PyTorch), or relevant projects is a significant advantage.
• Exceptional analytical and problem-solving abilities, with a strong capacity to understand complex business needs and translate them into effective technical solutions.
• Excellent leadership, team management, and mentoring capabilities.
• Superior verbal and written communication skills, with the ability to articulate complex technical concepts clearly to both technical and non-technical audiences.
• Strong collaboration and interpersonal skills, with a proven ability to work effectively with cross-functional teams.
• Highly proactive, results-oriented, and a strong commitment to delivering high-quality, innovative solutions.
• Ability to thrive and lead in an agile, dynamic, and fast-paced work environment.

Preferred Skills:

• Exposure to at least one major cloud provider (AWS, Azure, or GCP), specifically with their compute, storage, and data services (e.g., S3, ADLS, EMR, Databricks, Azure Synapse) preferred.
• Master's degree preferred

SGA is a technology and resource solutions provider driven to stand out. We are a women-owned business. Our mission: to solve big IT problems with a more personal, boutique approach. Each year, we match consultants like you to more than 1,000 engagements. When we say let's work better together, we mean it. You'll join a diverse team built on these core values: customer service, employee development, and quality and integrity in everything we do. Be yourself, love what you do and find your passion at work.