Data Engineer

Position Summary As a Data Engineer, you play a key role in building and maintaining the data architecture that supports our business needs. Your work includes developing scalable data pipelines to transform and move data from diverse sources, enabling analysts and data scientists to work with clean, accessible datasets. You'll focus on the secure and efficient management of data systems, ensuring information is stored effectively and retrieved quickly when needed. This includes designing, testing, and maintaining data architectures such as databases, data warehouses, data lakes, and large-scale processing systems. In this role, you'll assemble large, complex datasets to meet both functional and non-functional requirements. Expect to build high-performance algorithms, predictive models, and proofs of concept while using modern programming languages and tools to integrate systems and manage complex data workflows. A Glimpse into the Daily Routine of a Data Engineer A day in the life of a Data Engineer states with building and delivering high quality data architectures and pipelines that support clients, business analysts, and data scientists. A Data Engineer also interfaces with other technology teams to extract, transform, and load [ETL] data from a wide variety of data sources. Effectual Data Engineers continually improve ongoing reporting and processes, as well as automate or simplify self-service for our clients. Effectual Data Engineers develop, code, and deploy scripts written in the Python programming language, as Python is the language of Data. All Data Engineers are first and foremost Software Engineers with an understanding of the SDLC process. Essential Duties and Responsibilities Design and create the overall data architecture (databases, large scale processing systems). Define how data will be stored, consumed, integrated, and managed by different data entities and IT systems. Ensure the data strategies and architectures are in compliance with regulatory requirements. Work with business executives to ensure the data architecture aligns with business requirements. Data volumes. Ensure that data flows efficiently through the pipelines, making it usable for further analysis and reporting. Implement, maintain, and update data management tools and systems. Optimize data retrieval, develop dashboards, reports and perform database maintenance tasks. Collaborate closely with data scientists, helping to ensure that the organization's data infrastructure meets the requirements of complex data analytics. Qualifications Bachelor's or master's degree in Computer Science, Engineering or a related field 4-7 years experience working as a Data Engineer, preferably in a professional services or consulting environment Strong proficiency in programming languages such as Python, Java, or Scala, with expertise in data processing frameworks and libraries (e.g., Spark, Hadoop, SQL, etc.) In-depth knowledge of database systems (relational and NoSQL), data modeling, and data warehousing concepts Experience with cloud-based data platforms and services (e.g., AWS, Azure, Google Cloud) including familiarity with relevant tools and technologies (e.g., S3, Redshift, BigQuery, etc.) Computer Vision/Intelligent Document Processing. Proficiency in designing and implementing ETL processes and data integration workflows using tools like Apache Airflow, Informatica, or Talend Familiarity with data governance practices, data quality frameworks, and data security principles Strong analytical and problem-solving skills, with the ability to translate business requirements into technical solutions Excellent communication and collaborations skills, with the ability to effectively work with clients and cross-functional teams Self-motivated and proactive, with a passion for learning and staying updated with the latest trends and advancements in the field of data engineering Able to work with ambiguity and turn client wants and needs into working stories, epics which can be executed upon during a sprint. This means Data Engineers understand and know the 'agile' progress software delivery A firm understanding of the SDLC process An understanding of object-oriented programming Needs minimal direction AWS background, AWS Cloud Formation & Data Migration Services (DMS) Solution Engineer mindset Nice-to-Have Skills, Certifications and Experience A curious nature and inquisitive attitude when approaching problems Have the attitude of 'good is not good enough' for our clients Snowflake or Databricks certifications and/or hands-on-keyboard experience AWS Certified Solutions Architect – Associate AWS Certified Solutions Architect – Professional AWS Certified Developer – Associate AWS Certified Machine Learning – Specialty AWS Certified Data Engineer - Associate HashiCorp Certified: Terraform Associate Certified Associate in Python Programming (PCAP) Certified Entry-Level Data Analyst with Python (PCED) Snowflake SnowPro Core Databricks Certified Data Engineer Associate Company Offered Benefits Full-time employees are eligible to participate in our employee benefit programs: Medical, dental, and vision health insurances, Short term disability, long term disability and life insurances, 401k with Company match Paid time off (PTO) (120 hours PTO that accrue over one year) Paid time off for major holidays (14 days per year) These and any other employee benefit offerings are subject to management's discretion and may change at any time. Physical Demands and Work Environment The work is generally performed in an office environment. Physical demands include sitting, keyboarding, verbal communication, written communication. Employees are occasionally required to stand; walk; reach with hands and arms; climb or balance; and stoop, kneel, crouch, or crawl. The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this position. Reasonable accommodation may be made to enable individuals with disabilities to perform the functions. Salary Range for this position: $132,000-$166,000 CA ID IT10000879 Salary ranges provided are for informational purposes only and may vary depending on factors such as experience, qualifications, and geographic location. The final salary offer will be determined based on the candidate's skills and alignment with the role requirements. This job description may not be inclusive of all assigned duties, responsibilities, or aspects of the job described, and may be amended anytime at the sole discretion of the Employer. Duties and responsibilities are subject to possible modification to reasonably accommodate individuals with disabilities. To perform this job successfully, the incumbents will possess the skills, aptitudes, and abilities to perform each duty proficiently. This document does not create an employment contract, implied or otherwise, other than an "at will" relationship. Effectual Inc. is an EEO employer and does not discriminate on the basis of any protected classification in its hiring, promoting, or any other job-related opportunity. #J-18808-Ljbffr