Senior Data Engineer

Senior Data Engineer Remote Contract

Key Responsibilities:

• Data Pipeline Development: Design, implement, and maintain ETL processes to extract, transform, and load data from diverse sources into data storage systems.
• Data Integration: Integrate data from various structured and unstructured sources, including databases, APIs, logs, and external data feeds.
• Data Modeling: Develop and maintain data models that align with business requirements, ensuring data is stored efficiently and accurately.
• Data Quality Assurance: Implement data quality checks and validation procedures to ensure data accuracy, consistency, and completeness.
• Data Warehousing: Build and maintain data warehouses or data lakes, optimizing them for query performance and scalability.
• Documentation: Create and maintain documentation for data pipelines, data models, and infrastructure configurations.
• Collaboration: Collaborate with analysts, and other stakeholders to understand data requirements and provide data support for analytics and reporting.
• Monitoring and Troubleshooting: Set up monitoring systems to proactively identify issues and troubleshoot data pipeline failures or performance bottlenecks.

Qualifications:

• Bachelor's or higher degree in computer science engineering, information technology, or a related field.
• Strong programming skills in Python and associated libraries: Pandas, SQL Alchemy, Flask, Fast API etc.
• Proficiency in SQL and experience with relational and NoSQL databases.
• Knowledge of data warehousing technologies like Amazon Redshift, Google BigQuery, Snowflake, or similar is a plus.
• Experience with data pipeline orchestration tools like Apache Airflow, Luigi, or similar.
• Familiarity with cloud platform AWS is a plus.
• Understanding of data integration techniques, data modeling, and data architecture best practices.
• Excellent problem-solving and troubleshooting skills.
• Strong communication and collaboration skills.
• Knowledge of data security and compliance standards is a plus.
• Experience with big data technologies (e.g., Hadoop, Spark) is a bonus.