Senior Specialist - Data Engineering

Pipeline Development Design and implement highly modular automated Extract Transform Load ETL data pipelines using Python and related frameworksData Transformation Build Python logic using packages like Pandas or PySpark to clean format deduplicate and encrypt raw data into optimized staging areasOrchestration Automation Schedule and orchestrate complex workflows using tools like Apache Airflow Prefect or Dagster to minimize delays and prevent performance bottlenecksDatabase Warehouse Engineering Develop and tune SQL queries design data schemas and manage data ingestion into Relational Databases RDBMS or modern cloud data warehouses like SnowflakeMonitoring Error Handling Implement thorough logging frameworks exception handling and realtime ing systems to maintain pipeline integrity and data qualityAPI System Integration Extract data from heterogeneous systems including flat files thirdparty REST APIs web scraping and streaming sourcesDocumentation Maintain clear code documentation architectural mapping and pipeline workflow outlines