Big Data Architect

Job Title: Big Data Architect Location: Minneapolis, MN (Hybrid) Role Summary We are looking for a highly experienced Big Data Architect with 15+ years of overall technology experience and strong specialization in Azure Cloud, Hadoop, Apache Spark, Scala, and Snowflake. The ideal candidate will be responsible for designing, architecting, and optimizing enterprise-scale data platforms, batch and streaming data pipelines, data lakes, lakehouse architectures, and cloud-native analytics solutions. This role requires deep technical expertise, hands-on architecture experience, strong stakeholder engagement skills, and the ability to define scalable, secure, high-performance data solutions aligned with business and technology objectives. Key Responsibilities Define and own end-to-end big data architecture across ingestion, storage, processing, orchestration, governance, security, consumption, and operational monitoring layers. Design scalable batch and real-time streaming data pipelines using technologies such as Azure Data Factory, Azure Databricks, Apache Spark, Spark Structured Streaming, Kafka, Event Hubs, Hadoop ecosystem tools, and Snowflake. Architect cloud-native and hybrid data platforms on Microsoft Azure, including Azure Data Lake Storage, Azure Synapse Analytics, Azure Databricks, Azure Functions, Azure Key Vault, Azure Monitor, and related services. Lead modernization and migration initiatives from legacy Hadoop/on-premises platforms to Azure-based data lake, lakehouse, and Snowflake architectures. Design data models, storage structures, partitioning strategies, file formats, and schema evolution approaches to support high-volume analytics, reporting, and machine learning workloads. Provide technical leadership on data ingestion patterns, ETL/ELT design, metadata management, data lineage, data quality, and governance frameworks. Optimize Spark, Scala, Hadoop, and Snowflake workloads for performance, scalability, cost efficiency, reliability, and maintainability. Define reusable architecture patterns, reference implementations, coding standards, solution blueprints, and best practices for data engineering teams. Collaborate with business stakeholders, product owners, enterprise architects, data engineers, data scientists, BI teams, security teams, and operations teams to translate requirements into robust technical solutions. Review solution designs, conduct architecture audits, identify technical risks, and recommend improvement plans for platform stability, throughput, latency, and operational excellence. Guide teams on workload orchestration, dependency management, CI/CD implementation, DevOps practices, observability, alerting, and production support readiness. Mentor senior engineers and technical leads on big data engineering, distributed processing, cloud architecture, and performance tuning practices. Required Technical Skills Cloud Platform: Strong hands-on architecture experience with Microsoft Azure, including Azure Data Lake Storage, Azure Databricks, Azure Synapse Analytics, Azure Data Factory, Event Hubs, Azure Functions, Azure Monitor, and Azure security services. Big Data Platforms Deep expertise in Hadoop ecosystem components such as HDFS, Hive, MapReduce, YARN, Sqoop, Oozie, HBase, and related distributed data processing tools. Distributed Processing Expert-level experience in Apache Spark, Spark SQL, Spark Structured Streaming, performance tuning, memory management, job optimization, and cluster sizing. Programming Strong development experience in Scala, along with working knowledge of Python, SQL, shell scripting, and data engineering frameworks. Data Warehousing Strong Snowflake experience, including warehouse design, schema design, data loading, Snowpipe, performance tuning, clustering, query optimization, cost optimization, and integration with Azure data services. Streaming Technologies Experience designing and implementing streaming pipelines using Kafka, Azure Event Hubs, Spark Structured Streaming, and event-driven architectures. Storage Optimization Expertise in Parquet, ORC, Delta Lake, partitioning, bucketing, compaction, compression, indexing, archival strategies, and data lifecycle management. Data Architecture Experience with data lakes, lakehouse platforms, enterprise data warehouses, data marts, dimensional modeling, metadata management, data lineage, and data governance. DevOps and Automation Experience with Git, CI/CD pipelines, infrastructure automation, deployment practices, environment management, monitoring, and production operations. Security and Governance Understanding of role-based access control, encryption, data masking, auditing, secrets management, compliance controls, and enterprise data governance standards. Required Experience 15+ years of IT experience with strong background in data engineering, big data platforms, distributed computing, cloud data architecture, and enterprise analytics solutions. Minimum 8+ years of hands-on experience with Hadoop, Spark, Scala, and large-scale data processing platforms. Minimum 5+ years of experience architecting solutions on Azure Cloud data services. Strong experience designing and implementing batch pipelines, near-real-time pipelines, streaming ingestion frameworks, and event-driven data processing solutions. Proven experience with Snowflake architecture, data warehouse modernization, cloud data migration, and performance optimization. Experience handling large data volumes, complex transformations, SLA-driven workloads, high-throughput ingestion, and low-latency processing requirements. Strong background in performance tuning across Spark jobs, cluster configurations, SQL queries, storage layers, partitioning strategies, and Snowflake compute warehouses. Experience in defining architecture roadmaps, technology evaluation, proof of concepts, migration strategy, and production implementation plans. Prior experience working with cross-functional teams in agile delivery models and leading technical discussions with senior stakeholders. Preferred Qualifications Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field. Azure certifications such as Azure Solutions Architect Expert, Azure Data Engineer Associate, or Azure Database Administrator Associate are preferred. Snowflake certification or demonstrated Snowflake architecture experience is an added advantage. Experience with Databricks, Delta Lake, Unity Catalog, dbt, Apache Airflow, Terraform, Kubernetes, or containerized data platforms is preferred. Exposure to AI/ML data pipelines, feature engineering platforms, MLOps, and advanced analytics workloads is desirable. Key Competencies Strong solution architecture and technical leadership skills. Ability to translate business requirements into scalable data architecture and implementation roadmaps. Excellent analytical, problem-solving, and performance troubleshooting abilities. Strong communication skills with the ability to work with technical and non-technical stakeholders. Ability to mentor engineering teams and enforce architecture standards and best practices. Strong ownership mindset with focus on reliability, scalability, security, and cost optimization. Futran Tech Solutions Pvt. Ltd.