Big Data Architect

Big Data Architect

Location: Phoenix, AZ

We are seeking an experienced Senior Data Architect to modernize and optimize our existing data architecture, primarily focused on virtual assistant conversational data. This includes large data volumes, multi-table structures, and multi-modality data formats, currently hosted in a traditional RDBMS environment. The role involves rethinking the data schemas, fine-tuning performance, and migrating the data to modern technologies like BigQuery or other scalable, efficient platforms.

Technical Expertise:

Strong experience in designing and managing relational databases (e.g., MySQL, PostgreSQL, Oracle). Hands-on experience with cloud-based data platforms like Google BigQuery, Snowflake, AWS Redshift, or similar. Proficiency in Apache Flink, Kafka, SQL and data modeling tools.

Performance Optimization: Proven track record of fine-tuning large-scale databases, including indexing, partitioning, and query optimization. Experience in schema redesign and migration strategies.

Modern Data Solutions: Knowledge of multi-modality data handling and NoSQL solutions (e.g., MongoDB, DynamoDB). Familiarity with ETL/ELT pipelines and tools like Apache Airflow, DBT, or similar.

Key Responsibilities

1. Data Architecture Modernization:

Analyze the current RDBMS-based architecture for virtual assistant conversational data.

Redesign and modernize data schemas to support scalability, performance, and multi-modality use cases.

Incorporate emerging data storage technologies such as BigQuery, Snowflake, or other cloud-native platforms.

2. Optimization and Fine-Tuning:

Evaluate and improve indexing, partitioning, and sharding strategies to optimize query performance.

Refactor existing schemas and table structures for efficient data retrieval and storage.

Implement best practices for data normalization and denormalization as required by the use cases.

3. Migration Strategy:

Develop a detailed migration plan for transitioning data from the current RDBMS to modern platforms.

Ensure data consistency, integrity, and minimal downtime during migration.

Work with DevOps and engineering teams to automate migration processes and set up monitoring tools.

4. Support Multi-Modality Data Needs:

Design data models that can handle multi-modality data (text, images, audio, etc.) effectively.

Enable seamless integration of new data types into the existing architecture.

5. Collaboration and Governance:

Collaborate with engineering, analytics, and AI/ML teams to align the data architecture with their needs.

Define and enforce data governance, quality standards, and security policies.

Document architectural decisions and maintain up-to-date diagrams and schemas.

6. Performance Monitoring and Maintenance:

Implement tools to monitor database performance and identify bottlenecks.

Proactively recommend improvements to maintain high availability and reliability.

Plan for future data growth and evolving business requirements.