Data Engineer

Data EngineerLocation: San Francisco, CARequired Clearance: SecretSalary: CompetitiveJob DescriptionWe are looking for a skilled Data Engineer with a strong focus on AI and machine learning to join our dynamic team. The ideal candidate will play a critical role in designing, implementing, and optimizing data pipelines and infrastructure to support our AI-driven initiatives. You will collaborate closely with data scientists, machine learning engineers, and software developers to ensure robust data architecture and seamless integration of AI models.Key ResponsibilitiesDesign, build, and maintain scalable data pipelines and ETL processes to support AI and machine learning workflows.Collaborate with data scientists and machine learning engineers to understand data requirements and translate them into technical solutions.Implement data processing, transformation, and integration solutions to ensure high-quality, clean, and usable data for model training and evaluation.Optimize data storage and retrieval processes for performance and efficiency.Develop and maintain data infrastructure on cloud platforms (e.g., AWS, Google Cloud, Azure) to support AI model deployment and scaling.Implement monitoring and alerting mechanisms to ensure data pipeline reliability and availability.Ensure compliance with data governance and security standards throughout the data lifecycle.Stay current with industry trends and advancements in data engineering, AI, and machine learning.Participate in code reviews, team meetings, and contribute to a collaborative development environment.Document data architecture, processes, and workflows comprehensively.QualificationsBachelor's or Master's degree in Computer Science, Engineering, or a related field.Proven experience as a Data Engineer with a focus on AI and machine learning projects.Strong proficiency in programming languages such as Python, Scala, Java, or SQL.Experience with big data technologies and frameworks such as Hadoop, Spark, Kafka, etc.Solid understanding of data modeling, database design, and data warehousing concepts.Experience with cloud-based data services (e.g., AWS S3, Redshift, EMR, Glue, Google BigQuery, Azure Data Lake).Familiarity with AI and machine learning frameworks and libraries such as TensorFlow, PyTorch, scikit-learn, etc.Strong problem-solving skills and attention to detail.Excellent communication and teamwork skills.Preferred QualificationsExperience with containerization technologies such as Docker and orchestration tools like Kubernetes.Knowledge of DevOps practices and CI/CD pipelines.