Senior Data Engineer

LatentView Analytics is a leading global analytics and decision sciences provider, delivering solutions that help companies drive digital transformation and use data to gain a competitive advantage. With analytics solutions that provide a 360-degree view of the digital consumer, fuel machine learning capabilities and support artificial intelligence initiatives., LatentView Analytics enables leading global brands to predict new revenue streams, anticipate product trends and popularity, improve customer retention rates, optimize investment decisions, and turn unstructured data into valuable business assets Required Qualifications: 7+ years of experience in Data Engineering roles Extensive hands-on experience with Databricks (Delta Lake, Unity Catalog, cluster management, notebooks, job orchestration) Strong experience with AWS cloud services (S3, IAM, Glue, EMR, Lambda, Redshift, CloudWatch) Proven expertise in Apache Airflow for workflow orchestration and scheduling Strong programming skills in Python and SQL Solid understanding of Apache Spark (PySpark, Spark SQL, performance tuning) Experience with data modeling, warehousing concepts, and Lakehouse architecture Familiarity with version control (Git) and CI/CD practices Strong understanding of data governance, quality, and lineage principles Excellent problem-solving and communication skills Responsibilities Design, develop, and maintain scalable ETL/ELT pipelines using Databricks (PySpark/Spark SQL) Build, orchestrate, and monitor workflows using Apache Airflow (DAG design, scheduling, dependency management) Architect and manage data infrastructure on AWS (S3, Glue, Lambda, EMR, Redshift, IAM, etc.) Optimize data pipelines for performance, reliability, and cost-efficiency Implement data quality checks, testing frameworks, and observability/monitoring for pipelines Collaborate with Data Scientists, Analysts, and Product teams to understand data requirements Design and maintain data models (Lakehouse/Medallion architecture - Bronze/Silver/Gold layers) Ensure data security, access controls, and compliance across the data platform Document architecture, pipelines, and processes for team knowledge sharing Mentor junior data engineers and contribute to engineering best practices Required skills Databricks, AWS, Python, SQL, Pyspark, Genie, GenAI, Analytics, Cybersecurity Knowledge At LatentView Analytics, we value a diverse, inclusive workforce and provide equal employment opportunities for all applicants and employees. All qualified applicants for employment will be considered without regard to an individual's race, color, sex, gender identity, gender expression, religion, age, national origin or ancestry, citizenship, physical or mental disability, medical condition, family care status, marital status, domestic partner status, sexual orientation, genetic information, military or veteran status, or any other basis protected by federal, state or local laws. #J-18808-Ljbffr LatentView Analytics