Data Platform Manager- Hybrid (Applicants must reside in Pittsburgh, PA)
A.C.Coy Company
Data Platform Manager
Location: Hybrid, Pittsburgh, PA
Job Type: Full Time / Permanent
Work Authorization: No C2C or Sponsorship
The A.C.Coy company has an immediate opening for a Data Platform Manager. This role will be responsible for designing, building, and optimizing enterprise wide data platforms within the Data Warehouse.
Responsibilities
- Lead and mentor a team of data engineers, conducting code reviews and ensuring development standards
- Support troubleshooting and incident management for data-related issues in production
- Collaborate with business stakeholders, data scientists, and other team members to gather requirements and translate them into technical specifications
- Lead the design, development and deployment of scalable and high-performance data pipelines using Azure Databricks; ensuring the data integrity, availability, efficient extraction, transformation, and loading of data from various sources into the Azure Databricks Data Warehouse
- Collaborate with data scientists, analysts, and other engineering teams to deliver business-critical insights. Optimize pipeline performance, cost, and scalability in the Azure cloud environment
- Define best practices for data ingestion, processing, storage, and governance. Implement data quality checks and validation procedures to ensure the accuracy and integrity of data between various sources, including API's, databases and streaming platforms
- Collaborate with data scientists and analysts to operationalize and deploy machine learning models
- Architecture Design:
- Define the end-to-end Lakehouse architecture using Delta Lake, implementing medallion architecture (Bronze, Silver, Gold layers) for robust data processing
- Familiarity with data modeling and schema design principles
- Pipeline Engineering:
- Oversee the development of robust, scalable batch and streaming ETL/ELT pipelines using PySpark, Scala, and SQL and with minimal latency
- Implement data transformations, enrichment, and quality checks using PySpark/Scala within the Databricks environment
- Integrate real-time and batch data sources using Apache Kafka and ADF
- Support large-scale data pipelines using Apache Spark on Databricks, Kafka, Stelo, and Azure Data Factory (ADF)
- Data Governance & Security:
- Implement Unity Catalog for unified governance, data security, fine-grained access control (RBAC), privacy measures, and data lineage tracking
- Performance Optimization & Tuning:
- Tune Spark jobs and Databricks clusters to maximize throughput while maintaining cost efficiency through auto-scaling and cluster policies
- Expertise in indexing strategies, query optimization, execution plans, and partitioning/sharding
- Platform Integration:
- Orchestrate workflows by integrating Databricks with other Azure services like Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), and Azure DevOps for CI/CD pipelines
Qualifications
Required Education
- Bachelor's degree in Computer Science, Engineering, or a related field
Required Experience
- 5-7+ years hands-on data engineering or architecture, with at least 2-4 years specifically focused on Azure Databricks, including Azure cloud technologies
- 2-5 years experience in managing a team of data engineers, data scientists and/or analysts
- Certifications (Preferred): Microsoft Certified: Azure Data Engineer Associate (DP-203), Databricks Certified Data Engineer Professional, or Azure Solutions Architect Expert
- Database Architecture: Proficiency in both Relational (SQL) and NoSQL (Document, Key-Value, Graph, Columnar) databases. Develop and maintain data models and schemas to support data analysis and reporting requirements
- Distributed Systems: Knowledge of frameworks like Apache Hadoop, Spark, or Presto/Trino for optimizing and handling massive data volumes and retrieval mechanisms, ensuring the efficient processing of large datasets
- Storage Optimization: Understanding file formats like Parquet, Avro, or ORC and compression techniques
- Deep proficiency in programming languages: Python (specifically PySpark), SQL, PowerShell, and Scala
- Infrastructure: Hands-on experience with Azure Cloud infrastructure, including Networking (VNETs), Key Vault, and Identity Management
- Big Data Tools: Deep knowledge of Apache Spark runtime internals, MLflow for MLOps, and orchestration tools like Airflow
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Data Platform Manager- Hybrid (Applicants must reside in Pittsburgh, PA) in Pittsburgh, PA vacancy
- ...Role: Data Engineer (SQL, PySpark and Python) Location: Pittsburgh, PA Hybrid ( 3 to 4 days from office)... ...pipelines and big data applications using SQL, PySpark,... ...process, analyze, and manage large datasets... ...modern enterprise data platforms. Develop and maintain...PlatformContract workWork at officeRemote work
- ...conversion) Location: Pittsburgh, PA--Hybrid Only on W2 Manager Notes: This... ...Expertise in Kafka for data streaming Strong... ...JSON Google Cloud Platform (or similar such as AWS... ...and maintain scalable applications using Java with Spring...PlatformLong term contractContract work
$118k - $150k
...to join our team in Pittsburgh, PA , helping patients navigate... ...s) to assess and manage comorbid conditions... ...'s online training platform StarLearning.... ...at Colorado Residents: Please do not respond... ...questions in this initial application that may seek age-identifying...PlatformMinimum wageFull timeCasual workRemote workWork from homeHome officeMonday to FridayWeekend work$100.1k - $223.08k
...company's success. As a Data Manager within PNC's Data... ...will be based in Pittsburgh, PA; Dallas, TX or... ...on multiple data platforms and/or multiple applications. Develops strategies... ...candidates must demonstrate appropriate... .... California Residents Refer to the California...PlatformFull timeTemporary workPart timeWork experience placementWork at office$118k - $150k
...to join our team in Pittsburgh, PA, helping patients navigate... ...(CHEs) to assess and manage comorbid conditions... ...’s online training platform StarLearning. #LI-BB3... ...Learn more at Colorado Residents: Please do not... ...questions in this initial application that may seek age-...PlatformMinimum wageFull timeCasual workLocal areaRemote workWorldwideHome officeMonday to FridayWeekend work- ...Mission Operator - Pittsburgh, PA (Contract)... ..., cloud platforms, mapping, sensors... ...meets real-world application. As a Test... ...preferred Must have a valid driver... ...the state of residence and ability to... ...diagnostics and data logging tools... ...with work management tools like Jira...PlatformPermanent employmentContract workLocal areaLong distance
- ...to improve total applications and create a continual... ...: Manage job board postings... ...organization on varying platforms. Strategically place... ...of graduating residents/fellows into AHN.... ...uploading essential data into ATS. Maintain... ...information that must be protected at all...PlatformFor contractorsLocal area
- ...And Security Location: Hybrid, Pittsburgh, PA Job Type: Full Time / Permanent... ...will require experience managing enterprise technology... ...regulatory requirements, including data privacy and security... ...Own enterprise collaboration platforms (e.g., Microsoft 365/Teams,...PlatformPermanent employmentFull timeContract workWork at officeImmediate start
$80k - $217.1k
...Engineering Manager Senior within PNC's Data and Automation... ...-OH, Pittsburgh-PA, Denver, CO... ...science and application development... ...ensuring our ML platform is both powerful... .... • Drive hybrid/on-prem... ...candidates must demonstrate... ...California Residents Refer to...PlatformFull timeTemporary workPart timeWork experience placementWork at office- ...Assistant for the firm's Pittsburgh, PA office.... ...rooms Scanning and data entry into internal database... ...files from a variety of platforms Operate office phone... ...experience is preferred Must be organized and... ...documents properly - applicants must pass standard alphabetical...PlatformFull timeWork at office
$91k - $202.8k
.... As Product Manager Senior within... ...be based in Pittsburgh or Philadelphia, PA; Wilmington,... ...Strong grasp of data models, APIs,... ..., Streaming platforms, entitlements... ...through all applicable channels. Prioritizes... ...candidates must demonstrate... ...California Residents Refer to...PlatformFull timeTemporary workPart timeWork experience placementWork at office- Application Developer Lead (Agile, Scrum, Java, Bootstrap... ..., HTML5, Jenkins) in Pittsburgh, PA Agile, Angular JS,... ...container platforms, Openshift, Docker, Kubernetes... ...Kubernetes, iOS, Android, Hybrid HTML, Responsive Web... ...2.0, mobile device management. -DevOps CI/CD, GIT,...PlatformPermanent employmentFull timeRemote work
- ...Washington Place. Pittsburgh, Pennsylvania. 15... ...in Pittsburgh, PA. Candidates must be local or willing... ...Reports to: Manager of AI Delivery and... ...tools, preparing data, and monitoring automated... ...and automation platforms. The position... ...any employee or applicant for employment because...PlatformWork at officeLocal areaRelocation
- ...Data Manager Senior At PNC, our people are our greatest differentiator... ..., you will be based in Pittsburgh, PA, Denver, CO, Dallas, TX or... ...complex data types and hybrid platforms. Create easily accessible... ...Successful candidates must demonstrate appropriate knowledge...PlatformWork experience placementWork at office
- ...capabilities across data, edge,... ...infrastructure and applications, deep ecosystem... ...in cloud change management; and cloud-... ...understanding of Data & AI platforms. Minimum of... ...'s Degree, must have minimum 6... ..., private, hybrid, and sovereign... ...are a California resident, live in or plan...PlatformWork experience placementLive inWork at officeLocal area
- ...at Ross Park Mall in Pittsburgh, PA. How Do You Fit In?... ...alongside the Store Manager, store supervisors, and... ...climbing ladders. * Applicants must be 18 years of age or... ...deviations. Personal data and sensitive... ...through the approved platforms Teams or Zoom-never via...PlatformPart timeWork experience placementWeekend workAfternoon shift
- ...Senior Marketing Manager, Global Platform Robotics (Pittsburgh, PA) Life. Unlimited. At Smith+Nephew we design... ...Assistance Program Flexibility - Hybrid Work Model (For most professional... ...employment opportunities to applicants and employees without regard to race...PlatformFlexible hours
- ...integrated, AI-powered platform combining the... ...you to submit your application via the apply link... ...and/or families in Pittsburgh, PA Experience managing risk and responding... ...needed Full-time resident of the United States... ...contract position, your data will be processed...PlatformFull timeContract workFor contractorsPrivate practice
$171.61k
...Consulting - AI & Data - Enterprise... ...Engineering (Manager) (Multiple... ...Young U.S. LLP, Pittsburgh, PA. Demonstrate... ...tools and platforms. Design methodology... ...: Must have a Bachelor... ...leader-enabled hybrid model. Our expectation... ...EY accepts applications for this...PlatformFull timeWork experience placementSummer holidayImmediate startMonday to Friday- ...Description Title: Sr. Data Product Manager Reports To:... ...Location: Pittsburgh, PA or New York, NY American... .... Through the brand platform Live Your Life, we... ...dress code Hybrid schedule FOMO events... ...associates and applicants for employment because...PlatformFull timePart timeSummer workCasual workLocal area
- ...Business Banking Relationship Manager - Pittsburgh, PA Market Pittsburgh, Pennsylvania To proceed with your application, you must be at least 18 years of age. Acknowledge (... ...and services Familiarity with CRM platforms and other banking systems Desired Qualifications...PlatformTemporary workWork experience placementWork at officeShift workDay shift
- ...Assurance Lead in Pittsburgh, PA, for our direct client... ...apps and Power Platform.... ...tools. The manager is looking for at... ...of experience. Must Have Technical Skills... ...Teradata • Azure Data Factory • Azure... ...complying with all applicable laws, including taxation...PlatformContract workRemote workFlexible hours
- ...role covering the Pittsburgh, PA territory. Travel... ...selected candidate resides, and overnight travel... ...will develop and manage relationships with... ...analyze complex data and leverage insights... ...digital tools and platforms to engage with... ...and business need, must reside within 50 miles...PlatformWork experience placementLocal areaFlexible hoursNight shift
- ...Sciences at the University of Pittsburgh Medical Center/Magee Women's Hospital... .... Qualified candidates must possess an MD or equivalent with... ...to obtain an unrestricted PA license. H1B visa sponsorship... ...clinical programs serve as a platform for research and education, and...PlatformTemporary workH1bVisa sponsorshipRelocation packageWeekend work
- ...disabled individuals. Must possess good verbal and... ...program in Coraopolis, PA. Earn up to 19.00/... ..., and documents residents' attendance and progress... ...On the Goga well-being platform, featuring self-care tools... ...we serve and encourage applicants from underrepresented backgrounds...PlatformFull timeWork experience placementMonday to FridayFlexible hoursShift workDay shift
$79.6k - $156.7k
...Analyst - Local to Pittsburgh, PA Category... ...Document Management team, supporting... ...concerns. . Analyze data to support business... .... Candidate must have an agile mindset... ...management / ECM platforms such as: OpenText... ...world. Qualified applicants will receive...PlatformPermanent employmentFull timeLocal area$23.89 - $42.69 per hour
...Performance Consultant in Pittsburgh, PA and Surrounding... ...practices that manage a high volume of... ...pm EST If you reside withinPittsburgh... ...Execute applicable provider incentive... ...Activities include data collection, data... ...multiple system platforms to identify open...PlatformHourly payMinimum wageFull timeWork experience placementWork at officeLocal areaRemote workMonday to Friday- ...positive energy. They must combine initiative... ...Title: Account Manager, Immunology Location: Pittsburgh, PA Job... ...contracting and access platforms, and proven ability... ...accordance with all applicable company and regulatory... ...complex sales data/call reporting software...PlatformContract workWork experience placementFlexible hoursShift work
- THREE Application Developer Leads (BPM, Oracle, SOA, ESB, ETL, BI, Analytics) in Pittsburgh, PA Application Design, BPM, Business Intelligence... ...Transmissions, Vendor Management DESCRIPTION: Number of Positions... ...cross-functional, multi-platform application systems. -...PlatformPermanent employmentFull timeWork experience placementRemote work
- Sr Application Architect (Microservices Architecture, Containers, NOSQL, Big Data, Java, Spring frameworks, REST API, WebServices) in Pittsburgh, PA Big Data, Containers, Java, Microservices... ..., multi-platform, and Web-based... ...and effectively manages all of the risks...PlatformPermanent employmentFull timeRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Platform Manager- Hybrid (Applicants must reside in Pittsburgh, PA). Be the first to apply!
Related searches
- clinical data manager remote Pittsburgh, PA
- senior manager data analytics Pittsburgh, PA
- director data architecture Pittsburgh, PA
- senior data manager Pittsburgh, PA
- director data management Pittsburgh, PA
- senior manager data science Pittsburgh, PA
- data manager Pittsburgh, PA
- data integration manager Pittsburgh, PA
- director data analytics Pittsburgh, PA
- director data center Pittsburgh, PA

