Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Big Data Lead

Smart IT Frame LLC

Role: Big Data Lead

Location: Raleigh, NC (Hybrid)

Contract

Role Summary:

  • Senior Databricks Architect or Lead engineer with 10+ years in data engineering, platform engineering, including at least 3 years designing and running Azure Databricks platforms at enterprise scale for multiple business units
  • Understanding FinOps on Databricks in depth: what drives DBU consumption (all-purpose vs. job compute, SQL warehouses, serverless, Photon, DLT), how DBU pricing tiers work, and how DBU charges relate to the underlying Azure VM, storage and networking costs.
  • Designed and enforced a tagging and governance strategy using cluster and workspace tags, cluster policies, budget policies and Unity Catalog, so every unit of spend can be traced to a business unit, cost center, project or pipeline.
  • Hands-on experience building cost data pipelines from Databricks System Tables and Azure Cost Management exports, using PySpark and SQL to model clean, reconciled data for chargeback.
  • Able to deliver chargeback dashboards in Databricks AI/BI (Lakeview) or Power BI, define how shared and untagged costs get allocated, and work with finance and business-unit leaders on cost optimization and spend accountability

About Smart IT Frame:

At Smart IT Frame, we connect top talent with leading organizations across the USA. With over a decade of staffing excellence, we specialize in IT, healthcare, and professional roles, empowering both clients and candidates to grow together

Apply today or share profiles at View email address on click.appcast.io

Vacancy posted 2 hours ago
Similar jobs that could be interesting for youBased on the Big Data Lead in Chapel Hill, NC vacancy
  • A prominent software development firm is seeking an experienced professional for a full-time position requiring 7+ years in data architecture and engineering. Ideal candidates will have expertise in HDFS, Hive, and programming languages such as Java, Python, and Perl.... 
    Big data
    Full time

    Sonsoft Inc

    Durham, NC
    16 hours ago
  •  ...the Shutterfly family) is seeking a Merchandising Specialist II to lead preseason strategy for the Homeowner business, focusing on Decor...  ...to optimize product placement and customer engagement. Join a data-driven team that collaborates across merchandising, growth, ecom,... 
    Suggested
    Seasonal work

    Shutterfly

    Durham, NC
    2 days ago
  • $86.8k - $217.1k

    IQVIA is growing! Hiring multiple Senior Data Team Leads across our FSP (Functional Service Provider) and Traditional (Sponsor-Agnostic) delivery models.Apply now and have your resume considered for Senior Data Team Lead opportunities across multiple teams - allowing us... 
    Suggested
    Full time
    Part time
    Immediate start
    Remote work
    Work from home
    Worldwide

    IQVIA

    Durham, NC
    16 hours ago
  •  ...Job Title: Senior Enterprise MDM & Data Governance Transformation Lead Engagement: Contract Location: Durham / Research Triangle Park (RTP), North Carolina (Preferred) Other Locations: Open to candidates across the United States Work Arrangement: US-based;... 
    Suggested
    Contract work
    Work at office

    Global Applications Solution

    Durham, NC
    3 days ago
  • Manager, Advanced Data Analytics Note: Fidelity will not provide immigration sponsorship...  ...collaborative team member, helping develop and lead data and hypothesis driven initiatives....  ...a plus. Experience using and querying big data technologies such as Snowflake, Athena... 
    Big data
    Full time

    Fidelity Investments

    Durham, NC
    3 days ago
  •  ...- Proficiency in Fortran90 - Proficiency in R/Python and EPA guidance on model evaluation procedures - Experience in working with big data and parallel computing in the cloud environment with major vendors - Experience in developing and offering training to user community... 
    Big data
    Full time
    Work at office
    Remote work

    UNC-Chapel Hill

    Chapel Hill, NC
    6 days ago
  •  ...in developing Sqoop, Spark.Experience in processing structured data - Warehousing Concepts like de-duplication, cleansing, look ups,...  ...and desire to work in a Global delivery environment.Experience leading medium to large sized teams.CloudEra CertificationKnowledge of PLSQL... 
    Big data
    Permanent employment
    Full time
    H1b

    Sonsoft

    Durham, NC
    4 days ago
  • Data Analyst Manager The role involves facilitating requirement sessions with business partners, articulating business requirements into...  ...with relational databases (Oracle preferred) and familiar with Big Data NoSQL databases/technologies (Hive, Spark, MongoDB) Expert knowledge... 
    Big data
    Flexible hours

    Samprasoft

    Durham, NC
    3 days ago
  •  ...Senior Data Analyst Senior Data Analysts to work remotely or in the Durham NC office. The role includes working with a tech lead to analyze an existing database to determine how and what parts...  ...procedures in Oracle Comfortable with Big Data Workplace Investing (WI)... 
    Big data
    Work at office
    Remote work

    Software Technology Inc

    Durham, NC
    4 days ago
  •  ...functions of the job Primarily expertise in an emerging technology where there may be fewer available resources such as Hadoop, Actimize, Big Data, Dev Ops, Brio/BenXL/Blaze/Exadata/Hadoop/Hibernate/Netezza/RECON/RTC /SDL Tridion/SPS/SunGard/Sybase .Communication and... 
    Big data
    Permanent employment
    Full time
    H1b

    Sonsoft

    Durham, NC
    16 hours ago
  •  ...Design Networking Cloud Services Data Center Warehousing Storage Services Transportation Specialties Big Data Analytics NoSQL Streaming data...  ...model, shaping the value proposition, and leading the organization to deliver sustainable, profitable... 
    Big data
    Flexible hours

    Confidential

    Durham, NC
    3 days ago
  •  ...Different file formats like ORC, AVRO, and Parquet etc.Hand on experience in developing Sqoop, Spark.Experience in processing structured data - Warehousing Concepts like de-duplication, cleansing, look ups, transformation, data versioning etc.Hand on experience in... 
    Big data
    Permanent employment
    Full time
    H1b

    Sonsoft

    Durham, NC
    16 hours ago
  •  ...Different file formats like ORC, AVRO, and Parquet etc.Hand on experience in developing Sqoop, Spark.Experience in processing structured data - Warehousing Concepts like de-duplication, cleansing, look ups, transformation, data versioning etc.Hand on experience in... 
    Big data
    Permanent employment
    Full time
    H1b

    Sonsoft

    Durham, NC
    16 hours ago
  •  ...managing the acquisition, integrity, and quality of sales-related data while ensuring accurate reporting for both the sales organization...  ...industry or other manufacturing environmentExposure to “big data” and master data conceptsStrong project management & communication... 
    Big data
    Full time

    WolfSpeed

    Durham, NC
    2 days ago
  •  ...Advanced knowledge & experience with SQL and good understanding of Data warehousing and Relational database concepts & technologies...  ...analytics requirements Ability to analyze and synthesize a wide range of big data into valuable insights Data Analysis using SQL on data... 
    Big data
    Work experience placement
    Flexible hours

    Samprasoft

    Durham, NC
    3 days ago
  •  ...Position Description : Oversees the building of data pipelines and the creation and evaluation...  ...processing frameworks (Apache Spark). Leads the design and oversight of cross-...  ...performing data modeling and troubleshooting of big data applications, predictive analytics, and... 
    Big data
    Full time

    Fidelity Investments

    Durham, NC
    16 hours ago
  •  ...Master’s Degree in a technology related field (e.g. Engineering, Computer Science, etc.) 8+ years of proven experience in implementing Big data solutions in data analytics space 2+ years of experience in developing ML infrastructure and MLOps in the Cloud using AWS... 
    Big data
    Local area

    Samprasoft

    Durham, NC
    3 days ago
  • $152k - $241.5k

     ...needs and use cases.Analyzing performance bottlenecks.Supporting and maintaining large scale data platform with high QoS.Building infrastructure and microservices for various big-data scenarios.What we need to see:5+ years of experience.Bachelors or Masters in Computer... 
    Big data
    Full time

    Nvidia

    Durham, NC
    1 day ago
  • Software Engineer IV - Big Data Engineer Location: Durham, NC (Day 1 onsite) Duration: Long Term Responsibilities: Ability to assemble large, complex sets of data that meet non-functional and functional business requirements Ability to identify, design and implement internal... 
    Big data

    Samprasoft

    Durham, NC
    3 days ago
  • Lightship is a clinical trial site organization delivering on-site, hybrid, and virtual trials through a single, trusted partner. Our expanded reach model places each visit in the most practical setting, increasing access, strengthening enrollment and engagement, and improving...

    Lightship

    Durham, NC
    16 hours ago
  •  ...decisions and improve outcomes. Through our unparalleled science, data, technology and laboratory network, we advance diagnostics,...  ...deployment, and support activities.Evaluate emerging technologies and lead proof-of-concept initiatives to improve platform capabilities and... 
    Full time
    Temporary work
    Casual work
    Internship
    Monday to Friday
    Flexible hours
    Day shift

    Laboratory Corporation of America

    Durham, NC
    3 days ago
  •  ...Anticipates and predicts client needs and future behaviors using data, analytics, customer, research and AI solutions. Processes large...  ...to transform data into impactful insights. Leverages industry‑leading analytics approaches, database analytics, digital analytics, data... 
    Full time

    Fidelity Investments

    Durham, NC
    3 days ago
  •  ...Division of Finance and Operations at UNC‑Chapel Hill encompasses a broad range of departments (Facilities Services, Finance, Budget & Data Analysis, Real Estate & Campus Enterprises, Human Resources, and Organizational Strategy and Performance) that serve and support the... 
    Full time
    Live in
    Work at office
    Local area
    Remote work

    Inside Higher Ed

    Chapel Hill, NC
    16 hours ago
  • $79k - $157k

    Senior Clinical Data Manager page is loaded## Senior Clinical Data Managerremote type: Remotelocations: Durham, North Carolinatime type...  ...across multiple complex, global projects and programs, leading activities from study start-up through conduct and closeout. You... 
    Hourly pay
    Contract work
    Remote work
    Worldwide

    WORLDWIDE, INC.

    Durham, NC
    3 days ago
  • Student Information System Data Manager - High SchoolPerforms a variety of computer systems support functions on the Student Information System in a high school within Durham Public Schools.Audit data necessary to create principal monthly reports. Monitor and update daily... 
    Work at office

    Durham Public Schools

    Durham, NC
    3 days ago
  •  ...Senior Clinical Data Manager Lightship is a clinical trial site organization delivering on-site, hybrid, and virtual trials through...  .... The Senior Clinical Data Manager is responsible for leading all data management activities across one or more clinical trials... 
    Remote work

    Lightship

    Durham, NC
    22 hours ago
  • $61.03k - $101.96k

     ...Clinical Data Manager Work Arrangement: On-Site Location: Durham, NC, US, 27710 Personnel Area: MEDICAL CENTER School of Medicine Established in 1930, Duke University School of Medicine is the youngest of the nation's top medical schools. Ranked sixth among medical... 
    Contract work
    Apprenticeship
    Work experience placement
    Worldwide

    Duke University

    Durham, NC
    16 hours ago
  • At BioAgilytix, we are passionate about premier science and the impact it has on our world. Our team of highly experienced scientists and professionals deliver tailored services for supporting new medicine breakthroughs with best-in-class bioanalytical services. We are...
    Full time

    bioagilytix

    Durham, NC
    9 days ago
  •  ...Description Job Description The advent of automation requires robust data sources and ever-expanding technologies like IoT, machine...  ...today than ever before. As a data engineer, you know that organizing big data can yield pivotal insights when it's gathered from disparate... 
    Big data

    Expert In Recruitment Solutions

    Durham, NC
    26 days ago
  • About The Role The Senior Data Analyst, Clinical Programs will be an analytically driven team player, responsible for understanding business needs and objectives, eliciting requirements from stakeholders, and analyzing data and processes to identify opportunities for... 
    Local area
    Remote work
    Flexible hours

    Brighton Health Plan Solutions

    Chapel Hill, NC
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Big Data Lead. Be the first to apply!