Senior Data Engineer
NTT
The Senior Data Engineer role at NTT Ltd focuses on designing and optimizing scalable data pipelines with Databricks and related technologies to support AI-driven document processing. The position emphasizes structured extraction from unstructured and semi-structured documents, ontology-based modeling, and delivery of standardized JSON outputs for downstream underwriting workflows.
Key Responsibilities
- Design, develop, and optimize scalable data pipelines using Databricks , Python , PySpark , and SQL .
- Build ingestion, extraction, classification, validation, and transformation pipelines for documents including PDFs , emails , attachments , and Excel/XLSX files, as well as insurance applications and Statements of Values (SOVs) .
- Integrate LLMs and AI models into data-processing workflows to extract structured information from unstructured and semi-structured content.
- Troubleshoot false positives, incorrect metadata, and inaccurate field values produced by LLM-based extraction pipelines.
- Implement document classification, source-specific extraction rules, data validation, reconciliation, and quality-control mechanisms.
- Transform extracted information into standardized JSON structures for downstream applications and underwriting workflows.
- Design and implement ontology-based data models representing insurance entities, business concepts, and relationships.
- Map extracted document attributes to standardized ontology concepts and metadata structures.
- Optimize processing for large and complex Excel/XLSX and SOV files, including performance, scalability, and concurrency challenges.
- Implement reliable data persistence using PostgreSQL , Lakebase , or related database technologies.
- Develop and maintain APIs and data products to support downstream application integration.
- Collaborate with brokerage, underwriting, AI, and data engineering teams to align extraction outputs with business requirements.
- Monitor production pipelines, troubleshoot data-processing issues, and continuously improve reliability, performance, and data quality.
Required Qualifications
- Minimum 7+ years of experience in data engineering, software development, or production data pipeline development.
- Minimum 4+ years of hands‑on experience with Databricks and scalable data-processing pipelines.
- Minimum 5+ years of experience with Python , PySpark , and SQL .
- Minimum 3+ years of experience processing unstructured and semi-structured data, including PDFs , emails , Excel files , JSON , and business documents.
- Minimum 2+ years of experience integrating LLMs or AI models into document-processing or data-extraction pipelines.
- Minimum 3+ years of experience implementing structured data extraction, document classification, validation, reconciliation, and data-quality controls.
- Strong understanding of data modeling, schemas, metadata, entity relationships, and ontology or semantic modeling concepts.
- Minimum 2+ years of experience designing JSON outputs , APIs , and data products for downstream consumption.
- Minimum 2+ years of experience with PostgreSQL , Lakebase , or similar relational database technologies.
- Minimum 2+ years of experience troubleshooting production performance, scalability, concurrency, and data-processing issues.
- Strong understanding of cloud-based data architectures, distributed processing, and pipeline optimization.
- Strong analytical, problem‑solving, communication, and cross‑functional collaboration skills.
Technologies
Databricks , Python , PySpark , SQL , LLMs , AI models , PostgreSQL , Lakebase , JSON , APIs , Excel/XLSX , PDFs , emails
Education
Bachelor’s degree in Computer Science, Information Technology, Data Engineering, Engineering, or a related field, or equivalent.
Location
Charlotte, North Carolina, United States (onsite)
Travel
NA
Nice to Have
- Experience in insurance, commercial insurance, brokerage, underwriting, or risk management.
- Experience processing insurance applications, Statements of Values (SOVs), policy documents, and underwriting submissions.
- Experience with LLM frameworks, prompt engineering, structured outputs, and AI evaluation techniques.
- Experience building custom ontologies, knowledge graphs, or semantic data layers.
- Experience with document AI, OCR, information extraction, and document intelligence platforms.
- Experience with Delta Lake , Lakehouse architecture, Databricks Workflows , and Delta Live Tables .
- Experience with AWS , Azure , or Google Cloud .
- Experience optimizing large-scale document ingestion and distributed processing workloads.
- ...Senior Data Engineer - AWS Charlotte, NC - Onsite 6 Months+ Required Skills & Qualifications • Data engineering, backend development, or software engineering, with demonstrated hands-on development experience. • Strong hands-on experience with AWS data and cloud...Senior
- ...Senior Data Engineer – PySpark & Databricks Location: Charlotte, North Carolina (Hybrid)// Need only locals Experience: 10+ Years Job Type: Contract Job Summary We are seeking an experienced Senior Data Engineer with strong hands-on expertise in PySpark...SeniorContract workLocal area
$60 - $65 per hour
...DescriptionKforce has a client that is seeking a Senior AWS Data Engineer in Charlotte, NC.Summary:We are looking for a hands-on Senior AWS Data Engineer with strong experience in Java, Python, PySpark, SQL, AWS Data Engineering, API development, AWS Bedrock, and Model...Senior- ...Senior Data Engineer Work Location: Charlotte, NC 28203 (hybrid – 2 days onsite per week)(Locals) Contract duration: 12 months Target Start Date: 06 Dec 2025 Visa: USC and GC only Must Have Skills: Spark, Python, Big Data, AWS We are currently seeking a Senior...SeniorContract workLocal area2 days per week
- ...Job Title: Senior Data Engineer Location: West Des Moines, IA OR Charlotte, NC - hybrid Wells Fargo - W2 ONLY!!!!! Rate: $50.00/hr on W2 Duration: 12+-month Contract ONLY W2 CANDIDATES ELIGIBLE FOR THIS PROJECT Key...SeniorContract work
- ...Overview As a Senior Data Engineer at AvidXchange, you'll design and build high-performing data solutions that power analytics, reporting, and business insights across the organization. This role provides the opportunity to shape technical direction, lead delivery...SeniorRemote work
- ...Senior Data Engineer Job#: 3044243 Job Description: Senior Data Engineer Location: Charlotte, North Carolina (Onsite) Role Overview We are seeking a senior-level Data Engineer to support large-scale data ingestion and data warehousing initiatives within the Hadoop ecosystem...Senior
- ...career with opportunities to learn, grow, and make an impact. Join us! Job Description: This job is responsible for driving data engineering efforts to deliver enterprise-wide capabilities and complex data solutions. Key responsibilities include directing code design...SeniorWork experience placementWork at officeFlexible hoursShift workDay shift
- ...Skanska is searching for a Senior Data Engineer . This is a great opportunity to start a career with a company that builds things that matter and values its team. We are proud to share our culture of diversity and inclusion. Our work makes a clear contribution to society...SeniorSecond jobLocal areaVisa sponsorship
- ...Dimensional leverages the rapidly evolving state of the art to engineer the platforms that power the innovative, research-... ...products to improve our client’s financial lives. As a Senior Python Engineer within the Data Distribution team you will participate in the...SeniorFull timeWork experience placementRemote work
- ...and Sumitomo Mitsui Finance and Leasing Co., Ltd. Role Description We are currently seeking a talented and motivated Senior Data Engineer to join our Cloud data solutions team. This role will be responsible for designing, developing & deploying cloud data solutions...SeniorWork at officeLocal areaWork from homeWorldwide
- ...Senior Data Engineer NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Senior Data...SeniorWork at officeRemote workFlexible hours
$99k - $164.8k
...provide an experience to our stakeholders that delivers on our promises. Position Summary: We are seeking a highly skilled Senior Data Engineer to guide the design, development, and delivery of our enterprise data platforms. This role requires deep technical expertise...SeniorFull timeTemporary workFor contractorsCasual workWork at officeLocal areaRemote work$90k - $105k
...About the role As a Senior AWS Data Engineer , you will make an impact by designing and building scalable cloud-based data solutions, modern APIs, and Generative AI capabilities that drive business innovation. You will be a valued member of the Data Engineering team...SeniorFull timeTemporary workWork at office2 days per week- ...be remote based in the United States. Reporting This position will report to the VP, Engineering. Summary of Role You will lead the team that makes Palmetto's data usable by product teams, finance, operations, partners, and AI agents. Today much of our data...SeniorLocal areaRemote work
- FCG is seeking a Senior Data Engineer to work on our Central Data Platform team, utilizing Snowflake, dbt, and Fivetran, and responsible for end-to-end ELT development from ingestion of raw source system data through our integrated and data consumption layers in Snowflake...Senior
- ...Job Description Job Description Title: Senior AI Platform Engineer (Data and Analytics Cloud Engineer) Location: Atlanta, Charlotte and/or Raliegh - 1 day onsite Duration: 12 Months Description : Business Initiative/Purpose: (Goal, Business Impact, Accomplishments...Senior
$176k - $179.5k
...patients. You will design and manage scalable data pipelines and secure analytics platforms,... ...data scientists, machine learning engineers, and industry experts to develop impactful... ...McKinsey. There is flexibility to hire at the Senior Data Engineer I/II or Principal Data...SeniorApprenticeshipEasy workShift work$127.31k - $243.34k
...flexibility for active-duty military spouses consistent with applicable policy and business needs.The OpportunityAs a dedicated Data Engineer - Senior, you will be engaged in all phases of the data management lifecycle. This includes gathering and analyzing requirements,...SeniorFull timeH1bRemote workHome office$170k - $190k
...management solutions that advisors use in helping clients achieve wealth, independence, and purpose.The Job/What You'll Do:The Senior Data Engineer / Technical Lead is a pivotal, hands-on leadership role responsible for the end-to-end design, governance, and operational...SeniorFull timeWork at officeFlexible hours- ...About Persistent We are an AI-led, platform-driven Digital Engineering and Enterprise Modernization partner, combining deep... .... About Position: We are seeking a highly skilled Senior GCP Data Engineer with strong expertise in Google Cloud Platform, BigQuery...SeniorContract workWork at officeFlexible hours
- ...Glue, EMR (PySpark), and Lambda functions Implement real-time data streaming solutions using Confluent Kafka with source and sink... ...Qualifications Bachelor's degree in Computer Science, Data Engineering, Information Technology, or related field Strong understanding...SeniorContract workRemote workShift work
- Job Title As an Engineer on the full stack team, you will be dedicated to help design, implement, and maintain a modern, robust, and scalable platform which will enable the Operational Risk team to meet the increasing demands from the various trading desks.Senior
- ...Data Engineering And BI Engineering Position Translates complex cross-functional business requirements and functional specifications into logical program designs, code modules, stable application systems, and data solutions; partners with Product Team to understand...SeniorFlexible hours
$60 - $65 per hour
job summary: We are seeking a highly skilled Senior Data Engineer to join an innovative technology-driven team. This is an exceptional opportunity to enhance your career within the Data and Analytics industry, where you will play a pivotal role in evolving our data...SeniorHourly payContract workTemporary workWork experience placementSecond jobRemote work2 days per week- ...while leading the development and deployment of AI models and data architectures to improve business processes and system performance... ...’s degree in Computer Science, Data Science, AI, Software Engineering, a related field, or equivalent education, training, and work-related...SeniorPermanent employmentFull timePart timeH1bWork at officeWork visaShift workDay shift
- ...Job Description Job Description Job title: Data Center Migration Engineer Location: Charlotte, NC 28202 Hybrid role 12 months contract Experience level: Mid (5 - 7 years) Description: Summary: Provides technical solutions across systems, applications...Contract workWeekend workAfternoon shift
- ...Job Description Job Description Summary: The engineer in this position should have a wealth of experience engineering/installing... ...cohesive package. This would include the complete engineering of data centers including, but not limited to, infrastructure, overhead...For contractorsWork at officeLocal area
- ...Job Description Job Description Job Title: Data Engineer Location: Charlotte, NC, Iselin NJ(Hybrid) Duration: 5 months CW-IT Applications Developer IV Develops, implements and assesses specific new and emerging technologies, platforms, and services. Key...SeniorWork experience placement
$140k - $160k
...Description Job Summary: This job is responsible for driving data engineering efforts to deliver enterprise-wide capabilities and complex... ...and automation solutions to the CACR organization. The Senior Data Engineer role will be responsible for developing and implement...Senior
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer. Be the first to apply!
- junior data engineer remote Charlotte, NC
- data platform engineer Charlotte, NC
- principal data engineer Charlotte, NC
- data engineer analytics Charlotte, NC
- data science developer Charlotte, NC
- data engineer manager Charlotte, NC
- big data developer Charlotte, NC
- big data devops engineer Charlotte, NC
- data center engineer Charlotte, NC
- big data cloud engineer Charlotte, NC






