Zero-Shot Clinical Data Extraction from Pathology Reports Using Llama 3.1

Sunghyeon Park; Wona Choi; In Young Choi

doi:10.3233/SHTI251230

Zero-Shot Clinical Data Extraction from Pathology Reports Using Llama 3.1

Sunghyeon Park
, Wona Choi
, In Young Choi

Department of Medical Informatics

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This study evaluates the performance of the Llama 3.1 model with 70 billion parameters (70b) in extracting clinically significant information from free-text pathology reports under zero-shot learning conditions. The model achieved a remarkable accuracy of 98%, directly generating structured outputs in JSON format without requiring annotated examples. These findings demonstrate the potential of Llama 3.1 in automating data extraction from complex medical documents, streamlining clinical workflows, and enhancing the precision of information retrieval in healthcare settings.

Original language	English
Title of host publication	MEDINFO 2025 - Healthcare Smart x Medicine Deep
Subtitle of host publication	Proceedings of the 20th World Congress on Medical and Health Informatics
Editors	Mowafa S. Househ, Mowafa S. Househ, Zain Ul Abideen Tariq, Mahmood Al-Zubaidi, Uzair Shah, Elaine Huesing
Publisher	IOS Press BV
Pages	1818-1819
Number of pages	2
ISBN (Electronic)	9781643686080
DOIs	https://doi.org/10.3233/SHTI251230
State	Published - 7 Aug 2025
Event	20th World Congress on Medical and Health Informatics, MEDINFO 2025 - Taipei, Taiwan, Province of China Duration: 9 Aug 2025 → 13 Aug 2025

Publication series

Name	Studies in Health Technology and Informatics
Volume	329
ISSN (Print)	0926-9630
ISSN (Electronic)	1879-8365

Conference

Conference	20th World Congress on Medical and Health Informatics, MEDINFO 2025
Country/Territory	Taiwan, Province of China
City	Taipei
Period	9/08/25 → 13/08/25

Bibliographical note

Publisher Copyright:
© 2025 The Authors.

Keywords

Information Extraction
LLMs

Access to Document

10.3233/SHTI251230

Cite this

Park, S., Choi, W., & Choi, I. Y. (2025). Zero-Shot Clinical Data Extraction from Pathology Reports Using Llama 3.1. In M. S. Househ, M. S. Househ, Z. U. A. Tariq, M. Al-Zubaidi, U. Shah, & E. Huesing (Eds.), MEDINFO 2025 - Healthcare Smart x Medicine Deep: Proceedings of the 20th World Congress on Medical and Health Informatics (pp. 1818-1819). (Studies in Health Technology and Informatics; Vol. 329). IOS Press BV. https://doi.org/10.3233/SHTI251230

Park, Sunghyeon ; Choi, Wona ; Choi, In Young. / Zero-Shot Clinical Data Extraction from Pathology Reports Using Llama 3.1. MEDINFO 2025 - Healthcare Smart x Medicine Deep: Proceedings of the 20th World Congress on Medical and Health Informatics. editor / Mowafa S. Househ ; Mowafa S. Househ ; Zain Ul Abideen Tariq ; Mahmood Al-Zubaidi ; Uzair Shah ; Elaine Huesing. IOS Press BV, 2025. pp. 1818-1819 (Studies in Health Technology and Informatics).

@inproceedings{05bc8fbf3ea2490f8572313c672dfdbf,

title = "Zero-Shot Clinical Data Extraction from Pathology Reports Using Llama 3.1",

abstract = "This study evaluates the performance of the Llama 3.1 model with 70 billion parameters (70b) in extracting clinically significant information from free-text pathology reports under zero-shot learning conditions. The model achieved a remarkable accuracy of 98\%, directly generating structured outputs in JSON format without requiring annotated examples. These findings demonstrate the potential of Llama 3.1 in automating data extraction from complex medical documents, streamlining clinical workflows, and enhancing the precision of information retrieval in healthcare settings.",

keywords = "Information Extraction, LLMs",

author = "Sunghyeon Park and Wona Choi and Choi, \{In Young\}",

note = "Publisher Copyright: {\textcopyright} 2025 The Authors.; 20th World Congress on Medical and Health Informatics, MEDINFO 2025 ; Conference date: 09-08-2025 Through 13-08-2025",

year = "2025",

month = aug,

day = "7",

doi = "10.3233/SHTI251230",

language = "English",

series = "Studies in Health Technology and Informatics",

publisher = "IOS Press BV",

pages = "1818--1819",

editor = "Househ, \{Mowafa S.\} and Househ, \{Mowafa S.\} and Tariq, \{Zain Ul Abideen\} and Mahmood Al-Zubaidi and Uzair Shah and Elaine Huesing",

booktitle = "MEDINFO 2025 - Healthcare Smart x Medicine Deep",

address = "Netherlands",

}

Park, S, Choi, W & Choi, IY 2025, Zero-Shot Clinical Data Extraction from Pathology Reports Using Llama 3.1. in MS Househ, MS Househ, ZUA Tariq, M Al-Zubaidi, U Shah & E Huesing (eds), MEDINFO 2025 - Healthcare Smart x Medicine Deep: Proceedings of the 20th World Congress on Medical and Health Informatics. Studies in Health Technology and Informatics, vol. 329, IOS Press BV, pp. 1818-1819, 20th World Congress on Medical and Health Informatics, MEDINFO 2025, Taipei, Taiwan, Province of China, 9/08/25. https://doi.org/10.3233/SHTI251230

Zero-Shot Clinical Data Extraction from Pathology Reports Using Llama 3.1. / Park, Sunghyeon; Choi, Wona; Choi, In Young.
MEDINFO 2025 - Healthcare Smart x Medicine Deep: Proceedings of the 20th World Congress on Medical and Health Informatics. ed. / Mowafa S. Househ; Mowafa S. Househ; Zain Ul Abideen Tariq; Mahmood Al-Zubaidi; Uzair Shah; Elaine Huesing. IOS Press BV, 2025. p. 1818-1819 (Studies in Health Technology and Informatics; Vol. 329).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Zero-Shot Clinical Data Extraction from Pathology Reports Using Llama 3.1

AU - Park, Sunghyeon

AU - Choi, Wona

AU - Choi, In Young

PY - 2025/8/7

Y1 - 2025/8/7

N2 - This study evaluates the performance of the Llama 3.1 model with 70 billion parameters (70b) in extracting clinically significant information from free-text pathology reports under zero-shot learning conditions. The model achieved a remarkable accuracy of 98%, directly generating structured outputs in JSON format without requiring annotated examples. These findings demonstrate the potential of Llama 3.1 in automating data extraction from complex medical documents, streamlining clinical workflows, and enhancing the precision of information retrieval in healthcare settings.

AB - This study evaluates the performance of the Llama 3.1 model with 70 billion parameters (70b) in extracting clinically significant information from free-text pathology reports under zero-shot learning conditions. The model achieved a remarkable accuracy of 98%, directly generating structured outputs in JSON format without requiring annotated examples. These findings demonstrate the potential of Llama 3.1 in automating data extraction from complex medical documents, streamlining clinical workflows, and enhancing the precision of information retrieval in healthcare settings.

KW - Information Extraction

KW - LLMs

UR - https://www.scopus.com/pages/publications/105013324224

U2 - 10.3233/SHTI251230

DO - 10.3233/SHTI251230

M3 - Conference contribution

C2 - 40776247

AN - SCOPUS:105013324224

T3 - Studies in Health Technology and Informatics

SP - 1818

EP - 1819

BT - MEDINFO 2025 - Healthcare Smart x Medicine Deep

A2 - Househ, Mowafa S.

A2 - Tariq, Zain Ul Abideen

A2 - Al-Zubaidi, Mahmood

A2 - Shah, Uzair

A2 - Huesing, Elaine

PB - IOS Press BV

T2 - 20th World Congress on Medical and Health Informatics, MEDINFO 2025

Y2 - 9 August 2025 through 13 August 2025

ER -

Park S, Choi W, Choi IY. Zero-Shot Clinical Data Extraction from Pathology Reports Using Llama 3.1. In Househ MS, Househ MS, Tariq ZUA, Al-Zubaidi M, Shah U, Huesing E, editors, MEDINFO 2025 - Healthcare Smart x Medicine Deep: Proceedings of the 20th World Congress on Medical and Health Informatics. IOS Press BV. 2025. p. 1818-1819. (Studies in Health Technology and Informatics). doi: 10.3233/SHTI251230

Zero-Shot Clinical Data Extraction from Pathology Reports Using Llama 3.1

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this