This repository holds SoA extraction, resolution, and consolidation outputs produced with the soa2usdm pipeline, organised as collections of studies. Full protocol PDFs are not stored here: each protocol links to its source, and only its SoA pages are published (an excerpt PDF and the rendered page images).
An introduction to SoA2USDM, with two short videos and the schemas on one page, is at kerfors.github.io/soa2usdm.
Collections
usdm_data
Mirrors the protocol set published in data4knowledge/usdm_data. Protocol folders are named by NCT ID; the d4k folder name is carried as manifest metadata and provenance.
22 protocols · 22 processed
misc_studies
Assorted protocols outside the data4knowledge/usdm_data mirror. Protocol folders are named by NCT ID; source documents are the sponsor-posted clinical study protocols on ClinicalTrials.gov.
2 protocols · 2 processed
Each collection index is generated by the pipeline (from
studies_protocols.xlsx plus discovered outputs) and is the single source of truth for the study
list, per-layer pipeline status, and links to each protocol's source, extraction data, consolidated SoA,
review page and extraction log.
Pipeline layers
- Layer 1 — Extraction: SoA table(s) transcribed from the protocol PDF to
soa-table-extractionJSON (single-pass), ending with an extraction log of every judgement call. - Corrections: human review findings applied as an auditable sidecar
(
*_corrections.json) — the raw extraction is never overwritten. - Layer 2 — Resolve: IDs, hierarchy, and relationships derived per table.
- Layer 3 — Consolidate: cross-table integration, activity matching, annotation dedup.
- Review page: the extraction drawn on its source pages, with the open decisions as a worklist.
- Consolidated view: the unified SoA across tables, with each annotation marked where it applies.
USDM Instantiation — turning one protocol's consolidated SoA into a USDM v4 document — is a documented manual step, not a pipeline layer; see the soa2usdm repository.