Developer PortalChangelogPlatform StatusMore InformationJoin Discord

Extract Medical Entities from Clinical Content

User Intent

"How do I extract medical entities (conditions, drugs, procedures, tests) from clinical documents and research papers? Show me how to build medical knowledge graphs for healthcare applications."

Operation

SDK Methods: createWorkflow(), ingestUri(), isContentDone(), getContent(), queryObservables() GraphQL: Medical content ingestion + extraction of 12 medical entity types Entity: Medical Content → Observations → Medical Observables (Clinical Knowledge Graph)

Prerequisites

  • Graphlit project with API credentials

  • Medical/clinical documents (PDFs, research papers, clinical notes)

  • Understanding of medical entity types

  • Appropriate data privacy/HIPAA compliance measures

Complete Code Example (TypeScript)

import { Graphlit } from 'graphlit-client';
import { ModelServiceTypes, ObservableTypes, SpecificationTypes } from 'graphlit-client/dist/generated/graphql-types';
import {
  FilePreparationServiceTypes,
  ExtractionServiceTypes,
  ObservableTypes,
  ModelServiceTypes,
  OpenAIModels
} from 'graphlit-client/dist/generated/graphql-types';

const graphlit = new Graphlit();

console.log('=== Building Medical Knowledge Graph ===\n');

// Step 1: Create high-quality medical extraction workflow
console.log('Step 1: Creating medical entity extraction workflow...');

// Use GPT-4 for medical accuracy
const spec = await graphlit.createSpecification({
  name: "GPT-4 Medical Extraction",
  type: SpecificationTypes.Completion,
  serviceType: ModelServiceTypes.OpenAi,
  openAI: {
    model: OpenAIModels.Gpt4,    // Best quality for medical
    temperature: 0.1              // Low temperature for consistency
  }
});

const workflow = await graphlit.createWorkflow({
  name: "Medical Entity Extraction",
  preparation: {
    jobs: [{
      connector: {
        type: FilePreparationServiceTypes.ModelDocument  // PDFs, Word, etc.
      }
    }]
  },
  extraction: {
    jobs: [{
      connector: {
        type: EntityExtractionServiceTypes.ModelText,
        extractedTypes: [
          // All 12 medical entity types
          ObservableTypes.MedicalCondition,         // Diseases, symptoms, diagnoses
          ObservableTypes.MedicalDrug,              // Medications, pharmaceuticals
          ObservableMedicalDrugClass,         // Drug categories (antibiotics, etc.)
          ObservableTypes.MedicalProcedure,         // Surgeries, treatments
          ObservableTypes.MedicalTest,              // Lab tests, diagnostics
          ObservableTypes.MedicalStudy,             // Clinical trials, research
          ObservableMedicalDevice,            // Medical equipment, implants
          ObservableMedicalTherapy,           // Therapies, treatments
          ObservableMedicalGuideline,         // Clinical guidelines, protocols
          ObservableMedicalIndication,        // Reasons for treatment
          ObservableMedicalContraindication,  // Reasons to avoid treatment
          
          // Also extract non-medical entities for context
          ObservableTypes.Person,                   // Patients, doctors, researchers
          ObservableTypes.Organization              // Hospitals, pharma companies
        ]
      }
    }]
  },
  specification: { id: spec.createSpecification.id }
});

console.log(`✓ Workflow: ${workflow.createWorkflow.id}\n`);

// Step 2: Ingest clinical research paper
console.log('Step 2: Ingesting clinical research paper...');
const paper = await graphlit.ingestUri('https://example.com/papers/clinical-trial.pdf', "Clinical Trial: Drug X for Condition Y", undefined, undefined, undefined, { id: workflow.createWorkflow.id  });

console.log(`✓ Ingested: ${paper.ingestUri.id}\n`);

// Step 3: Wait for extraction
console.log('Step 3: Extracting medical entities...');
let isDone = false;
while (!isDone) {
  const status = await graphlit.isContentDone(paper.ingestUri.id);
  isDone = status.isContentDone.result;
  
  if (!isDone) {
    console.log('  Processing...');
    await new Promise(resolve => setTimeout(resolve, 3000));
  }
}
console.log('✓ Extraction complete\n');

// Step 4: Retrieve extracted entities
console.log('Step 4: Retrieving medical entities...');
const paperDetails = await graphlit.getContent(paper.ingestUri.id);
const content = paperDetails.content;

console.log(`✓ Document: ${content.name}`);
console.log(`  Pages: ${content.document?.pageCount}`);
console.log(`  Total entities: ${content.observations?.length || 0}\n`);

// Step 5: Analyze by medical entity type
console.log('Step 5: Analyzing medical entities...\n');

const medicalTypes = [
  ObservableTypes.MedicalCondition,
  ObservableTypes.MedicalDrug,
  ObservableMedicalDrugClass,
  ObservableTypes.MedicalProcedure,
  ObservableTypes.MedicalTest,
  ObservableTypes.MedicalStudy,
  ObservableMedicalDevice,
  ObservableMedicalTherapy
];

medicalforEach(type => {
  const entities = content.observations?.filter(obs => obs.type === type) || [];
  const unique = new Set(entities.map(e => e.observable.name));
  
  if (unique.size > 0) {
    console.log(`${type} (${unique.size}):`);
    Array.from(unique).slice(0, 5).forEach(name => {
      console.log(`  - ${name}`);
    });
    if (unique.size > 5) {
      console.log(`  ... and ${unique.size - 5} more`);
    }
    console.log();
  }
});

// Step 6: Build drug-condition relationships
console.log('Step 6: Analyzing drug-condition relationships...\n');

const drugs = content.observations?.filter(obs => 
  obs.type === ObservableTypes.MedicalDrug
) || [];

const conditions = content.observations?.filter(obs =>
  obs.type === ObservableTypes.MedicalCondition
) || [];

// Co-occurrence analysis
const relationships: Array<{ drug: string; condition: string; confidence: number }> = [];

drugs.forEach(drug => {
  conditions.forEach(condition => {
    // Check if they appear on same pages
    const drugPages = new Set(drug.occurrences?.map(occ => occ.pageIndex));
    const condPages = new Set(condition.occurrences?.map(occ => occ.pageIndex));
    
    const sharedPages = Array.from(drugPages).filter(p => condPages.has(p));
    
    if (sharedPages.length > 0) {
      // Calculate average confidence
      const avgConf = (
        (drug.occurrences?.reduce((sum, occ) => sum + occ.confidence, 0) || 0) /
        (drug.occurrences?.length || 1) +
        (condition.occurrences?.reduce((sum, occ) => sum + occ.confidence, 0) || 0) /
        (condition.occurrences?.length || 1)
      ) / 2;
      
      relationships.push({
        drug: drug.observable.name,
        condition: condition.observable.name,
        confidence: avgConf
      });
    }
  });
});

console.log('Drug-Condition relationships:');
relationships
  .sort((a, b) => b.confidence - a.confidence)
  .slice(0, 5)
  .forEach(({ drug, condition, confidence }) => {
    console.log(`  ${drug} ↔ ${condition} (confidence: ${confidence.toFixed(2)})`);
  });

// Step 7: Query medical knowledge graph
console.log('\nStep 7: Querying medical knowledge graph...\n');

// Get all conditions across all documents
const allConditions = await graphlit.queryObservables({
  filter: { types: [ObservableTypes.MedicalCondition] }
});

console.log(`Total conditions in knowledge graph: ${allConditions.observables.results.length}`);

// Get all drugs
const allDrugs = await graphlit.queryObservables({
  filter: { types: [ObservableTypes.MedicalDrug] }
});

console.log(`Total drugs in knowledge graph: ${allDrugs.observables.results.length}`);

console.log('\n✓ Medical knowledge graph complete!');

Step-by-Step Explanation

Step 1: Understanding Medical Entity Types

Graphlit supports 12 medical entity types (all fully supported, not beta):

Core Clinical Entities:

  1. MedicalCondition:

    • Diseases, symptoms, diagnoses

    • Examples: "Type 2 diabetes", "hypertension", "chest pain", "COVID-19"

    • Schema.org: @type: "MedicalCondition"

  2. MedicalDrug:

    • Specific medications, pharmaceuticals

    • Examples: "metformin", "lisinopril", "aspirin", "Pfizer-BioNTech vaccine"

    • Schema.org: @type: "Drug"

  3. MedicalDrugClass:

    • Categories of drugs

    • Examples: "antibiotics", "beta-blockers", "statins", "ACE inhibitors"

    • Schema.org: @type: "DrugClass"

  4. MedicalProcedure:

    • Surgeries, treatments, interventions

    • Examples: "coronary artery bypass", "hip replacement", "chemotherapy"

    • Schema.org: @type: "MedicalProcedure"

  5. MedicalTest:

    • Diagnostic tests, lab tests

    • Examples: "HbA1c test", "MRI scan", "blood pressure measurement"

    • Schema.org: @type: "MedicalTest"

Advanced Medical Entities:

  1. MedicalStudy:

    • Clinical trials, research studies

    • Examples: "Phase III trial", "randomized controlled trial", "cohort study"

    • Schema.org: @type: "MedicalStudy"

  2. MedicalDevice:

    • Medical equipment, implants

    • Examples: "pacemaker", "insulin pump", "surgical robot", "stent"

    • Schema.org: @type: "MedicalDevice"

  3. MedicalTherapy:

    • Therapies, treatment approaches

    • Examples: "physical therapy", "radiation therapy", "cognitive behavioral therapy"

    • Schema.org: @type: "MedicalTherapy"

  4. MedicalGuideline:

    • Clinical guidelines, protocols

    • Examples: "WHO guidelines", "treatment protocol", "diagnostic criteria"

    • Schema.org: @type: "MedicalGuideline"

  5. MedicalIndication:

    • Reasons for treatment

    • Examples: "indicated for hypertension", "approved for diabetes management"

    • Schema.org: @type: "MedicalIndication"

  6. MedicalContraindication:

    • Reasons to avoid treatment

    • Examples: "contraindicated in pregnancy", "not for use with kidney disease"

    • Schema.org: @type: "MedicalContraindication"

  7. MedicalRiskFactor (if supported):

    • Risk factors for conditions

    • Examples: "smoking", "obesity", "family history"

Step 2: Model Selection for Medical Content

GPT-4 (Recommended for Medical):

  • Highest accuracy for medical terminology

  • Best understanding of clinical context

  • Lower false positive rate

  • More expensive but worth it for healthcare

GPT-4o:

  • Good balance for less critical medical content

  • Faster processing

  • Lower cost

  • Acceptable for research papers, general medical content

Claude 3.5 Sonnet:

  • Good alternative to GPT-4

  • Strong medical knowledge

  • Handles long clinical documents well

NOT Recommended:

  • Gemini: Less accurate for medical terminology

  • GPT-3.5: Too many medical errors

Step 3: Clinical Document Types

Research Papers:

// PubMed, ArXiv medical papers
extractedTypes: [
  ObservableTypes.MedicalCondition,
  ObservableTypes.MedicalDrug,
  ObservableTypes.MedicalStudy,
  ObservableTypes.MedicalProcedure,
  ObservableTypes.Person,  // Authors, researchers
  ObservableTypes.Organization  // Institutions
]

Clinical Notes (HIPAA considerations):

// Patient records, clinical summaries
extractedTypes: [
  ObservableTypes.MedicalCondition,  // Diagnoses
  ObservableTypes.MedicalDrug,       // Medications
  ObservableTypes.MedicalProcedure,  // Treatments
  ObservableTypes.MedicalTest        // Lab results
  // NOTE: Do NOT extract Person for patient privacy
]

Drug Information Sheets:

// Prescribing information, package inserts
extractedTypes: [
  ObservableTypes.MedicalDrug,
  ObservableMedicalDrugClass,
  ObservableMedicalIndication,
  ObservableMedicalContraindication,
  ObservableTypes.MedicalCondition  // What it treats
]

Clinical Guidelines:

// Treatment protocols, best practices
extractedTypes: [
  ObservableMedicalGuideline,
  ObservableTypes.MedicalProcedure,
  ObservableTypes.MedicalTest,
  ObservableTypes.MedicalCondition
]

Step 4: Medical Entity Relationships

Drug-Condition Relationships:

  • Co-occurrence on same pages

  • "Drug X is indicated for Condition Y"

  • "Patients with Condition Y treated with Drug X"

Procedure-Condition Relationships:

  • "Procedure X performed for Condition Y"

  • Diagnostic procedures for conditions

Drug-Drug Interactions:

  • Contraindications between drugs

  • Combination therapies

Test-Condition Relationships:

  • Diagnostic tests for conditions

  • Monitoring tests for treated conditions

Step 5: Confidence Scoring for Medical Entities

High Confidence (>=0.9):

  • Explicit medical terminology

  • Standard nomenclature (ICD, SNOMED CT terms)

  • Clear clinical context

Medium Confidence (0.7-0.9):

  • Common medical terms

  • Some ambiguity in context

  • Abbreviations with context

Low Confidence (<0.7):

  • Ambiguous terms

  • Incomplete information

  • Uncertain context

Recommended Threshold: >=0.75 for medical applications (higher than general content)

Configuration Options

Precision vs Recall Tradeoff

High Precision (fewer false positives):

// Use GPT-4, high confidence threshold
specification: {
  model: OpenAIModels.Gpt4,
  temperature: 0.05  // Very low temperature
}

// Filter results
const highConfidence = observations.filter(obs =>
  obs.occurrences?.every(occ => occ.confidence >= 0.85)
);

High Recall (fewer false negatives):

// Extract all possible entities, filter later
extractedTypes: [
  // All 12 medical types
  ...allMedicalTypes
]

// Lower confidence threshold
const allEntities = observations.filter(obs =>
  obs.occurrences?.some(occ => occ.confidence >= 0.6)
);

Domain-Specific Extraction

Cardiology:

extractedTypes: [
  ObservableTypes.MedicalCondition,  // Heart diseases
  ObservableTypes.MedicalProcedure,  // Cardiac procedures
  ObservableMedicalDevice,     // Pacemakers, stents
  ObservableTypes.MedicalDrug,       // Cardiac medications
  ObservableTypes.MedicalTest        // ECG, stress tests
]

Oncology:

extractedTypes: [
  ObservableTypes.MedicalCondition,  // Cancer types
  ObservableMedicalTherapy,    // Chemotherapy, radiation
  ObservableTypes.MedicalDrug,       // Cancer drugs
  ObservableTypes.MedicalStudy,      // Clinical trials
  ObservableTypes.MedicalProcedure   // Surgeries, biopsies
]

Pharmacology:

extractedTypes: [
  ObservableTypes.MedicalDrug,
  ObservableMedicalDrugClass,
  ObservableMedicalIndication,
  ObservableMedicalContraindication,
  ObservableTypes.MedicalCondition
]

Variations

Variation 1: Drug Information Database

Build comprehensive drug knowledge base:

// Ingest drug information sheets
const drugDocs = [
  'https://example.com/drugs/metformin-info.pdf',
  'https://example.com/drugs/lisinopril-info.pdf',
  // ... more drugs
];

const drugWorkflow = await graphlit.createWorkflow({
  name: "Drug Information Extraction",
  extraction: {
    jobs: [{
      connector: {
        type: EntityExtractionServiceTypes.ModelText,
        extractedTypes: [
          ObservableTypes.MedicalDrug,
          ObservableMedicalDrugClass,
          ObservableMedicalIndication,
          ObservableMedicalContraindication,
          ObservableTypes.MedicalCondition
        ]
      }
    }]
  }
});

// Ingest all drug docs
await Promise.all(
  drugDocs.map(uri =>
    graphlit.ingestUri({ uri, workflow: { id: drugWorkflow.createWorkflow.id } })
  )
);

// Query drug database
const metformin = await graphlit.queryObservables({
  search: "metformin",
  filter: { types: [ObservableTypes.MedicalDrug] }
});

// Find what conditions it treats
const conditions = await graphlit.queryContents({
  filter: {
    observations: [
      { type: ObservableTypes.MedicalDrug, observable: { id: metformin.observables.results[0].observable.id } },
      { type: ObservableMedicalIndication, observable: { /* any indication */ } }
    ]
  }
});

Variation 2: Clinical Trial Analysis

Analyze clinical trial results:

const trialWorkflow = await graphlit.createWorkflow({
  name: "Clinical Trial Extraction",
  extraction: {
    jobs: [{
      connector: {
        type: EntityExtractionServiceTypes.ModelText,
        extractedTypes: [
          ObservableTypes.MedicalStudy,
          ObservableTypes.MedicalDrug,
          ObservableTypes.MedicalCondition,
          ObservableTypes.MedicalProcedure,
          ObservableTypes.Person,          // Principal investigators
          ObservableTypes.Organization     // Sponsors
        ]
      }
    }]
  }
});

// Ingest clinical trial paper
const trial = await graphlit.ingestUri('https://clinicaltrials.gov/study/NCT12345678/document.pdf', undefined, undefined, undefined, undefined, { id: trialWorkflow.createWorkflow.id  });

// Wait and analyze
const trialDetails = await graphlit.getContent(trial.ingestUri.id);

// Extract trial metadata
const studyType = trialDetails.content.observations
  ?.find(obs => obs.type === ObservableTypes.MedicalStudy);

const drugTested = trialDetails.content.observations
  ?.find(obs => obs.type === ObservableTypes.MedicalDrug);

const conditionTreated = trialDetails.content.observations
  ?.find(obs => obs.type === ObservableTypes.MedicalCondition);

console.log(`Study: ${studyType?.observable.name}`);
console.log(`Drug: ${drugTested?.observable.name}`);
console.log(`Condition: ${conditionTreated?.observable.name}`);

Variation 3: Adverse Event Monitoring

Track drug side effects and adverse events:

// Process adverse event reports
const adverseWorkflow = await graphlit.createWorkflow({
  name: "Adverse Event Extraction",
  extraction: {
    jobs: [{
      connector: {
        type: EntityExtractionServiceTypes.ModelText,
        extractedTypes: [
          ObservableTypes.MedicalDrug,
          ObservableTypes.MedicalCondition,  // Side effects
          ObservableMedicalContraindication
        ]
      }
    }]
  }
});

// Ingest multiple adverse event reports
// ... (similar to above)

// Query for drug-side effect relationships
const drugId = 'drug-observable-id';
const adverseEvents = await graphlit.queryContents({
  filter: {
    observations: [{
      type: ObservableTypes.MedicalDrug,
      observable: { id: drugId }
    }]
  }
});

// Extract side effects co-occurring with drug
const sideEffects = new Map<string, number>();
adverseEvents.contents.results.forEach(report => {
  report.observations
    ?.filter(obs => obs.type === ObservableTypes.MedicalCondition)
    .forEach(obs => {
      sideEffects.set(
        obs.observable.name,
        (sideEffects.get(obs.observable.name) || 0) + 1
      );
    });
});

console.log('Common side effects:');
Array.from(sideEffects.entries())
  .sort((a, b) => b[1] - a[1])
  .slice(0, 10)
  .forEach(([effect, count]) => {
    console.log(`  ${effect}: ${count} reports`);
  });

Variation 4: Medical Literature Review

Build knowledge base from research papers:

// Process PubMed papers on specific topic
const reviewWorkflow = await graphlit.createWorkflow({
  name: "Literature Review Extraction",
  extraction: {
    jobs: [{
      connector: {
        type: EntityExtractionServiceTypes.ModelText,
        extractedTypes: [
          ObservableTypes.MedicalCondition,
          ObservableTypes.MedicalDrug,
          ObservableTypes.MedicalProcedure,
          ObservableTypes.MedicalStudy,
          ObservableTypes.Person,          // Authors
          ObservableTypes.Organization     // Institutions
        ]
      }
    }]
  }
});

// Ingest collection of papers
const papers = [
  'https://pubmed.ncbi.nlm.nih.gov/paper1.pdf',
  'https://pubmed.ncbi.nlm.nih.gov/paper2.pdf',
  // ... more papers
];

await Promise.all(
  papers.map(uri =>
    graphlit.ingestUri({ uri, workflow: { id: reviewWorkflow.createWorkflow.id } })
  )
);

// Analyze trends
const allConditions = await graphlit.queryObservables({
  filter: { types: [ObservableTypes.MedicalCondition] }
});

// Find most researched conditions
const researchCounts = new Map<string, number>();

for (const condition of allConditions.observables.results) {
  const papers = await graphlit.queryContents({
    filter: {
      observations: [{
        type: ObservableTypes.MedicalCondition,
        observable: { id: condition.observable.id }
      }]
    }
  });
  
  researchCounts.set(condition.observable.name, papers.contents.results.length);
}

console.log('Most researched conditions:');
Array.from(researchCounts.entries())
  .sort((a, b) => b[1] - a[1])
  .slice(0, 10)
  .forEach(([condition, count]) => {
    console.log(`  ${condition}: ${count} papers`);
  });

Variation 5: Treatment Protocol Assistant

RAG-based clinical decision support:

// After ingesting clinical guidelines and protocols
const conversation = await graphlit.createConversation({
  name: "Treatment Protocol Assistant"
});

// Query for treatment recommendations
const response = await graphlit.promptConversation({
  prompt: "What is the recommended treatment protocol for a patient with Type 2 diabetes and hypertension?",
  id: conversation.createConversation.id
  // RAG will search across all ingested guidelines
});

console.log('Treatment Recommendation:');
console.log(response.message.message);

// Extract structured treatment plan
const structured = await graphlit.promptConversation({
  prompt: "Based on the guidelines, provide a structured treatment plan with: 1) First-line medications, 2) Monitoring tests, 3) Lifestyle modifications, 4) Follow-up schedule. Format as JSON.",
  id: conversation.createConversation.id
});

console.log('\nStructured Plan:');
console.log(structured.message.message);

Common Issues & Solutions

Issue: Medical Abbreviations Not Recognized

Problem: "HTN", "DM", "CHF" not extracted as conditions.

Solution: Medical abbreviations may have low confidence. Either:

  1. Use lower confidence threshold (>=0.6)

  2. Expand abbreviations in preprocessing

  3. Train on medical-specific model (future feature)

Issue: False Positives on Common Terms

Problem: "Cold" extracted as MedicalCondition when discussing weather.

Solution: Context-aware filtering:

// Check surrounding context or confidence
const validConditions = conditions.filter(cond =>
  cond.occurrences?.some(occ => occ.confidence >= 0.8)
);

Issue: Missing Drug-Condition Relationships

Problem: Drug and condition mentioned but not linked.

Solution: Use co-occurrence analysis (same page) or RAG queries:

// Find relationships via RAG
const relationship = await graphlit.promptConversation({
  prompt: "What conditions is Drug X used to treat according to this document?",
  filter: { contents: [{ id: documentId }] }
});

Issue: HIPAA Compliance Concerns

Problem: Patient names being extracted from clinical notes.

Solution: Don't extract Person entities from patient records:

extractedTypes: [
  ObservableTypes.MedicalCondition,
  ObservableTypes.MedicalDrug,
  ObservableTypes.MedicalProcedure
  // DO NOT include ObservableTypes.Person for patient records
]

Also implement proper data handling:

  • Encrypt data at rest

  • Access controls

  • Audit logging

  • BAA with Graphlit (if processing PHI)

Developer Hints

Medical Entity Quality by Source

  • High quality: Published research papers, drug information sheets

  • Medium quality: Clinical guidelines, review articles

  • Variable quality: Clinical notes (abbreviations, typos)

Model Recommendations by Use Case

  • Clinical decision support: GPT-4 (highest accuracy required)

  • Research literature review: GPT-4o (good balance)

  • General medical knowledge: Claude 3.5 Sonnet

Confidence Thresholds

  • Regulatory/clinical use: >=0.85

  • Research/analysis: >=0.75

  • Exploratory/discovery: >=0.65

HIPAA and Privacy

  • Graphlit is HIPAA-compliant when properly configured

  • Sign BAA (Business Associate Agreement)

  • Use encryption, access controls

  • Don't extract identifiable patient information

  • Consider de-identification before ingestion

Performance Optimization

  • Medical extraction is slower (complex terminology)

  • Expect 20-30% longer processing than general content

  • Batch process overnight for large volumes

  • Cache commonly queried entities

Production Patterns

Healthcare Use Cases

  • Clinical decision support: Query guidelines by condition

  • Drug information lookup: Interactive drug database

  • Adverse event monitoring: Track side effects across reports

  • Literature review: Automated systematic reviews

  • Treatment protocol matching: Match patients to protocols

  • Medical education: Interactive medical knowledge base

Compliance Considerations

  • PHI (Protected Health Information) requires HIPAA compliance

  • De-identify data when possible

  • Implement access controls

  • Audit all queries

  • Regular security assessments

  • Data retention policies

Last updated

Was this helpful?