> For the complete documentation index, see [llms.txt](https://docs.graphlit.dev/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.graphlit.dev/api-guides/use-cases/knowledge-graph/knowledge-graph-from-pdf-documents.md).

# Build Knowledge Graph from PDF Documents

## Use Case: Build Knowledge Graph from PDF Documents

### User Intent

"How do I extract entities from PDF documents to build a knowledge graph? Show me a complete workflow from PDF ingestion to querying entities."

### Operation

**SDK Methods**: `createWorkflow()`, `ingestUri()`, `isContentDone()`, `getContent()`, `queryObservables()`\
**GraphQL**: Complete workflow + ingestion + entity querying\
**Entity**: PDF → Content → Observations → Observables (Knowledge Graph)

### Prerequisites

* Graphlit project with API credentials
* PDF documents to process (local files or URLs)
* Understanding of entity types
* Basic knowledge of workflows

***

### Complete Code Example (TypeScript)

```typescript
import { Graphlit } from 'graphlit-client';
import {
  FilePreparationServiceTypes,
  EntityExtractionServiceTypes,
  ObservableTypes,
  EntityState
} from 'graphlit-client/dist/generated/graphql-types';

const graphlit = new Graphlit();

console.log('=== Building Knowledge Graph from PDF ===\n');

// Step 1: Create extraction workflow
console.log('Step 1: Creating extraction workflow...');
const workflow = await graphlit.createWorkflow({
  name: "PDF Entity Extraction",
  preparation: {
    jobs: [{
      connector: {
        type: FilePreparationServiceTypes.ModelDocument  // PDF, Word, Excel, etc.
      }
    }]
  },
  extraction: {
    jobs: [{
      connector: {
        type: EntityExtractionServiceTypes.ModelText,
        extractedTypes: [
          ObservableTypes.Person,
          ObservableTypes.Organization,
          ObservableTypes.Place,
          ObservableTypes.Event
        ]
      }
    }]
  }
});

console.log(`✓ Created workflow: ${workflow.createWorkflow.id}\n`);

// Step 2: Ingest PDF
console.log('Step 2: Ingesting PDF document...');
const content = await graphlit.ingestUri('https://arxiv.org/pdf/2301.00001.pdf', "Research Paper", undefined, undefined, undefined, { id: workflow.createWorkflow.id  });

console.log(`✓ Ingested: ${content.ingestUri.id}\n`);

// Step 3: Wait for processing
console.log('Step 3: Waiting for entity extraction...');
let isDone = false;
while (!isDone) {
  const status = await graphlit.isContentDone(content.ingestUri.id);
  isDone = status.isContentDone.result;
  
  if (!isDone) {
    console.log('  Processing... (checking again in 2s)');
    await new Promise(resolve => setTimeout(resolve, 2000));
  }
}
console.log('✓ Extraction complete\n');

// Step 4: Retrieve content with entities
console.log('Step 4: Retrieving extracted entities...');
const contentDetails = await graphlit.getContent(content.ingestUri.id);
const observations = contentDetails.content.observations || [];

console.log(`✓ Found ${observations.length} entity observations\n`);

// Step 5: Analyze entities by type
console.log('Step 5: Analyzing entities...\n');

const byType = new Map<string, Set<string>>();
observations.forEach(obs => {
  if (!byType.has(obs.type)) {
    byType.set(obs.type, new Set());
  }
  byType.get(obs.type)!.add(obs.observable.name);
});

byType.forEach((entities, type) => {
  console.log(`${type} (${entities.size} unique):`);
  Array.from(entities).slice(0, 5).forEach(name => {
    console.log(`  - ${name}`);
  });
  if (entities.size > 5) {
    console.log(`  ... and ${entities.size - 5} more`);
  }
  console.log();
});

// Step 6: Query knowledge graph
console.log('Step 6: Querying knowledge graph...\n');

// Get all unique people
const people = await graphlit.queryObservables({
  filter: {
    types: [ObservableTypes.Person],
    states: [EntityState.Enabled]
  }
});

console.log(`Total people in knowledge graph: ${people.observables.results.length}`);

// Get all organizations
const orgs = await graphlit.queryObservables({
  filter: {
    types: [ObservableTypes.Organization],
    states: [EntityState.Enabled]
  }
});

console.log(`Total organizations in knowledge graph: ${orgs.observables.results.length}`);

// Step 7: Find entity relationships
console.log('\nStep 7: Analyzing entity co-occurrences...\n');

const cooccurrences: Array<{ person: string; organization: string; count: number }> = [];

observations
  .filter(obs => obs.type === ObservableTypes.Person)
  .forEach(personObs => {
    observations
      .filter(obs => obs.type === ObservableTypes.Organization)
      .forEach(orgObs => {
        // Check if they appear on same pages
        const personPages = new Set(
          personObs.occurrences?.map(occ => occ.pageIndex) || []
        );
        const orgPages = new Set(
          orgObs.occurrences?.map(occ => occ.pageIndex) || []
        );
        
        const sharedPages = Array.from(personPages).filter(p => orgPages.has(p));
        
        if (sharedPages.length > 0) {
          cooccurrences.push({
            person: personObs.observable.name,
            organization: orgObs.observable.name,
            count: sharedPages.length
          });
        }
      });
  });

console.log('Top person-organization relationships:');
cooccurrences
  .sort((a, b) => b.count - a.count)
  .slice(0, 5)
  .forEach(({ person, organization, count }) => {
    console.log(`  ${person} ↔ ${organization} (${count} pages)`);
  });

console.log('\n✓ Knowledge graph analysis complete!');
```

***

## Run

asyncio.run(build\_kg\_from\_pdf())

````

### C#
```csharp
using Graphlit;
using Graphlit.Api.Input;

var graphlit = new Graphlit();

Console.WriteLine("=== Building Knowledge Graph from PDF ===\n");

// Step 1: Create workflow
Console.WriteLine("Step 1: Creating extraction workflow...");
var workflow = await graphlit.CreateWorkflow(
    name: "PDF Entity Extraction",
    preparation: new WorkflowPreparationInput
    {
        Jobs = new[]
        {
            new WorkflowPreparationJobInput
            {
                Connector = new FilePreparationConnectorInput
                {
                    Type = FilePreparationServiceDocument
                }
            }
        }
    },
    extraction: new WorkflowExtractionInput
    {
        Jobs = new[]
        {
            new WorkflowExtractionJobInput
            {
                Connector = new ExtractionConnectorInput
                {
                    Type = ExtractionServiceModelDocument,
                    ExtractedTypes = new[]
                    {
                        ObservableTypes.Person,
                        ObservableTypes.Organization,
                        ObservableTypes.Place,
                        ObservableTypes.Event
                    }
                }
            }
        }
    }
);

Console.WriteLine($"✓ Created workflow: {workflow.CreateWorkflow.Id}\n");

// Step 2: Ingest PDF
Console.WriteLine("Step 2: Ingesting PDF document...");
var content = await graphlit.IngestUri(
    name: "Research Paper",
    uri: "https://arxiv.org/pdf/2301.00001.pdf",
    workflow: new EntityReferenceInput { Id = workflow.CreateWorkflow.Id }
);

Console.WriteLine($"✓ Ingested: {content.IngestUri.Id}\n");

// (Continue with remaining steps...)
````

***

### Step-by-Step Explanation

#### Step 1: Create Extraction Workflow

**Document Preparation**:

* `FilePreparationServiceDocument` handles PDFs, Word, Excel, PowerPoint
* Extracts text, tables, images
* Preserves page structure and layout
* Handles encrypted PDFs (if password provided)

**Vision-Based Extraction**:

* `ExtractionServiceModelDocument` uses vision models
* Analyzes visual layout (charts, diagrams, tables)
* Better for scanned PDFs
* Extracts from images within PDFs

**Entity Type Selection**:

* Choose types relevant to your domain
* More types = longer processing time
* Start with Person, Organization, Place, Event

#### Step 2: Ingest PDF Document

**Ingestion Options**:

```typescript
// From URL
await graphlit.ingestUri('https://example.com/document.pdf', undefined, undefined, undefined, undefined, { id: workflowId  });

// From local file (base64 encoded)
const fileBuffer = fs.readFileSync('./document.pdf');
const base64 = fileBuffer.toString('base64');

await graphlit.ingestEncodedFile({
  name: 'document.pdf',
  data: base64,
  mimeType: 'application/pdf',
  workflow: { id: workflowId }
});

// From cloud storage (via feed)
const feed = await graphlit.createFeed({
  type: FeedTypes.Site,
  site: {
    type: FeedServiceTypes.AzureFile,
    // ... Azure Blob Storage config
  },
  workflow: { id: workflowId }
});
```

#### Step 3: Poll for Completion

**Processing Timeline**:

* Small PDF (<10 pages): 30-60 seconds
* Medium PDF (10-50 pages): 1-3 minutes
* Large PDF (50-200 pages): 3-10 minutes
* Very large PDF (200+ pages): 10-30 minutes

**Polling Strategy**:

```typescript
// Efficient polling with backoff
let retries = 0;
const maxRetries = 60;  // 2 minutes max
const delay = 2000;     // 2 seconds

while (retries < maxRetries) {
  const status = await graphlit.isContentDone(contentId);
  if (status.isContentDone.result) break;
  
  await new Promise(resolve => setTimeout(resolve, delay));
  retries++;
}
```

#### Step 4: Retrieve Extracted Entities

**Full Content Details**:

```typescript
const content = await graphlit.getContent(contentId);

// Access entity observations
const observations = content.content.observations || [];

// Access content metadata
console.log(`Pages: ${content.content.document?.pageCount}`);
console.log(`File size: ${content.content.fileSize}`);
console.log(`Created: ${content.content.creationDate}`);
```

#### Step 5: Analyze Entities

**Group by Type**:

```typescript
const entityGroups = observations.reduce((groups, obs) => {
  if (!groups[obs.type]) {
    groups[obs.type] = [];
  }
  groups[obs.type].push(obs.observable);
  return groups;
}, {} as Record<string, Observable[]>);
```

**Deduplicate**:

```typescript
const uniqueEntities = new Map<string, Observable>();
observations.forEach(obs => {
  uniqueEntities.set(obs.observable.id, obs.observable);
});
```

#### Step 6: Query Knowledge Graph

After entities are extracted, they're available globally:

```typescript
// All people across all content
const allPeople = await graphlit.queryObservables({
  filter: { types: [ObservableTypes.Person] }
});

// Search for specific person
const kirkEntities = await graphlit.queryObservables({
  search: "Kirk Marple",
  filter: { types: [ObservableTypes.Person] }
});
```

#### Step 7: Analyze Relationships

**Co-occurrence Analysis**:

* Entities on same page likely related
* Frequency indicates relationship strength
* Build relationship graph from co-occurrences

**Cross-document Relationships**:

```typescript
// Find content mentioning both entities
const relatedContent = await graphlit.queryContents({
  
    observations: [
      { type: ObservableTypes.Person, observable: { id: personId } },
      { type: ObservableTypes.Organization, observable: { id: orgId } }
    ]
  });
```

***

### Configuration Options

#### Choosing Text vs Vision Extraction

**Use Text Extraction (`ModelText`) When**:

* PDFs are text-based (born-digital)
* No important visual elements
* Want faster/cheaper processing
* Content is primarily textual

**Use Vision Extraction (`ModelDocument`) When**:

* PDFs are scanned documents
* Contains important charts/diagrams
* Mixed text and visual content
* OCR quality is poor with text extraction

#### Model Selection for Quality vs Speed

```typescript
// High quality (slower, more expensive)
const specGPT4 = await graphlit.createSpecification({
  name: "GPT-4 Extraction",
  type: SpecificationTypes.Completion,
  serviceType: ModelServiceTypes.OpenAi,
  openAI: { model: OpenAIModels.Gpt4, temperature: 0.1 }
});

// Balanced (recommended)
const specGPT4o = await graphlit.createSpecification({
  name: "GPT-4o Extraction",
  type: SpecificationTypes.Completion,
  serviceType: ModelServiceTypes.OpenAi,
  openAI: { model: OpenAIModels.Gpt4o, temperature: 0.1 }
});

// Fast and cost-effective
const specGemini = await graphlit.createSpecification({
  name: "Gemini Extraction",
  type: SpecificationTypes.Completion,
  serviceType: ModelServiceTypes.Google,
  gemini: { model: GeminiModels.Gemini15Pro, temperature: 0.1 }
});
```

***

### Variations

#### Variation 1: Legal Contract Analysis

Extract parties, dates, obligations from legal documents:

```typescript
const legalWorkflow = await graphlit.createWorkflow({
  name: "Legal Contract Extraction",
  extraction: {
    jobs: [{
      connector: {
        type: EntityExtractionServiceTypes.ModelDocument,
        extractedTypes: [
          ObservableTypes.Person,        // Parties
          ObservableTypes.Organization,  // Companies
          ObservableTypes.Place,         // Jurisdictions
          ObservableTypes.Event          // Effective dates, deadlines
        ]
      }
    }]
  }
});

// Analyze extracted obligations
const contractContent = await graphlit.getContent(contentId);
const events = contractContent.content.observations
  ?.filter(obs => obs.type === ObservableTypes.Event) || [];

console.log('Contract deadlines:');
events.forEach(event => {
  console.log(`  - ${event.observable.name}`);
  event.occurrences?.forEach(occ => {
    console.log(`    Page ${(occ.pageIndex || 0) + 1}, Confidence: ${occ.confidence}`);
  });
});
```

#### Variation 2: Research Paper Citation Network

Build academic citation graphs:

```typescript
const researchWorkflow = await graphlit.createWorkflow({
  name: "Research Paper Extraction",
  extraction: {
    jobs: [{
      connector: {
        type: EntityExtractionServiceTypes.ModelText,
        extractedTypes: [
          ObservableTypes.Person,        // Authors
          ObservableTypes.Organization,  // Institutions
          ObservableTypes.Category       // Topics, keywords
        ]
      }
    }]
  }
});

// Build author collaboration network
const papers = await graphlit.queryContents({
   workflows: [{ id: researchWorkflow.createWorkflow.id }] });

const collaborations = new Map<string, Set<string>>();

papers.contents.results.forEach(paper => {
  const authors = paper.observations
    ?.filter(obs => obs.type === ObservableTypes.Person)
    .map(obs => obs.observable.name) || [];
  
  // Record co-authorships
  for (let i = 0; i < authors.length; i++) {
    for (let j = i + 1; j < authors.length; j++) {
      const key = [authors[i], authors[j]].sort().join(' & ');
      if (!collaborations.has(key)) {
        collaborations.set(key, new Set());
      }
      collaborations.get(key)!.add(paper.name);
    }
  }
});

console.log('Top collaborations:');
Array.from(collaborations.entries())
  .sort((a, b) => b[1].size - a[1].size)
  .slice(0, 10)
  .forEach(([authors, papers]) => {
    console.log(`  ${authors}: ${papers.size} papers`);
  });
```

#### Variation 3: Invoice/Receipt Processing

Extract vendors, amounts, dates from financial documents:

```typescript
const invoiceWorkflow = await graphlit.createWorkflow({
  name: "Invoice Extraction",
  extraction: {
    jobs: [{
      connector: {
        type: EntityExtractionServiceTypes.ModelDocument,  // Vision for logos/stamps
        extractedTypes: [
          ObservableTypes.Organization,  // Vendor, customer
          ObservableTypes.Place,         // Billing/shipping address
          ObservableTypes.Event,         // Invoice date, due date
          ObservableTypes.Product        // Line items
        ]
      }
    }]
  }
});

// Extract invoice metadata
const invoice = await graphlit.getContent(invoiceId);
const vendor = invoice.content.observations
  ?.find(obs => obs.type === ObservableTypes.Organization);

console.log(`Vendor: ${vendor?.observable.name}`);
```

#### Variation 4: Medical Records Analysis

Extract medical entities from clinical documents:

```typescript
const medicalWorkflow = await graphlit.createWorkflow({
  name: "Medical Records Extraction",
  extraction: {
    jobs: [{
      connector: {
        type: EntityExtractionServiceTypes.ModelText,
        extractedTypes: [
          ObservableTypes.Person,              // Patients, doctors
          ObservableTypes.MedicalCondition,    // Diagnoses
          ObservableTypes.MedicalDrug,         // Medications
          ObservableTypes.MedicalProcedure,    // Treatments
          ObservableTypes.MedicalTest          // Lab tests
        ]
      }
    }]
  }
});

// Analyze patient record
const record = await graphlit.getContent(recordId);
const conditions = record.content.observations
  ?.filter(obs => obs.type === ObservableTypes.MedicalCondition) || [];
const drugs = record.content.observations
  ?.filter(obs => obs.type === ObservableTypes.MedicalDrug) || [];

console.log('Diagnoses:', conditions.map(c => c.observable.name));
console.log('Medications:', drugs.map(d => d.observable.name));
```

#### Variation 5: Batch PDF Processing

Process multiple PDFs efficiently:

```typescript
const pdfUrls = [
  'https://example.com/doc1.pdf',
  'https://example.com/doc2.pdf',
  'https://example.com/doc3.pdf',
  // ... more PDFs
];

// Ingest all PDFs
const contentIds = await Promise.all(
  pdfUrls.map(uri =>
    graphlit.ingestUri({ uri, workflow: { id: workflowId } })
      .then(result => result.ingestUri.id)
  )
);

console.log(`Ingested ${contentIds.length} PDFs`);

// Wait for all to complete
const waitForAll = async () => {
  let allDone = false;
  while (!allDone) {
    const statuses = await Promise.all(
      contentIds.map(id => graphlit.isContentDone(id))
    );
    allDone = statuses.every(s => s.isContentDone.result);
    
    if (!allDone) {
      console.log('Processing...');
      await new Promise(resolve => setTimeout(resolve, 5000));
    }
  }
};

await waitForAll();
console.log('All PDFs processed');

// Query all extracted entities
const allEntities = await graphlit.queryObservables({
  filter: { types: [ObservableTypes.Person, ObservableTypes.Organization] }
});

console.log(`Total entities extracted: ${allEntities.observables.results.length}`);
```

***

### Common Issues & Solutions

#### Issue: No Entities Extracted from Scanned PDF

**Problem**: PDF is scanned image, text extraction fails.

**Solution**: Use vision model + proper preparation:

```typescript
preparation: {
  jobs: [{
    connector: {
      type: FilePreparationServiceTypes.ModelDocument,
      document: {
        includeOCR: true  // Enable OCR for scanned docs
      }
    }
  }]
},
extraction: {
  jobs: [{
    connector: {
      type: EntityExtractionServiceTypes.ModelDocument  // Vision model
    }
  }]
}
```

#### Issue: Encrypted PDF Won't Process

**Problem**: PDF is password-protected.

**Solution**: Provide password in preparation:

```typescript
preparation: {
  jobs: [{
    connector: {
      type: FilePreparationServiceTypes.ModelDocument,
      document: {
        password: 'document-password-here'
      }
    }
  }]
}
```

#### Issue: Missing Entities from Images/Charts

**Problem**: Text-based extraction misses visual elements.

**Solution**: Use vision model extraction:

```typescript
extraction: {
  jobs: [{
    connector: {
      type: EntityExtractionServiceTypes.ModelDocument,  // Analyzes images
      extractedTypes: [/* ... */]
    }
  }]
}
```

#### Issue: Processing Takes Too Long

**Problem**: Large PDF processing exceeds timeout.

**Solutions**:

1. Split large PDFs into smaller chunks
2. Use faster model (GPT-4o instead of GPT-4)
3. Reduce number of entity types
4. Process in background, poll asynchronously

```typescript
// Async processing pattern
const content = await graphlit.ingestUri(pdfUrl, undefined, undefined, undefined, undefined, { id: workflowId  });

// Don't block - return ID immediately
console.log(`Processing started: ${content.ingestUri.id}`);
// Poll in background job or webhook
```

***

### Developer Hints

#### PDF Processing Best Practices

1. **Check file size first**: >50MB PDFs may need special handling
2. **Test with sample page**: Validate extraction quality before batch
3. **Use appropriate model**: Vision for scanned, text for born-digital
4. **Monitor confidence scores**: Filter entities with confidence <0.7
5. **Handle failures gracefully**: PDFs can be corrupt or malformed

#### Vision Model Selection

* **GPT-4o Vision**: Best balance (recommended)
* **Claude 3.5 Sonnet**: Good for complex layouts
* **GPT-4 Vision**: Highest quality but slower/expensive

#### Cost Optimization

* Text extraction much cheaper than vision
* GPT-4o significantly cheaper than GPT-4
* Extract only needed entity types
* Batch processing for volume discounts

#### Performance Optimization

* Parallel ingestion up to 10 PDFs simultaneously
* Poll every 2-5 seconds (not more frequently)
* Cache entity results to avoid re-querying
* Use collections to organize large sets of PDFs

***

### Production Patterns

#### Pattern from Graphlit Samples

`Graphlit_2024_09_13_Extract_People_Organizations_from_ArXiv_Papers.ipynb`:

* Ingests ArXiv research papers (PDFs)
* Extracts Person (authors), Organization (institutions)
* Filters by confidence >=0.7
* Builds citation network from entities
* Exports to CSV for analysis

#### Pattern from Legal Tech

* Process contracts in batch (100s of PDFs)
* Extract parties, dates, obligations
* Build contract database with entity index
* Enable search by party or jurisdiction
* Alert on approaching deadlines

***