mateclaw/mateclaw-server/src/main/resources/skills/pdf/SKILL.md

name	description	dependencies	platforms
pdf	Use this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill.	commands tools python3 skillScriptTool skillFileTool	macos linux windows

Important: All scripts/ paths are relative to this skill directory. Use run_skill_script tool to execute scripts, or run with: cd {this_skill_dir} && python scripts/...

PDF Processing Guide

Prerequisites

• pypdf: core PDF reading and writing
• pdfplumber: text and table extraction
• reportlab: PDF creation
• pdftotext (poppler-utils): command-line text extraction
• pdftoppm (poppler-utils): PDF-to-image conversion
• qpdf: PDF manipulation (merge, split, rotate, decrypt)

Tool Selection Decision Table

Choose the right approach before starting:

Input	Condition	Recommended Tool
URL	PDF accessible via URL	web_extract(url) — fastest, no download needed
Local file	Text-native PDF (generated by software)	pymupdf — ~25 MB install, instant extraction
Local file	Scanned/image-only PDF (no selectable text)	marker-pdf — OCR with layout preservation (~5 GB, needs GPU or CPU)
Local file	Form filling or page manipulation	pypdf / pdfplumber + form scripts
Local file	NLP editing or semantic search	nano-pdf — sentence-level operations

URL-first rule: If the user provides a URL, always try URL extraction first before downloading.

Overview

This guide covers essential PDF processing operations using Python libraries and command-line tools.

URL-First Extraction

If the user provides a URL pointing to a PDF, extract it without downloading:

web_extract(url="https://example.com/report.pdf")

Fall back to download + local processing only if web_extract returns empty or errors.

---

Fast Extraction: pymupdf (fitz)

Best for: Text-native PDFs (digital, not scanned). Install: pip install pymupdf (~25 MB).

import fitz  # pymupdf

doc = fitz.open("document.pdf")
print(f"Pages: {doc.page_count}")

# Extract all text (fast)
full_text = "\n".join(page.get_text() for page in doc)

# Extract with layout blocks (tables, columns)
for page in doc:
    blocks = page.get_text("blocks")  # (x0,y0,x1,y1,text,block_no,block_type)
    for block in blocks:
        print(block[4])  # text content

# Extract images
for page in doc:
    for img in page.get_images():
        xref = img[0]
        base = doc.extract_image(xref)
        with open(f"img_{xref}.{base['ext']}", "wb") as f:
            f.write(base["image"])

pymupdf is 5-10× faster than pypdf for text extraction and preserves layout better.

---

OCR Extraction: marker-pdf

Best for: Scanned PDFs, image-only PDFs, or documents where pymupdf returns garbled text. Install: pip install marker-pdf (~5 GB with models).

# Single file
marker_single document.pdf output_dir/ --batch_multiplier 2

# Batch
marker input_dir/ output_dir/ --workers 4

Outputs Markdown with preserved headings, tables, and code blocks.

Decision signal: Run pymupdf first. If extracted text has <50% printable characters or looks like garbage, switch to marker-pdf.

---

NLP Editing: nano-pdf

Best for: Semantic search, sentence-level edits, keyword replacement in text-native PDFs. Install: pip install nano-pdf.

from nano_pdf import NanoPDF

doc = NanoPDF("document.pdf")

# Search sentences
results = doc.search("termination clause", top_k=5)
for r in results:
    print(r.page, r.text, r.score)

# Replace text (produces new PDF)
doc.replace("old phrase", "new phrase", output="modified.pdf")

---

Python Libraries

pypdf - Basic Operations

Merge PDFs

from pypdf import PdfWriter, PdfReader

writer = PdfWriter()
for pdf_file in ["doc1.pdf", "doc2.pdf", "doc3.pdf"]:
    reader = PdfReader(pdf_file)
    for page in reader.pages:
        writer.add_page(page)

with open("merged.pdf", "wb") as output:
    writer.write(output)

Split PDF

reader = PdfReader("input.pdf")
for i, page in enumerate(reader.pages):
    writer = PdfWriter()
    writer.add_page(page)
    with open(f"page_{i+1}.pdf", "wb") as output:
        writer.write(output)

Extract Metadata

reader = PdfReader("document.pdf")
meta = reader.metadata
print(f"Title: {meta.title}")
print(f"Author: {meta.author}")

Rotate Pages

reader = PdfReader("input.pdf")
writer = PdfWriter()
page = reader.pages[0]
page.rotate(90)  # Rotate 90 degrees clockwise
writer.add_page(page)
with open("rotated.pdf", "wb") as output:
    writer.write(output)

pdfplumber - Text and Table Extraction

Extract Text with Layout

import pdfplumber

with pdfplumber.open("document.pdf") as pdf:
    for page in pdf.pages:
        text = page.extract_text()
        print(text)

Extract Tables

with pdfplumber.open("document.pdf") as pdf:
    for i, page in enumerate(pdf.pages):
        tables = page.extract_tables()
        for j, table in enumerate(tables):
            print(f"Table {j+1} on page {i+1}:")
            for row in table:
                print(row)

Advanced Table Extraction

import pandas as pd

with pdfplumber.open("document.pdf") as pdf:
    all_tables = []
    for page in pdf.pages:
        tables = page.extract_tables()
        for table in tables:
            if table:
                df = pd.DataFrame(table[1:], columns=table[0])
                all_tables.append(df)

if all_tables:
    combined_df = pd.concat(all_tables, ignore_index=True)
    combined_df.to_excel("extracted_tables.xlsx", index=False)

reportlab - Create PDFs

Basic PDF Creation

from reportlab.lib.pagesizes import letter
from reportlab.pdfgen import canvas

c = canvas.Canvas("hello.pdf", pagesize=letter)
width, height = letter
c.drawString(100, height - 100, "Hello World!")
c.line(100, height - 140, 400, height - 140)
c.save()

Subscripts and Superscripts

IMPORTANT: Never use Unicode subscript/superscript characters in ReportLab PDFs. The built-in fonts do not include these glyphs, causing them to render as solid black boxes.

Use ReportLab's XML markup tags instead:

from reportlab.platypus import Paragraph
from reportlab.lib.styles import getSampleStyleSheet
styles = getSampleStyleSheet()
chemical = Paragraph("H<sub>2</sub>O", styles['Normal'])
squared = Paragraph("x<super>2</super> + y<super>2</super>", styles['Normal'])

PDF Form Processing

Check if PDF has fillable fields

python scripts/check_fillable_fields.py document.pdf

Extract form field info

python scripts/extract_form_field_info.py document.pdf

Extract form structure (non-fillable PDFs)

python scripts/extract_form_structure.py document.pdf

Fill form fields

python scripts/fill_fillable_fields.py document.pdf output.pdf --fields '{"field_name": "value"}'

Fill with annotations (non-fillable PDFs)

python scripts/fill_pdf_form_with_annotations.py document.pdf output.pdf --data '{"x,y": "text"}'

Validate bounding boxes

python scripts/check_bounding_boxes.py document.pdf

Convert PDF to images

python scripts/convert_pdf_to_images.py document.pdf output_dir/ --dpi 150

Create validation image with overlays

python scripts/create_validation_image.py document.pdf output.png

Command-Line Tools

pdftotext (poppler-utils)

pdftotext input.pdf output.txt              # Extract text
pdftotext -layout input.pdf output.txt      # Preserve layout
pdftotext -f 1 -l 5 input.pdf output.txt   # Pages 1-5

qpdf

qpdf --empty --pages file1.pdf file2.pdf -- merged.pdf     # Merge
qpdf input.pdf --pages . 1-5 -- pages1-5.pdf               # Split
qpdf input.pdf output.pdf --rotate=+90:1                    # Rotate
qpdf --password=mypassword --decrypt encrypted.pdf out.pdf  # Decrypt

Common Tasks

Extract Text from Scanned PDFs (OCR)

import pytesseract
from pdf2image import convert_from_path

images = convert_from_path('scanned.pdf')
text = ""
for i, image in enumerate(images):
    text += f"Page {i+1}:\n"
    text += pytesseract.image_to_string(image)
    text += "\n\n"

Add Watermark

from pypdf import PdfReader, PdfWriter

watermark = PdfReader("watermark.pdf").pages[0]
reader = PdfReader("document.pdf")
writer = PdfWriter()

for page in reader.pages:
    page.merge_page(watermark)
    writer.add_page(page)

with open("watermarked.pdf", "wb") as output:
    writer.write(output)

Password Protection

from pypdf import PdfReader, PdfWriter

reader = PdfReader("input.pdf")
writer = PdfWriter()
for page in reader.pages:
    writer.add_page(page)
writer.encrypt("userpassword", "ownerpassword")
with open("encrypted.pdf", "wb") as output:
    writer.write(output)

Quick Reference

Task	Best Tool	Command/Code
URL → text	web_extract	web_extract(url=...)
Fast text extraction	pymupdf	fitz.open(...).get_text()
Scanned / OCR	marker-pdf	marker_single doc.pdf out/
Semantic search/edit	nano-pdf	NanoPDF(...).search(...)
Merge PDFs	pypdf	writer.add_page(page)
Split PDFs	pypdf	One page per file
Extract text (layout)	pdfplumber	page.extract_text()
Extract tables	pdfplumber	page.extract_tables()
Create PDFs	reportlab	Canvas or Platypus
Fill forms	scripts	fill_fillable_fields.py