pdfplumber vs PyMuPDF for CAM Table Extraction

CAM and operating-expense reconciliation tables are the hardest financial grids in a commercial lease to pull cleanly: they mix ruled year-over-year columns, footnoted line items, gross-up rows, and pro-rata share percentages, and a single misaligned cell silently corrupts a tenant’s reconciled balance. The decision this page resolves — inside the broader Table Extraction Strategies work — is narrow and practical: for a CAM/opex reconciliation table, do you reach for pdfplumber or PyMuPDF (fitz), and how do you stop guessing per-document by routing each table to whichever engine its layout actually favors. The answer is not “pick a winner”; it is a confidence-gated hybrid that plays each library to its strength.

This is a sibling to extracting multi-column rent schedules from PDFs, which handles step-up rent grids. That page cares about escalation math down a time axis; this one cares about reconciling expense pools — controllable vs non-controllable CAM, admin fees, gross-ups to occupancy, and the pro-rata share that turns a landlord’s total into a tenant’s line — where the table shape is far less predictable.

Architectural context

Table extraction sits between raw ingestion and the canonical model. Upstream, the PDF/DOCX ingestion pipelines stage the file and using pdfplumber for commercial lease text extraction at scale already establishes pdfplumber as the workhorse for flowing lease prose. When a page has no recoverable text layer — a scanned reconciliation statement — neither library helps until the OCR preprocessing workflows rasterize and recognize it first. Downstream, the typed cells this page emits feed field mapping strategies and align with the CAM taxonomy described in handling CAM charge variations in lease taxonomy design. The engine choice here is a single question: given this table’s geometry, which parser reconstructs the cell grid with the fewest errors?

pdfplumber vs PyMuPDF: the head-to-head

Both libraries read the PDF’s text layer and its vector graphics, but they reconstruct tables from opposite starting assumptions. pdfplumber treats ruling lines as the primary evidence of cell boundaries; PyMuPDF’s find_tables leans on text clustering and whitespace, then optionally uses lines. For CAM reconciliation grids — which arrive both as fully-ruled accountant exports and as borderless landlord memos — that difference decides accuracy.

The practical read: neither wins outright on a real portfolio. A landlord’s year-end reconciliation packet often contains both a ruled controllable/non-controllable summary and a borderless narrative of pro-rata adjustments on the next page. Committing to one engine for the whole document guarantees you fight its weakness on half the tables.

A confidence-gated hybrid

The robust pattern tries pdfplumber’s lattice strategy first when ruling lines are present, falls back to PyMuPDF’s find_tables when lattice yields a sparse or ragged grid, coerces every money cell to Decimal, and scores the result before trusting it. Confidence is the arbiter — not which library produced the grid.

from __future__ import annotations

from dataclasses import dataclass
from decimal import Decimal, InvalidOperation

import pdfplumber
import fitz  # PyMuPDF


@dataclass
class ExtractedTable:
    rows: list[list[Decimal | str]]
    engine: str
    confidence: float


_CURRENCY = str.maketrans("", "", "$,  ")


def _to_decimal(raw: str) -> Decimal | str:
    """Coerce a CAM cell to Decimal; parenthesized values are negative."""
    if raw is None:
        return ""
    text = raw.strip()
    negative = text.startswith("(") and text.endswith(")")
    cleaned = text.strip("()").translate(_CURRENCY)
    if not cleaned:
        return text
    try:
        value = Decimal(cleaned)
    except InvalidOperation:
        return text  # keep label cells (e.g. "Property taxes") verbatim
    return -value if negative else value


def _score(rows: list[list]) -> float:
    """Reward rectangular grids where numeric columns actually parsed."""
    if not rows or len(rows) < 2:
        return 0.0
    width = len(rows[0])
    if width < 2:
        return 0.0
    rectangular = sum(1 for r in rows if len(r) == width) / len(rows)
    numeric_cells = sum(
        1 for r in rows for c in r if isinstance(c, Decimal)
    )
    density = numeric_cells / (len(rows) * width)
    return round(0.6 * rectangular + 0.4 * min(density * 2, 1.0), 3)


def _pdfplumber_lattice(page) -> list[list[Decimal | str]]:
    settings = {"vertical_strategy": "lines", "horizontal_strategy": "lines"}
    table = page.extract_tables(settings)
    grid = table[0] if table else []
    return [[_to_decimal(c) for c in row] for row in grid]


def _pymupdf_stream(page: "fitz.Page") -> list[list[Decimal | str]]:
    found = page.find_tables(strategy="text")
    if not found.tables:
        return []
    raw = found.tables[0].extract()
    return [[_to_decimal(c) for c in row] for row in raw]


def extract_cam_table(pdf_path: str, page_index: int, min_conf: float = 0.72) -> ExtractedTable:
    """Route a CAM reconciliation table to the engine its geometry favors."""
    with pdfplumber.open(pdf_path) as pdf:
        page = pdf.pages[page_index]
        has_rules = bool(page.lines) or bool(page.rects)
        lattice_rows = _pdfplumber_lattice(page) if has_rules else []

    lattice_conf = _score(lattice_rows)
    if lattice_conf >= min_conf:
        return ExtractedTable(lattice_rows, "pdfplumber-lattice", lattice_conf)

    # Fall back to PyMuPDF's whitespace-driven detector
    doc = fitz.open(pdf_path)
    try:
        stream_rows = _pymupdf_stream(doc[page_index])
    finally:
        doc.close()

    stream_conf = _score(stream_rows)
    if stream_conf >= lattice_conf:
        return ExtractedTable(stream_rows, "pymupdf-stream", stream_conf)
    return ExtractedTable(lattice_rows, "pdfplumber-lattice", lattice_conf)

The engine label rides along with every result so downstream review — and the field mapping strategies that map these rows to canonical fields — can weight a pymupdf-stream extraction differently from a clean lattice hit. Money is Decimal from the first coercion, never float, so a rounded gross-up never desyncs a reconciliation total.

Edge cases specific to CAM tables

CAM reconciliation grids break the naive “read rows, split columns” assumption in ways rent schedules rarely do:

• Multi-year reconciliation columns. Statements routinely show Budget, Actual, and Prior Year side by side. pdfplumber’s lattice keeps these separated when borders exist; PyMuPDF can merge two tight numeric columns into one cell, which _score catches as a width irregularity so the fallback re-runs the other engine.
• Gross-up rows. A “grossed-up to 95% occupancy” line inserts an adjusted subtotal between the raw pool and the tenant share. Keep it as a labeled row — the label cell stays a string via _to_decimal’s fallback — so downstream logic can distinguish an adjustment from a base expense.
• Pro-rata share percentages. The tenant’s share (e.g. 4.7823%) sits in the same column as dollar figures. Strip the percent and store it as a Decimal fraction at the mapping layer, not here, so _score still reads it as numeric.
• Footnoted line items. Superscript markers like Insurance¹ attach to the label; PyMuPDF sometimes emits the marker as a stray adjacent cell. Reconcile footnotes to their notes block after extraction, never by column position.
• Parenthesized negatives and currency noise. Credits appear as (1,204.55) and figures carry $, thin spaces ( ), and thousands commas. _to_decimal normalizes all of these and flips parenthesized values negative — the single most common source of sign errors in reconciled balances.

When to escalate

The hybrid is deliberately conservative: two engines, one confidence gate, no heroics. Escalate out of the automated path when:

• Both engines score below min_conf. The table geometry is genuinely ambiguous — send the page through OCR preprocessing workflows to re-render and re-detect, then queue the result for manual review rather than committing a low-confidence grid.
• The page has no recoverable text layer. A scanned reconciliation statement returns empty page.lines and empty text; there is nothing for either library to parse. OCR is mandatory before extraction, not a fallback.
• The reconciliation total doesn’t tie. If the summed line items diverge from the stated pool total beyond a cent tolerance, the extraction is wrong even at high confidence — route it to review and align the CAM categories against handling CAM charge variations in lease taxonomy design.

Frequently asked questions

Should I default to pdfplumber or PyMuPDF for CAM tables? Neither as a blanket default. Route by geometry: pdfplumber’s lattice strategy for tables with visible ruling lines, PyMuPDF’s stream detector for borderless whitespace tables. The hybrid tries lattice first and falls back only when the score is low, so you get pdfplumber’s border precision and PyMuPDF’s whitespace inference on the same document.

How do I keep parenthesized credits from posting as positive amounts? Detect the leading and trailing parentheses before coercion, strip currency symbols, thin spaces, and commas, then negate the resulting Decimal. Doing this at the cell-coercion boundary means every downstream consumer sees a correctly signed value and no reconciliation total silently inflates.

Why score confidence instead of trusting whichever engine returns a table? Both libraries will happily return a malformed grid — merged columns, dropped rows, ragged widths — without raising. Scoring rectangularity and numeric-cell density gives an objective gate that decides whether to accept the result, try the other engine, or escalate to OCR and review.

What makes CAM tables harder than rent schedules? Rent schedules are regular time-axis grids; CAM reconciliation tables mix ruled and borderless layouts, gross-up adjustment rows, pro-rata percentages sharing a column with dollars, and footnoted labels. The variety is why a single engine underperforms and a routed, confidence-gated hybrid wins.

Related

• Extracting multi-column rent schedules from PDFs — the sibling technique for step-up rent grids, where the table shape is regular and the axis is time.
• Table Extraction Strategies — the parent guide covering detection models, engine selection, and validation across all lease table types.
• Using pdfplumber for commercial lease text extraction at scale — establishes pdfplumber as the workhorse for flowing lease prose upstream of table work.
• OCR Preprocessing Workflows — the mandatory pre-step when a CAM statement is scanned or both engines score below threshold.
• Handling CAM charge variations in lease taxonomy design — how the extracted rows map onto controllable, non-controllable, and grossed-up CAM categories.

← Back to Table Extraction Strategies

← Back to Table Extraction Strategies