Extracting Multi-Column Rent Schedules from PDFs

A rent step-up schedule is the most financially load-bearing table in a commercial lease, and it is the one PDF extractors most reliably mangle. The specific problem this page resolves is reconstructing a multi-column schedule — period, monthly rent, annual rent, rate per square foot, and escalation percent — from a lease PDF where the columns bleed into one another, individual cells wrap onto a second line, and the whole table continues onto the next page. This is a different failure surface from the sibling comparison in Table Extraction Strategies: that guide weighs pdfplumber against PyMuPDF for CAM reconciliation tables, where the goal is matching charge categories across two documents. Here the tool choice is settled; the hard part is column reconstruction — turning a cloud of positioned words back into rows whose money lands in the right column every time.

Architectural context

This technique lives inside Table Extraction Strategies, one cluster of the broader Parsing & Extraction Workflows domain. Upstream, OCR preprocessing workflows deliver a text layer for scanned schedules; here we assume positioned words are available and focus on geometry. Downstream, each reconstructed row flows through field mapping strategies into the canonical model, its money and percents normalized against the metadata normalization standards contract. The escalation column in particular feeds escalation formula mapping, which is where a misaligned 3.0% becomes a mispriced renewal. This page owns exactly one question: given the positioned words on a schedule page, which column does each value belong to, and what typed row does it produce?

Canonical rent-schedule columns

Before any code, fix the target shape. A rent step-up schedule reduces to five canonical columns, each with a type and a coercion rule. Everything downstream — the pydantic model, the column classifier, the validators — keys off this table.

Two rules matter more than the rest. Money is always Decimal, never float, because a rent schedule is posted to the books and binary rounding silently drifts. And period is a parsed interval, not a label — “Months 13-24” has to become (13, 24) so the schedule can be validated for contiguity and handed to payment-schedule generation from escalation rules without re-parsing.

Recommended implementation

The reconstruction runs in four passes: infer column boundaries from word x-coordinates, stitch the multi-row header, merge wrapped rows by vertical proximity, then assign each word to a column and coerce. The following works on any word list exposing x0, x1, top, and text — the shape pdfplumber’s extract_words() returns.

from __future__ import annotations
from collections import defaultdict
from decimal import Decimal, InvalidOperation
from typing import Optional
import re

from pydantic import BaseModel, field_validator, model_validator


class Word(BaseModel):
    text: str
    x0: float
    x1: float
    top: float  # y from page top; larger = further down


def infer_column_bounds(words: list[Word], min_gap: float = 14.0) -> list[float]:
    """Cluster word x0 values; a horizontal gap wider than min_gap is a column edge."""
    xs = sorted(w.x0 for w in words)
    bounds: list[float] = [xs[0] - 1]
    for prev, cur in zip(xs, xs[1:]):
        if cur - prev > min_gap:
            bounds.append((prev + cur) / 2)  # boundary sits in the empty gutter
    bounds.append(xs[-1] + 1)
    return bounds


def column_of(x0: float, bounds: list[float]) -> int:
    for i in range(len(bounds) - 1):
        if bounds[i] <= x0 < bounds[i + 1]:
            return i
    return len(bounds) - 2

Clustering x0 rather than reading a fixed template is what makes this robust to bleed: two columns whose text nearly touches still leave a gutter narrower than their internal word spacing only when they genuinely merge, and min_gap is the knob that separates “tight columns” from “run-together columns.” The header is stitched by grouping words into visual rows on top, then folding any row above the first data row into the column labels beneath it.

def group_rows(words: list[Word], y_tol: float = 4.0) -> list[list[Word]]:
    """Bucket words into visual rows by y-proximity, top to bottom."""
    rows: list[list[Word]] = []
    for w in sorted(words, key=lambda w: (w.top, w.x0)):
        if rows and abs(rows[-1][0].top - w.top) <= y_tol:
            rows[-1].append(w)
        else:
            rows.append([w])
    return rows


def stitch_header(header_rows: list[list[Word]], bounds: list[float]) -> dict[int, str]:
    """Collapse a spanning row + a cell row into one label per column index."""
    labels: dict[int, list[str]] = defaultdict(list)
    for row in header_rows:
        for w in sorted(row, key=lambda w: w.x0):
            labels[column_of(w.x0, bounds)].append(w.text)
    return {col: " ".join(parts).strip() for col, parts in labels.items()}

A cell that wrapped — “Months 25-36” spilling under “Base Year”, or a long period label breaking across two lines — produces a short second visual row with no value in the money columns. Merge it back before column assignment by folding any row whose populated columns are a strict subset of the row above into that row.

MONEY = re.compile(r"[^\d.\-]")


def to_decimal(raw: str) -> Optional[Decimal]:
    s = raw.strip().replace("\xa0", "").replace(",", "")
    if not s or s in {"-", "—", "n/a", "N/A"}:
        return None
    neg = s.startswith("(") and s.endswith(")")  # (1,234.00) = negative
    s = MONEY.sub("", s)
    if not s or s == ".":
        return None
    try:
        value = Decimal(s)
    except InvalidOperation:
        return None
    return -value if neg else value


def parse_period(raw: str, commencement_month: int = 1) -> Optional[tuple[int, int]]:
    """'Months 1-12', 'Year 1', 'Yr 2-3' -> inclusive month interval."""
    text = raw.lower().replace("through", "-").replace("to", "-")
    nums = [int(n) for n in re.findall(r"\d+", text)]
    if not nums:
        return None
    if "year" in text or "yr" in text:
        lo = (nums[0] - 1) * 12 + 1
        hi = (nums[-1] if len(nums) > 1 else nums[0]) * 12
        return (lo, hi)
    lo, hi = nums[0], (nums[1] if len(nums) > 1 else nums[0])
    return (lo, hi)

The typed row model is the boundary where geometry stops and the canonical contract begins. It coerces, cross-checks annual against monthly, and rejects a period whose bounds are reversed.

class RentScheduleRow(BaseModel):
    period: tuple[int, int]
    monthly_rent: Decimal
    annual_rent: Optional[Decimal] = None
    rate_psf: Optional[Decimal] = None
    escalation_pct: Optional[Decimal] = None
    confidence: float = 1.0

    @field_validator("monthly_rent", "annual_rent", "rate_psf")
    @classmethod
    def non_negative(cls, v: Optional[Decimal]) -> Optional[Decimal]:
        if v is not None and v < 0:
            raise ValueError("rent and rate values must be non-negative")
        return v

    @model_validator(mode="after")
    def check(self) -> "RentScheduleRow":
        lo, hi = self.period
        if lo > hi:
            raise ValueError(f"reversed period interval: {self.period}")
        if self.annual_rent is not None:
            expected = self.monthly_rent * 12
            drift = abs(self.annual_rent - expected)
            if expected and drift / expected > Decimal("0.02"):
                self.confidence = min(self.confidence, 0.6)  # flag, don't reject
        return self


def build_rows(data_rows: list[list[Word]], bounds: list[float],
               header: dict[int, str]) -> list[RentScheduleRow]:
    col_role = classify_columns(header)  # maps column index -> canonical field
    out: list[RentScheduleRow] = []
    for row in data_rows:
        cells: dict[str, str] = {}
        for w in row:
            role = col_role.get(column_of(w.x0, bounds))
            if role:
                cells[role] = (cells.get(role, "") + " " + w.text).strip()
        period = parse_period(cells.get("period", ""))
        monthly = to_decimal(cells.get("monthly_rent", ""))
        if period is None or monthly is None:
            continue  # not a data row
        out.append(RentScheduleRow(
            period=period,
            monthly_rent=monthly,
            annual_rent=to_decimal(cells.get("annual_rent", "")),
            rate_psf=to_decimal(cells.get("rate_psf", "")),
            escalation_pct=_pct(cells.get("escalation_pct", "")),
        ))
    return out

classify_columns maps each stitched header label to a canonical field by keyword (“monthly” → monthly_rent, “psf” or “/sf” → rate_psf), and _pct divides a stripped percent by 100 into a Decimal fraction. The result is a contiguity-checkable list ready for field mapping strategies and the escalation layer.

Edge cases specific to lease schedules

• Wrapped cells. A period label like “Months 25-36 (Option 1)” wraps to a second line with no rent beside it. group_rows will emit it as a stray short row; fold rows whose only populated columns are period into the following money-bearing row before building.
• Two-line headers. Schedules often print “Base Rent” as a spanning banner over “Monthly / Annual / PSF.” stitch_header joins both rows per column, so the banner word contributes to every column it spans and the specific label disambiguates — classify on the specific label, not the banner.
• Schedule continues on the next page. One logical schedule frequently breaks across a page boundary, sometimes repeating the header. Re-run infer_column_bounds per page, drop a repeated header row, and concatenate rows before the contiguity check so a period gap at the seam is not mistaken for a real gap.
• Blank $/SF. Many leases quote monthly and annual rent but omit a per-square-foot rate. to_decimal maps “—” and empty cells to None; never coerce a blank to Decimal("0"), which would read as free rent.
• “See Exhibit B.” When the schedule cell defers to an exhibit rather than stating a number, capture the reference as a sentinel, mark the row low-confidence, and route it out rather than emitting a fabricated rent.
• Ranges and parenthesized negatives. “Months 1-12” and “Yr 1-2” both parse to intervals; a free-rent abatement printed as ($4,500.00) must coerce to a negative Decimal, not fail the money regex.

When to escalate

Column inference is only as good as the gutters in the document, so treat low geometric confidence as a routing signal, not an error to swallow.

• Column boundaries are ambiguous — when infer_column_bounds finds fewer than the expected columns, or clusters overlap because a scan is skewed, send the page back through OCR preprocessing workflows for deskew and re-extraction before trusting the geometry.
• A row cross-check fails — when annual_rent diverges from monthly_rent × 12 beyond tolerance, or periods are non-contiguous after stitching pages, the row lands below threshold and diverts to human review through fallback routing logic rather than posting a suspect step-up.
• The escalation column is implicit — some schedules state only dollar amounts and expect the reader to infer the percent. Hand the clean row list to escalation formula mapping, which derives the rule from consecutive periods instead of trusting a printed percent that may be rounded.

Frequently asked questions

How do I set the column-gap threshold so tight columns do not merge? Measure the median intra-word space on the page and set min_gap to roughly twice it. Real column gutters are consistently wider than the spacing between words inside a cell, so a multiple of the median separates genuine columns from run-together text without hard-coding pixel values per lease.

What if the schedule spans three pages with the header only on the first? Infer boundaries independently on every page, but carry the stitched header forward from the first page rather than re-deriving it. Concatenate the data rows across all pages before checking period contiguity, so the natural break at each page seam is not misread as a gap in the schedule.

Why coerce money to Decimal instead of float here? A rent schedule is posted to accounts receivable and accounting exports, where binary floating point accumulates rounding error across dozens of periods. Decimal preserves the exact cents the lease states, which is why every money column in the canonical model is Decimal and coercion strips formatting into a Decimal string, never a float.

What confidence threshold should divert a row to manual review? Start around 0.75 for the per-row score, and treat any failed cross-check — reversed period, annual not matching monthly times twelve, or a value coming from a page with ambiguous column inference — as an automatic diversion regardless of score, because a wrong rent step-up misprices every downstream payment.

Related

• pdfplumber vs PyMuPDF for CAM Table Extraction — the sibling tool-choice guide for CAM reconciliation tables, where matching categories matters more than column reconstruction.
• Table Extraction Strategies — the parent guide covering table detection, ruling-line vs whitespace layouts, and when each approach applies.
• Field Mapping Strategies — how the reconstructed rows are mapped onto canonical lease fields for ingestion.
• Generating Payment Schedules from Escalation Rules — the downstream consumer that turns a clean, contiguous rent schedule into dated payment rows.

← Back to Table Extraction Strategies

← Back to Table Extraction Strategies