Implementing Fallback Routing for Missing Lease Metadata Fields

The precise decision this page resolves: when a single lease field arrives null, malformed, or below its confidence threshold, which substitute value should the pipeline emit — a static portfolio default, the value inherited from a prior lease version, a cross-referenced figure from an adjacent system, or nothing at all because the record belongs in a human queue? Picking one strategy per field, deterministically and auditably, is the whole job. Guessing wrong silently overstates net operating income or posts a charge against the wrong tenant; failing loudly on every gap halts a portfolio-scale ingestion run. Fallback routing is the narrow layer that turns an extraction miss into a typed, traceable outcome instead of either failure mode.

Architectural context

This technique lives inside fallback routing logic, the stage within Core Architecture & Lease Taxonomy that catches records before they reach the canonical store. It runs after the input boundary has applied metadata normalization standards — so dates are already ISO-8601 and amounts already typed Decimal — and after a clause classification system has tagged each span. By the time a payload hits this router, a field is missing for one of three reasons: the extractor returned None, the value failed schema validation, or the confidence score dropped below the per-field threshold. The router’s only question is what to substitute, and its output feeds straight into lease data models for storage or diverts to a review queue. It never re-extracts and never reclassifies.

Choosing a fallback strategy per field

There is no single correct fallback — the right one depends on the field’s financial impact and how stable it is across a lease’s lifetime. A rent_escalation_cap must never be invented from a portfolio average, because a wrong cap compounds across the term; a parking_allocation_sqft can safely inherit an asset-class default. The table below is the decision matrix used to assign one primary strategy per field class.

The governing rule: the higher a field’s downstream financial blast radius, the further right its strategy should sit. When in doubt, a field escalates rather than inherits. Anything feeding an escalation formula — caps, floors, base indices — defaults to manual_prompt unless a high-confidence prior version exists.

Declaring the routing rules

Externalize the matrix into a version-controlled YAML specification so thresholds and strategies change without redeploying the ingestion service. The router compiles this once at startup into an ordered chain per field; the order is the precedence.

fallback_routing:
  rent_escalation_cap:
    primary_source: "llm_extractor"
    confidence_threshold: 0.85
    fallback_chain:
      - source: "historical_version"
        strategy: "inherit_if_valid"
      - source: "market_default"
        value: 0.03
        strategy: "static_override"
    escalation_target: "ops_review_queue"
    audit_flag: true

  cam_reconciliation_date:
    primary_source: "ocr_engine"
    confidence_threshold: 0.90
    fallback_chain:
      - source: "tenant_portal_sync"
        strategy: "cross_reference"
      - source: "property_manager_input"
        strategy: "manual_prompt"
    escalation_target: "data_steward_dashboard"
    audit_flag: true

Recommended implementation

The router is a side-effect-free decision layer: it takes one normalized field payload, walks the chain in declared order, and returns a typed outcome. It uses pydantic v2 for strict validation of both the config and the payload, and Python’s standard logging for the audit trail. The strategy executors are injected, so the resolution logic stays testable and the network calls live behind a clear seam — wrap those in circuit breakers and exponential backoff in production, the same pattern the error-handling and retry logic layer uses for flaky OCR APIs.

import logging
from enum import Enum
from typing import Any, Callable, Optional
import yaml
from pydantic import BaseModel, Field, model_validator

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
)
logger = logging.getLogger("lease_fallback_router")


class FallbackStrategy(str, Enum):
    INHERIT_IF_VALID = "inherit_if_valid"
    STATIC_OVERRIDE = "static_override"
    CROSS_REFERENCE = "cross_reference"
    MANUAL_PROMPT = "manual_prompt"


class FallbackStep(BaseModel):
    source: str
    strategy: FallbackStrategy
    value: Optional[Any] = None

    @model_validator(mode="after")
    def _static_needs_value(self) -> "FallbackStep":
        if self.strategy is FallbackStrategy.STATIC_OVERRIDE and self.value is None:
            raise ValueError(f"static_override step '{self.source}' must define a value")
        return self


class FieldRoutingRule(BaseModel):
    primary_source: str
    confidence_threshold: float = Field(ge=0.0, le=1.0)
    fallback_chain: list[FallbackStep]
    escalation_target: str
    audit_flag: bool = True


class RoutingConfig(BaseModel):
    fallback_routing: dict[str, FieldRoutingRule]


class LeaseFieldPayload(BaseModel):
    lease_id: str
    field_name: str
    extracted_value: Optional[Any] = None
    confidence_score: float = Field(ge=0.0, le=1.0)
    source_system: str


# Strategy executors are injected so resolution stays pure and unit-testable.
# In production each one wraps a DB query or API call behind a circuit breaker.
StrategyFn = Callable[[LeaseFieldPayload, FallbackStep], Optional[Any]]


class FallbackRouter:
    def __init__(self, config_path: str, executors: dict[FallbackStrategy, StrategyFn]):
        with open(config_path, "r", encoding="utf-8") as fh:
            self.config = RoutingConfig(**yaml.safe_load(fh))
        self.executors = executors
        logger.info("Fallback routing configuration compiled (%d fields)",
                    len(self.config.fallback_routing))

    def resolve(self, payload: LeaseFieldPayload) -> dict[str, Any]:
        rule = self.config.fallback_routing.get(payload.field_name)
        if rule is None:
            logger.warning("No routing rule for field '%s'", payload.field_name)
            return {"status": "unmapped", "value": payload.extracted_value}

        # Primary extraction wins only if present AND above threshold.
        if payload.extracted_value is not None and payload.confidence_score >= rule.confidence_threshold:
            logger.info("Accepted primary for %s (score %.2f)",
                        payload.field_name, payload.confidence_score)
            return {"status": "accepted", "value": payload.extracted_value,
                    "source": payload.source_system}

        # Walk the chain in declared precedence order.
        for step in rule.fallback_chain:
            executor = self.executors.get(step.strategy)
            resolved = executor(payload, step) if executor else None
            if resolved is not None:
                logger.info("Fallback resolved %s via %s (%s)",
                            payload.field_name, step.source, step.strategy.value)
                return {"status": "fallback_resolved", "value": resolved,
                        "source": step.source, "strategy": step.strategy.value}

        # Chain exhausted: escalate with full context for the reviewer.
        logger.warning("Exhausted fallback for %s on lease %s -> %s",
                       payload.field_name, payload.lease_id, rule.escalation_target)
        return {"status": "escalated", "value": None,
                "lease_id": payload.lease_id, "field_name": payload.field_name,
                "target": rule.escalation_target, "requires_audit": rule.audit_flag}

The manual_prompt strategy is deliberately a no-op executor: it returns None, falls through, and the record reaches the escalation branch carrying its lease_id, field_name, and audit flag — exactly the context a reviewer needs. Every branch returns a discriminated status, so downstream code never has to guess whether a value is real, synthesized, or pending.

Edge cases specific to commercial leases

Real lease portfolios break naive fallback in ways generic data pipelines never see:

• Amendment overrides. inherit_if_valid must inherit from the governing version, not merely the chronologically latest document. A 2019 amendment can reinstate a term a 2021 letter agreement struck. Resolve precedence against effective dates in the lease data models layer before inheriting, never on filename order.
• Null versus zero versus absent. A percentage_rent_breakpoint of 0 is a real, negotiated number; None means the clause was unreadable. Coerce these distinctly at the normalization boundary — collapsing them sends a genuine zero-breakpoint lease down the fallback chain and overwrites it with a portfolio default.
• CAM reconciliation timing. A missing reconciliation date is the classic cross_reference win, because the tenant portal and the GL both carry it. But the two systems use different fiscal calendars; corroborate on the normalized ISO date, not the raw string, or a December-31 calendar year silently disagrees with a June-30 fiscal one.
• OCR drift on scanned riders. When a value is low-confidence because a scanned addendum shifted columns, the fix is upstream, not in routing. Tag these for OCR preprocessing re-runs rather than substituting a default, so a fixable extraction problem is not masked by a synthesized value. The mechanics of detecting that drift are covered in handling OCR drift and layout shifts.

When to escalate instead of substitute

Fallback substitution is the wrong move — and the chain should end in manual_prompt — whenever the field carries direct financial blast radius or the substitute cannot be attested. Concretely, escalate to a human queue when: the field is base rent, a CAM cap, a termination trigger, or any input to an escalation formula; the only candidate is a static_override and the lease’s asset class is an outlier; or a cross_reference returns a value that disagrees with the extracted-but-low-confidence figure, which signals a real conflict rather than a simple gap. Set a confidence threshold of 0.90 or higher on these fields so they reach the queue eagerly, and attach an SLA — 48 hours is a common commercial baseline — after which the escalation pings the asset-management lead. The point of the router is not to eliminate human review; it is to ensure humans only see the records where their judgment actually changes the books.

Frequently asked questions

Related

• Fallback Routing Logic — the parent stage that defines confidence gating and escalation across all lease fields.
• Metadata Normalization Standards — the ISO-8601 and typed-decimal contract that lets the router trust null-versus-zero distinctions.
• Handling OCR Drift and Layout Shifts — why low-confidence scanned fields route to re-extraction rather than substitution.
• Handling CAM Charge Variations in Lease Taxonomy Design — the recovery taxonomy whose reconciliation dates are a prime cross-reference target.

← Back to Fallback Routing Logic

← Back to Fallback Routing Logic