Fallback Routing Logic

Fallback routing logic is the component that decides where an incomplete lease record goes next — recalculated, re-extracted, exempted, or queued for a human — so that a single missing field never stalls an entire abstraction pipeline. This page sits inside Core Architecture & Lease Taxonomy, and it picks up exactly where validation ends: once a payload has been parsed and scored, the router reads which required fields are absent, malformed, or low-confidence and dispatches the record down a deterministic path instead of failing it outright. Commercial portfolios aggregate documents from legacy PDFs, broker spreadsheets, scanned addenda, and email threads, so missing metadata is the normal case, not the exception — and routing it intelligently is what keeps throughput high without flooding analysts with false alarms.

The specific challenge this page solves: when a rent commencement date, CAM reconciliation method, or escalation cap is missing, how do you decide — automatically and auditably — whether the gap is recoverable by computation, exempt by lease type, or genuinely requires human judgment? The answer used throughout production lease pipelines is a layered router: a schema layer establishes what must be present, a semantic layer reads the lease’s classification to decide what is contextually required, and a confidence gate diverts everything it cannot resolve into a review queue rather than guessing. Everything below is the engineering of that router.

Scope and where this fits

Routing is one stage in a longer chain, and keeping its responsibilities narrow is what makes it testable:

• Upstream of it: document ingestion, OCR preprocessing for scanned leases, clause classification, and field-level validation. By the time a payload reaches the router it should already carry a lease-type tag and per-field confidence scores.
• This stage: inspect required-field presence, attempt deterministic recovery, apply lease-type exemptions, and either accept the record or divert it — emitting an audit entry for every decision.
• Downstream of it: accepted records flow into lease data models and, for financial provisions, the escalation formula mapping engine; diverted records land in a manual-review dashboard or a dead-letter queue.

A router that also tries to parse an escalation rate or normalize a tenant name becomes impossible to reason about. It should answer one question — “given what is missing, where should this record go?” — and hand off everything else. The normalization it depends on is owned by the metadata normalization standards contract upstream.

Prerequisites and environment setup

The reference implementation targets Python 3.11+ and a deliberately small dependency set so the router stays fast and trivial to vendor into a worker image. It runs with zero third-party packages for the core decision logic; pydantic is used only at the output boundary to enforce the validated record contract.

Install with pip install "pydantic>=2.6" structlog. Two environment assumptions matter: each payload arrives already classified (carrying a classification_tag such as NNN or FSG) and already normalized to canonical types, so the router compares values, not raw strings. Routing rules — thresholds, required-field sets, computable pairs — should be externalized and version-controlled alongside the schema, so a rule change and the validation it implies move together. Compile that ruleset once at startup; never rebuild it inside the hot path.

Routing decision logic

Routing is a layered decision, not a single null check. The table below is the operational contract the implementation encodes — read it as the spec, the code as the proof.

The pattern that makes this robust is that no single stage is allowed to be authoritative. A computed value does not bypass validation; a lease-type exemption is applied only when the classification is trusted; and a record that survives every stage with fields still missing is diverted, never force-completed. That deliberate tension is what the layered design exists to manage.

Validation thresholds should be configurable per asset class. A Class-A multifamily portfolio may enforce strict date formatting and monetary precision, while a mixed-use industrial portfolio tolerates broader variance in common-area definitions. Semantic routing also accommodates jurisdictional variation: a triple-net (NNN) lease can legitimately omit landlord_maintenance_responsibility, whereas a full-service gross (FSG) lease cannot — so the same missing field routes differently depending on the classification tag the upstream stage attached.

Primary implementation

The router below is engineered for the lease domain: a required-field scan, a computational-fallback pass for derivable monetary fields, lease-type exemptions for structurally optional clauses, a diverting confidence gate, and a deterministic audit trail on every decision. The output is a pydantic-validated RoutedLease so no downstream consumer can receive a half-resolved record.

import logging
import hashlib
from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
from typing import Any, Dict, List, Optional, Set, Tuple

from pydantic import BaseModel, Field, field_validator

logging.basicConfig(level=logging.INFO, format="%(asctime)s | %(levelname)-8s | %(message)s")
logger = logging.getLogger("fallback_router")


class RoutingDestination(str, Enum):
    CALCULATION_ENGINE = "calculation_engine"
    CLAUSE_EXTRACTION = "clause_extraction"
    MANUAL_REVIEW = "manual_review"
    REJECT = "reject"  # dead-letter


@dataclass
class LeasePayload:
    lease_id: str
    raw_data: Dict[str, Any]
    classification_tag: Optional[str] = None          # e.g. "NNN", "FSG"
    field_confidence: Dict[str, float] = field(default_factory=dict)
    routing_history: List[Dict[str, Any]] = field(default_factory=list)


class RoutedLease(BaseModel):
    """The only shape a downstream consumer is allowed to receive."""
    lease_id: str
    destination: RoutingDestination
    reason: str = Field(..., min_length=3)
    confidence: float = Field(..., ge=0.0, le=1.0)
    resolved_via: str = "none"

    @field_validator("confidence")
    @classmethod
    def round_confidence(cls, v: float) -> float:
        return round(v, 4)  # keep audit arithmetic reproducible


class FallbackRouter:
    REQUIRED_FIELDS = {"base_rent_per_sqft", "lease_start_date", "rentable_area_sqft"}
    CONFIDENCE_FLOOR = 0.70  # a present-but-low-confidence field counts as missing

    # (numerator, denominator, derivable_target) — order encodes precedence.
    CALCULABLE_PAIRS: List[Tuple[str, str, str]] = [
        ("annual_base_rent", "rentable_area_sqft", "base_rent_per_sqft"),
        ("monthly_base_rent", "rentable_area_sqft", "base_rent_per_sqft"),
    ]

    # Fields a given lease class may legitimately omit.
    SEMANTIC_EXEMPTIONS: Dict[str, Set[str]] = {
        "NNN": {"landlord_maintenance_responsibility"},
    }

    def __init__(self, strict_mode: bool = False):
        self.strict_mode = strict_mode  # strict portfolios dead-letter instead of reviewing

    def evaluate(self, payload: LeasePayload) -> Tuple[RoutingDestination, str, float, str]:
        """Layered routing: scan -> compute -> exempt -> gate."""
        missing = self._missing_required(payload)
        if not missing:
            return RoutingDestination.CALCULATION_ENGINE, "All required fields present.", 0.99, "schema"

        # Stage 2: computational fallback for derivable monetary fields.
        resolved_via = "none"
        for numerator, denominator, target in self.CALCULABLE_PAIRS:
            if target in missing and numerator in payload.raw_data and denominator in payload.raw_data:
                try:
                    value = round(float(payload.raw_data[numerator]) / float(payload.raw_data[denominator]), 2)
                    if value <= 0:
                        raise ValueError("non-positive PSF")
                    payload.raw_data[target] = value
                    missing.discard(target)
                    resolved_via = "calculation"
                    logger.info("Computed %s=%.2f for lease %s", target, value, payload.lease_id)
                except (ValueError, TypeError, ZeroDivisionError):
                    logger.warning("Could not compute %s for lease %s", target, payload.lease_id)

        if not missing:
            return RoutingDestination.CALCULATION_ENGINE, "Missing fields resolved via calculation.", 0.92, resolved_via

        # Stage 3: semantic exemptions by lease classification.
        exempt = self.SEMANTIC_EXEMPTIONS.get(payload.classification_tag or "", set())
        if exempt:
            missing -= exempt
            if not missing:
                return (RoutingDestination.CLAUSE_EXTRACTION,
                        f"{payload.classification_tag} exemption applied.", 0.85, "semantic")

        # Stage 4: confidence gate — divert, never force-complete.
        if self.strict_mode:
            return RoutingDestination.REJECT, f"Strict-mode violation: {sorted(missing)}", 0.50, resolved_via
        return RoutingDestination.MANUAL_REVIEW, f"Missing critical fields: {sorted(missing)}", 0.50, resolved_via

    def route(self, payload: LeasePayload) -> RoutedLease:
        """Execute the decision, append a deterministic audit entry, and emit a validated record."""
        destination, reason, confidence, resolved_via = self.evaluate(payload)
        payload.routing_history.append({
            "timestamp": datetime.now(timezone.utc).isoformat(),
            "lease_id": payload.lease_id,
            "destination": destination.value,
            "reason": reason,
            "classification_tag": payload.classification_tag,
            "resolved_via": resolved_via,
        })
        logger.info("Routed %s -> %s | %s", payload.lease_id, destination.value, reason)
        return RoutedLease(
            lease_id=payload.lease_id,
            destination=destination,
            reason=reason,
            confidence=confidence,
            resolved_via=resolved_via,
        )

    def _missing_required(self, payload: LeasePayload) -> Set[str]:
        """A field is 'missing' if null OR present but below the confidence floor."""
        missing = set()
        for f in self.REQUIRED_FIELDS:
            if payload.raw_data.get(f) is None:
                missing.add(f)
            elif payload.field_confidence.get(f, 1.0) < self.CONFIDENCE_FLOOR:
                missing.add(f)
        return missing


if __name__ == "__main__":
    router = FallbackRouter(strict_mode=False)
    samples = [
        # Complete -> calculation engine
        LeasePayload("L-1001", {"base_rent_per_sqft": 32.50, "lease_start_date": "2024-01-01", "rentable_area_sqft": 5000}),
        # PSF missing but derivable -> calculation engine
        LeasePayload("L-1002", {"annual_base_rent": 195000, "lease_start_date": "2024-03-15", "rentable_area_sqft": 6000}),
        # NNN lease omitting an exempt field -> clause extraction
        LeasePayload("L-1003", {"base_rent_per_sqft": 28.00, "lease_start_date": "2024-06-01", "rentable_area_sqft": 4500},
                     classification_tag="NNN"),
        # Genuinely missing a required field -> manual review
        LeasePayload("L-1004", {"lease_start_date": "2024-09-01", "rentable_area_sqft": 3200}),
    ]
    for p in samples:
        result = router.route(p)
        print(f"[{result.destination.value.upper()}] conf={result.confidence:.2f} via={result.resolved_via} :: {result.reason}")

The final sample is intentionally unrecoverable: no computation can derive base_rent_per_sqft without a rent figure, no exemption applies, and the record is diverted to MANUAL_REVIEW rather than being shipped with a fabricated value. That behavior — failing loudly into review instead of quietly into the wrong financial posting — is the single most important property of a financial-grade router.

Validation and quality gates

Producing a routing decision is not the same as producing a trustworthy one. Three gates stand between the router and the rest of the platform.

Schema enforcement. Every output is a pydantic RoutedLease; the min_length on reason and the ge/le bounds on confidence make malformed decisions impossible to construct. A record whose serialized shape fails validation does not vanish — it goes to a dead-letter queue with its original payload intact so it can be replayed after a fix. The accepted shape must match the contract every downstream consumer parses against, defined in the lease data models layer.

Confidence gating and review routing. The CONFIDENCE_FLOOR is a circuit breaker, not a cosmetic field: a present-but-low-confidence value is treated as missing, so an OCR misread of an escalation cap never silently feeds automated billing. Strict portfolios flip strict_mode and dead-letter instead of reviewing; the cost is more replays, the benefit is zero ambiguous records reaching production.

Idempotency and dead-letter handling. Re-routing the same payload must produce the same decision and update the same record, not create a duplicate. Derive a deterministic id from a hash of the stable inputs rather than a random UUID, and key the dead-letter queue by that same id:

def deterministic_lease_id(document_id: str, version: int) -> str:
    digest = hashlib.sha256(f"{document_id}:{version}".encode("utf-8")).hexdigest()
    return f"LSE-{digest[:16]}"

def route_or_deadletter(router: FallbackRouter, payload: LeasePayload, dead_letter: list) -> Optional[RoutedLease]:
    try:
        return router.route(payload)
    except Exception as exc:  # pydantic ValidationError or upstream type errors
        dead_letter.append({"lease_id": payload.lease_id, "payload": payload.raw_data, "error": str(exc)})
        logger.error("Dead-lettered %s: %s", payload.lease_id, exc)
        return None

Routing thresholds and the computable-pair table should be version-controlled alongside the schema so staging and production resolve identical decisions, and so a labeled holdout set can be replayed before any threshold change ships.

Troubleshooting

Concrete failure modes that surface in real lease corpora, with the diagnostic and the fix.

• A present field is still wrong, so the router accepts garbage. A date parsed as 0001-01-01 is non-null, so the required-field scan passes. Diagnostic: log accepted records whose values fall outside sane ranges. Fix: attach per-field confidence and a range check upstream; the CONFIDENCE_FLOOR then treats the bad value as missing and diverts it.
• Computational fallback divides by a zero or string area. rentable_area_sqft arrives as "0" or "5,000" and the PSF computation raises or yields nonsense. Diagnostic: watch for the “Could not compute” warning spiking from one source. Fix: the try/except already guards division; enforce numeric coercion and thousands-separator stripping in the metadata normalization standards layer before routing.
• A lease-type exemption masks a genuinely required field. An FSG lease is mis-tagged NNN, so landlord_maintenance_responsibility is wrongly exempted. Diagnostic: audit exemptions whose source classification has low confidence. Fix: apply exemptions only when the classification tag itself clears a confidence threshold; otherwise treat the field as required.
• Strict mode dead-letters records that review could have salvaged. A new portfolio onboarded with strict_mode=True floods the dead-letter queue. Diagnostic: a reject-rate spike on first ingest of a source. Fix: default new sources to review, promote to strict only after the field-completeness rate stabilizes.
• Duplicate records from retried batches. A network retry re-routes the same payload and a random UUID creates a second record. Diagnostic: two routing-history entries for one document version. Fix: derive the id with deterministic_lease_id so a retry updates the existing record instead of duplicating it.
• Amendment overrides a field the router still reads from the base lease. An amendment supplies a corrected commencement date but the base payload is routed. Diagnostic: check for an amendment effective-dated after the base document. Fix: the router labels destinations; let the lease data models layer resolve amendment precedence — do not encode override logic here.

Performance and scale notes

For portfolio-scale runs the router itself is rarely the bottleneck — a required-field scan and a couple of divisions are microseconds — but the surrounding orchestration is. Keep the router stateless and compile its ruleset once at startup so each worker pays the cost only at import, then fan it out freely. For multi-thousand-document batches, push routing into the same worker fabric described in async batch processing: stream payloads rather than loading whole portfolios into memory, cap in-flight tasks to bound memory, and make routing endpoints idempotent so an orchestration retry never double-processes. Let transient failures retry through the platform’s error handling and retry logic instead of being swallowed inside route. Because the decision is pure and deterministic, results cache cleanly keyed on the deterministic lease id, so re-running an unchanged document version skips work entirely. Where payloads carry tenant-identifying data, enforce the isolation rules in security and access boundaries before fanning records out to shared review queues. For field-by-field configuration of the hierarchy itself, see implementing fallback routing for missing lease metadata fields.

Frequently asked questions

Related

• Implementing Fallback Routing for Missing Lease Metadata Fields — field-by-field configuration of the routing hierarchy, queue management, and dead-letter handling.
• Clause Classification Systems — the upstream stage that tags each clause and diverts low-confidence spans into this router.
• Lease Data Models — where accepted records land and how amendment precedence is resolved.
• Escalation Formula Mapping — the financial engine that consumes records once their required fields are present.
• Metadata Normalization Standards — the canonicalization contract the router relies on before it compares values.

← Back to Core Architecture & Lease Taxonomy

← Back to Core Architecture & Lease Taxonomy