← All projects

LeagueFinder

In development

Answers one question for an adult who wants to go do something: what can I join near me, when does it meet, and where do I actually sign up?

Web AppDjangoPostGISData Provenance

At a glance

  • Outcome: A directory of organized adult recreation where every published fact traces to the evidence it came from. The part that’s built is the hard part: source bytes are retained, each field value becomes an assertion carrying its own evidence and valid time, and versioned publication contracts decide whether that value is fit to publish. The participant-facing surface comes last on purpose, since a directory that can’t prove it’s current is just a slower search engine.
  • Status: In development. Portland–Vancouver is the chosen pilot market. Nothing is live: there’s no serving projection, search page, or activity page yet, parsing currently reads one field from one source, and no collection runs on a schedule.
  • Role: Solo.
  • Stack & libraries: Django 5.2 LTS as a modular monolith on PostgreSQL/PostGIS, httpx plus lxml/selectolax for collection, content-addressed evidence on a local root keyed the way an object store will key it, and Docker images built by GitHub Actions to GHCR. Celery and Redis are configured and the worker runs, but no task is defined anywhere in the codebase yet. Browser collection is a planned, permission-gated worker: Playwright is not a dependency and browser_allowed is false on all 83 source manifests.
  • Limitations: Not launched, and the two scoreboards are deliberately kept apart. Source review is finished; participant coverage is zero. One source has ever been collected by code, and no coverage cell has been defined, audited, or measured.

The actual problem

Finding an adult rec league is worse than it should be. The information exists, but it’s scattered across organizer sites, half of it is last season’s, and the listing sites that aggregate it don’t tell you which parts are stale. You find a league, get excited, and discover registration closed in March.

The thing being indexed is the organization layer above facilities. Not that a court, gym, field, rink or trail exists, but that an identifiable organizer runs a joinable activity there. Plenty of maps will show you a gym. None of them tell you there’s a Tuesday-night volleyball league in it.

So the durable thing here isn’t the website. It’s the maintained graph underneath: organizers, activity series, offerings, occurrences, venues, registration windows, the observations each fact came from, and the review decisions made about them. The public site is a projection of that graph.

What counts as one thing you can join

The central record is an activity offering: the thing a participant can actually select. Successive seasons share an activity_series, a dated session is an occurrence, and an independently selectable division becomes a leaf offering under at most one container, never a separate kind of thing. Otherwise “the thing you select” would sometimes be an offering and sometimes a division, and every search and review query would have to branch on which.

The rule was written before any inventory existed, with the thresholds that would reopen it stated up front. It was measured against 21 leaf offerings hand-annotated from nine real program pages: 0 of 4 containers had a single child (reopen above 10%), and the busiest organizer-season had 10 leaves (reopen above 40). Neither trigger fired, so the rule stood.

The worked example written before collection got the shape right and the breadth wrong. It modeled two volleyball divisions where the real season has five.

Provenance is one table

Every field value is a row in one append-only assertion table: one typed value column, an authority class, a valid period, and an expiry.

Seeing the same value again doesn’t mint a second claim, because the database permits only one open claim per value. It adds a support row instead. So the assertion count grows with real change rather than with how often the collector runs. Replaying a parser stamps its own clock on the replay and leaves observed_at with the original fetch, so a claim is never fresher than the evidence behind it and re-parsing old bytes can’t make stale data look current. A correction supersedes rather than overwrites, and the row it replaced stays readable.

The alternatives were a table per value kind and a shadow table per entity. Both were rejected for the same reason: publication, conflict detection, expiry, reviewer explanation, and projection would each have to be written once per kind or once per table, and that logic is the product.

What the code enforces today

  • Every published field traces back to evidence, enforced in the data rather than by convention.
  • Expired or contradicted evidence fails closed. An offering whose supporting evidence has expired has its occurrences withheld rather than cancelled, and an unresolved contradiction between sources suppresses the record instead of picking a winner.
  • Lifecycle, registration, verification, and publication are separate states, so “we know this exists” never gets confused with “you can sign up today.”
  • Registration and payment stay on the organizer’s own site. This sends people to the real thing; it doesn’t sit in the middle of it.

What the release gates still require

Stating these as gates is what makes the list above believable.

  • Nothing yet notices staleness on its own. There’s no verified_at column and nothing schedules a revisit; recency is measured absolutely, so a 2023 season page isn’t yet stale merely for being about 2023; and a run isn’t compared against its predecessor, so count drift, field drift, and a listing that quietly disappears from a source all go undetected. “Goes stale loudly” is the gate, not current behavior.
  • Coverage and freshness have to be measured before the pilot expands anywhere. No coverage cell has been defined yet.

The access review

This is the largest finished piece of work in the repo, and it’s the unusual one: most people scrape first and read terms never.

All 83 registered sources have a dated attempt report and none is still queued: 39 open for collection, 31 needing permission, 4 manual-records routes, 8 blocked outright, 1 implemented. The eight are blocked because the vendor’s terms say so: several major league-management and event platforms each prohibit systematic retrieval, and a number of league sites turned out to be tenancies of those platforms on custom domains.

The honesty clamp matters here and the repo states it plainly: thirty-nine access reviews found no obstacle, not thirty-nine collectors. One source has ever been collected from by code.

Why the scope is deliberately narrow

The pilot is one metro area and one audience: someone who wants to participate, not run a league. Expansion waits until the pilot can actually measure its own coverage, freshness, and what it costs to keep accurate. A directory that can’t prove it’s current is just a slower search engine.