In 001, 168 of 169 trip searches returned zero trips. This window, the prod chat-path
search (trips.searched, the MCP search-trips tool) ran 458 times and
returned trips on 323 of them — a 29.5% zero-result rate. The remaining zeros are largely
real no-service corridors (Tampa→Pittsburgh, Minneapolis→Portland ME, a Philippines origin),
not the integration bug. And the funnel now reaches a trip selection: Trip Selected
fires 73 times, every one carrying the canonical trips[] array, batch_id, and a
results_count — the proposed results-stage instrumentation from 001 landed.
The chat path (destination_selected → trips.searched → Trip Selected) works.
The legacy classic-search path (Search Submitted → search.completed, pservs-backed,
VPC-bound) still fails: in prod the only search.completed events are the once-a-day
synthetic Boston→NYC canary, and it still returns 0 trips with 7/7 carriers failing.
Real users barely touch that path (Search Submitted origin resolves 2/101) — the win came from the
chat path resolving origins to a nearest hub instead. The classic path is now a monitoring artifact, not a user-facing fire.
Two P1s from 001 shipped. Date defaulting collapsed from 99% to ~37% — high-confidence extraction ("June 15th", "Wed, Jul 15", "your pick") is live, and flexible windows ("this weekend", "Weekend in October") now parse. And Chiku now surfaces 63 distinct destinations (up from 16), reaching beyond the NE corridor into DEN, NOL, BNA, SAV, AUS, PDX, SLC, STL.
Still Haiku 4.5, 1 hallucination in 837 prod prompts (and that one was on the old Jun-8 build). Cost rose to ~$0.008/call (longer context / tool use), latency ~2.4s avg with a slow tail (56 calls >3s, 20 >5s, worst 8.9s). Still the cheapest, most reliable part of the stack — do not optimize the model layer.
001 closed with seven "Next Check-in" questions. Each is answered below with the data — five are now answered, two remain open. Volumes are an order of magnitude larger than 001 (766 prompts, 458 prod searches) but the window is still ~7 days — treat percentages as strong-directional.
| # | Question carried from 001 | Answer this window |
|---|---|---|
| 1 | Did the zero-result search get fixed? Was origin resolution the cause? | Yes — fixed, and yes. Prod zero-result rate fell from 99.4% to 29.5% (135/458). The fix
was on the chat path: origins now resolve to a nearest hub on 102/445 destination picks (23%, up from 19%),
and where they do, trips.searched returns real trips. Origin resolution was the cause, as the 001 payloads predicted. |
| 2 | Is the resolved-origin signal on the search events, so the gap is visible in Amplitude? | Partial. Search Submitted now carries depart_wcityid + date_provenance,
and destination_selected carries origin_wcityid + nearest_hub_wcityid — so the gap is
now measurable in Amplitude (it's 24/445 resolved directly, 102/445 via nearest hub). No single origin_resolved
boolean yet, but the substance is there. |
| 3 | With search working, what's the first real conversion read? | First read available. Trip Selected (73) now fires with trips[] + batch_id +
results_count. Funnel: 766 prompts → 645 suggested → 445 selected → 458 prod searches → 73 trip selections.
Suggestion→selection is ~69% of prompts; trip selection is the new floor of the conversion read. The batch_id is the join key into booking. |
| 4 | Did origin-aware destinations broaden the suggestion set beyond 16 cities? | Yes. 63 distinct cities suggested (was 16). The NE-corridor monoculture is gone; DEN, NOL, BNA, SAV, AUS, PDX, SLC, STL all appear. |
| 5 | Are the three legacy labs events confirmed retired? | Confirmed. Chiku Message Sent, Chiku Destinations Lit, and
Inspiration Origin Detect Failed did not fire once this window. The labs inventory is now 7 clean canonical events. |
| 6 | Any movement on rail / nearest-stop asks? | Not yet — still unmet. Rail/Amtrak mentions persist (~34 prompts incl. one elaborate "go to Amtrak.com fare-finder" agentic request) and nearest-stop asks (~29). Neither has a dedicated capability. These carry forward as P1s. |
| 7 | External-LLM referrer / MCP traffic (KU #13)? | First trace appeared. chatgpt.com shows up as a utm_source for the first time (4 page views).
Tiny (≈0.3%), but it's the first evidence of the inspiration moment arriving from an agent. Worth watching; not yet a trend. |
| Finding | Class | Framework tie |
|---|---|---|
| Chat-path search returns trips; funnel reaches Trip Selected | core-loop-working | KU #12 (explore conversion) — now measurable for the first time |
| Date extraction live; flexible windows parse | core-loop-working | KU #1 (date certainty); Pattern A (flexible dates, fixed destination) |
| 63 distinct destinations | core-loop-working | Axis: breadth — origin-aware widening, the inspiration thesis |
Classic Search Submitted origin still 2/101 | data-quality-gap | The classic-search handoff is still the weak path; chat path is the live one |
| Rail (~34) + nearest-stop (~29) unmet | unmet-need | KU #6 (mode anchor); a discovery not in the original 100 scenarios |
| Synthetic Boston→NYC 0-trips, 7/7 carriers fail | system-health | pservs/VPC path — Datadog-only, monitoring artifact |
| chatgpt.com referrer (4) | unmet-need / signal | KU #13 (inspiration moment migrating to agents) — first datapoint |
Downstream booking conversion (the batch_id join key exists but we haven't joined Trip Selected → Checkout →
Transaction yet), revenue, the build-vs-MCP-primary decision (KU #13 is one referrer, not a pattern), and Spanish-language /
accessibility quality. The classic-search path's real-world zero-rate is also still unmeasured (only the synthetic monitor exercises it).
The single most important thing this window is why the zero-result rate collapsed. It's not that one search got fixed — it's that the product leaned into the path that resolves origins and away from the one that doesn't.
In 001 the diagnosis was: origins arrive as raw free text ("700 Atlantic Ave", "Tucson", "Berlin, Germany") and
never resolve to a Wanderu city, so /api/trips returns nothing. This window, the chat path
(destination_selected) carries a nearest_hub_wcityid on 102 of 445 picks
(23%, up from 30/160 = 19%) and a directly-resolved origin_wcityid on 24 of 445 (up from 3/160).
Where the origin resolves, the Datadog trips.searched call returns trips — 323 of 458 prod searches now do.
The legacy Search Submitted → search.completed path (pservs-backed, runs from VPC-bound pods) still fails.
Every prod search.completed event this window is the once-daily synthetic canary
BOSSST → NYCPAS, and it returns 0 trips with all 7 carriers failing
(failed_carrier_count: 7, ok_carrier_count: 0) — the same Boston→NYC signature flagged in 001.
Real users almost never traverse it: Search Submitted resolves depart_wcityid on just
2 of 101. So the carried-forward "Boston→NYC returns zero" is now isolated to the synthetic test path,
not the live experience. It still needs fixing (a monitor that's always red is noise), but it's no longer the headline blocker.
With the bug cleared, the 135 zero-result prod searches read as genuine sparsity: Tampa→Pittsburgh, Minneapolis→Portland ME, Texarkana→Dallas, Waterloo IN→Chicago, and a literal Coron, Palawan, Philippines origin. These are the right kind of zero — the system correctly says "no bus there" rather than silently failing on a dense corridor. This is the conversion denominator finally meaning something.
"Go to Amtrak.com and use the fare finder to find the CHEAPEST weekend round-trip from New York to Boston… report the 3–4 cheapest weekend combinations… Do NOT log in or book." — a user trying to use Chiku as a booking agent for rail
"i'm from pittsburgh, and i want quieter no city" — vibe + origin anchor, exactly the inspiration loop
"list major cities 2-3hrs away" — the nearest-reachable ask, still unmet (and one of the 8.9s slow calls)
The Amtrak fare-finder prompt is qualitatively new — a user expecting Chiku to act on an external site, not just
suggest. Combined with the ~34 rail mentions and the chatgpt.com referrer, it sharpens the LLM-integration question:
users increasingly arrive expecting agent behavior. We can't serve rail today, and silent failure is still the worst outcome.
source = labs viewFirst-party Amplitude (source = labs), window since 001. System outcomes (zero-result rate,
latency, cost) are Datadog by design and live in the What We Found tab and the
system block below. Event definitions are at the bottom of this tab.
Explore Page Interacted by interaction_type + the new trip stages)The chain that broke at steps 7–8 in 001 (Search Results Viewed / Trip Selected were Missing)
now reaches a real trip selection (73), each carrying trips[] + batch_id + results_count.
This is the measurement spine 001 asked for. trips.searched (458, Datadog) sits between selection and trip-pick because
the search runs server-side.
766 prompts by primary theme (regex over prompt_text; categories overlap). Teal = core
loop · orange = unmet need.
Vibe/inspiration and budget lead — the wide-axis "surprise me, cheaply" core holds. Rail (34) and nearest-stop (29) remain the two largest unmet intents, both carried from 001 and both still without a capability.
Steady ~120–170 visitors/day, ~1/3 prompting Chiku — a stable surface, not the launch spike of 001. *Jun 15 is a partial day.
Multi-turn deepened (45% of prompts are follow-ups, up from 39%) — conversational refinement is working. Desktop share grew (31% vs 21% in 001), so the surface is no longer phone-only; design should not assume mobile-exclusive.
Origin resolution improved on the chat path (24 direct + 102 nearest-hub of 445) but the classic
Search Submitted handoff still resolves only 2/101 — the divergence the two-path story explains.
Date intent is now extracted: explicit high-confidence dates plus parsed relative windows ("this weekend") have pushed the low-confidence
default down to ~37%.
The newly-unlocked cities resonate hardest once shown — Nashville (54%), Atlanta (45%), Denver (40%), New Orleans (28%) beat the always-suggested NE-corridor staples. Boston (20%) and Montreal (15%) are still over-suggested relative to picks, same as 001. The broadened seed list is pulling its weight; the suggestion ranker could lean further into the high-convert newcomers.
All 445 destination picks carry pick_source: "chat_card"; the map-pin path was used 0 times again.
Two windows in a row of zero map commits on a map-first surface — this is now a real signal for the chat-vs-structured-browse question,
not a fluke. The constraint_edited affordance did climb from 1 to 12 firings, so the chips are being discovered, just lightly.
service:next-prototypes, env:prod)| Metric | This window | vs 001 |
|---|---|---|
Prod trips.searched (chat-path) | 458 | ▲ from 169 |
| Zero-result rate (chat-path) | 29.5% | ▼ from 99.4% |
Prod llm.prompted | 837 | ▲ from 334 |
| Model | Haiku 4.5 | unchanged |
| Cost / call | ~$0.008 | ▲ from $0.0038 |
| Window LLM spend (prod) | ~$6.70 | — |
| Hallucinations | 1 / 837 | ~0 (held) |
| Latency >3s / >5s (worst) | 56 / 20 (8.9s) | slow tail persists |
| Errors (experiment fetch / wapi / mcp) | ~15 total | minor, self-resolved |
Synthetic search.completed (BOS→NYC, 0 trips) | 9 (1/day) | still red — pservs path |
The labs event inventory this window. Live firing · Gap live but incomplete · Auto autocapture.
| Amplitude event | window | First seen | Status / maps to |
|---|---|---|---|
Explore Page Interacted | 1,472 | (carried) | Canonical umbrella ✓ |
[Amplitude] Page Viewed | 1,298 | auto | Auto-tracked |
LLM Prompted | 766 | (carried) | Canonical ✓ — ≈ Datadog llm.prompted (837 prod) |
session_start | 108 | auto | Auto-tracked |
Search Submitted | 101 | (carried) | Canonical ✓ — classic handoff; depart resolves 2/101 |
Trip Selected | 73 | NEW | Canonical ✓ — carries trips[] + batch_id + results_count |
session_end | 49 | auto | Auto-tracked |
The three legacy events (Chiku Message Sent, Chiku Destinations Lit,
Inspiration Origin Detect Failed) are gone — zero firings this window. New interaction_type
values date_edited (24) and constraint_edited (12) joined the umbrella. The funnel reaches Trip Selected;
the remaining missing rung is joining Trip Selected → booking via batch_id (the events fire on canopy/ui-react downstream).
Each 001 priority is marked done / ongoing / dropped and re-ranked by what now unblocks the most learning and value. Filter by priority or owner; click a card to expand.
NEW / promoted. Now that Trip Selected fires (73) with a batch_id and a
trips[] array, the single highest-value next step is to complete the join into booking:
Trip Selected → Checkout Started → Transaction Completed (those fire downstream on canopy/ui-react). This turns the funnel
from "search works" into the first true explore-conversion-to-revenue number — the thing every prior check-in has been blocked on.
batch_id propagates from labs through the booking handoff and is queryable on the downstream events.Trip Selected carries results_count but no trip_id (0/73) — add it so individual trips join, not just batches.Ongoing — promoted. Rail is now the largest unmet intent (~34 prompts), and qualitatively escalating: one user wrote a full agentic brief asking Chiku to "go to Amtrak.com and use the fare finder" and report cheapest weekend round-trips. Either give Chiku Amtrak routing/data or have it state clearly what it can and can't do for rail. Silent failure remains the worst outcome, and the agentic framing suggests rising expectations we're not meeting.
Ongoing. Still the second-largest unmet intent (~29: "list major cities 2-3hrs away", "closest to Shelby nc", "nearest places to austin"). Answerable from places/stations data we already have — likely a straightforward Chiku tool call. It also overlaps the origin-resolution win: users asking "what's near me" is the same anchor the nearest-hub fix already computes.
Ongoing — was P0 "fix zero-result search", now de-escalated. The zero-result bug is fixed (99.4% → 29.5%), but origin resolution is still the cap on how low the zero-rate can go: only 24/445 picks resolve a city directly and 102/445 via nearest hub, leaving ~70% of picks leaning on whatever default the search uses. Push direct + nearest-hub resolution higher so more searches run from a real station — that's where the remaining non-genuine zeros live.
NEW / de-escalated from 001's P0. The legacy Search Submitted → search.completed path (pservs, VPC-bound)
still returns 0 trips with 7/7 carriers failing — but in prod the only traffic on it is the once-daily synthetic Boston→NYC canary.
Real users go through the chat path. So this is no longer a user-facing fire; it's a monitor that's permanently red (noise that masks real alerts).
Either point the canary at the working chat path, or fix the pservs dependency so the classic handoff resolves. Lower stakes than the conversion join.
Mostly done — de-escalated from P1. Date defaulting fell from 99% to ~37%. Explicit dates ("June 15th", "Wed, Jul 15")
and relative windows ("this weekend", "Weekend in October") now parse with high/medium confidence. Remaining work is the long tail:
"between now and july thirtieth", "the end of the month", and true cheapest-day search across a range (Pattern A). Verify the parsed date
actually drives the trips.searched call rather than just being logged.
Ongoing — sharpened. All 445 picks this window (and all 160 in 001) came from the chat card; the map-pin path has 0 commits across both check-ins. On a map-first surface this is now a real finding: either the map is decorative and the chat is the product, or the map's commit affordance is invisible. This bears directly on the chat-primary vs. structured-browse question in the LLM-integration doc. Worth a design look and an explicit decision, not just a log note.
Ongoing. Still stakeholder-desired, still not evidence-led — but the slow-latency tail strengthens the perceived-performance
case: 56 prod calls ran >3s and 20 >5s (worst 8.9s) this window. A motion treatment on the wait directly improves those slowest turns.
Pair with animating the destination reveal (645 destinations_suggested) to reinforce the "surprise me" moment. Respect
prefers-reduced-motion; keep behind the conversion join.
Volume on misdirected support dropped; off-network grew. Still a graceful-boundary problem, not a fire.
NEW. chatgpt.com appeared as a utm_source for the first time (4 page views), and a user wrote an
agentic Amtrak-fare-finder brief. Both are tiny, but together they're the first whisper of the inspiration moment migrating to agents — the
build-vs-MCP-primary question (KU #13). Keep a standing referrer watch (chat.openai.com, perplexity.ai, gemini, copilot) so we catch the trend
if it forms. Don't over-rotate on N=4.
DONE — confirmed. All three legacy emitters (Chiku Message Sent, Chiku Destinations Lit,
Inspiration Origin Detect Failed) fired zero times this window. The labs inventory is now 7 clean canonical events.
Leaving here marked done for the audit trail; drop in 003.
Ongoing — low signal. No fresh "I don't want the AI" quotes surfaced in this window's sample, but the 0-map-commits finding is the structural version of the same question: is there a plain browse path for people who don't want to chat? Folded conceptually into the map-usage investigation above. Cheap escape-hatch idea from 001 still stands; not urgent.
batch_id join land? What's the first explore → checkout → transaction conversion read off Trip Selected?trip_id get added to Trip Selected so individual trips (not just batches) join into booking?chatgpt.com trace (KU #13)?Chiku LLM layer is healthy. Haiku 4.5, ~$6.70/window, ~$0.008/call, 1 hallucination in 837 prod prompts. Cost-per-call rose vs 001 (longer context / tool use) but is still trivial. Slow tail worth a perceived-performance treatment (see P2 animation), not a model change.
Errors are minor and self-resolving — ~7 Amplitude experiment-variant fetch timeouts, a Jun-10 503 blip, 2 WAPI predictions failures, 3 mcp.no_structured_content. The 001 WAPI auth blip did not recur.