The prototype is a working in-product conversational search surface (Surface 1 from the LLM-integration doc), doing exactly the job we designed it for: people arrive, set an origin, and ask open-ended "where can I go" questions. Haiku is cheap, fast, and effectively never hallucinates. Do not spend effort optimizing the model layer.
Only Chicago→Detroit (5 trips) came back with results. Boston→NYC returning empty 9 times is not
a real-world result — that's the densest intercity corridor in the country. Amplitude points
at the cause: the origin almost never resolves to a Wanderu city (0/48 on Search Submitted,
3/160 on destination picks) while the destination always does — see the
Amplitude tab. Until it's fixed we cannot measure conversion,
validate revenue, or rank any candidate surface. Everything else is secondary.
The prototype produced our first first-party natural-language prompt distribution (known unknown #14). We now know what users actually type when handed a freeform box — see the Intents → model tab.
Each finding tied back to the questions Round 1 was trying to answer. Volumes are small (315 messages, 169 searches) — treat every percentage as directional.
| KU | Question | What the logs say now |
|---|---|---|
| #14 NL input distribution | What do users type when freed from the form? | Answered (directionally). Origin-setting dominates (~140/315), then vibe (~26) + budget (~19) + date (~24). The "anchor + open range" shape is confirmed in the wild. |
| #15 Model performance | Where do models hallucinate / mis-route / refuse? | Strong signal. 1 hallucination in 334; ~0 off-list destinations. Failure mode is capability gaps (rail, station detail), not bad generation. |
| #3 Destination certainty | Fixed vs open destination? | Most messages are destination-open ("somewhere warm", "anywhere", "more"). Validates the inspiration thesis: people want the system to pick. |
| #6 Mode/access anchoring | How many need a specific mode? | Rail is a real anchor — ~14 Amtrak/train messages, one user rephrased 4× (Chiku failed the task). |
| #5 Budget anchoring | Do users shop with a budget cap? | Yes — ~19 budget-led ("cheapest", "round trip under 50"). Budget is a first-class entry anchor. |
| #2 Origin choice | Current location vs other? | 34 IP-detected origins vs ~140 origin-setting messages → users overwhelmingly type their own origin rather than accept geolocation. |
| #12 Explore conversion | What's the conversion rate? | Unmeasurable — search returns nothing, so there is no conversion to measure yet. Blocked on the zero-result fix. |
Conversion, revenue, the build-vs-MCP-primary question (#13 — no external-LLM referrer data here), or Spanish-language / accessibility quality. Those remain open.
315 production messages by primary intent. Teal = core loop working as designed · orange = real unmet need · grey = out of scope / misdirected.
| Live intent | Pattern | Anchor | Read |
|---|---|---|---|
| Setting a location (~140) | precursor to all | Origin only | Base-page premise holds: people lead with origin, then open everything else. |
| Vibe / inspiration (~26) | B / C | Origin only | Wide-axis "surprise me" — the heart of the inspiration page. |
| Budget-led (~19) | C / E | Budget | Budget-anchor entry warranted (slider, not buried field). |
| Date / timing (~24) | A / B | Calendar | Date flexibility is real; "in August" / "today" both appear. |
| Vague follow-ups (~24) | multi-turn | — | Conversational refinement is happening — "More", "Anywhere". Multi-turn state must hold. |
| Nearest-stop / station (~16) | n/a | — | New, unmet. Not in the 100 scenarios — a discovery from real traffic. |
| Train / Amtrak (~14) | E | Mode/access | Mode anchor confirmed; currently unserved. |
| Off-network / intl (~10) | n/a | — | Scope-boundary questions; needs a graceful "here's our coverage" answer. |
| Logistics + support (~16) | n/a | — | Misdirected; needs handoff, not a product surface. |
Nearest-stop / boarding detail (~16) wasn't a first-class job in the 100 scenarios, but it's the 6th-largest intent in real traffic. Worth adding to the scenario set.
"Hello I'm waiting on my bus … haven't seen no Wanda bus where is my bus" — lost passenger, no handoff
"Just give me the options like old pen I don't wanna see the ai" — the anti-AI minority
source = labs viewWhat real users did on the explore surface, from first-party Amplitude events
(source = labs, last 30 days). Event definitions and funnel coverage are at the
bottom of this tab — most readers can stay up here.
LLM Prompted (326) ≈ Datadog llm.prompted (334). Manual origin entry
(origin_set_from_autocomplete, 54) > geolocation (use_my_location_clicked, 37),
confirming KU #2: users type their own origin. And LLM Prompted carries
prompt_text + turn_number — so the intent distribution and multi-turn behavior
are reproducible from Amplitude going forward, not just a one-off log scrape.
Explore Page Interacted by interaction_type)The suggestion→selection loop works (293 → 160, and all 160 picks were Chiku
suggestions). Most "trip searches" the Datadog report counts originate here — from
destination_selected (160) — far more than the classic Search Submitted
handoff (48). The constraint_edited affordance fired once in 30 days —
effectively undiscovered. UX flag.
The prototype is effectively ~5 days old — 514 of the 506 uniques landed Jun 4–8. Read every "30-day" number through that lens. *Jun 8 is a partial day.
utm_source on page viewsBy referrer, 582 arrivals came from within www.wanderu.com — explore
is reached by navigating the main site, not as a direct landing. No LLM/chat referrers appear yet
(ChatGPT, Perplexity, etc.), so KU #13 (the inspiration moment migrating to agents) is ≈ 0 on this surface today.
90% of prompts produce destination suggestions and ~half lead to a pick — the
suggestion→selection loop is healthy. The drop to Search Submitted is partly real
hand-off attrition and partly that the event was only instrumented Jun 4.
Mobile-first, conversational (4 in 10 prompts are refinements), and two stark data-quality gaps that drive the experience problems: the origin almost never resolves to a Wanderu city, and the search date is almost always a low-confidence default rather than what the user asked for.
Chiku surfaces Chicago / NYC / Miami most often, but Providence (41%) and LA (31%) resonate hardest once shown, while Miami and Boston are over-suggested relative to what users pick. A signal for tuning the suggestion set.
All 160 destination picks came from the chat card (pick_source: "chat_card"); the map-pin
path was used 0 times. On a surface built map-first, the map isn't where users commit —
a data point for the chat-vs-structured-browse question.
On Search Submitted, the destination resolves to a Wanderu city ID 100% of the
time (48/48) but the origin resolves 0% (0/48). On destination_selected, only
3 of 160 carried a resolved origin_wcityid, and nearest-hub substitution
fired on just 30 of 160. Origins arrive as raw free text — "700 Atlantic Ave, Boston",
"Tucson", "Berlin, Germany", even "Cortina d'Ampezzo, Italy".
Leading hypothesis: the trip search runs without a resolved origin station, so
/api/trips returns nothing. (Amplitude doesn't log the search result — that's
Datadog by design — so this needs confirming, but ~98% unresolved origins vs ~99% empty results is a
tight correlation with an obvious mechanism.)
trips array — the explore funnel never sees its resultsZero of the 9 labs events carry the canonical trips array (or f_pricing).
That's correct for these pre-results events — but the explore surface has
no results-stage event at all, so what trips a search surfaced (or that it surfaced
none) lives only in Datadog, and the explore funnel can't join the booking funnel.
Proposal: reuse the canonical Search Results Viewed (with the
trips array + a required origin_resolved flag, page_type: "explore")
so the zero-result rate becomes an Amplitude metric and trips join via batch_id / trip_id.
See proposals/explore-trips-surfacing.md →
inspiration_base, now explore):
Chiku Message Sent → LLM Prompted; Chiku Destinations Lit
("Lit" is UI-mechanism naming, an anti-pattern) → Explore Page Interacted [destinations_suggested];
Inspiration Origin Detect Failed — a failure event that belongs in Datadog, not
Amplitude. Confirm all three emitters are gone.depart_wcityid (or an
origin_resolved boolean) to Search Submitted / destination_selected so
the origin-resolution gap is visible in Amplitude — today it's only inferable.Search Submitted instrumented Jun 4 — the Amplitude search funnel is only
~4 days deep; Datadog holds the full 30. Treat Amplitude search counts as a short window until it matures.amplitude_user_id is null on all labs events; use
device/amplitude ID for unique-user counts.Reference for the analytically inclined — the labs event inventory and where the funnel is (and isn't) instrumented.
| Amplitude event | 30d | First seen | Status / maps to |
|---|---|---|---|
[Amplitude] Page Viewed | 678 | May 29 | Auto-tracked |
Explore Page Interacted | 634 | Jun 2 | Canonical umbrella ✓ |
LLM Prompted | 326 | Jun 2 | Canonical ✓ — ≈ Datadog llm.prompted (334) |
session_start | 60 | May 29 | Auto-tracked |
Search Submitted | 48 | Jun 4 | Canonical ✓ — instrumented late |
session_end | 36 | — | Auto-tracked |
Chiku Destinations Lit | 4 | May 29 | Legacy — retired Jun 1 |
Chiku Message Sent | 4 | May 29 | Legacy — retired Jun 1 |
Inspiration Origin Detect Failed | 3 | May 29 | Legacy — retired Jun 1 |
Live firing today · Gap live but incomplete · Missing not instrumented · Auto Amplitude autocapture.
| # | Event | What it captures | 30d | Status |
|---|---|---|---|---|
| 1 | [Amplitude] Page Viewed | Landing on /explore. Carries referrer + UTM — the entry-attribution signal (KU #13) | 678 | Auto |
| 2 | Explore Page Interacted· origin_set_from_autocomplete / use_my_location_clicked | User sets origin — manual (54) vs geolocation (37). Setup step, not forward progress | 91 | Live |
| 3 | LLM Prompted | User asks Chiku. prompt_text, turn_number, origin_known — the intent + multi-turn signal | 326 | Live |
| 4 | Explore Page Interacted· destinations_suggested | Chiku surfaces destinations on the map. suggested_wcityids, suggested_count | 293 | Live |
| 5 | Explore Page Interacted· destination_selected | User picks a destination → kicks off a trip search. The explore conversion signal. But date is defaulted 99% of the time | 160 | Gap |
| 6 | Search Submitted | Classic-search handoff (O-D, date, pax). Origin never resolves to a city ID (0/48) | 48 | Gap |
| 7 | Search Results Viewed | Trips surfaced. Proposed: trips[], results_count (incl. zero), origin_resolved — makes the zero-result rate an Amplitude metric | — | Missing |
| 8 | Trip Selected | User picks a surfaced trip. Proposed: trips[] + batch_id — joins the explore funnel to booking via Hold-Constant | — | Missing |
| 9 | Checkout Started … Transaction Completed | Downstream booking funnel — fires on canopy/ui-react after handoff, not labs | n/a | Downstream |
Steps 1–6 are live; the chain breaks at 7–8. Until Search Results Viewed
+ Trip Selected land (with a batch_id join key), we can't read explore conversion or
see zero-results in Amplitude. See the trips-surfacing proposal →
Ranked by what unblocks the most learning and value. Filter by priority or owner; click a card to expand.
168/169 searches returned empty. Boston→NYC (9 searches) returning zero is not a real-world result. This is an integration/config bug, not genuine no-service.
Amplitude now points at the cause: the origin never resolves. On labs
Search Submitted the destination resolves to a Wanderu city ID 100% of the time (48/48)
but the origin resolves 0% (0/48); on destination_selected only 3 of 160 carried a
resolved origin_wcityid, and nearest-hub substitution fired on just 30/160. Origins
arrive as raw free text ("700 Atlantic Ave", "Tucson", "Berlin, Germany"). Leading hypothesis: the
trip search runs without a resolved origin station → /api/trips returns nothing. See the
Amplitude tab.
/search (pservs) failed all 7 carriers on Boston→NYC and only ran from dev pods (VPN/VPC). Determine whether Explore shares that infra dependency.Until this is fixed, no future check-in can report conversion. Must move before 2026-06-10.
The Amplitude funnel exists (destinations_suggested → destination_selected →
Search Submitted) but can't yet reach conversion. Per the data dictionary, search
outcome (result count, success/failure) lives in Datadog by design — so don't
duplicate it into Amplitude. Instead, connect the two:
/api/trips outcome to booking, so "search → results → trip clicked → checkout" is answerable across systems. (Refines the Round 1 conversation-ID ask.)depart_wcityid or an origin_resolved boolean to Search Submitted / destination_selected so the origin-resolution gap shows up in the Amplitude funnel itself.Search Submitted only began firing Jun 4; the Amplitude search funnel is ~4 days deep until it matures.This is the measurement spine for the whole check-in cadence.
Only 16 destinations ever appear because they come from a fixed seed list, while origins sprawl (60 distinct). A user from Tucson, Omaha, or Hattiesburg gets NE-corridor suggestions that may have no service from where they are — which also feeds the zero-result problem. Inspiration value collapses when the answer is the same 16 cities for everyone.
Lots of users are looking for specific dates or trying other dates — and the surface is largely
ignoring them. 158 of 160 destination picks (99%) ran on date_source: "default —
2 weeks out" with date_confidence: "low" — only 2 used anything else.
Meanwhile the prompts are full of date intent: "august 24-27", "Cheap from dc today",
"LA to Atlanta July 12", "Check dates around June 14-16",
"The date doesn't have to be the 21", "a weekend in july", "Any date will work".
Two distinct fixes:
Independent of the zero-result bug: even when a search returns trips, a defaulted low-confidence date means it's not the trip the user asked for.
Repeat, specific, answerable from data we already have (places/stations). "Where will it stop in DC", "closest stop to Long Island." Likely a straightforward Chiku capability + tool call.
One user rephrased the rail question 4× — an unambiguous UX-failure signal that Chiku isn't answering it. Either give Chiku Amtrak routing/data or have it state clearly what it can and can't do for rail. Silent failure is the worst outcome.
Stakeholder desire to bring motion to the explore surface. Not evidence-backed by this check-in and lower priority than the search / origin-resolution work — but two placements would pull double duty as UX and perceived-performance wins, so worth scoping:
destinations_suggested, 293 in 30d) reinforces the "surprise me" moment, the core
of the inspiration loop.Needs scoping: which animation, where, and a motion spec aligned to the design tokens
(duration.fast 100ms / duration.normal 150ms; respect
prefers-reduced-motion). Keep it behind the P0/P1 work — don't let polish jump the
queue while searches still return nothing.
"Just give me the options like old pen I don't wanna see the ai." Only ~7 messages — don't over-rotate — but it speaks directly to the chat-primary vs. structured-browse question in the LLM-integration doc. Cheap mitigation: a visible escape hatch to a plain map/list view. Log how often it's used; it's a data point for the build-vs-paradigm decision, not a fire to fight.
The constraint_edited interaction fired exactly once in 30 days, against 160 destination picks and 326 prompts. The affordance to edit origin/destination chips is effectively undiscovered. Either users don't need it (they re-prompt instead) or it's invisible. Worth a quick design look — but lower stakes than the search and origin-resolution work.
The labs surface migrated to canonical event names ~Jun 1–2. Confirm the three legacy emitters are fully removed:
Chiku Message Sent → superseded by LLM Prompted.Chiku Destinations Lit → superseded by Explore Page Interacted [destinations_suggested] ("Lit" was UI-mechanism naming, an anti-pattern).Inspiration Origin Detect Failed → a failure event that belongs in Datadog, not Amplitude. Verify /api/whereami failures are captured in Datadog instead.All three last fired Jun 1 with the dead page_type: "inspiration_base", so this is likely confirm-and-close — but worth verifying so dashboards don't double-count.
depart_wcityid / origin_resolved) now on the search events, so the gap is visible in Amplitude?Chiku Message Sent, Chiku Destinations Lit, Inspiration Origin Detect Failed) confirmed retired?Chiku LLM layer is healthy. Haiku 4.5, $1.28/30d, 1.9s avg latency, 1 hallucination in 334 (chiku-system-base-v5). The cheapest, most reliable part of the stack.
WAPI auth blip on 2026-06-01 (3 timeouts, 4 downstream autocomplete fails) self-resolved. Watch for recurrence; no action now.