# Round 1 goals: data to gather before prioritizing inspiration-page surfaces > Companion to [`scratchpad.md`](scratchpad.md), [`axes.options.md`](axes.options.md), > and [`llm-integration.options.md`](llm-integration.options.md). > Round 1 is data-gathering only — no feature work. The goal is to size the > demand for each surface so we can rank by expected revenue with real numbers, > not hypotheses. **Updated to assume the inspiration page lives in the realm > of LLM integrations** — both Wanderu-owned LLM surfaces and external LLM > clients (ChatGPT, Claude, Gemini) consuming Wanderu via MCP. ## What this round needs to answer Round 1 informs two decisions: 1. **Which `(breadth × resolution × anchor)` surface from [`axes.options.md`](axes.options.md) gets built first.** 2. **Which paradigm — owned LLM surface on `/inspiration/`, LLM-augmented structured search, or external-LLM (MCP) entry — gets the primary build investment.** See [`llm-integration.options.md`](llm-integration.options.md) for the strategic ordering question. To answer both we need to know — for each candidate surface — how many people hit it today, how many bounce, how much revenue is on the table, and how much of the inspiration moment has already migrated to external LLM clients. That decomposes into four top-level unknowns: 1. **How open-ended is current Wanderu demand?** What % of sessions today are *latently* open on date, destination, return, mode, or budget — even though the UI forces them to enter a fixed value? 2. **What audience is currently structurally excluded?** Who tries to search and gets "no results," abandons in confusion, or never starts because the product doesn't speak their constraint (accessibility, language, payment, mode)? 3. **Where does the open-ended demand prefer to enter the funnel?** From their current location, a different city, a date window, a budget cap, an event, or a person? 4. **How much of the inspiration moment is already LLM-mediated?** How many sessions arrive via chat referrers, MCP, or generative-search engines — and how fast is that share growing? Each known unknown maps to one or more of those four. --- ## Known unknowns ### Called out explicitly 1. **Date certainty** — How many people are uncertain about which date they want to travel? Signals: empty/default date fields, broad date ranges, repeated date-picker visits, post-results date changes, multiple searches with same origin+destination but different dates. 2. **Origin choice** — Do people want to search from their current location or another? Signals: how often the auto-detected origin is overridden, mobile vs desktop split, repeat searches with different origins. ### Additions that affect prioritization 3. **Destination certainty** — What % of sessions have a fixed destination vs. open destination? Signals: empty destination field, multiple searches with same origin but different destinations, Explore page entries. 4. **Return-date certainty** — What % of round-trip bookers pick a specific return vs. would have used a window? Signals: round-trip vs one-way ratio, return-date edit patterns, time elapsed between outbound book and return book in two-leg bookings. 5. **Budget anchoring** — Do users actually shop with a budget cap in mind? Signals: results sort order chosen, scroll depth past price thresholds, no-booking exits when cheapest visible price exceeds median. 6. **Mode/access anchoring** — How many users need a specific mode (train only, no transfers) or access feature (wheelchair, Spanish-only, cash)? Signals: filter usage today, "no results" segmentation, CS ticket categorization, Intercom theme mining. 7. **Carrier/loyalty anchoring** — How many users are searching for a specific carrier's network? Signals: carrier-named search queries, Explore filter usage. 8. **Person/relationship anchoring (proxy bookers)** — How many sessions are one person booking for another? Signals: payment-name vs ticket-delivery email mismatch, multi-passenger bookings with non-matching surnames, Intercom mentions of "for my son/dad/boss." 9. **Event-driven demand** — How much demand clusters around named events (NCAA, festivals, Día de los Muertos)? Signals: search-volume spikes by route, seasonal corridor analysis. 10. **Comparison behavior mid-funnel** — How many sessions search 2–3 different destinations before booking? Signals: same-session repeat searches with different destination, multi-tab behavior, time-on-results. 11. **Lead time distribution by scenario shape** — Are open-ended searchers more last-minute or more advance? Signals: book-date minus depart-date histogram, segmented by inferred scenario shape. 12. **Explore awareness and conversion** — What % of Wanderu users know Explore exists? Of those who use it, what's the conversion rate vs A→B search? ### LLM-specific additions 13. **LLM-mediated traffic share** — How many sessions originate from external LLM clients (ChatGPT, Claude, Gemini, Perplexity, Copilot) today, and how is that share trending? Signals: referrer headers, UTM analysis, MCP query logs, conversation-ID propagation from agent calls. 14. **Natural-language input distribution** — When users get a freeform input field, what do they actually say? Signals: prompts logged by the `labs` Amplitude source (meags-prototypes), any existing chat surfaces, intercept surveys with "describe your trip in your own words." 15. **Current-model performance on Wanderu scenarios** — Where do frontier LLMs hallucinate, mis-route, or refuse on the 100 scenarios? Signals: eval-set runs against Claude/GPT/Gemini both with and without Wanderu MCP tool access. See [`scenarios.evals.md`](scenarios.evals.md). 16. **Trust calibration by persona** — What do users trust an LLM to do — and what do they want to keep manual? Signals: surveys segmented by persona, qualitative interviews, in-chat acceptance rates of LLM recommendations in any current prototype. 17. **In-language LLM quality** — How well do candidate models handle Spanish, voice-transcribed Spanglish, and Wanderu-specific corridor names that transcribe poorly? Signals: Spanish-language eval-set scoring, error rates on station-name resolution. --- ## Quantitative data points ### Tier 1 — already in our data, just needs to be queried | Data point | Source | Why it matters | |---|---|---| | Distribution of search date-range width (single day vs 2–7 days vs 7+) | Search-event logs, Snowflake `SNOWPLOW_SESSIONS` | Quantifies date flexibility today | | % of sessions where auto-detected origin is overridden | Snowplow + IP geolocation log | Answers "current location vs other" directly | | % of sessions returning "no results" — by origin, destination, date, mode filter | Search-event logs | Sizes the latent excluded audience | | Funnel abandonment by step (date picker, results, checkout) | Amplitude | Reveals where open-ended searchers drop out | | Repeat-search behavior within a session — what changes between searches? | Snowplow session reconstruction | Detects comparison and date-shopping behavior | | Round-trip vs one-way ratio; return-date edit patterns | FACT_TRANSACTION + search logs | Sizes "open return" demand | | Lead time (book-date → depart-date) distribution by persona inference | FACT_TRANSACTION | Confirms or rejects the lead-time priors in archetypes doc | | AOV and booking frequency by route, by persona, by mode | FACT_TRANSACTION + DIM tables | Weights revenue by audience | | Mobile vs desktop split of open-ended search shapes | Amplitude device | Informs which surface to build for which platform | | Explore page (`/en-us/explore/`) entry volume and conversion | Snowplow + FACT_TRANSACTION | Tells us if the lever is build vs promote | | **Referrer-based LLM-traffic share** — `chat.openai.com`, `claude.ai`, `gemini.google.com`, `copilot.microsoft.com`, `perplexity.ai`, generative-search referers | Snowplow referrer logs | Sizes external-LLM-mediated arrivals today | | **MCP query volume and conversion** — queries hitting the Wanderu MCP server, what they asked, whether they converted | Wanderu MCP server logs | Direct measure of agent-driven demand | | **Natural-language prompt distribution from the `labs` source** — what shape are user prompts in the meags-prototypes surface? | Amplitude with `source: "labs"` | First-party evidence of what users say given freeform input | ### Tier 2 — requires light new instrumentation | Data point | What to add | Effort | |---|---|---| | Whether the user accepted, edited, or cleared the auto-detected origin | Event on origin field interaction | Low | | Whether the user opened the date picker but didn't change the date | Event on picker open/close without change | Low | | Multi-tab signal — duplicate sessions from same anonymous ID with different queries within N minutes | Server-side session correlation | Medium | | Payment-name vs traveler-name mismatch flag at checkout | Boolean on booking | Low | | Whether `noResults` was reached, and which filter caused it (mode, accessibility, language, cash) | Structured no-results event | Low–medium | | Time-to-first-search after landing on `/inspiration/` or Explore | Event on first search submit | Low | | **Conversation-ID propagation from MCP / agent sessions through to booking** | Pass conversation ID as a session parameter; record on booking | Medium | | **LLM-prompt logging on any chat-enabled prototype, with structured-anchor extraction recorded alongside the raw prompt** | Prompt + extracted anchors saved to a queryable store | Medium | | **Search-tool-call vs free-generation flag on any LLM response** | Boolean on LLM response indicating whether a Wanderu search was actually executed | Low | ### Tier 3 — external / longer-lead | Data point | Source | Notes | |---|---|---| | SEO search volume for openness phrases — "weekend trips from X," "where can I go from X," "buses from X" | Ahrefs, Google Search Console | Sizes intent that never reaches Wanderu | | Competitor traffic share of Explore-style entry points | Similarweb, Ahrefs | Benchmarks the prize | | Skyscanner/Google Flights Explore-Everywhere usage rates (public data) | Industry reports | Lower bound on willingness to browse | | Volume of Spanish-language travel-search queries in Wanderu corridors | SEO tooling, Spanish-language ad platforms | Sizes the excluded Spanish audience | | Accessibility-traveler population in Wanderu corridors | DOT, ACS, advocacy orgs | Sizes the excluded accessibility audience | | **Public usage rates of ChatGPT / Claude / Perplexity for travel-planning** | Industry reports, OpenAI / Anthropic public disclosures | Estimates the migration of inspiration moment to agents | | **Competitor MCP server presence** — does Omio, Busbud, Rome2rio, FlixBus expose an MCP? | Manual audit | Indicates whether Wanderu is ahead or behind | --- ## Qualitative data points Quantitative data reveals *what* people do; qualitative reveals *why* and *what they wish existed*. Both needed before ranking. | Method | Audience | What it should tell us | |---|---|---| | **5–8 user interviews per persona group** | Student, family connector, retired explorer, proxy parent, accessibility, premium seeker | Walk through last open-ended search; capture where they actually started (date? destination? person? event?) and what they wished the product asked them first | | **Diary study** (1–2 weeks, 3–5 participants) | Proxy bookers (Susan, Amanda, Priya, Claudia, Rosa) | Capture the iterative back-and-forth that doesn't show up in a single session | | **Co-browsing observation** (live or recorded) | 5–10 mixed users on Wanderu today | See where they actually hesitate, change inputs, switch tabs, give up | | **Cart-abandonment intercept survey** | Users who bounce after results | "What stopped you?" — categorize by openness type | | **Recent-booker survey** | Last 30 days of bookers | "When you started, was your date fixed, flexible, or unknown?" Same for destination, return, mode | | **Spanish-language user research** | Recruited separately through community channels | This is its own study — English-speaking research will miss it | | **CS ticket and Intercom theme mining** | Last 6 months of contacts | Categorize by openness type: "couldn't find anywhere to go," "wanted to compare," "no Spanish carrier," etc. | | **Reddit / Trustpilot mining** | r/Amtrak, r/Greyhound, Wanderu Trustpilot, FlixBus Trustpilot | Capture open-ended language and unmet needs in the user's own words | | **5-second tests on `/inspiration/` mockups** | Mixed-intent traffic | Which entry-point anchor feels most natural — date, destination, budget, person? | | **LLM trust survey by persona** | Same persona groups as interviews | "Would you let an AI book your trip? Recommend? Compare? Plan an itinerary?" — segment results by persona | | **"Describe your trip in your own words" intercept** | Visitors to Wanderu landing pages | Captures natural-language utterance shape distribution; feeds [`scenarios.evals.md`](scenarios.evals.md) | | **Voice vs text vs structured preference** | Mobile-heavy personas | Which input mode works best per persona | | **Eval-set scoring sessions** | Cross-functional team (PM, design, eng, support) | Run candidate models against [`scenarios.evals.md`](scenarios.evals.md); score grounding, hallucination, anchor accuracy | | **Accessibility-specific LLM evaluation** | Accessibility-dependent users + advocates | Manually verify that any LLM-mediated response on accessibility is grounded and accurate. Critical safety check. | --- ## Prioritization of Round 1 work Cost-to-collect × decision-leverage, roughly: | Priority | Item | Why first | |---|---|---| | **P0** | Query Tier 1 quantitative data points from existing logs | Cheap, fast, biggest unknown unlocks | | **P0** | CS ticket + Intercom mining for excluded-audience themes | Cheap, surfaces structurally-excluded audiences invisible in session data | | **P0** | LLM-referrer and MCP-traffic share analysis | Answers the build-vs-MCP-primary strategic question; cheap once referrer logs are queried | | **P0** | Run [`scenarios.evals.md`](scenarios.evals.md) against Claude/GPT/Gemini with and without Wanderu MCP | Establishes baseline LLM quality on Wanderu-specific tasks; no shipping required | | **P1** | Recent-booker survey on date/destination/return certainty | Directly answers the two unknowns called out; can ship in <2 weeks | | **P1** | Add Tier 2 instrumentation (including conversation-ID propagation and prompt logging) | Need this running for 30+ days before P2 surveys can leverage it | | **P1** | Mine the `labs` Amplitude source for natural-language prompt distribution | First-party data on what users actually type when freed from the form | | **P2** | User interviews (5–8 per persona group) | Highest insight, longest lead time | | **P2** | LLM trust survey by persona | Tells us which surfaces fit which personas; informs which to ship first | | **P2** | Cart-abandonment intercept | Needs traffic volume to be meaningful | | **P3** | Spanish-language and accessibility-specific research + LLM evaluation | Requires separate recruiting; defer if scoping bandwidth — but flag as the highest-risk LLM area | | **P3** | Diary study for proxy bookers | High insight but long-running | --- ## Out of scope for Round 1 - Any UI build or A/B test on `/inspiration/`. Round 2 will design the tests. - Defining the inspiration-page IA. We need the data first to know which anchors deserve primary entry. - Re-architecting Explore. Round 1 answers whether the lever is build vs. promote. - Shipping a customer-facing LLM surface. Eval-set work is internal-only in Round 1. ## Round 1 deliverables - A populated data table for each of the 17 known unknowns with current numbers from Tier 1 - A categorized list of CS / Intercom themes mapped to scenario types - A survey report from the recent-booker study - A qualitative findings memo from interviews - A **baseline LLM scorecard** across the [`scenarios.evals.md`](scenarios.evals.md) eval set for at least 3 candidate models, with and without Wanderu MCP - A **build-vs-MCP-primary recommendation** based on LLM-mediated traffic trends and competitor MCP presence - A revised ranking of surfaces in [`axes.options.md`](axes.options.md) with evidence-backed `Δ Revenue` estimates instead of hypotheses, segmented by paradigm (UI-native, LLM-augmented, MCP-only) ## Open questions for Round 1 scoping - Do we have enough Spanish-speaking session data to draw conclusions from logs alone, or do we need dedicated user research from the start? - Is the Wanderu app's session data clean enough to detect multi-tab comparison behavior, or do we need server-side session stitching first? - How much of the proxy-booker audience is detectable from existing booking metadata (name/email mismatch) vs. requires direct survey? - Should Round 1 include a *single* low-cost experiment — e.g. promoting Explore in the main nav for one week, or exposing the existing MCP server more broadly — to bound the build-vs-promote-vs-MCP question early? - Is the existing Wanderu MCP server (`mcp__claude_ai_Wanderu_Booking__search-trips`) enough to serve the inspiration use case, or does it need extension before the eval set can be run meaningfully? - What's the budget for eval-set runs? Frontier-model API costs add up over a 150-eval set × multiple models × multiple prompt configurations.