AI tour guide with a custom retrieval pipeline

Overview

An AI travel assistant for Saudi Arabia. A user asks about places to visit, stay or eat, and the assistant answers from a dataset of real locations rather than from whatever the model happened to memorise during training.

I built the whole thing: the data collection, the retrieval layer, the fine-tuning, the agent and the API. It reached roughly 70% of the planned scope before the engagement moved on, with the messaging integration as the remaining piece.

What I own

Data acquisition, data modelling, the retrieval pipeline, model fine-tuning, prompt design, the agent, and the Express API around it.

Complexities tackled

The dataset did not exist, so building it was the first project. No available API gave the coverage or the fields this needed. I wrote a custom scraper that walked map regions by coordinate polygon, and ran it for weeks across remote machines to collect over 35,000 location records. People underestimate this part constantly: an AI product with no proprietary data is a thin wrapper over a public model, and the data pipeline is usually the longer and less glamorous half of the work.

Retrieval quality is the product, not the model choice. Embedding a dataset and doing one nearest-neighbour lookup produces mediocre answers, which is where most retrieval projects stop. This one expands each user question into several search queries before retrieving, then puts the results through a relevance filtering pass, and only then answers. Those two extra stages are the difference between a demo and something a person would actually use twice.

Knowing what fine-tuning is for. Fine-tuning and retrieval solve different problems, and conflating them wastes months. Fine-tuning taught the model how to answer: tone, structure, the shape of a good response. Retrieval supplied what to answer with: current, specific, local facts. Trying to teach facts by fine-tuning would have been expensive and stale on arrival.

Designing a system that is allowed to not know. A tourism assistant confidently inventing a restaurant is worse than one that declines. The agent decides per question whether to search the dataset or answer directly, is scoped explicitly to one country, and has defined behaviour for questions outside that scope. Deciding where the boundaries sit, and enforcing them in the design rather than hoping the model behaves, is most of the safety work in an assistant like this.

Advising the client away from work I could have billed. Partway through this relationship the client proposed a larger system built on hotel and visitor data for a government body. Before any development I pushed back on the foundation: whether that data would actually be made available, whether privacy would allow it, and what the buyer would do with the result if it were incomplete. My position was that if the data was not there, everything built on it was wasted. Telling a paying client that their idea is not ready is uncomfortable, and it is what a technical partner is for.

A relationship rather than a job. Across roughly two years I built four systems for this direct client: a social media monitoring and reporting platform that collected from around fifty accounts daily, a football scouting data platform, a city-scale business data scraper, and this. Each one came from having delivered the last.

The corpus is four separate collectors, not one scraper. There is a place collector and a district collector, each with its own source and its own output, both driven by a real browser rather than by HTTP requests, because the sources render their content client side and a request based scraper gets an empty shell back. Thirty five thousand records is not one pass over one site.

Places have geometry, not just coordinates. Districts are polygons, and deciding whether a place belongs to a district is a point in polygon question rather than a distance comparison. That is done with a geospatial library rather than by hand, which is what makes “restaurants in this district” a query the assistant can actually answer correctly near a boundary.

There is a front end. The assistant is not an API with a curl example attached. It has a browser client, which is what turned it from a pipeline into something a person could be handed and asked to try.

Stack

Node.js and Express serving the retrieval and chat layer, LangChain with OpenAI embeddings, MongoDB with Atlas Vector Search, Mongoose for the models. Collection runs on Playwright with a geospatial library for the district geometry. Python for the analysis and fine-tuning pipelines. A Vue client on the front.