30,000 datasets, one click: how the GOAT AI-powered Catalog is built

Every time we start working on a project, the first step is to look for data. Every city, every region, every country has built its own portal. All accessible, but all spread out. Nowadays, with AI, it might seem easier. You ask an AI for bus stops in Bavaria. It gives you a link to a portal. Maybe the right one, maybe one that no longer exists. Then the real work starts: check what is there, read the metadata, check the licence, download it, unzip it, check the projection, find the boundary layer, ask the AI again, open another portal, work out if this is the right dataset, download it, add it to your project. And now you can start working. Sorry for the long list. It was meant to be tiring. Because even with AI, getting data into a QGIS or GOAT project still is.

That is why we rebuilt our Catalog. It used to hold about 60 to 70 datasets that we maintained by hand. Now it holds more than 30,000. And all the work above is already done: every dataset is converted, checked, and comes with its metadata and licence. One click and it is in your project, with no download and no import.

The Catalog in GOAT: one search instead of five portals.

What the Catalog holds: the data you need, curated for you

The Catalog brings together two kinds of data: open data harvested from public portals, and datasets Plan4Better prepares itself.

Public administrations publish large amounts of geodata in open data portals. Using it normally means knowing which portal to look in, reading its metadata, downloading a file and converting it before any analysis can begin. GOAT harvests these portals so the data is ready to use. Among others, it comes from:

• GovData: Germany’s national open data portal, where federal, state and municipal administrations publish their data.
• data.gv.at: Austria’s open government data portal.
• geocat.ch: Switzerland’s geodata catalogue, run by the Federal Office of Topography (swisstopo).
• data.europa.eu: the official portal for European data, run by the EU. The Catalog covers part of it.

How do we choose which portals to harvest? By hand. That is what makes the data in the Catalog trustworthy. We don’t collect the whole internet, the way a general AI search does. We start from official open data portals that we know and trust, and we use what we have learned from years of planning work to decide which ones are worth reading.

But choosing good portals is only the first step. Even official portals publish much more than planning needs: budget tables, legal texts, historic maps. So we don’t ask “what data exists?” We ask “which data answers a planning question?” Every dataset is checked against that question, and only the ones that pass go into the Catalog. A land-use plan for a whole municipality stays. A development plan for a single plot, a historic map or live parking data does not.

Of about 554,000 datasets we collected, more than 70% did not pass in the beginning. How much is filtered depends a lot on the portal. On a catalogue made for geodata, most of the entries will be relevant for planning. But on a general open data portal that publishes everything an administration produces, a really small fraction is.

We also remove duplicates. Some sources publish the same data once per district and again for the whole administrative region. If the regional layer contains everything in the district layers, we keep the regional one and drop the rest. We also find layers with identical content and keep only one. This saves you time. Imagine searching for a street network for Bavaria and getting fifty versions of it. You would spend more time choosing than analysing. In the Catalog, that choice has already been made.

Datasets we prepare ourselves

Not everything planners need is on a public portal in a usable form. So the Catalog also holds datasets we build and maintain ourselves. Points of interest come from OpenStreetMap. Its raw tags are detailed but messy, so our pipeline corrects them and groups them into categories that make sense for planning, such as services or tourism and leisure. Public transport stops, stations and lines come from the GTFS timetable data of transport operators, and cover local, regional and long-distance services.

We built the Catalog we wanted to use ourselves. We are planners too, and we work with municipalities and consulting firms every day. So we know that what matters is not having millions of datasets. It is having ones you can trust, and that cover the whole area you are working on.

The datasets Plan4Better prepares, found by filtering the Catalog by publisher.

How you use the Catalog: you filter what you need

More than 30,000 datasets is still a lot, so the Catalog is built to help you narrow things down quickly. You can open it from the Workspace, or directly from + Add Layer while you are working in a project.

You can narrow it down further by typing what you are looking for, like “noise map” or “cycling network”, and filtering by category, licence or format. To look somewhere else, search for another town or region, or draw your own area on the map.

Filters for location, category, publisher, licence and more.

But if you open the Catalog from inside a project, it already knows where you are working. It shows you the datasets available for the area on your map first, so the first results are the ones that are useful for you.

The Catalog finds the data for the area on your map.

Some datasets you will use again and again. You can mark them as favourites, and they are waiting for you the next time you open the Catalog. When you find the right one, add it to your project with one click. There is nothing to download and nothing to import. The layer is on your map straight away, ready for a Catchment Area or any other tool in GOAT.

Why a catalog, if you can ask an AI?

It’s a fair question. AI search is good at pointing. Ask for bus stops in Bavaria and it gives you a link. But it cannot promise that the link works, that the dataset is the right one, or that it is the same one your colleague used last month. In the Catalog, every dataset comes with its metadata, so you can see where it comes from, who published it and under which licence.

Every dataset with its metadata and a sample of its data, before you add it.

A catalog also shows you data you did not know to ask for. An AI answers the question you type. A catalog you can browse shows you what exists, and sometimes the most useful dataset is one you were not looking for.

And to be honest, we use AI too. Checking 554,000 datasets for planning relevance by hand would take years, so a language model reads each title, description and keywords and decides. The difference is that it does this once, for everyone, instead of every planner asking again for every project. It is not perfect. When we checked samples, we found it had dropped data that planners do need, such as official standard land values in North Rhine-Westphalia and electric bus data in Hamburg. A dataset that is wrongly dropped is the worse mistake, because nobody ever sees it. So we keep checking samples and improving the instructions we give the model.

AI can tell you where the data might be. The Catalog is the data, ready to use.

What’s next: AI that searches for you, and more data of course

We are not done yet. The next step is bringing AI into the search itself. Instead of setting filters one by one, you will be able to describe what you need in your own words, for example “noise maps for Freiburg” or “cycling networks in the municipality I’m working on”, and get the matching datasets with their content, licence and date. We are also adding a way to get data directly from OpenStreetMap and Overture Maps. With AI, you will be able to ask for the available data no matter where in the world you are.

So yes: lots of changes, lots of data, and now it is up to you what to do with it. Meanwhile, open GOAT and try the Catalog for free. Open a project, click + Add Layer and search for the first dataset you need. GOAT is free to start. If you need inspiration, you can start from a template or look at our projects in the gallery.

Frequently asked questions

What is the GOAT Catalog?

The GOAT Catalog is a collection of more than 30,000 open geodatasets inside GOAT, the WebGIS by Plan4Better. You can search and filter it, and add any dataset to a project with one click, without downloading or importing anything.

Where does the data in the GOAT Catalog come from?

Most of it comes from official open data portals in Germany, Austria, Switzerland and the EU, such as GovData, data.gv.at, geocat.ch and data.europa.eu. The Catalog also includes datasets Plan4Better prepares itself, such as points of interest from OpenStreetMap and public transport stops and lines from GTFS timetable data.

How does GOAT decide which datasets go into the Catalog?

Every dataset collected from the portals is checked for planning relevance. Simple rules first remove outdated, single-site and duplicate entries, then a language model reads each title, description and keywords. Datasets that pass are downloaded, converted, checked for quality and licence, and published. Of about 554,000 datasets collected, about 32,000 were published.

Is the data in the GOAT Catalog free to use?

The Catalog only publishes datasets with a usable open licence. Datasets whose licence does not allow sharing are left out. Each dataset shows its licence in its metadata, so you can check the exact conditions before you use it.

Can I use the GOAT Catalog in QGIS?

Yes. The Catalog follows the STAC standard (SpatioTemporal Asset Catalog), so you can also search it from STAC clients such as QGIS or pystac-client, not only from the GOAT interface.

Can AI assistants search the GOAT Catalog?

Yes. The Catalog has its own MCP server, so any AI assistant that supports the Model Context Protocol can search it in natural language. It uses the same search and the same prepared data as the GOAT interface.

Why use a data catalog if AI can search for data?

AI search can point to where data might be, but it cannot promise the link works or that the dataset is the right one. The Catalog holds the data itself, already selected, converted and documented, so it is ready to use in an analysis.