A paid, real-world, location-based game similar to Pokémon, using AI (GPT-4o, LLMs) for real-time animal identification, taxonomy generation, game data creation, and sprite generation.

Firstly, apologies, it's not free. It would be difficult to support this for free, it's a paid game.I will now share the technical details, which will probably be most of interest for HN readers.I previously made a carbon footprint tracking app where you photo objects and it tells you the carbon footprint by using an LLM to estimate the data on the fly, e.g. 32kg CO2e / kg of beef, in the UK. At some point, I realised that it is possible to make a Pokémon-style game, but capturing real animals in the real world.This is now possible because:
- image recognition is cheap, i.e. identifying animals, and the models (gpt-4o) can detect a (surprisingly) large number of animals and output their exact species.
- LLMs can output a species' full taxonomy, pretty reliably. And, more importantly, they can generate game data quickly, on the fly.It would unfeasible to generate the game sprites (images) for every species (millions, worldwide) and their full evolution chain, e.g. caterpillar, chrysalis, butterfly, ahead of time. I realised it's possible to do this in real time.General game flow:
- photo animal
- send to gpt-4o
- return species
- send species to LLM, create evolution chain, plus attributes, types and moves.
- in parallel, create sprites.All data is cached.The aim of the game is to build up your team and compete with other players to take over gyms.The game is based in the real world, I had to come up with a way to have health centres and shops. These must both have decent coverage, globally. The solution is health centres are places of worship, e.g. churches, mosques, temples etc and shops are real world grocery stores. Every country as far as I can tell has places of worship, with good distribution, which was surprising. Gyms are located in every park worldwide.Challenges:How to get players outside:
- I use openstreetmap for the game map, but I overlay my game design on top of it.
- To physically make players go out into nature: I use openstreetmap area types to only allow capturing animals when your GPS location is in natural areas, e.g. woodland, parks etc. The aim of the game is to get you out into nature and appreciating animals.
- Level system: The solution I came up with is to set the animal levels based on the proximity to built-up areas, e.g. Every ~500 meters you go away from built-up areas, the animal level bands increase by 5 levels.
- It would be expensive to render the entire physical world in my game map, so I instead render the map on the fly, deterministically.I also fetch animal calls in real time so that when they enter battle you hear a pigeon cooing, for example, which is pretty cool. I also fetch the animals conservation status, i.e. how endangered is it, and give you more reward (leaves, in-game currency) for capturing rarer animals.I "launched" the game about a month ago, but have not really been publicising it as I've been working on various updates and improvements, but now I am sharing it more openly. It's got about 20 players so far, from around the world, and around 500 unique animal species have already been encountered.Challenges have been keeping the costs low. Servers cost about $200 / month, text-gen is basically free as I get free tokens from OpenAI for sharing data, it's not privacy-related, and image-gen costs about $0.04 per sprite (2 per animal).My background: not a programmer, originally a mechanical engineer and then business development manager, then started learning programming and building apps with AI in the last few years.Feel free to ask me any technical details, happy to share.