Playable 3D World From a Text Prompt on AMD Radeon

Playable 3D world generated from a text prompt on an AMD Radeon GPU

AMD has published a full recipe for building a playable 3D world from one text prompt on a single consumer-class GPU. The pipeline runs locally on a Radeon AI PRO R9700 with ROCm software and ends in a small game where a thief steals a pie.

The goal is to show the extra steps that turn a generated video into something you can walk around in: real 3D geometry, characters that understand speech, and agents that act sensibly in a place no designer laid out.

What the demo does

You play a traveller in a plaza, seen from behind. You talk to villagers by typing or by push-to-talk. Soon after you greet Anna the baker, a thief named Leo grabs a pie from her stall and bolts. He ducks behind cover and out of view, so chasing him solo rarely works. A better move is to enlist Tom, another villager: order him to catch Leo and tell him where the thief was last seen.

Whoever gets close enough catches Leo, who gives the pie back, and Anna reacts. If nobody gets him in time, he escapes. There are no dialogue menus. Everything runs through conversation.

The parts of the pipeline

Four pieces make up the system:

  • The world starts as a video from the Wan2.2 world model. Depth Anything 3 and AMD’s ROCm port of gsplat convert that footage into 3D Gaussians, which are millions of tiny translucent colored points that can redraw the scene from any angle.
  • The villagers run on an LLM, Qwen3-8B served by vLLM, which turns plain English into game actions. Voices come from Qwen3-TTS, and speech input uses Whisper.
  • The thief’s escape behavior comes from a policy trained with reinforcement learning (PPO) inside the generated world itself.
  • The game runs in Godot 4, which draws the Gaussians alongside regular meshes and characters.

Hardware needed for a playable 3D world

The test machine used an AMD Radeon AI PRO R9700, which has 32 GB of memory and RDNA 4 architecture (gfx1201). Any Radeon card needs enough memory to hold a 5B-parameter video model. The post used Linux, Docker, ROCm 10.0 and the GPU device files /dev/kfd and /dev/dri, plus Godot 4. Readers should know PyTorch, Docker and a game engine.

The work happens in three containers sharing the GPU, all started from a current ROCm vLLM image that ships a ROCm build of PyTorch. The team used the ROCm 10.0 build, which includes PyTorch 2.12. A separate container for vLLM keeps the language model server steady, so the other two images can pin their own library versions. The voice container layers Qwen3-TTS and a lightweight web service on top, while the world container carries the video, depth and splatting libraries.

Build once, play locally

The first five stages and the RL training run a single time, offline, and produce files. The game reads them at startup. Live play needs just Godot, the LLM server and the voice server. Godot calls the two servers asynchronously over localhost, so a villager can think or talk without freezing the game. The trained policy is a compact network that Godot runs on its own, needing no additional service.

CPU offload for the video model applies to the world container only, which is done before launch. Once you play, the language and voice models remain in GPU memory.

Licensing details

The software mix is mostly permissive. Wan2.2, VideoX-Fun, Depth Anything 3 Base, gsplat and its ROCm port, OWLv2, Qwen3 and vLLM are Apache-2.0. Whisper, Godot, gdgs and Microsoft Rocketbox avatars are MIT, and Poly Haven assets are CC0. Larger Depth Anything 3 checkpoints carry a non-commercial license, which matters for anyone planning a commercial product.

Why it matters for developers

A single workstation card now covers video generation, 3D reconstruction, a language model, speech synthesis and speech recognition, all in one project. For indie developers and researchers, the interesting part is that the generated scene isn’t just a picture. Because Gaussian splats can be rendered by Godot, characters can stand in the world and the policy can train inside it.

What to watch next

The recipe is a proof of concept, not a product, and it leaves open how well the approach scales to larger or more complex scenes. The recipe is meant to be reproducible. Readers who own a supported Radeon card can reproduce the playable 3D world step by step and swap in their own prompts, models and game logic. The post also doubles as a showcase for running ROCm on Radeon hardware.

Source: ROCm Blogs

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.