Like everybody in tech nowadays, Sony and its PlayStation brand have AI on their minds. On the one hand, the company is looking for ways to improve visuals and performance for anybody who spent $700 on a PlayStation 5 Pro—which is objectively a good thing. On the other hand, Sony is desperately trying to crowbar AI NPCs into games—and it looks and sounds exactly as bad as you imagine it does.
A leaked video first reported by The Verge showed how Sony put generative AI into a rendering of Aloy from Horizon: Forbidden West. In the video, Sony’s director of software engineering, Sharwin Raghoebardajal, talked with fake-Aloy with voice prompts. Thankfully, the AI wasn’t trying to replicate voice actor Ashley Burch. Instead, it conversed with an unnervingly robotic tone while its mouth moved with inhuman over-emphasis for each syllable.
When asked about the story of Horizon, the AI Aloy spoiled many of the mid-game reveals from the first game. In-game, Raghoebardajal talked with Aloy about where the character was in the game world, then received a response after an awkward several-second delay. The AI was using AI vision capabilities to explain what was happening on screen, including the ability to talk about the time of day.
“It’s the afternoon, and it’s quite hot under the sun,” fake Aloy said, taking an awkward second after it was prompted to respond.
Fake-Aloy incorporated AI-generated speech from OpenAI’s Whisper model. Its speech was generated with Llama 3 from Meta and OpenAI’s GPT-4 large language model. The facial animation came from Sony’s own Mockingbird suite, which is one of the few models that’s not widely available. All this software was then funneled into the Decima engine, which was used for both Horizon games from Guerilla Games.
Does Sony Imagine That Players Want to Talk to an AI Aloy?
© Photo: Kyle Barr / GizmodoRaghoebardajal said this is “a glimpse of what’s possible,” though the demo doesn’t exactly leave the viewer spellbound. The original video was taken down by a company affiliated with Sony, according to The Verge. You can find other mirrors on Reddit and elsewhere. It’s nothing we haven’t seen before. Nvidia has been experimenting with AI NPCs for well over a year. The company’s first iteration lets you convince an AI-controlled cyberpunk barman you can hack their Ramen noodles. A later demo shown off in the middle of 2024 involved more constraints.
The largest gaming companies are scrambling to develop ways to shove AI into their upcoming products, whether players want them or not. In a February blog post, Microsoft said it was working on a generative AI model for “gameplay ideation” called Muse. That tool generated an Overwatch-like hero shooter with control-mapped powers and timer-based ultimates. It created this specific game because it was trained on assets from the Bleeding Edge, a real game developed by Ninja Theory. The developers stopped working on that game in 2021 after it failed to gain traction. Would an AI do any better? Why don’t we ask the developers of fellow hero shooter Concord?
AI is inherently derivative, so its use in an artistic medium is necessarily just a copy-paste job based on other people’s assets. AI Aloy may be a Sony side project, but it touches upon many issues cropping up in the industry. Publishers have openly talked about taking the vocals of a voice actor like Burch and replicating them with AI if nothing else than to make them cheaper to produce. Players can tell if something is done by a professional versus a bot. These deep learning models are better when you don’t know they’re there. The funny thing is that game companies have been using AI for years in a form that was popular even before ChatGPT came on the scene.
Improved AI Upscaling Is Coming to PlayStation 5 Pro, Eventually
Mark Cerny, the imperturbable, ASMR-voiced lead designer behind the PlayStation 5 architecture, told Digital Foundry the company will vastly improve the PS5 Pro’s AI upscaling capabilities. The plan is to port AMD’s newfangled FidelityFX Super Resolution upscaler to PlayStation 5 games that already support PlayStation Super Resolution (PSSR). According to Cerny, FSR 4 capabilities could come to the $700 console by 2026 as the “next evolution of PSSR. ”
This move is part of the recently revealed “Project Amethyst” collaboration between Sony and AMD. This can be seen in a huge chunk of the around 30 games that support FSR 4 at launch, which are all the PS5-to-PC ports, such as the recent Marvel’s Spider-Man 2. Cerny said the plan is to integrate PSSR with the current list of PS5 Pro-supported titles before porting FSR 4 to PS5. Before its latest iteration, FSR was hardware agnostic, but that changed with the RDNA 4 GPU architecture. FSR 4 requires users to have the Radeon RX 9070 or RX 9070 XT, as those are the only current graphics cards using AMD’s RDNA 4 design.
The latest AMD graphics card and Sony’s buffed-up PS5 GPU are different beasts, and it will take a lot of elbow grease to crowbar AMD’s machine-learning tech into Sony’s mainline console. Still, Cerny said the company has already implemented some of the FSR 4 neural network into PS5 Pro. The PS5 Pro can do 300 8-bit TOPS (trillions of operations per second), which Cerny said is comparable to AMD’s latest GPUs. FSR 4 is simply “the next evolution” of PlayStation’s own upscaler.
AI upscaling takes each frame rendered at a lower resolution, and then, using AI algorithms, the software shows that frame at a higher resolution, all while keeping the performance of the lower resolution. FSR 4 is the first machine learning-based branch of FidelityFX. In our tests, it offers better performance and sharper visuals for supported games compared to FSR 3.1. Sony’s PSSR should work on all games with the “enhanced” label on PS5 Pro. The performance gains in some of these titles are difficult to track, though in cases like Call of Duty: Black Ops 6, the game can support an uncapped 80 to 100 FPS compared to 60 on the base PS5. Some games, like The Last of Us Part I, explicitly use 1440p to 4K upscaling on supporting TVs to maintain the capped 60 FPS.
PC games increasingly rely on AI upscaling from the likes of Intel with XeSS, AMD’s FSR, and Nvidia’s ever-expanding DLSS suite. The PS5 is still a console, and you won’t have nearly as many graphics options as you do on the PC, even with the Pro. FSR on PS5 Pro will be integrated into the background so that most players won’t recognize anything, but they may see an increase in frame rates or improved graphical details.
Some gaming purists resent the overreliance on AI upscaling and the rise of generated or “fake” frames from both FSR and DLSS frame generation. The thing is, if hardware improves to make gaming more accessible, then that’s a positive development for players everywhere. If Sony should focus its AI efforts anywhere, it’s to help developers deliver games that play at a higher fidelity, not to make games nobody wants to play.
English (US) ·