The future of Cloud Gaming: Command Streaming as the alternative to video streaming
We live in a world where Netflix and Spotify have won the battle for digital media consumption. Thanks to these streaming services buying CDs and downloading content are a distant memory – and yet that is what we have to do before we can play a game. Why can we instantly watch and listen but cannot instantly play?
The problems with video streaming for games
Music and video streaming platforms provide their service using standard cloud servers and content delivery networks because the content is created offline and then played on our client device. But game streaming platforms require a more powerful server, one with a graphics card (GPU), because the game is played and rendered on the cloud server itself, in real-time. Our client receives only a video representation of the game.
One of the problems with requiring GPUs is that they are hard to find in the cloud. The first image shows the combined availability zones of cloud providers such as Microsoft Azure, IBM Cloud, Amazon AWS, Google Cloud, and Alibaba Cloud. The second image is the combined availability zones for the same providers that also support GPUs:
Of Google Cloud’s 61 zones, only 27 have GPUs[1]. Clients who are farther from one of those 27 zones will experience decreased reach and increased latency.
GPUs in the cloud are also expensive, in some cases, nearly 50x the cost of CPU-only servers. The estimated monthly cost of Google Cloud’s cheapest CPU is $3.88, while the cheapest CPU with a GPU costs $182.73.
One of the main causes for this cost difference is power consumption:
With great power comes great cooling.
A GPU needs more power than a CPU to cool and run at peak performance which in turn requires plenty of electricity. More cloud gaming platforms have been killed by GPU electricity bills than by latency.
At this point, it might seem that cloud gaming systems using video streaming can’t work. Playstation Now, however, is proof that they do. But “just working for certain regions” isn’t enough. Netflix provides a seamless user experience to more than 97% of the Earth across any device, at any time [4]. For cloud gaming systems to operate at the same level, they have to reach everyone with the same quality – the real barriers are cost and scale.
What is Command Streaming?
Command Streaming is an alternative to video streaming that does not require GPUs in the cloud. It works by intercepting the graphics commands the game issues to a graphics API – such as DirectX or OpenGL – and then compressing and sending them over the network for the client to render using its local GPU.
Advantages of Command Streaming
The processing of the game requires only the use of a standard cloud server, which means command streaming can use any of the availability zones in the public or private cloud. The result is increased reach and scale as well as reduced cost and latency. To display the game on the client, command streaming can use the GPUs available on our desktops, laptops, tablets, phones and even ‘smart’ TVs, because GPUs are cheap and abundant… except in the cloud.
Command streaming doesn’t need changes to the game to work – even older games and emulators will run, which helps with video game preservation. Another advantage is that the streaming bandwidth does not depend on the resolution. Google Stadia recommends 10 Mbps to stream 1080p video and 35 Mbps to stream 4K content. What about 8K…? Instead, with command streaming, a 1080p stream uses the same bandwidth as an 8K stream because graphics commands are resolution-independent, and pixels aren’t.
Cloud gaming is often criticized because of the additional network latency. Networked games reduce latency by using techniques such as state interpolation, input prediction, and lag compensation. Video-based cloud gaming platforms undo this effort. In this article detailing latency mitigation strategies John Carmack made a call for a more advanced type of cloud gaming system:
“If the cloud gaming client was made more sophisticated, time-warping could be performed locally, which could theoretically reduce the latency to the same levels as local applications”
Command Streaming answers that call for more sophistication. The client doesn’t receive compressed pixels – it receives graphics data, which includes depth buffers and render targets. The client can also keep a history of how they evolve. This type of data is essential in enabling prediction algorithms that can mitigate latency, such as time warping.
Disadvantages of Command Streaming
Reduced cost, lower latency, bigger scale, increased reach, higher quality, resolution independence… What’s the catch?
Keeping bandwidth low is a harder challenge for command streaming than for video. Graphics data is complex (e.g. textures, vertex buffers, index buffers) and streams uncompressed at a variable rate of up to multiple GBps. Uncompressed video data is regular (pixels) and streams at 24 × 1920×1080 × 60 = 372.5 MBps for a 1080p 60fps video. This means Command Streaming cannot use a standard compression codec with decades of research and hardware support, such as H.264, and instead requires research into new compression algorithms.
Also, the client needs to know how to interpret the graphics commands sent by the server. This means the client GPU should be capable and performant enough to render the game. Or as an alternative, the cloud gaming system needs to become increasingly complex, to adapt the game to different client profiles.
And that is the real tradeoff, complexity. A client that can only use OpenGL could render a DirectX game using a translation layer; features of newer graphics APIs could be emulated; GPU bound workloads could make use of dynamic resolution and variable rate shading; heavy shaders could be replaced with lighter versions, even at runtime. The options are as endless as they are complex. How hard can it be?
Enabling new business models
Looking beyond technical complexity, the main benefit of command streaming is in enabling a business model that is sustainable. When game streams cost cents instead of dollars per hour, cloud gaming platforms can finally aim to be default alive. Besides, aspiring platforms don’t have to build their own GPU data centers, developers and publishers can provide instant and continuous access to their games without source code changes, and players can rejoice in not having to live near a GPU in the cloud to be free of update screens.
Follow Nuno on Twitter @nunopleiria
References
[1] https://cloud.google.com/compute/docs/regions-zones/
Accurate as of 2020
[2] https://cloud.google.com/compute/all-pricing https://cloud.google.com/products/calculator/
Prices reflect spot pricing. A cloud gaming platform would optimize components and take advantage of economies of scale to reduce costs but there is still an order of magnitude difference between the cost of a server with a GPU and one without, especially when taking CPU virtualization into account.
[3] https://www.hpcwire.com/2016/08/23/2016-important-year-hpc-two-decades/
[4] https://help.netflix.com/en/node/14164
Netflix is available in 190 out of 195 countries