Developing In-House Tools: The Polystream PPU
At Polystream, as part of our work on command streaming technologies and the metaverse, we’ve been developing an in-house tool: Polystream Packaging Utility (PPU).
I’m Charlie Tizzard Ó Kevlahan, and as a platform engineer, I spend a lot of my time problem solving! In this blog I want to introduce you to our PPU tool, highlighting why we created it to overcome challenges onboarding new applications to our platform, and how we use it today to stream applications from a developer to a server which is serving that application to an end user somewhere in the world.
Anyone who talks about the metaverse being interoperable and interactive at a global scale needs to figure out how they get their application into the cloud. To solve that problem, we’ve come up with a few solutions over the years.
To stream an application we really have to do three things:
- Prepare an appropriate application for streaming
- Get that application onto the Polystream platform
- Deploy that application to Agents (servers) across the world as needed for users
We’ve tweaked and experimented with this process on over 100 applications now, and we’ve tried it in a lot of different ways. The three core challenging questions we keep tackling along the way are:
- How do we store those applications and get them onto our Agents?
- How does the Agent know how to install and run an application?
- Then, how do we enable Developers to publish their applications onto the Polystream platform?
Charlie Tizzard Ó Kevlahan
What is an Application Package in Polystream terms?
The core of any streaming experience is the application. Whether you’re streaming Unity’s Editor, Alien: Isolation, or Room Zero – you’re streaming an application. No application, no stream!
In Polystream terms, an application package isn’t just the application, it also includes Polystream specific metadata such as the Application Id, Name, Asset type, Publisher, Developer, Description, and the Developer’s tenant information. You can think of an application package as coming in two parts; first the application with all it’s necessary assets, and secondly the run-config which is a JSON formatted file that describes everything the Agent needs to know to install, prepare, and launch the application for streaming it to the end user. Really, other than the application itself, the run-config is the most vital piece of information when it comes to streaming an application on Polystream. Without one or the other there is no stream, so we need to get them both onto the platform.
“The core of any streaming experience is the application.”
Some applications are instantly great for streaming and work straight away, especially those that are built to be cloud-native. Others need a little bit more thought and investigation.
For example, Polystream can stream hundreds of applications which weren’t necessarily developed, or at least released, with the intention of being able to be duplicated across and run on servers without any user input. But, it’s sometimes the case that we have to weave a little magic when working with applications which, for example, don’t have launchers that need to be clicked through, don’t have licensing which needs to be set-up per instance after it’s installed onto a server, or don’t have DRM mechanisms which leads us to having a Steam account (or equivalent) per server.
The issue of running Steam dependent applications is one that we’ve been experimenting with for a while now. Part of the issue is that beyond just logging into Steam, there are a slew of other changes or ‘hacks’ we need to do like:
- Confirm the account has the correct entitlements
- Make changes to the Steam registry values
- Update the installed applications Steam manifest so that it corresponds with the logged in account,
- Convince Steam to recognise the application in a location it doesn’t expect it to be (this is because the application isn’t installed through Steam, it’s installed by Polystream).
This process has historically been done manually as a series of PowerShell Scripts and finally we trimmed it down to a CLI tool which can be bundled with Steam and provides a usable interface to set-up Steam with minimal input. There are still problems we’re working on, for example we don’t have an ideal way to automate retrieving a valid Steam Account implemented yet – drop us a line if you’re working on something similar and let’s swap notes!
Okay! Back to packaging up applications onto the Polystream platform. In the past this has been a largely manual process which restricted a lot of what we could do.
First of all, we’d log-in to our internal Management Portal and register a new application package, and enter all of the attached metadata. We’d then have to prepare an application for streaming, and write the run-config, but the problem was this used to be done in a completely different place to where we would prepare the application!
The run-config would be developed and stored in our own GitHub repository where pushing a new run-config would mean triggering a DevOps pipeline which would then validate the run-config and push it up to the Polystream platform, ready to be deployed to an agent. But we’d still need to deploy the core application, ie, the tricky bit! In our earliest days, we used to have a single PC in the office which was fitted out with all the best hardware and high speed internet, and we would RDP onto this machine from our work PC. This PC would hold all of the preparation work for 100+ applications, and from here we would run the script to deploy the application to the Polystream platform.
Well, that’s one way to do it…
All of these assets – the run-config, the application, and any extras that are needed – are stored in Azure BlobStorage. The way things currently work, we register a new asset with the Polystream platform, then the platform will return a shared access signature (SAS) which is a signed URI pointing to the appropriate place in Azure BlobStorage. Our deployment scripts would then chunk the assets (currently the maximum chunk size available is 100 MB) and upload them piece by piece to Azure. When this is complete, we send a final message to Azure with a BlockList describing the order of blocks so that Azure can piece it back together as a single asset.
We’ve largely carried this process over to PPU and we’ve made some improvements, for example greater parallelisation using System.Threading.Channels and better reliability around transmission issues due to a wider variety of internet connections, so we’re largely happy with this solution and will be continuing to use it going forward.
Once that’s done we can install the application package onto an Agent, and the Agent retrieves all of the individual components of the package: the application, any other assets, and the run-config. The Agent then installs the application following the provided instructions and then, whenever a stream request is received, launches the application in the same way, and abracadabra the application is streaming across the globe!
So, to recap, we’ve packaged and published a brand new application. We registered a new application package in the Polystream Management Portal, we prepared an application and uploaded it using deployment scripts from an internal machine, then finally we’ve written a run-config and deployed it through GitHub and DevOps.
That’s three different processes in three different places, just to publish a single application
Next Steps – the Polystream PPU
All of that is a bit of a hassle! It also isn’t very ‘shippable’ i.e. there’s no way we could (or should) work with external developers or open up our platform to customers with that process. We wanted to make the process much more simple, flexible, and one that we could fully own without having to rely on GitHub or Azure DevOps being an integral part of the process.
It also needed to be available to anyone, anywhere (assuming they have authentication). Especially given that over the past year Polystream has been taking big technical steps towards building the metaverse, the whole team required an easy to use, flexible tool that allows us to quickly develop and iterate versions of Room Zero, that we can also start to share externally.
That’s why we decided to develop the Polystream Packaging Utility (PPU). PPU has been able to provide us with a significant number of benefits over past methods, the most important of which is using PPU to develop the application and the run-config into a combined single location; a folder which we initialise using ppu init to turn it into what we term a ‘Polystream initialised application’. This works in much the same way as Git where you have a hidden .git folder that has all the information Git needs to manage the directory. We instead have a .polystream folder which holds authentication data, metadata for the application, the run-config, past builds, and PPU logs.
We’ve also streamlined the run-config development process using the ppu config provided with information about the application e.g. architecture, dll location, launch arguments. This is able to generate a basic run-config that will work for most applications. If the application is a bit more complicated, we might need to take extra steps during the install or streaming process but at least now we have already generated a framework to work with. We can then validate this config file while we’re working on it using ppu lint, which validates the JSON schema, referenced steps, and resources.
We also no longer need to go to the Polystream Management Portal to manually register a new application package, PPU does that all automatically with the information provided when a new application package is published to the platform with ppu publish which gives us back time to do more experiments!
Because PPU is just a CLI it can be integrated into whatever build process our internal team chooses to use. It can be a part of DevOps, CodeFresh, or any build process that supports running the published executable. Our internal team has already integrated PPU into their Perforce pipeline.
Obviously, there are more benefits and opportunities going forward with PPU than these quality of life improvements for our team 😀 PPU is the beginning of us providing tools for the community to really begin developing a new era of cloud-native, interactive applications for the Polystream platform, testing Command Streaming, and by extension the metaverse as a whole. If we want the metaverse to truly succeed, we need to start providing the tools for everyone to begin creating.