Primitive space system Raised $ 8 million for its subsidiary Real-time interactionAnd will use it to make a technology called GPEG, like MPEG format The format used for video playback, but for graphics.

But GPEG (a content streaming protocol) is a different way of data visualization, and its creators hope increase the appeal of the game and make people like they can be part of an animated TV show. Instant Interactive, on the one hand, wants to use its GPEG technology to stream games more efficiently, and on the other, it wants to transform passive video entertainment into more interactive and attractive products.

This is probably where most people stop reading this story. But I think there are many aspects to this technology. The idea of ​​the Geometric Pump Engine Group (GPEG) was derived from the people of Instant Interactive co-founder Barry Jenkins (Medical Doctor who became a graphics expert), John Scott (predecessor of Epic Games) and Solomon Luo (a medical vision expert), considering years of The challenge was to set up startup Primal Space Systems and Instant Interactive, a gaming-focused division.

Investors include a variety of seed and angel investors, including ImeveGroup co-founder Steve Charles. The capital will support the development and first release of GPEG, which can be used with game engines such as Epic's Unreal. As a company, Instant Interactive has been working on this technology since 2015. It has only 7 employees.

Jenkins has invested in this technology for more than a decade. The company has 11 patents for technology that can better play games and make TV shows into interactive entertainment. Bill Freeman, the company ’s president, said that while MPEG (short for Moving Picture Experts Group) provides us with technology to compress video so that it can be easily viewed across the network, GPEG can make certain very expensive gaming and entertainment technologies practical. Interview with GamesBeat.

GamesBeat Summit April 28-29 | Two Circuses | Los Angeles, California. A new generation of dawn. Join more than 500 game leaders for 2-day inspiring talks and unparalleled social networks.

Above: Bill Freeman is president and chief operating officer of Instant Interactive.

Image source: Real-time interaction

"This technology enables interactive content on top shows such as interactive Netflix shows," Freeman said.

For cloud gaming and interactive TV, GPEG replaces video-based streaming by using pre-encoded content packets, which can be streamed more efficiently using GPEG middleware technology. Packets are pre-fetched to eliminate latency while reducing overall streaming costs. GPEG's middleware solutions are designed to work with all existing content delivery networks and integrate into any game engine, including Epic's Unreal Engine 4, to effectively deliver truly instant personalized content.

Jenkins said in an interview with GamesBeat: "We think it is possible to introduce entertainment to interactivity, break the interaction between players and create content that everyone can consume."

The possibility of interactive entertainment

Above: Netflix's "Black Mirror: Bandersnatch" show lets you choose effects.

Image source: Netflix

If you want to know everything, you may have heard the story of Epic Games' Unreal Engine (mainly used for making games) used to make movie special effects for the Disney + TV show Mandalorian. GPEG can be used on top of Unreal Engine, it enables TV shows, allows you to participate in the action, and directs the story in the direction you want.

With GPEG, the interactive entertainment industry Black Mirror: Bandas Natch Showing where you can choose your own story can improve your technical skills. Jenkins envisions GPEG as a more efficient way to create content for use across media or across multiple media such as comics, movies, and games.

See also  Mars Express images show boundaries between hemispheres

Today's games often have richly visualized movie sequences or " cuts" that are presented in real-time in the game. Advanced real-time rendering effects now provide games with more complex movie effects than ever before. At the same time, the art of interactive narrative has come a long way since Professor Henry Jenkins (then MIT, then University of Southern California) talked about cross-media.

The focus of narrative-driven games such as Detroit Becomes Human and Life Strange 2 is to allow users to actively discover and participate in conflict and resolution cycles that are critical to the story.

And, if you focus on streaming, today's edge-centric content delivery network (CDN) infrastructure can deliver pre-encoded video streams at a very low cost per user and on a global scale. However, this infrastructure is not sufficient to support video-based cloud gaming systems, such as Stadia or GeForce Now, which rely on expensive game server hardware in dedicated data centers, Freeman said.

Although most of today's game content is delivered by CDNs, this delivery comes in the form of slow game downloads, which usually requires users to wait a few minutes or hours to start the game. New ways to stream game engine and VR content to gaming consoles, gaming PCs and mobile devices
Devices can eliminate these download delays and provide instant access to interactive content.

Using the existing CDN infrastructure, this new stream can change the way games are delivered over broadband and wirelessly, and it can also provide new instant interactive content for wired and over-the-top (OTT, considered Netflix) viewers.

Jenkins says you should imagine that an animation program or special effects sequence would use
Applications such as "HBO Max Interactive" or "Netflix Interactive". Such a program will convey the story-driven "backward-looking" entertainment experience expected from a video, but will also allow users to selectively choose a game controller and customize characters, explore different narrative arcs, accept short-term challenges or take other Way to "slim down". More engaging experience than the simple branch videos of Bandersnatch.

Such programming naturally attracts gamers, and it can also attract non-gamers and more mainstream audiences, resulting in unique and compelling entertainment value. This type of programming can enhance traditional cross-media approaches by blending movie experience with the immersive and engaging media experience brought by modern game engines.

Instant Interactive wants to revolutionize graphics

Above: The output of the GPEG encoder for a single viewcell is shown. Calculate fully visible triangles (blue) using a conservative out-of-region visibility pre-calculation method that is faster and more accurate than ray tracing.

Image source: Real-time interaction

Instant Interactive is leading the development of the game engine middleware protocol GPEG, which can stream interactive content to gaming consoles, PCs, mobile devices and next-generation set-top boxes.

Primal Space Systems itself uses this technology to enable drones to transmit data more efficiently as they fly over a place and capture image and location data. The US Army is using the technology. But Instant Interactive uses the technology to stream games more efficiently and transform passive video entertainment into something more interactive and engaging.

See also  “Louvre Agreement” will establish new partnership between CS: GO team, ESL and Dreamhack

Freeman said, "We all grew up in MPEG (Video Format)." "This is not MPEG. This is a new way to encode and stream 3D data. Games have always been our core, but we see GPEG go beyond games, Bringing interactivity to content with passive content that will move forward. "

Cloud gaming and interactive entertainment need help. In theory, placing games in the cloud so that they can be processed using heavy servers is a great way to handle games. You can then stream the video of the game to the user's computer or console. As the user interacts with the controller, the input is sent back to the data center, where the impact is calculated, and the new scene is sent back to the user's computer as a video. This allows heavy work to be done in the cloud, so high-end games can run on low-end laptops.

The problem is that this data stream consumes a lot of bandwidth, and cable modem systems have only recently been able to transmit data at high enough speeds to enable cloud gaming services such as Google Stadia and Nvidia GeForce Now. But Jenkins claims that with GPEG, data can be reduced and transmitted using a fraction of the bandwidth required today.

"We know what OnLive was trying to do a few years ago," Freeman said. "However, with today's infrastructure, we can stream fully interactive content without delay, loss, frame loss or download, and provide the highest quality instant interactive experience at a much lower cost. Provided. What we are most excited about is interactivity. Therefore, with our technology, VR, MR, AR content can indeed be achieved. And because we have never downloaded everything, piracy is eliminated. This is not the traditional cloud adopted by others Games. This is a new way to stream data. "

Crazy doctor

Above: Barry Jenkins is the co-founder of Instant Interactive.

Image source: Real-time interaction

Jenkins received his medical degree from Harvard Medical School. But he was very interested in computational models of human vision, computer vision, and real-time graphics. He wrote software for large-scale 3D dataset processing, and defense contractor Northrop Grumman used it to render images.

"We focus on the 3D data itself, not the video data, which is exactly what Stadia and GeForce Now do." "They run the entire game, of course in the data center, then compress the video frames and send them to Users. As you know, this was possible many years ago, but not necessarily feasible. Indeed, the cost per user is very high, while the cost per user in the data center is almost unprecedented. We estimate that Stadia will Give a user $ 1,500 worth of equipment. "

Compressing game content into video is not very good, so it takes up a lot of bandwidth. In cloud games, the player's laptop sends input, then enters the data center, performs calculations there, and then sends it back to the computer in the form of a video, so there is a lot of latency or interaction latency. . This delay is okay at 720p resolution. But today's 4K TVs don't.

See also  Chinese hacking frenzy will last for decades

How to fix the game stream

Above: In the top-down view of the game level, the green part is the part of the that the user may see.

Image source: Real-time interaction

Jenkins said that because Instant Interactive's software has been integrated into the Unreal Engine, it can solve this problem and enable game publishers, game developers, and game publishers to provide a better experience for their users.

"In fact, we are streaming the game engine content itself to the PC or mobile device on the game console in a very flexible way," Jenkins said. "We have taken away the game itself, all the cards that make up the game. We process them offline through a GPEG encoder and then convert it into data, which is GPEG. The packet data can be located on any server. It does not require a GPU and basically does not require any CDN server.

He added: "Its purpose is to stream data into our GPEG client software, which is a plug-in to the Unreal Engine. Therefore, we have integrated it into the Unreal Engine. It has been written to be integrated into other Engine. Basically, we just download the data needed for the current game, not the entire game. All are prefetched. So you do n’t experience any delays. "

The software takes a given scenario and breaks it down into units. The software then accurately pre-calculates which textured triangle surfaces are visible in each box. We then code the changes between the boxes. All of this data is on the server, which predicts where the user is within the game and then only shows the part of the that the user can see. It also predicts what users will see in the next few minutes and pre-calculates it for display to users. The game engine does not need to render the rest of the scene.

Jenkins said: "So instead of downloading the Call of Duty game with 140 GB downloads, it can really start with tens of megabytes of data." Here is a real opportunity to manage it with great precision The game engine actually needs to render content at any point in time to improve the performance of the game. "

Go to the past

Above: The 1996 earthquake

Image source: id Software

Computer scientists have studied this concept for decades. John Carmack, a graphics guru who worked on games like Doom and Quake at id Software, was the first Quake game engine in the 1990s to actually use this technology in games.

"In fact, he used this pre-calculated visibility to increase the frame rate of the game by to four times," Jenkins said.

When others abandoned the technology, Jenkins did not. He applied it to the modern world.

Instant Interactive hopes to license the middleware it creates to other companies.

"We want to see it used by many different game publishers and producers from Valve to Epic to Activision and EA," Freeman said. "Bandwidth requirements are actually much more reasonable than video-based game streams because We are not using any compression. In fact, we are using the structure of the game data itself. "

Jenkins said that GPEG allows for better interactivity, instant access, efficient and lossless transmission, and fast delivery of 4K and VR content.

"It's really middleware designed to help game engines run better," Jenkins said. "And you're watching Netflix and decide to jump off the photographer's camera track for other narratives, or take a brief challenge and let I ’m more addicted to the story myself. Video-based streaming does n’t really let you do that. In a matter of seconds, you are actually playing something else. Playing a 22-minute video clip every week may take an hour or more Long time. "