Using Artificial Neural Networks to Compensate Negative Effects of Latency in Commercial Real-Time Strategy Games

Abstract

Cloud-based game streaming allows gamers to play Triple-A games on any device, anywhere, almost instantly. However, they entail one major disadvantage - latency. Latency, the time between input and output, worsens the players’ experience and performances. Reduc same game experience as in local gaming. Previous work demonstrates that deep learning-based techniques can compensate for a game’s latency if the artificial neural network has access to the game’s internal state during inference. However, it is unclear if deep learning can be used to compensate for the latency of unmodified commercial video games. Hence, this work investigates the use of deep learning-based latency compensation in commercial video games. In a first study, we collected data from 21 participants playing real-time strategy games. We used the data to train two artificial neural networks. In a second study with 12 participants, we compared three different scenarios: (1) playing without latency, (2) playing with 50 ms of controlled latency, and (3) playing with 50 ms latency fully compensated by our system. Our results show that players associated the gaming session with less negative feelings and were less tired when supported by our system. We conclude that deep learning-based latency compensation can compensate the latency of commercial video games without accessing the internal state of the game. Ultimately, our work enables cloud-based game streaming providers to offer gamers a better and more responsive gaming experience.