Voice will be the new standard of how you interact with your gaming console
The announced PS5 controller and several patents by Sony hint at a new development in the area of gaming and consoles. In this article I want to describe how a rather new way of interaction is going to influence your gaming experience.
We’re all used to handheld controllers. Walking through amazing landscapes and endless open worlds. Pushing buttons to do actions like speaking to NPCs or riding on a horse through nature with the control stick. Don’t be afraid — most probably this will stay for the next few years. But Sony has increasingly started to file patents for different kinds of voice interactions within their console and games. They are going to introduce an assistant called “ PlayStation Assist”. You’ll be able to ask this assistant different kinds of questions, interact with it, influence gameplay and it might even be able to recognize emotions and much more.
General Game Assistance
“Where can I find this resource? How can I craft this item? How can I defeat this boss? Which armor and weapons fit my playing style?”
Soon you might get answers just by asking your PS5 and the according game. In this patent from September 2019 Sony describes “Voice Help System using Artificial Intelligence” and another one filed in May 2018 an “In-Game Information Platform.”
(…) providing gaming assistance in response to a player’s query relating to a gaming application, wherein the response is provided to a device of a player (…), wherein the response may provide assistance during game play of the user (…)
In this patent from January 2020 Sony gets into more detail and describes an “In-Game Resource Surfacing Platform, which helps players specifically in terms of finding resources, crafting tools, armory or weaponry.
Embodiments of the present disclosure relate to methods and systems for searching and identifying in-game resources that assist a player based on contextual data and data from a community of similar players.
Furthermore the assistant can analyze how other players managed a certain situation and could give you instructions on how you can successfully complete the current challenge as well. It also describes a “virtual store” from where the player could select resources. If this is going to become a new level of microtransactions (which I’m not particularly a fan of, if the game is based on pay-to-win model) or not, I can not tell. I hope game studios will use this option carefully and not abuse the opportunity.
In another embodiment, a computer-implemented method includes an operation for receiving a request to access a virtual store that enables the player to select resources for use while playing a plurality of video games.
The gaming console interacting and knowing about real life context
The already mentioned patent of the “In-Game Information Platform” also mentions a case in which the console knows about “user activity” connected through a “user profile module”. The figure shows a person playing a level supposedly taking 45 minutes to complete and the system giving information that a TV show the user wants to watch is going to air in 30 minutes.
(…) a user profile module for identifying a user activity associated with a predetermined time of day; a scheduling manager for determining that the player will not complete the game course before the predetermined time of day associated with the user activity based on a current time and the estimated time (…)
Games reacting to the emotional state of the user
Another patent filed 18th June 2018, which was just granted 26th May 2020, describes a whole new level of interaction with games. Through “receiving input from camera showing face of user” and/or receiving “audible input from user via microphone” it can “tailor audible assistance based on location/section, first level of confidence, user’s speech, user’s gameplay ability, difficulty level of the game, user’s identified emotion(s), objective of user, user inability to complete task within average time, etc.”
In my opinion this is huge news and will shift gameplay to new dimensions!
Let me sketch a few scenarios I could imagine being enabled through this patent:
- In an open world game the game could respond to your emotional state. If you seem to be lost and randomly talking to yourself “Where can I find that?”, it could proactively give you assistance. If you’re amazed by the landscape you’re walking through it could tell you more about it or shift the soundscape according to your behavior.
- In a shooter it could give you audible assistance if you made a mistake, help you to become better in certain situations and it could also suggest to change weapons.
- While playing a sports game, let’s say soccer, there could be an optional game mode which reacts to you insulting the other player. Let’s say you play an online match, the referee decides to give your opponent a penalty and you’re outrageously screaming at your TV several swear words. The game could react and give you a yellow card.
Conclusion
The recent rise of voice assistants like Alexa and Google Assistant (and yes, Siri is slowly improving as well), the incredible improvements in NLP (Natural Language Processing) and people getting more and more used to this rather new interface, it only makes sense to introduce it in gaming consoles as well. The technology opens up endless opportunities for game studios and developers, who can design incredible new gaming experiences, which are even more immersive.
What do you think about this topic? Let’s discuss!
Originally published at https://www.linkedin.com.
