In spring 2020 Kuflex studio began an experimental project Kusmos Live. The purpose of the experiment was to upgrade the Kusmos system in order to create a interactive online home shows.
In 2018-2019 the Russian audiovisual studio experimented with the innovative tool together with SILA SVETA studio on the Therr Maitz concert, Caprices Festival 2018, Nina Kraviz experimental performance at Coachella 2019.
Scene functions build a 3D model using the data provided by the Tracker and then we can layer all kinds of features with effects on the model and also transform and distort it according to the artistic concept. We also captured video from the laptop’s webcam and sometimes showed its picture. 3D scene acting as the artist surrounding, a set of virtual cameras for capturing from different viewpoints and visual effects were set up in advance in the Unreal Engine. Then the OBS program captures the video of the launched scene and sends it to the video streaming server.
During our second live experiment we tested some new features for the viewers interactive communication with the stream. While Leksha (Smolensk, Russia) was playing his ambient-set, a VJ (Moscow, Russia) was controlling visual effects with the use of commands via YouTube chat in real-time mode. Our team has implemented this function between the concerts and decided not to tell the viewers about it.
During the stream, noticing weird messages in the chat, some of the viewers started to realize that they could not only send a message to the chat but to even affect visualization. In the end the concert has turned into a digital quest. Some viewers picked up effect control by sending particular commands to the chat. We have yet to comprehend how to develop this function in the future.
KUFLEX: Kusmos is a universal software tool, with the possibility of variability of visual and interactive solutions. As a rule, our team creates a virtual stage specifically for the performance of the musician. Of course, we want to upgrade the program creating a database with different scenes, effects. In this case, the user will be able to construct the scene himself and combine the effects for his live/stream.
KUFLEX: Regarding Kusmos Live project, the Kuflex team is collaborating with various musicians. We wanted to support the performers. So this approach determined the emphasis on the figure of the musician on the virtual stage, under whose musical personality, sound we come up with a visual solution. We do not just shoot a video with a musician, as is often done in broadcasts, but create a digital avatar that changes depending on the script, music and VJ control.
We try to achieve the effect of real interaction with the viewer as well. Our team is developing a function of interaction through chat – viewers’ comments fall into the scene, they can affect the content through certain chat commands. But Kusmos can be used by artists of other genres. In the near future we want to try to create a dance performance. Now we are discussing this idea with one Russian choreographer.
Technically, the performer will find himself in different areas of camera scanning, on the screen we will observe how his digital avatar changes. Again, it will all be like shooting a movie in one shot and in real time!
We intentionally did not talk about this function in advance to get the quest. As a result, some viewers guessed and began to help in managing the scene. We explore different possibilities about other ways of interaction.
In the future we want to create a client application for connecting to the broadcast via a mobile phone, desktop PC screen or VR. We intend to develop Kusmos as an art tool. Our team believes in a power of collective interaction. We want to give a palette of visual solutions, effects. Let’s all together create beauty here and now! This idea is a sincere inspiration for us.
We wrote a special function for our software that receives data from chat on YouTube using the Google API. We came up with several commands, for example: cam1, cam2, skin1, skin2, electric noise, lasers and the like. And when someone in the chat wrote one of these words, then a certain visual effect or a corresponding camera was included in our program.
In general, we have an idea to expand the number of commands and their appearance, so that it looks more like live coding. For example, add numerical arguments to the commands, which will additionally specify the parameters of a particular visual effect.
KUFLEX: Yes, this is the main object of research for us. Usually, a limited number of people can come to the offline exhibition. So we want to overcome any space frames. With Kusmos we don’t have any restrictions online! We can find ourselves in amazing digital worlds that are impossible in the physical world.
Now that Kuflex Lab and its creation Kusmos entered our radar we will most definitely keep following their progression, as always supporting innovation and creativity in the audiovisual art world.