Применение технологии видео 360° для презентации культурного наследия Соловецкого монастыря

1. Introduction

At the present time virtual reality technologies and their applications have developed intensively [1]. Using these technologies for Arts and Culture [2] allows for a wide variety of opportunities due to providing the user with the effect of presence in the very center of cultural events. One of the most prospective approaches of creating content for virtual reality systems is the Video 360° Technology [3]. This technology allows the creation of panoramic videos with different grades of interactivity where the users changes an angle shot (a camera angle) of the video accordingly to their desire. The video can be viewed through virtual reality headsets (for example, Oculus Rift [4]) as well as on a smartphone, via a special app when the user rotates the video “around himself” by moving his head or turning a smartphone. The video can be watched on computers also. In this case, the user manages the foreshortening by using a mouse or a keyboard.

Many big companies (Facebook, Nokia, Samsung, Google and others) develop video cameras for shooting video 360°, virtual reality garnitures for different smartphones and personal computers as well as audio recording devices that provide the binaural sound and the realization of the technology Multimedia 360°.

The panoramic video plays an important role for preservation and provision of access to cultural heritage. The technology provides the unique opportunity to see not only closed for public access archives of museums but also reconstructed historical monuments that were destroyed by time or circumstances (3D virtual reconstruction). Moreover, virtual reality provides disabled people a unique chance for exploring to different parts of the planet.

The project of development and applying of the Video 360° Technology to culture and arts has been realized in the Center for Design and Multimedia of ITMO University [5]. Certain results that are necessary for creating high-quality content for virtual reality were achieved within the frames of the project, starting with original devices for filming video 360° and recording audio to software for special task solutions in this field [5, 6].

2. The Video 360° Technology

At present video cameras for filming video 360° are only being developed. However, it is possible to use regular video cameras with special equipment and software for this purpose.

One of the outstanding examples of this approach is the project 360Heros (http://www.360heros.com/) that offers special rigs (video gears) for GoPro cameras HERO 4. The amount of cameras dependent on the tasks in the rig is from 6 to 14 (the stereo version). After a shooting video streams are combined into one spherical video with the help of the special software.

Moreover, several big IT-companies and electronic manufacturers have announced their own fully-automated solutions for shooting content for virtual reality — Google Jump + GoPro Odyssey (https://gopro.com/odyssey), Samsung Beyond (http://thinktankteam.info/beyond/), Nokia Ozo (https://ozo.nokia.com/).

The team of project “video360production.com” films 360° videos with the help of its own invention – the camera 360°. This camera provides spherical quality up to 24K and creates several video streams that are afterwards combined into one panoramic video with the help of a special sequence of algorithms developed by programmers. This approach provides a high quality product decreasing parallax errors in panoramic videos that is a complex issue to resolve. In the field of video content for virtual reality, “stitching” quality is the most challenging feature to maintain with the only help of the software. Often parallax errors can be eliminated only during the postproduction.

The team of project “video360production.com” unites experts from two fields - engineering (engineers, programmers, hardware and software experts) and art (stage directors, videographers). While one part of the team develops solutions for maintaining a high quality of video 360, the other part provides a creative approach to every case during the film production. It means that the employees of the Center for Design and Multimedia do not use only the unique equipment and software but also a unique approach for every video shooting depending on a set of factors such as interior, exterior, weather, season, time of day and people in a shot.

Video 360° is a new and upcoming trend that doesn’t have any standard direction. It gives a wide variety of opportunities for new ideas and methods. The effect of full immersion is achieved due to the camera 360° that shoots everything happening around with the help of several lenses. However, it causes some challenges — for instance, a vast majority of film editing rules and film grammar do not exist in video 360 (for example, screen direction). The main distinction of video 360° in virtual reality glasses is the fact that the audience has the opportunity to look in any direction “around itself” which means that the producer will give an opportunity of dynamic choice of viewing angle without focusing the viewer on a specific part of the screen.

One of the solutions is a detailed scriptwriting with special attention to changes of paradigm — 3D filming. Every video project requires not only a standard preparing (such as setting up lights, choosing the best camera angle, creating a scenario and a composition) but also individual approaches (unique for every case camera mounts, binaural audio recorders, and special ideas for video 360°). For instance, for videos of musical concerts, the recorded production sound track from a camera or stereo microphones is often combined with other elements such as effects or noise of the audience in order to reach full immersion into a video story.

3. Presentations of Virtual Excursions with the help of the Video 360° Technology

The team has filmed more than 50 different events for virtual reality glasses such as concerts, musical festivals, excursions in historical places, performances in theaters and different cultural events [6]. Furthermore, the video360production project works with leading St. Petersburg theatres. Videos 360° of historical attractions in the city and excursions to museums provide a unique opportunity for users to see the main sights of the city at any time with effect of presence. As for the Peter the Great Museum of Anthropology and Ethnography (Kunstkamera) that is one of the biggest museums in Saint Petersburg, a hall with the Greater Academic Globe, a diplomatic gift to Peter I on the Holstein Duke Carl Friedrich during the Northern War, was filmed before closing for reconstruction [7]. Consequently, now it is the only chance to visit virtually this hall.

Video 360° technologies have many fruitful applications for virtual cultural tourism [6, 8, 9]. The tour was filmed during the excursion on the rivers and canals of St. Petersburg (Fig.1). The virtual canal tour in Saint-Petersburg, which was published in public access on the YouTube channel, [10] has become incredibly popular.

The team of ITMO University has started the realization of the project ‘The point of presence’ with the State Hermitage. Within the frames of the project, the team will represent different events in halls of the museum with the help of the panoramic video. The first story of this series of video 360° films  is ‘The Peacock Clock’ (Fig.2).

4. The Presentation of the Solovetsky Monastery Cultural Heritage using Video 360° Technology

The special attention our team was paid to the filming of different events in the Solovetsky Monastery in the format of video 360°. The Solovetsky Monastery played the unique and important role in Russian history and culture. The Monastery was founded in XV century as a place for religious reclusion of Sergius of Radonezh’s (‘the father of Russian monasticism’) progenies. In a short space of time the Monastery has become the religious, cultural, political and economic capital of the Far North.

Due to the works of the Solovetsky monastics, the Russian Church was given a great number of ascetics. As a result, Christianity developed on the Russian North.

The Solovetsky Monastery was the main factor which attracted people to the region of the White Sea. The Monastery had the great influence on the development of economics of the Seaboard. It was the main prerequisite for Russian seafarers to explore the Arctic Ocean which turned Russia into the Sea Empire.

The location of the Solovetsky Monastery is the main obstacle for a vast majority of people to take part in its religious events and celebrations. The access to these events can be provided by the shooting them with the help of the Video 360° Technology. By putting on the virtual reality glasses, the user can virtually visit the Monastery.

As a result of the team’s work, more than ten shootings were performed on Solovetsky Islands: the excursion in the Ascension church on the Sekirnaya Hill, the Celebration in the Solovetsky Monastery, the Christmas Liturgy, the Night Service, the Christmas celebration for children, the Religious Koledari (the Slavic tradition), different views of the Monastery and the Solovetsky Island (Fig. 3).

Special features of 360° video format require a different approach both to the technical and the creative part of filming and editing. It’s imperative to keep in mind that the viewer of a 360° video  is allegedly taking the place of the cameraman or his camera, and can thus encompass the entire surrounding space that the 360° video  can encompass. It’s essential to take this particular aspect into account while shooting a video in this technique.

Solovetsky Fortified Monastery is a fairly spacious object, both outdoor and indoor. While filming an excursion, with the guide in the frame, it’s important to position him or her frontally, so that after the cut she would properly remain in the frame. Thus it’s important to select the right frontal position of the object (Fig 4).

The number and the level of cameras used for filming people depend on the location (outdoor or indoor), as well as on the information embedded into the scene. For instance, the filming of a show for young spectators includes both stage action and the dynamics of behavior of the children and the choir. It’s important to find the right height and fixtures for the cameras, so that they could track both the stage action and the response of the audience, without distorting the human figures (Fig 5).

While filming the tour of architectural highlights of the monastery, the length of each sequence and scene was taken into account, to ensure comfortable viewing and processing of both the information and the picture. It’s important to take into account the specifics of filming, editing and sound mixing, as well as the viewer’s ability to embrace the interactive virtual reality of  video 360°  within a certain time space. Currently, 2—15 minutes is believed to be a comfortable length of a  video 360°  movie.

Shots filmed from a moving camera, especially the one without a stabilizing fixture, can cause discomfort. Besides, perception is influenced by a number of other dynamic processes: walks, flights, change of size and height of objects within a frame etc. It’s imperative to keep in mind that in 360° video  the size of the frame is determined by the positioning of the camera in front of an object only. None of the currently available software can adjust the frame size. While editing the moving shots, it’s important to watch the size of the frontal picture at the cuts. What can help are the well-known combinations of the frame sizes and static images that prevent the viewer from spinning around and losing the sense of real presence. That’s why tracking shots on the winter lakes and canals of Solovetsky Islands were sequenced as brief dynamic shots (Fig. 5).

It’s important to decide on the type of editing before the shooting takes place. As a rule, basic types of editing are used, i. e. linear, associative, and cross cutting of the shots, with the option of in-frame editing with the use of inbuilt augmenter reality. The inserts used are photos, videos, iconography, 3D, captions, signs. This type of data representation, through in-frame editing, was used in Mount Sekirnaya documentary, specifically in the episodes Angel’s Chapel, By the Cross, Executions, Isolation Ward in a Church. Newsreel footage, paintings and photographs were incorporated into the guided tour sequences (Fig. 6).

The approaches to managing the viewer’s attention while filming the tours of Solovetsky Islands have been developed over the last two years, with encouraging results. Using the 360° video  technology, we have filmed a number of virtual tours for our project’s website, Multimedia information system "Architectural ensemble of the Solovetsky monastery in the period of its highest prosperity (XVI-XVII centuries)” ( http://solovky.ifmo.ru).

Now, to tackle the higher level of creative challenges, with the use of virtual reality technologies, we need to search for new modes of expression. That means, above all, a new level of augmented reality – spherical Chroma Key, innovative approach to transitions and drama, through the introduction of new image processing software and plug-ins that would take into account the equidistant projection of the panoramic video.

All these projects and a great number of other cultural events, which were filmed with the help of the  Video 360° Technology, are published in public access on the YouTube Channel [14].

This work was supported by the project Multimedia information system "Architectural ensemble of the Solovetsky monastery in the period of its highest prosperity (XVI-XVII centuries)" (Grant 16-01-12022, Russian Foundation for Humanities).