The Heritage Large-scale Immersive eXperience Contest
The first Contest to scale up ambitious immersive projects for European cultural heritage.
Heritage-LIX is a springboard to co-produce, distribute or finance sustainable and viable immersive experiences.
The 2022 cohort
- Participated in a dedicated workshop in Stereopsia EUROPE, the Immersion Forum
- Challenged their business model with about 30 Belgian operators and renowned international experts
- Pitched their product to European decision-makers in tourism and innovation
- Had the chance to get one of the 3 creation grants to highlight their concept during the BRIGHT Brussels festival
INTERVIEW SERIES - MEET THE MENTORING PRIZE WINNERS
XR4Europe sat down with 3 of the Mentoring Prize winners in an exclusive interview series:
At El Gabinete, we work towards delivering content and providing wayfinding to every visitor profile to heritage venues using the technologies vastly available for the public reducing any frictions on the delivery of that experience.
Meet Carlos Mondragón
Carlos is co-founder of El Gabinete which is focused on creating relevant content enriched with AI technology delivered through IoT devices, and AR wayfinding tools in natural and cultural heritage venues,
Previously he founded a company specialised in the use of technology for science and art popularisation.
1. Could you tell us how your project came about, the vision and goals underpinning its concept and creation?
We are two co-founders. We have been working for Heritage organisations for more than 15 years now as we’re passionate about Heritage and bringing the stories and the knowledge behind it to all visitors. Having realised that every visitor has unique interests and needs, we founded EL GABINETE in Portugal three years ago to provide products and services to meet this diversity. With the adoption of smartphones booming – among other factors, we saw huge business opportunities in our field. Our company’s vision lies in designing and delivering technology products with a rich content base for Heritage venues.
2. How would you describe your solution? What makes it unique?
EL GABINETE is a B2B company (also B2C soon) that aims to boost better informed, inclusive, and meaningful visits to Heritage places for all kinds of visitors. For instance, visitors may speak different languages, have varying timeframes to visit the site, or have visual or hearing impairments. In parallel, it is of great importance to manage and deliver visitor-specific content that can easily be accessed by one’s smartphone. With this in mind, we create interactive content for each visitor profile (eg. a family) on a dedicated Heritage site.
We have developed a Wayfinding tool, that powers customised maps to locate relevant points of interest and an AR function to show directions to the visitors, via our Web app. Indeed, we found that visitors usually find it difficult to know where to go when they stroll around in extensive heritage sites, or what is of interest on their route. We use artificial intelligence for content creation, and for scene and object recognition. If you open the Web app and focus on a scene or painting, you can directly know which scene you have in front of you as content is generated accordingly.
3. How does XR/your tech change the way of being creative, rather than other displays?
Many have thought of our technology as Pokemon-go. This is similar, but with content specific to visitors: receiving an AR route that triggers content depending on who you are. If you speak Italian, we can translate and produce audios in real-time in the appropriate language. So this is interesting in terms of creativity.
4. What were the most prominent challenges (storytelling, technical, IP, funding, commercial etc.) you faced prior to the development of the project, and/or during?
We are two co-founders with highly complementary expertise and strengths. Mar Fuentes, CEO, has been working in the Heritage sector for over 25 years. For the last 15 years, she’s been working in interactive content and visitor experience. Our fields of expertise complement each other nicely, as I am an engineer myself.
Our most prominent challenge has been business development. Moving the company from Mexico to Portugal has been both a great opportunity as Heritage and tourism is a huge field in Europe, but also, unchartered territory in many ways. As we arrived there in 2019 right before the COVID-19 pandemic struck, it was difficult to find customers at first. Now we’ve developed business in Portugal, Spain, and soon in Italy, with a B2C opportunity in Germany.
Entering a new market that forms part of a wider ecosystem with specific authorities and bodies, was also challenging. At first, we thought the best way to enter the Heritage market was to approach the Portuguese Ministry of Culture and cultural authorities. The bureaucratic burden was heavy and complex to navigate. It came to our attention then, that to enter the Heritage market, our direct contact points should be entities linked to tourism, as they showed a strong interest in our visitor-centric services.
5. Could you tell us a bit more about your role in the project? What are the main lessons learned from undertaking this journey? Any specific trade-off?
I am co-founder, with an engineering background, and responsible for designing and developing the products. Listening carefully has been one of the most important lessons learnt. When we opened here in Portugal, listening to clients and various cultural and tourism stakeholders to identify the needs was a very rewarding experience. The Wayfinding tool was developed as we saw a real need for our product: the machine learning tool for translating languages. As a startup, we’ve also learnt from ‘failing fast’: if we have an idea, we try to test it to see if there is a future or not as soon as possible.
Our business was also confronted with the pandemic. Against this backdrop, we considered using our technology for other customer segments, and were given opportunities to collaborate with other public venues such as airports or universities. But we decided not to move forward with this idea. Instead, we kept our focus on the Heritage niche and great content, because that’s what we do best!
“Metropolis” is the collective term for a multi-player 6DOF experience in which people can jointly visit spectacular places in history. Joined by an experienced guide, be it an historian, a celebrity …, users will discover these worlds together.
Their first location is Ancient Rome, which can be entered and experienced in photorealistic quality for the first time in 2000 years.
Meet Maria Courtial
Maria Courtial is co-founder, CEO and producer at Faber Courtial – studio for digital worlds, which she established in 1998 (together with her husband and director Joerg Courtial) after graduating in industrial design.
Under Maria’s leadership the company developed into one of the leading VR and VFX studios in Germany. Fascinated by new technologies and their possibilities, Maria embraced immersive media and set a new focus to XR.
1. Could you tell us how your project came about, the vision and goals underpinning its concept and creation?
You might say the project METROPOLIS – Rome actually started 6 years ago, when our studio, Faber Courtial, was commissioned to build 3 time periods of Ancient Rome for an exhibition entitled “The Popes and the Unity of the Latin World” in a German museum. Over the next years, sequences produced with these models have been used in different films and documentations. With each project, the model became more detailed. And since we were always keen on crawling into our models, we started creating virtual reality implementations. Initially, our studio focused mostly on VR film, but for two years now, with a modified pipeline, we have been able to produce interactive experiences. And the first one we focused on, was ancient Rome in all its glory.
That´s why METROPOLIS – Rome has been developed in two stages: the first was the reconstruction and detailing of certain places of Ancient Rome, which required meticulous and iterative work; and the second aimed to imagining and creating an interactive virtual tour that should also allow multiplayer interaction.
METROPOLIS is the overarching term for this kind of experience – a multi-player interactive experience in which people can jointly visit spectacular places in history or places that are not easily accessible. Indeed, the project’s underpinning concept is replicable to other spaces and time periods. For instance, the same concept could apply to the Pyramids in Ancient Egypt.
METROPOLIS – Rome represents the first in a series of projects that aims to explore new territories virtually, along with other history enthusiasts.
2. How would you describe your project? What makes it unique?
METROPOLIS – Rome offers a highly photorealistic 3D model of Ancient Rome experienced during a live guided interactive virtual tour. This solution is unique in combining the detailed and photorealistic digital reconstruction of Ancient Rome with an experience easily accessible with standalone headsets. No connection to a computer is needed. Nothing alike exists.
To experience the virtual guided tour, two options are possible: either on-site with standalone headsets for up to 8 participants including the tour guide; or at home with your headset via an app – experiencing Rome alone, with the possibility to be joined by friends or a small group of people.
As regards the on-site experience, it can be deployed in venues such as museums, educational institutions, and as part of specific events. In this case, the venue receives a special package that includes 8 VR standalone headsets and a Wifi router. Once all participants put on their headset, they get immersed into ancient Rome and can see each other as avatars, as Roman busts. The tour guide communicates insightful information and thrilling stories to the visitors throughout the virtual journey in which they get to explore 9 locations, and exchange on their shared and unique experience with their fellow visitors. From the first glance, people will immediately feel like a captivated witness of the time and completely forget that they have just travelled 2000 years into the past. The experience is created in a very intuitive manner for both the tour guide and visitors to smoothly discover the different locations.
Accessing the experience from one’s home via the app gives the user more options. You can experience Rome together with a virtual tour guide (filmed and inserted as kind of a hologram in front of you) and listen to exciting stories, with the possibility to be simultaneously joined by friends.
3. How does XR/your tech change the way of being creative, rather than other displays?
The very concept of METROPOLIS – Rome live guided virtual tour is only made possible thanks to XR technologies. XR offers a novel immersive and interactive way to experience places and even interact with others as if you were actually there.
This virtual historical reconstruction of Ancient Rome combined with the multiplayer feature enables users/visitors to get fully immersed, engage actively with the content, and converse about something they are passionate about. XR brings interactivity that 2D documentaries or traditional formats do not offer.
4. What were the most prominent challenges (storytelling, technical, IP, funding, commercial etc.) you were faced with prior to the development of the project, and/or during?
METROPOLIS – Rome took shape in parallel with the development of numerous projects. As it wasn’t funded, the team could only get involved in its development when other projects were sufficiently advanced. Hence, kick-starting METROPOLIS – Rome was rather slow. Setting some key milestones and deadlines to drive the project forward was essential to its success.
We were faced with several challenges. We had set ourselves the goal to develop a tool that would enable us to maintain high quality content by reducing data size significantly to run the content on standalone untethered headsets. For this reason, we dedicated much time developing in-house a specific toolbox called PHALANX, that enables to reduce data while upholding photorealistic content. Taking the time to explore and finetune this process contributed greatly to METROPOLIS – Rome as we know it.
Interoperability is also a challenge as we aim to bring people with different headset brands to the multiplayer experience. Hence, developing a product requires continuous efforts to figure out how to “do it right”, and bringing an “overarching” solution that enables people using different headsets to access the experience together, in the best possible way.
5. Could you tell us a bit more about your role in the project? What are the main lessons learned from undertaking this journey? Any specific trade-off?
I work as producer at Faber Courtial, for multiple projects. For the project METROPOLIS – Rome in particular, my main role was to push forward the product launch and readiness, identify relevant events where to present the product, as well as devising a business model and strategy. Of course, it also entails co-directing the project with my husband, by looking into how to improve our project, and make sure the immersive multiplayer experience can be seamlessly deployed both online and offline.
Building a strong business model and the early consistent thinking through of all operational levels are key lessons we learned at the end of the project. Thinking about the whole process leading to the finished product is paramount. Despite having always known what the content of the project would be, we had not sufficiently developed our thinking regarding the finished product across different channels (online and offline), which ultimately required us to develop it as one app, but with different potential use cases. In short, how the experience should look like and how it will work, as well as how to make sure it is deployed both online and offline (on-site) are key aspects we should have further dived into earlier. But often, ideas or innovations emerge along the development process, that’s important and a beautiful thing!
Augmenta is a technology that helps audiovisual companies to bring interactivity in any type of immersive space, regardless its size and the number of people. It’s a complete environment that combines hardware, sensors and a software suite to deploy, automatize, monitor and scale immersive experiences in the most efficient way.
Meet David-Alexandre Chanel
Artist, engineer and entrepreneur, David-Alexandre Chanel co-founded THÉORIZ Studio with Jonathan RICHER in 2014. Combining an education in music and an engineering school, David-Alexandre Chanel uses art and science to create innovative and poetic art installations.
Since 2011, he co-signed many digital art installations seen in international renowned art festivals and received several international awards. In 2022, he founds a new company Augmenta, a tracking technology intended for the creative industries.
1. Could you tell us how your project came about, the vision and goals underpinning its concept and creation?
It all started about 10 years ago, when we were using new technologies for art and entertainment at my Art & Technology creative studio THEORIZ. The technology we came up with didn’t exist at the time. So what we started to do in the early years is deploying some interactive videos that would move with the dancers’ movements for contemporary art shows. We started building a tech prototype for this purpose. Gradually, we improved the technology until the demand for interactivity grew for such projects.
Since then, we launched a spin-off company AUGMENTA to help people deploy interactivity in the space and democratise this new format for the audiovisual and creative industries. We want creators to focus on their creative process and craft, while supporting them with our tracking technology to add interactivity.
2. How would you describe your project? What makes it unique?
AUGMENTA aims to easily deploy interactivity in the space within audiovisual, entertainment, and cultural industries. We develop an augmented tracking technology to efficiently deploy interactivity for the audiovisual media professionals (audiovisual integrators and interactive studios), that consists of both hardware and software. Our approach is similar to that of video mapping but with an interactivity component (via 3D cameras and sensors).
We’re the only studio that provides such technology. If you want to do interactivity or tracking in public spaces, many existing technologies will always or too often require people to wear sensors (ie. electronic devices). With AUGMENTA, we overcome this barrier by enabling people to enjoy the creative content in the space, without sensors.
Through our expertise, technical equipment, algorithms and tools, we are highly agile in responding to a wide range of requirements and constraints for different venues. AUGMENTA’s technology is compatible with all content creation software available on the audiovisual market. Our unique value lies in enabling people to learn about and leverage all the creative possibilities emerging from this technology, so they can start imagining creative interactivity content.
3. How does XR change the way of being creative, rather than other displays?
Since interactivity in public spaces is something new, it means that creators must adapt to the technology and learn how to harness and use it within their creative process – the same way creative directors had to understand the rules for crafting impactful cinematic VR for instance.
Given the newness of the field, we see that people are starting to design interactivity meaningfully. In a sense, our work is transforming the surrounding space into a “giant Ipad”. When it comes to the Ipad, content makers are used to designing great applications on there. But now that we can also transpose that into physical spaces, it’s a challenge for audiovisual professionals to imagine interactivity in the space, beyond the usual 2D videomapping. They need to adapt their practices and skillset to new contents that include different interaction possibilities.
4. What were the most prominent challenges (storytelling, technical, IP, funding, commercial etc.) you were faced with prior to the development of the project, and/or during?
One of the main challenges was the development of such technology itself, as it is not an easy endeavour. It requires much expertise in various domains. This tech is real-time and low-latency, and relies on different types of sensors linked to various software. In 8 years of technical development, we have gradually lifted all those challenges one by one.
We are the first startup to offer this type of technology. Hence, we need to make sure that cultural and audiovisual players know about it and understand what it can offer them, as many might think that the technology cannot resolve specific technical obstacles they face. Making the technology known globally is our goal.
5. Could you tell us a bit more about your role in the project? What are the main lessons learned from undertaking this journey? Any specific trade-off ?
I am the co-founder, creative director and managing director of THEORIZ. I am also COO and CTO of AUGMENTA. With my 10-year professional experience in the audiovisual and cultural sectors, I’ve learnt a lot as to how people work in those fields, how roles are defined, notably the ones of the technology and service providers there. . As a creator of interactive media for ten years, I am intimately familiar with the constraints and needs of these industries.
Signal processing is my technical background, which is related to building algorithms that can analyse images from cameras and other sensors. Since I’ve worked in laboratories, startups, and in software development, I’ve gained a deep understanding of the tools, sensors, algorithms and calculations necessary to create an efficient real-time technology.
One of the main eye openers of that journey was how to best navigate and juggle between both full-time roles as Creative Director at the studio, and launching the spin-off AUGMENTA – focused on the development of a new technology. For accelerating the development of the technology, I needed to dedicate all my time on managing AUGMENTA as product owner and manager, and turn down some creative projects at THEORIZ. Staying focused on the development of AUGMENTA has been key.