Celebrating São João do Nordeste in Brazil with MPEG-H Audio

Every year in June, Brazil celebrates the birth of John the Baptist and the end of the rainy season. Millions of people tune in when the popular festivities of São João do Nordeste are transmitted in conference from various locations in the country. Globo Recife, the local headquarter of Grupo Globo in northeastern Brazil, set up a live streaming service with MPEG-H Audio immersive and personalized sound in parallel to their regular broadcast.

This decision was in preparation for upcoming transmissions of MPEG-H Audio in compliance with the current Brazilian digital terrestrial TV system TV 2.5. This system includes MPEG-H Audio as additional audio system to enhance the user experience with immersive and personalized sound. MPEG-H Audio has also recently been adopted as the sole audio system for the Brazilian TV 3.0 standard, which will be implemented over the next few years. A team from Fraunhofer IIS supported the Globo staff through the setup, production, and transmission which served as proof of concept for delivering an immersive and interactive streaming experience. The live stream was received by a selected audience via personalized Globo mobile apps for iOS and Android devices. Right after the event, highlights were made accessible as video-on-demand content with MPEG-H Audio.

“Such a complex broadcast situation isn’t easy to handle, even under the usual circumstances. As proof of concept for enabling a new audio system and low-latency live-streaming technologies, as well as working with multiple partners, distributed content sources, and a fast-paced switching schedule, it was a real challenge,” says Yannik Grewe Senior Engineer for Audio in Broadcast and Media Systems at Fraunhofer IIS. “But we were confident that we were up to the challenge, and indeed, it was a very successful production and a great experience for everyone involved.”

On June 26, each event location in the northern part of Brazil transmitted the audio signals into the central Globo hub in Recife, where the production took place. From the incoming signals, the Globo team created a 5.1+4H immersive sound live mix. Commentary, crowd, and audio description were added as separate audio objects using Linear Acoustics’ real-time MPEG-H Authoring and Monitoring Systems (AMS). Through this, enhanced personalization options for users were achieved. A particularly novel approach was the real-time creation of the audio description: For this, the video and audio signals were transmitted to São Paulo, where the audio description was created live by Showcase and sent back to Recife to be integrated into the live stream.

The Globo team created the underlying MPEG-H Audio experience, including four presets that the audience could choose from: a standard preset, a version with enhanced dialog, Audio Description, and Live Atmosphere. Finally, the video and the MPEG-H Audio scene were encoded with ATEME’s Titan Live Encoder. The encoder can export two output formats simultaneously: A classic broadcast signal using a MPEG-2 Transport Stream was transmitted locally to an MPEG-H Audio immersive experience room; a live-streaming service using the Common Media Application Format (CMAF) for HLS and DASH made the content available for streaming to the selected audience using Amazon Web Services (AWS). With binaural rendering support, the personalized Globo mobile apps provided a convenient way to enjoy immersive sound on standard mobile devices using regular stereo headphones.

“At Globo, we are dealing with challenges every day and we wanted to make sure that the new technology is up to anything that might happen on our program. We know that it is quite a jump from a laboratory setting to a very challenging real-life scenario, and we are impressed with the potential of MPEG-H Audio to enhance the audio experience,” says Luiz Augusto, Head of Technology at Globo Recife. “This PoC proved that immersive and interactive sound can be implemented in our workflows, creating a fantastic sound experience. We are looking forward to more productions and transmissions with MPEG-H Audio in the future.”