How Peter Jackson's Beatles Documentary Used Custom AI To Remove Background Noise

Peter Jackson's seven-hour documentary "Get Back" (now streaming on Disney+) edits footage from the Beatles' ambitious recording sessions for their 1970 album Let It Be. But long-time Slashdot reader MattSparkes writes that the whole documentary "would have been impossible without custom-made artificial intelligence, say sound engineers." Sixty hours of footage were recorded but most of the audio was captured by a single microphone that picked up the musicians' instruments in a noisy jumble rather than a carefully crafted mix. It also recorded background noise and chatter, which made much of the footage unusable. The team scoured academic papers on using AI to separate audio sources but realised that none of the previous research would work for a music documentary. They consulted with Paris Smaragdis at the University of Chicago and started to create a neural network called MAL (machine assisted learning) and a set of training data that was higher quality than datasets used in academic experiments. The Washington Post describes it as "a sort of sonic forensics," adding that the name MAL was a deliberate homage to the HAL computer in 2001: a Space Odyssey — and to the Beatles' beloved road manager and principal assistant, Mal Evans. Using MAL, Jackson and his colleagues were able to painstakingly and precisely isolate each and every audio track — be it musical instrumentation, singing or studio chatter — from the original mono recordings made for most of "Let It Be." "What we've managed to do is split it all apart in a way that is utterly clean and sounds much better," Jackson said. Other interesting observations from the Post: "Get Back" tapped nearly 120 hours of previously unheard audio recordings. Jackson and his team started work in 2017. Jackson's team also "carefully restored, upgraded and enlarged the grainy original 16-millimeter" footage from the 1969 documentary Let It Be "so that it now pops with vibrant color." Jackson's documentary "was originally set to open in theaters last year as a two-and-a-half hour feature film, but was pushed back by the pandemic. With more time unexpectedly on his hands, Jackson transformed his feature film into the six-hour epic...." Jackson would also like to release an expanded director's cut sometime in the future, "but there are no current plans to do so." "At one point, Jackson's favorite version of his Get Back film clocked in at 18 hours..."

Read more of this story at Slashdot.

from Slashdot https://ift.tt/3prfGiM

Share on FacebookPlus on Google+

SUBSCRIBE TO OUR NEWSLETTER