Researchers are increasingly using AI to transform historical footage (such as the Apollo 16 moon landing and the 1895 Lumière Brothers movie "Arriving at the Train at La Ciotat Station") into high-resolution, high-frame-rate video that looks just like Like shooting with modern equipment. This is a boon for protectionists, and in addition, the same technology can be applied to security screening, television production, film production and other similar situations. To simplify the process, researchers at the University of Rochester, Northeastern University, and Purdue University have recently proposed a framework that can generate high-resolution slow-motion videos from low-frame-rate, low-resolution videos. They say their method- Spatio-temporal video super-resolution (STVSR) -Not only is it better than existing methods in terms of quality and quality, but it is also three times faster than previous latest AI models.
In some ways it makes jobs Published by Nvidia in 2018, it describes an AI model that can apply slow motion to any video-regardless of the frame rate of the video. And similar high-resolution technology has been applied in the field of video games. Last year, fans of Final Fantasy used A $ 1 billion software called A.I. To increase the resolution of the Final Fantasy VII background.
STVSR learns both temporal interpolation (ie, how to synthesize non-existent intermediate video frames between original frames) and spatial super-resolution (how to reconstruct high-resolution frames from the corresponding reference frame and its adjacent supporting frames). In addition, with the help of the convolutional long short-term memory model, it can use video context and time alignment to reconstruct frames from aggregated features.
The researchers trained STVSR using a dataset of more than 60,000 7-frame clips from Vimeo, and divided the fast-, medium-, and slow-motion sets using separate assessment corpora to measure performance under various conditions. In their experiments, they found that STVSR achieved "significant" improvements in fast-moving videos, including those with "challenge" moves, such as basketball players moving quickly on the court. In addition, it demonstrates the ability of a "visually appealing" frame reconstruction with more accurate image structure and fewer blurring artifacts, while being four times smaller and at least twice as fast as the baseline model.
"With this single-stage design, our network can well explore the intrinsic relationship between temporal interpolation and spatial super-resolution in tasks," the co-author of the paper wrote. Preprinted paper Describe work. "It enables our model to adaptively learn to take advantage of useful local and global time contexts to alleviate large motion problems. A large number of experiments have shown that our … framework is more efficient and efficient than existing … networks, and suggests The feature-time interpolation network and deformable (model) are capable of processing very challenging fast-moving videos. "
The researchers intend to release the source code this summer.