As their popularity on YouTube can attest, first-person time-lapse videos can be mesmerizing. Unfortunately, they can also be shaky, headache-inducing viewing experiences.
An innovation from Microsoft Research, called Hyperlapse, may spell the end to unwatchable time-lapse videos.
The project began after Johannes Kopf, a Microsoft researcher as well as a mountain climber, noticed that compressing hours of climbing footage from a GoPro camera into a time lapse yielded a bouncy video that proved difficult to watch. Kopf, with the help of the Interactive Visual Media Group at Microsoft Research, developed software called Hyperlapse that stabilizes time lapses and virtually eliminates camera shake.
The Interactive Visual Media Group deals primarily with “computer vision, image processing and statistical signal processing,” according to Microsoft. The unit specializes in 3D reconstructions and image-stitching algorithms, among other modeling and rendering technologies.
In a blog post, Microsoft describes how Hyperlapse delivers superior results when compared to current stabilization techniques. “Standard video stabilization crops out the pixels on the periphery to create consistent frame-to-frame smoothness. But when applied to greatly sped up video, it fails to compensate for the wildly shaking motion,” noted Microsoft spokesman David Chen.
In contrast, the team’s technology “reconstructs how a camera moves throughout a video, as well as its distance and angle in relation to what’s happening in each frame,” said Chen. “Then it plots out a smoother camera path and stitches pixels from multiple video frames to rebuild the scene and expand the field of view.”
As the project’s Web page describes, the software accomplishes this by generating a rough 3D reconstruction of the scene and plotting a smooth path through the new model. Finally, Hyperlapse renders a smooth video by projecting imagery that is taken from video onto the approximate geometry of the 3D model.
“Put another way, it’s akin to the human brain’s ability to fill in blind spots by ‘hallucinating’ on the person’s behalf,” said Chen.
A YouTube video of the technology in action shows the dramatic difference between sped-up GoPro video and Hyperlapse. Instead of jerky footage of a cityscape, Hyperlapse produces a smooth video that appears to have been shot on rails.
While impressive, the Hyperlapse is not without its flaws. Keen observers may notice elements that morph into one other, warping effects and other artifacts. Nonetheless, the tech produces convincing results.
Chen reported that Kopf’s team is working “on reducing the number of visual artifacts and reducing the time it takes to render a Hyperlapse video.” Their software may be headed toward commercialization, he added. “Eventually, they hope to release Hyperlapse to the public, making it easier for people to share their digital memories and weekend exploits.”
If Hyperlapse is released as a commercial product, it could add a new wrinkle to Microsoft’s partnership with GoPro, whose namesake first-person cameras have proven popular with online videographers. Last month, the companies announced the launch of the GoPro Channel App for the Xbox One. “User-generated content is huge on Xbox,” said the Xbox Wire team in a recent blog post, before predicting that it will “continue to grow as GoPro joins other popular apps like YouTube and Twitch.”