Magically remove objects, watermarks or expand the field of view in Videos
Flow edge guided video completion
Made By: Chen Gao Ayush Saraf Jia-Bin Huang Johannes Kopf
The Flow edge guided video completion project allows users to seamlessly remove objects and watermarks from casually captured videos. Not just that, it also offers the ability to expand the field of view for a video.
This project has been developed by Chen Gao, Ayush Saraf, Jia-Bin Huang, Johannes Kopf along with Virginia Tech and Facebook research team.
Compared to previous flow completion methods, this approach is able to retain and preserve the sharpness of motion boundaries resulting in very clean object removals.
How does it work?
A video completion is a task of filling a given space-time region with newly synthesized content.
To properly remove objects and watermarks from a video, the newly generated content should embed seamlessly in the video, and the alteration should be as imperceptible as possible.
There are a few challenges associated with video completion, mainly:
To ensure that the result is temporally coherent (does not flicker)
Respect dynamic camera motion as well as complex object motion in the video.
Most methods used patch-based synthesis techniques which were often slow and had limited ability to synthesize new content because they can only remix existing patches in the video.
The flow-based approach has been quite successful in the video completion problem because it synthesizes color and flow jointly and propagates the color along flow trajectories to improve temporal coherence, which alleviates memory problems and enables high-resolution output. The above-mentioned flow guided video completion method also follows a similar approach.
But previous flow-based approaches falter in complex situations due to their way of working. For example, it propagates color values directly. However, the colors often change with every scene in the video due to, shadows, lens vignetting, auto exposure, and white balancing. This leads to artifacts.
The Flow guided video completion method alleviates such limitations by providing the following solutions:
Flow edges: By explicitly completing flow edges, it obtains piecewise-smooth flow completion.
Non-local flow: It helps handle regions that cannot be reached through transitive flow (e.g., periodic motion, such as walking)
Seamless blending: Avoid visible seams in the results through operating in the gradient domain.
Memory efficiency: Handles videos with up to 4K resolution, while other methods fail due to excessive GPU memory requirements.
Who can benefit from this project?
If you want to remove an object from a motion video easily or want to remove watermarks then this solution works really efficiently with a very less memory footprint. Also, if you want to extend the field of view for a video then again this flow-based approach helps in extending the view and add content automatically.
About Qblocks
A distributed computing company building affordable supercomputers. Built by a team of scientists and engineers to help our tribe get super-fast computing results. Read the vision for a brief background and the secret master plan.
GPU Power
Launching a GPU powered the virtual computer on Q Blocks Peer to Peer Computing Platform is really easy.
1. Train AI models faster
2. Select a GPU Instance
3. Select an AI Framework of your choice
4. Select Access Method: CLI or Jupyter Lab
5. Get a Supercomputer at your service!
6. One more thing: For $20, get 100 GPU hours
Why Qblocks:
1. 10X Better
2. Breakthrough Computing Paradigm
3. Community of people with a deep love for supercomputing
4. Crafted with love for the Crazy Ones!
What is Q Blocks?
Imagine the uber of computing.
A way to use millions of underutilized sources of computing to build your next AI model.
Gone are the days, when supercomputers were just limited to scientific institutes or governments.
By connecting together these computing sources spread across the world,
Q Blocks envisions bringing access to supercomputers in your hands.
