GrabCut NoteBook implementation for Image Matting

Follow along the article with the complete code implementation on GitHub. Open the notebook in Google Colab, import your image, run the cells!

Image matting is an extremely interesting task where the goal is to find any object of interest, or human, in a picture and remove its background. This task is hard to achieve due to its complexity, finding the person, people, or objects with the perfect contour. This post reviews a very interesting technique using basic computer vision algorithms to achieve this task. The GrabCut algorithm. It is extremely fast but not very precise for complex objects like humans or animals. Nonetheless, it can be handy in specific contexts and is a perfect applied first project to start in computer vision and python! As mentioned above, the implementation uses Google Colab, thus having no requirements or setup needed, making it a very interesting project to duplicate for learning.

This GrabCut algorithm implementation basically estimates the color distribution of the foreground item and the background using a gaussian mixture model. We draw a rectangle over the object of interest (the foreground) and iteratively tries to improve the results by drawing over the parts the algorithm failed to add pixels to the foreground or remove a set of pixels from the foreground. This is why we often use a “green screen”, helping the algorithms to remove only the green pixels and leave the rest into the final results. But the results are not so great when we do not have access to such a green screen.

This algorithm uses a rectangle made by the user to identify the foreground item. Then, the user can edit to add or remove objects to the foreground. Once done, it removes the background and makes it transparent (adds alpha channel), and saves it.

This is a simple notebook implementing the GrabCut algorithm shown in the video I made about ModNet, a new technique for image matting.

How does it work?

1. Upload your image. [3rd cell]
2. Create a rectangle over your image, including all your foreground objects. [4th cell]
3. Remove the background using the GrabCut algorithm. [5th cell]
4. If improvement is needed, draw on the image, left-click (white) = foreground and right-click(black) = background. [6th cell]
5. Remove the background using the GrabCut algorithm. [6th cell]
6. Return to step 4 or end. [6th cell]
7. Save your results. [7th cell]

Note: The [cells] refer to the notebook implementation

Video demos

Example with one iteration needing only the first pass of the algorithm

This is an example where the algorithm works on the first trial, and we only need steps 1 to 3. We basically upload the image, draw a rectangle around the object of interest (our dog), and that's it! You can save the image with the alpha channel in png.

Example with a more challenging picture and multiple iterations needed

This is an example where the algorithm works on the first trial, and we only need steps 1 to 6. We basically upload the image, draw a rectangle around the object of interest (our second dog), and enter the fine-tuning loop. Here, we just draw random lines using the left click to further improve the results. Once you are done, you can save the image with the alpha channel in png using the rest of the code.

Implement it yourself!

The complete code implementation is available on GitHub! You can easily open it in a Google Colab machine and run it right away, without any pre-requisites or setup needed. Just import your pictures in Google Colab and run the code!

louisfb01/iterative-grabcut

This algorithm uses a rectangle made by the user to identify the foreground item. Then, the user can edit to add or remove objects to the foreground. Then, it removes the background and makes it tr...

GitHublouisfb01

Concrete results

Come chat with us in our Discord community: Learn AI Together and share your projects, papers, best courses, find Kaggle teammates, and much more!

If you like my work and want to stay up-to-date with AI, you should definitely follow me on my other social media accounts (LinkedIn, Twitter) and subscribe to my weekly AI newsletter!

To support me:

• The best way to support me is by being a member of this website or subscribe to my channel on YouTube if you like the video format.
• Support my work financially on Patreon

References

Interactive Foreground Extraction using GrabCut Algorithm, OpenCV, (2013), https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_grabcut/py_grabcut.html