Mask R-CNN using Tensorflow and OpenCV to increase inference performances on NVidia GPU
Mask RCNN is a deep neural network for instance segmentation. In other words, it can separate different objects in a image or a video. You give it a image, it gives you the object bounding boxes, classes and masks.
You can read full paper explaining Mask RCNN here : https://arxiv.org/abs/1703.06870
Basically a Mask RCNN neural netwotk model is an extension of faster R-CNN SSD object detection network neaural network. The model generates bounding boxes and segmentation masks for each instance of an object in the image. It’s based on Feature Pyramid Network (FPN) and a ResNet101 backbone.
You can find an implementation of Mask R-CNN on Python 3, Keras, and TensorFlow. The model generates bounding boxes and segmentation masks for each instance of an object in the image. It’s based on Feature Pyramid Network (FPN) and a ResNet101 backbone here : https://github.com/matterport/Mask_RCNN/
With this implementation you can easily train or transfer learning Mask R-CNN with your own dataset on Tensorflow / Keras , I will talk about Mask R-CNN training with Python3 anf TF/Keras in another article.
For this example we are going to use default Mask R-CNN weights trained with COCO Dataset wich is included in OpenCV 4.2.0.
First of all you have to install sources and compile OpenCV 4.2.0.
My workstation is based on Unbuntu 18.04 with Nvidia Geforce RTX 2080 nvidia dirvers 440.59 cuda 10.2 and cudnn 7.5.0 which is a minimum requirement to build OpenCV 4.2.0

Create a directory for example mkdir OpenCV-4.2.0 in your home and get inside
Download both opencv and opencv_contrib :
wget -O opencv.zip https://github.com/opencv/opencv/archive/4.2.0.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.2.0.zip
Unzip both archives and rename directories to opencv and opencv_contrib :
mv opencv-4.2.0 opencv
mv opencv_contrib-4.2.0 opencv_contrib
cd opencv
mkdir build
cd build
Now you are ready to cmake Opencv-4.2.0 inside build directory, you need a cmake version 3.10 or newer or compilation will fail, update cmake in your system before proceed.
These are the flags I used on my system for cmake command to build with Opencv-4.2.0 with Cuda and Cudnn. I have excluded python builds because not interested in python environment on this machine, but you can turn INSTALL_PYTHON_EXAMPLES=ON to build them :
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local/opencv-4.2.0 \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D INSTALL_C_EXAMPLES=ON \
-D OPENCV_ENABLE_NONFREE=ON \
-D WITH_CUDA=ON \
-D WITH_CUDNN=ON \
-D OPENCV_DNN_CUDA=ON \
-D ENABLE_FAST_MATH=1 \
-D CUDA_FAST_MATH=1 \
-D CUDA_ARCH_BIN=7.0 \
-D WITH_CUBLAS=1 \
-D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules \
-D BUILD_EXAMPLES=ON ..
If cmake succefully recognized cuda and cudnn you should see something like this :
– NVIDIA CUDA: YES (ver 10.2, CUFFT CUBLAS FAST_MATH)
– NVIDIA GPU arch: 70
– NVIDIA PTX archs:
– cuDNN: YES (ver 7.5.0)
now you can run make command (suggest to check your proc capabilities on your machine using nproc command to speed up Opecv build on my machine I can go up to make -j24)
Once compilation is end you can install OpenCv 4.2.0 in your machine using command :
sudo make install
sudo ldconfig
Now that you have OpeCV installed on your system you can download marsk-rcnn.cpp OpenCV source demo here : https://github.com/spmallick/learnopencv/tree/master/Mask-RCNN
Git clone repository or download archive and follow instruction on page to dowload Tensorflow Mask-RCNN model and weights trained on COCO Dataset.
You have to modify modify mask_rcnn.cpp to enable GPU and CUDNN network inference here on Network loading :
//net.setPreferableBackend(DNN_BACKEND_OPENCV);
//net.setPreferableTarget(DNN_TARGET_CPU);
net.setPreferableBackend(DNN_BACKEND_CUDA);
net.setPreferableTarget(DNN_TARGET_CUDA);
You can download my mask_rcnn.cpp modified and with a bug in BuondingBox draw corrected here : https://github.com/MarcoGonnelli74/DeepLearning/blob/master/mask_rcnn.cpp
To build you can use Eclipse or any other IDE you are used or just compile with command line command
Remember to include in your include paths : – I /usr/local/include/opencv4
Link following libraries -l :
opencv_imgproc
opencv_videoio
opencv_imgcodecs
opencv_core
opencv_dnn
opencv_highgui
Using library search path -L : /usr/local/opencv-4.2.0/lib
then run test program :
./mask_rcnn.out –video=<path to your video file>
Here’s my video sample running test program on Nvidia RTX 2080 GPU with 20-25 fps performance using cuda and cudnn acceleration enjoy :