Caffe: How to convert Mat from opencv to caffe format

Created on 13 Jun 2015 · 4Comments · Source: BVLC/caffe

Hello all, I am using opencv to crop face from my camera. And then I used caffe to predict that image belongs to male or female. I have a original code that load image from static image. However, I want to use image from camera for it. This is original code in caffe

        model = caffe.Classifier(...)
        image_path = './static_image.jpg'
        input_image = caffe.io.load_image(image_path )
        prediction =model.predict([input_image])

Now, I will use opencv to capture frame and call predict method

      val, image = cap.read()    
      image = cv2.resize(image, (320,240))
      gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
      faces = face_cascade.detectMultiScale(gray, 1.3, 5, minSize=(30,30))
      for f in faces:
          x,y,w,h = f
          cv2.rectangle(image, (x,y), (x+w,y+h), (0,255,255))        
           face_image = gray[y:y+h, x:x+w]
           resized_img = cv2.resize(face_image, (45,45))/255.

After having resized_image, I will predict that image based on above code. I just replace above predict code by =model.predict([resized_img]). But I got error that is

IndexError: tuple index out of range

How to fix my code? Thank you for help!

Source

mjohn123

Most helpful comment

In the training stage, caffe‘s c++ implementation uses imread in OpenCV to read image, which by default reads the color channels in order of B G R. However, in the python interface, caffe uses skimage to read image, which by default reads the color channels in order or R G B.

If you are using the python version of OpenCV to read image, you need to do the following conversion:
1
2
3

img = cv2.imread(path/to/image)
img = img / 255.
img = img[:,:,(2,1,0)]

However, it is always better to use caffe’s skimage wrapper:
1

img = caffe.io.load_image(path/to/image)

mjohn123 on 17 Jun 2015

👍6

All 4 comments

Hi mJohn, did you ever resolve this issue? If so, what did you end up doing?

Thanks in advance,
NC

nealchecka on 17 Jun 2015

If you are using the python version of OpenCV to read image, you need to do the following conversion:
1
2
3

img = cv2.imread(path/to/image)
img = img / 255.
img = img[:,:,(2,1,0)]

However, it is always better to use caffe’s skimage wrapper:
1

img = caffe.io.load_image(path/to/image)

mjohn123 on 17 Jun 2015

👍6

@mjohn123 Can you please describe the part:

img = cv2.imread(path/to/image)
img = img / 255.
img = img[:,:,(2,1,0)]

azamattokhtaev on 6 Jul 2016

i have meet this issue and i have solved this problem by the inspiration of "examples/cpp_classification/classification.cpp". i think they have did a great job, instead of loading the data into blob pixel by pixel, they use one trick.

Here is what they did:
First they initialise a channels object to point to the input blob

std::vectorcv::Mat channels;
float* data = mean_blob.mutable_cpu_data();

for (int i = 0; i < num_channels_; ++i) {

(Here the trick) cv::Mat channel(mean_blob.height(), mean_blob.width(), CV_32FC1, data);
channels.push_back(channel);
data += mean_blob.height() * mean_blob.width();
}

According to OpenCV website:
C++: Mat::Mat(int rows, int cols, int type, void* data, size_t step=AUTO_STEP)
data – Pointer to the user data. Matrix constructors that take data and step parameters do not allocate matrix data. Instead, they just initialize the matrix header that points to the specified data, which means that no data is copied. This operation is very efficient and can be used to process external data using OpenCV functions. The external data is not automatically deallocated, so you should take care of it.

Then to load img into, they simply just do this:

cv::split(sample_normalized, input_channels);