Caffe: How to convert binaryproto to npy (like ilsvrc_2012_mean.npy)?

Created on 3 Apr 2014  ·  9Comments  ·  Source: BVLC/caffe

When I run the pretrained ImageNet model, I found the python wrapper (wrapper.py) will read this file:
python/caffe/imagenet/ilsvrc_2012_mean.npy
To my understanding, the file is the numpy version of file "imagenet_mean.binaryproto".

  1. Do anyone know how to convert "imagenet_mean.binaryproto" to "ilsvrc_2012_mean.npy"?
  2. If the mean file ("ilsvrc_2012_mean.npy") is a parameter in imagenet_deploy.prototxt, will it be better?

Thank you!

question

Most helpful comment

808

import caffe
import numpy as np
import sys

if len(sys.argv) != 3:
print "Usage: python convert_protomean.py proto.mean out.npy"
sys.exit()

blob = caffe.proto.caffe_pb2.BlobProto()
data = open( sys.argv[1] , 'rb' ).read()
blob.ParseFromString(data)
arr = np.array( caffe.io.blobproto_to_array(blob) )
out = arr[0]
np.save( sys.argv[2] , out )

All 9 comments

1: For binaryproto/numpy conversion see convert.py and its utility functions. Load the binaryproto in python, yielding a blob, then call blobproto_to_array, and save the numpy array however you like (for instance as a npy).

2: The mean would ideally be embedded in the net definition, so that it doesn't need to be carried around elsewhere. There is ongoing work #148 #244 and a planned change to a channel mean that could be easily written in prototxt (that is, each channel would have a space invariant mean value, which has comparable performance to a full image mean and simplifies processing).

Dear Evan Shelhamer,

Thank you for you help! I have read convert.py and found that the function 'blobproto_to_array'. To convert the format, I think it should be:
Step1: dat = to_load_datum('imagenet_mean.binaryproto')
Step2: arr = blobproto_to_array(dat)
Step3: ny.save('ilsvrc_2012_mean.npy', arr)
Because I'm not familiar with datum and python enough, I don't know how to load the 'imagenet_mean.binaryproto' file. Could you kindly give me some lines of code for doing that? Thank you.

The binaryproto file is a protobuf for BlobProto. Refer to the protobuf python tutorial and do from caffe.proto import caffe_pb2 to make a blob = caffe_pb2.BlobProto().

As a tip for people who are reading this now, the file: convert.py does not exist anymore, instead it's function blobproto_to_array moved into the caffe.io module.

I think @mezN means the caffe/python/caffe/io.py module

808

import caffe
import numpy as np
import sys

if len(sys.argv) != 3:
print "Usage: python convert_protomean.py proto.mean out.npy"
sys.exit()

blob = caffe.proto.caffe_pb2.BlobProto()
data = open( sys.argv[1] , 'rb' ).read()
blob.ParseFromString(data)
arr = np.array( caffe.io.blobproto_to_array(blob) )
out = arr[0]
np.save( sys.argv[2] , out )

where are all these files? 404 or doesn't exist now

The utility functions to read protofiles and to write them into other formats are in the io module. You can find the code here:
CPP:
https://github.com/BVLC/caffe/blob/master/src/caffe/util/io.cpp
Python:
https://github.com/BVLC/caffe/blob/master/python/caffe/io.py

Great thanks to @sg90 for sharing such a nice snippet :+1:

Was this page helpful?
0 / 5 - 0 ratings

Related issues

serimp picture serimp  ·  3Comments

FreakTheMighty picture FreakTheMighty  ·  3Comments

malreddysid picture malreddysid  ·  3Comments

OpenHero picture OpenHero  ·  3Comments

lixin7895123 picture lixin7895123  ·  3Comments