Caffe: Check failed: error == cudaSuccess (2 vs. 0) out of memory in solver phase.

Images larger than 1280x800? Are you trying to make deepdream images? If so, the batchsize is already as small as it can be (i.e. 1), and you need either a larger GPU (e.g. the TITAN X), or you need to make your input images smaller. 2GB of RAM is pretty tiny for deep learning. You could also modify the code to take the input in smaller chunks; you'll need to do something clever to handle the seams between chunks though.

If you're training a model for a task, you can reduce the batchsize and correspondingly increase iter_size (default: 1) by the same factor, in the solver.prototxt file. This will give you an equivalent result, since gradients are accumulated.

In the future, please ask questions like this on the caffe-users group (https://groups.google.com/forum/#!forum/caffe-users). Github is for code issues, not software usage.

Thanks for the info! I tried to post in the Google group a while ago but it seems that it takes some time before the post actually appears on the page (it's being moderated I guess). I asked this in the Gitter chatroom as well, but did not get any responses.

I tried this model with deepdream too, but I faced the same issue as with my application. I am trying to implement a recognition algorithm that highlights buildings in videos, so it might actually be possible to split the frames into smaller images and process them separately. Another approach would be to scale the images down, process them and output a "mask" with the highlights that can be scaled up and applied to the original images.

I will get a better GPU soon, but I am hoping to get this running on Tegra devices as well, so I will definitely have to optimize a lot of things for better portability.

Thanks again! Closing issue.

Hello @seanbell. First of all thanks to you and all BVLC contributors for this amazing project :)

I'm doing an extensive research on this error because we are a lot of people to encounter it, all are Caffe related.

In the future, please ask questions like this on the caffe-users group (https://groups.google.com/forum/#!forum/caffe-users). Github is for code issues, not software usage.

This is where I disagree : "Check failed: error == cudaSuccess (2 vs. 0) out of memory in solver phase." is a pretty valid code issue.

Caffe should not break while in GPU mode at all, because it does not on CPU mode.
While having a GTX970 instead of a 1500$ TITAN should make your program way slower, it should not break it. I'm sorry...