I tried to install PyTorch, but I can't use it. It can't link dynamic library libomp.dylib
Steps to reproduce the behavior:
from torch.utils.data import Dataset
Traceback (most recent call last):
File "/Users/kaspersapala/Documents/Airly/new-pollution-forecast/Datasets/__init__.py", line 3, in <module>
from torch.utils.data import Dataset
File "/Users/kaspersapala/.local/share/virtualenvs/new-pollution-forecast-BSetKF9E/lib/python3.7/site-packages/torch/__init__.py", line 79, in <module>
from torch._C import *
ImportError: dlopen(/Users/kaspersapala/.local/share/virtualenvs/new-pollution-forecast-BSetKF9E/lib/python3.7/site-packages/torch/_C.cpython-37m-darwin.so, 9): Library not loaded: /usr/local/opt/libomp/lib/libomp.dylib
Referenced from: /Users/kaspersapala/.local/share/virtualenvs/new-pollution-forecast-BSetKF9E/lib/python3.7/site-packages/torch/lib/libshm.dylib
Reason: image not found
Being able to use and import PyTorch
conda
, pip
, source): pipenvbrew install libomp
solves the problem.
cc @JerryShih
cc @pjh5 @kostmo
brew install libomp
solves the problem.
@ezyang
How to add the libomp dependency in pytorch pip installation?
Will all pytorch dependent libraries be backed into wheel package?
If yes, I will add the libomp files like these code: https://github.com/pytorch/pytorch/blob/2356fac9a5205a2e11f1abf6105fdb3124246d41/setup.py#L738-L825
How about other dependent libraries? Are they all in third_party folder and built from source?
pip doesn't know anything about brew, you can't ask it to brew install a package. Also, while this resolution "solves" the issue, I would love to better understand how this dependency came to be (in particular, why are we building binaries with a dependency on brew libraries? That sounds bad.)
In some cases where we can't assume end users have a library installed, we distribute the library with our binaries. However, I'm not sure that we should do this for libomp.dylib. cc @soumith
this is something that needs to be fixed. I can take a look at fixing binaries and re-uploading.
The problem is that our macos CI installs libomp over here: https://github.com/pytorch/pytorch/blob/0da0c4be48c675d61c39b4554131e1dbe9177cab/.circleci/verbatim-sources/header-section.yml#L294 , I think for CI / testing. But binary builds also pick it up and use it.
I can remove this dependency for the binaries. I'll put out a PR soon
brew install libomp solves the proble
this solve my problem
I agree with @ezyang that a pytorch installation via pip should not depend on brew. PyTorch is a dependency for my library and my users may or may-not have/care about brew.
So just to clarify, the solution was to remove the brew install libomp
step && correctly package up the platform specific libraries of libomp (.dylib, .so, etc) ? Is that correct?
If I'd like to grab the latest nightly with this fix...that'd be a pip install torch_nightly -f blah.html
?
@ruohoruotsi all of that is correct
Just for reference: If using macports the issue can be worked around by creating a symlink using sudo ln -s /opt/local/lib/libomp/libomp.dylib
in /usr/local/opt/libomp/lib/
.
Just for reference: If using macports the issue can be worked around by creating a symlink using
sudo ln -s /opt/local/lib/libomp/libomp.dylib
in/usr/local/opt/libomp/lib/
.
the issue should be fixed from PyTorch by looking libraries form system instead of only checking /usr/local/opt/
brew install libomp
solves the problem.
It work!
when trying to upgrade pytorch from 1.0.1 to 1.1, I ran into the same issue and found this thread
the update on linux server worked fine but not on my macbook (OSX)
I tried brew install libomp but the brew formula could not be found, hence this did not fix the issue and I had to downgrade back to 1.0.1 on OSX to be able to import torch again
did anyone have the same issue regarding both the pip install of pytorch and the brew install of libomp ?
@soumith
I can take a look at fixing binaries and re-uploading.
Is there still any plan to do this, or are new binary releases going to wait for the next PyTorch release?
when trying to upgrade pytorch from 1.0.1 to 1.1, I ran into the same issue and found this thread
the update on linux server worked fine but not on my macbook (OSX)
I tried brew install libomp but the brew formula could not be found, hence this did not fix the issue and I had to downgrade back to 1.0.1 on OSX to be able to import torch again
did anyone have the same issue regarding both the pip install of pytorch and the brew install of libomp ?
You need to install brew first, go to brew site and follow install instructions.
thanks for your reply, I had brew installed but it was not enough up to date
so I updated everything and now it works
note for other users, when I did brew install libomp, it went through but other applications did not work anymore (Spyder IDE) ; at this moment my OS was still 10.11 (Capitan) and I had to upgrade to 10.12 (Sierra) to be able to solve the problem
so from my experience, I would recommend upgrading to Sierra at least before installing pytorch 1.1 and libomp (I use pip and brew for all install)
uploaded new binaries that remove dependency on libomp.dylib
Now I get clang: error: unsupported option '-fopenmp'
error.
@sukuya That sounds like a different problem, since it sounds like you're trying to compile PyTorch yourself, but this issue concerned a problem with the pre-compiled binaries.
this issue still present in torch 1.1.0.post2, do you know when the fix will be available on stable release?
@torrmal How are you making sure that you are using 1.1.0.post2? In my testing the changes introduced by 1.1.0.post2 fixed this problem completely, so I wonder whether you have a 1.1.0 non-post2 lying around somewhere that's getting picked up inadvertently.
@sukuya That sounds like a different problem, since it sounds like you're trying to compile PyTorch yourself, but this issue concerned a problem with the pre-compiled binaries.
@alex-xnor I was compiling a c++ program which used prebuilt libtorch
and I get those errors.
@sukuya can you make a new issue with more details on your problem? In particular, can you include the cmake/makefile that you are running and the compiler command that you are using?
brew install libomp
solves the problem.
Thx bro I've solved my problem
Does anyone have a suggestion for when brew install libomp doesn't work? I've reinstalled, relinked, but to no avail.
put this in terminal,hope it works. It took me 2hrs to fix this and I hope you can fix it faster, jeeez...
install_name_tool -change @rpath/libc++.1.dylib /usr/lib/libc++.1.dylib /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/_cvxcore.cpython-37m-darwin.so
Most helpful comment
brew install libomp
solves the problem.