Pytorch: I can't import PyTorch, libomp.dylib can't be loaded.

Created on 1 May 2019  ·  27Comments  ·  Source: pytorch/pytorch

🐛 Bug

I tried to install PyTorch, but I can't use it. It can't link dynamic library libomp.dylib

To Reproduce

Steps to reproduce the behavior:

  1. pipenv install torch torchvision
  2. from torch.utils.data import Dataset
Traceback (most recent call last):
  File "/Users/kaspersapala/Documents/Airly/new-pollution-forecast/Datasets/__init__.py", line 3, in <module>
    from torch.utils.data import Dataset
  File "/Users/kaspersapala/.local/share/virtualenvs/new-pollution-forecast-BSetKF9E/lib/python3.7/site-packages/torch/__init__.py", line 79, in <module>
    from torch._C import *
ImportError: dlopen(/Users/kaspersapala/.local/share/virtualenvs/new-pollution-forecast-BSetKF9E/lib/python3.7/site-packages/torch/_C.cpython-37m-darwin.so, 9): Library not loaded: /usr/local/opt/libomp/lib/libomp.dylib
  Referenced from: /Users/kaspersapala/.local/share/virtualenvs/new-pollution-forecast-BSetKF9E/lib/python3.7/site-packages/torch/lib/libshm.dylib
  Reason: image not found

Expected behavior

Being able to use and import PyTorch

Environment

  • PyTorch Version (e.g., 1.0): 1.1.0
  • OS (e.g., Linux): Mac OS Mojave
  • How you installed PyTorch (conda, pip, source): pipenv
  • Python version: 3.7.1
high priority binaries build triaged

Most helpful comment

brew install libomp solves the problem.

All 27 comments

brew install libomp solves the problem.

cc @JerryShih

cc @pjh5 @kostmo

brew install libomp solves the problem.

@ezyang
How to add the libomp dependency in pytorch pip installation?

Will all pytorch dependent libraries be backed into wheel package?
If yes, I will add the libomp files like these code: https://github.com/pytorch/pytorch/blob/2356fac9a5205a2e11f1abf6105fdb3124246d41/setup.py#L738-L825

How about other dependent libraries? Are they all in third_party folder and built from source?

pip doesn't know anything about brew, you can't ask it to brew install a package. Also, while this resolution "solves" the issue, I would love to better understand how this dependency came to be (in particular, why are we building binaries with a dependency on brew libraries? That sounds bad.)

In some cases where we can't assume end users have a library installed, we distribute the library with our binaries. However, I'm not sure that we should do this for libomp.dylib. cc @soumith

this is something that needs to be fixed. I can take a look at fixing binaries and re-uploading.
The problem is that our macos CI installs libomp over here: https://github.com/pytorch/pytorch/blob/0da0c4be48c675d61c39b4554131e1dbe9177cab/.circleci/verbatim-sources/header-section.yml#L294 , I think for CI / testing. But binary builds also pick it up and use it.

I can remove this dependency for the binaries. I'll put out a PR soon

brew install libomp solves the proble
this solve my problem

I agree with @ezyang that a pytorch installation via pip should not depend on brew. PyTorch is a dependency for my library and my users may or may-not have/care about brew.

  • So just to clarify, the solution was to remove the brew install libomp step && correctly package up the platform specific libraries of libomp (.dylib, .so, etc) ? Is that correct?

  • If I'd like to grab the latest nightly with this fix...that'd be a pip install torch_nightly -f blah.html ?

@ruohoruotsi all of that is correct

Just for reference: If using macports the issue can be worked around by creating a symlink using sudo ln -s /opt/local/lib/libomp/libomp.dylib in /usr/local/opt/libomp/lib/.

Just for reference: If using macports the issue can be worked around by creating a symlink using sudo ln -s /opt/local/lib/libomp/libomp.dylib in /usr/local/opt/libomp/lib/.

the issue should be fixed from PyTorch by looking libraries form system instead of only checking /usr/local/opt/

brew install libomp solves the problem.

It work!

when trying to upgrade pytorch from 1.0.1 to 1.1, I ran into the same issue and found this thread

the update on linux server worked fine but not on my macbook (OSX)

I tried brew install libomp but the brew formula could not be found, hence this did not fix the issue and I had to downgrade back to 1.0.1 on OSX to be able to import torch again

did anyone have the same issue regarding both the pip install of pytorch and the brew install of libomp ?

@soumith

I can take a look at fixing binaries and re-uploading.

Is there still any plan to do this, or are new binary releases going to wait for the next PyTorch release?

when trying to upgrade pytorch from 1.0.1 to 1.1, I ran into the same issue and found this thread

the update on linux server worked fine but not on my macbook (OSX)

I tried brew install libomp but the brew formula could not be found, hence this did not fix the issue and I had to downgrade back to 1.0.1 on OSX to be able to import torch again

did anyone have the same issue regarding both the pip install of pytorch and the brew install of libomp ?

You need to install brew first, go to brew site and follow install instructions.

thanks for your reply, I had brew installed but it was not enough up to date
so I updated everything and now it works

note for other users, when I did brew install libomp, it went through but other applications did not work anymore (Spyder IDE) ; at this moment my OS was still 10.11 (Capitan) and I had to upgrade to 10.12 (Sierra) to be able to solve the problem

so from my experience, I would recommend upgrading to Sierra at least before installing pytorch 1.1 and libomp (I use pip and brew for all install)

uploaded new binaries that remove dependency on libomp.dylib

Now I get clang: error: unsupported option '-fopenmp' error.

@sukuya That sounds like a different problem, since it sounds like you're trying to compile PyTorch yourself, but this issue concerned a problem with the pre-compiled binaries.

this issue still present in torch 1.1.0.post2, do you know when the fix will be available on stable release?

@torrmal How are you making sure that you are using 1.1.0.post2? In my testing the changes introduced by 1.1.0.post2 fixed this problem completely, so I wonder whether you have a 1.1.0 non-post2 lying around somewhere that's getting picked up inadvertently.

@sukuya That sounds like a different problem, since it sounds like you're trying to compile PyTorch yourself, but this issue concerned a problem with the pre-compiled binaries.

@alex-xnor I was compiling a c++ program which used prebuilt libtorch and I get those errors.

@sukuya can you make a new issue with more details on your problem? In particular, can you include the cmake/makefile that you are running and the compiler command that you are using?

brew install libomp solves the problem.

Thx bro I've solved my problem

Does anyone have a suggestion for when brew install libomp doesn't work? I've reinstalled, relinked, but to no avail.

put this in terminal,hope it works. It took me 2hrs to fix this and I hope you can fix it faster, jeeez...

install_name_tool -change @rpath/libc++.1.dylib /usr/lib/libc++.1.dylib /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/_cvxcore.cpython-37m-darwin.so

Was this page helpful?
0 / 5 - 0 ratings

Related issues

soumith picture soumith  ·  3Comments

miguelvr picture miguelvr  ·  3Comments

szagoruyko picture szagoruyko  ·  3Comments

dablyo picture dablyo  ·  3Comments

bartvm picture bartvm  ·  3Comments