Nokogiri: libiconv is missing

Created on 3 Apr 2011  ·  109Comments  ·  Source: sparklemotion/nokogiri

We're running Ruby 1.8.7 (through RVM) on OS X 10.6.6. Trying to install nokogiri 1.4.4 using bundler results always the following error:

Installing nokogiri (1.4.4) with native extensions /Users/administrator/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/site_ruby/1.8/rubygems/installer.rb:533:in `build_extensions': ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError)

/Users/administrator/.rvm/rubies/ruby-1.8.7-p302/bin/ruby extconf.rb 
checking for libxml/parser.h... yes
checking for libxslt/xslt.h... yes
checking for libexslt/exslt.h... yes
checking for iconv_open() in iconv.h... no
checking for iconv_open() in -liconv... no
-----
libiconv is missing.  please visit http://nokogiri.org/tutorials/installing_nokogiri.html for help with installing dependencies.
-----
*** extconf.rb failed ***
[…]

Following the instructions here in the wiki as well as around the net we've used MacPorts to install libxml2 and libxslt, which results in the following situation:

sudo port install libxml2 libxslt
Password:
Error: Cannot install libxml2 for the arch(s) 'x86_64' because
Error: its dependency libiconv is only installed for the archs 'i386 ppc'.
Error: Unable to execute port: architecture mismatch

We've build libxml2 and libxslt from source and tried both to install the gem with the flags for the MacPorts and the build as described in the wiki as well as a few variations found online. — The results remains the same. We've even downloaded the gem, changed extconf.rb as suggested here: https://github.com/tenderlove/nokogiri/issues#issue/381, and compiled it locally but the final outcome is the same.

What are we missing? We're pretty much stuck with this situation.

All 109 comments

I asked this same question, and was given this response:

Install MacPorts: http://www.macports.org
Add /opt/local/bin to your PATH (bashrc or zshrc or whatever you use).
sudo port install libxml2 and sudo port install libxslt
Then sudo gem install nokogiri --no-rdoc --no-ri should run with no issues. That's all I had to do for the system ruby (1.8.7 on OSX 10.6) and 1.9.2 via rvm.

I was having the same issues on 10.6. I re-installed MacPorts, port installed libxml2 and libxslt again and now I'm no longer experiencing these issues.

Assuming you installed libxml2 and libxslt with MacPorts, then you might still be getting this error due to a mismatch of the order of include paths and link paths used by the call to have_func('iconv_open', 'iconv.h')

Simple (patch) solution: delete libiconv.* in /usr/local/lib

Suggestion: have extconf.rb perform slightly more sophisticated checking of what iconv libs are actually installed, and give the user a better clue. In this case libiconv is NOT missing. There's actually TWO libiconvs, the correct header was being compiled, and an old library was being linked. Seems like a very common problem that the install file error message could give a little more help with.

And I think this might be an issue with mkmf and the way it handles library paths? The failing compile check for me was:

```"gcc -o conftest -I. -I/usr/local/lib/ruby/1.8/i686-darwin10.3.0 -I. -I-I-I/opt/local/include -I-I-I/usr/local/include -I-I-I/usr/local/include -I-I-I/usr/include -I-I-I/usr/include/libxml2 -I/opt/local/include/libxml2 -I/usr/local/include/libxml2 -I/usr/local/include/libxml2 -I-I/opt/local/include -I-I/usr/local/include -I-I/usr/local/include -I-I/usr/include -I-I/usr/include/libxml2 -I/opt/local/include -I/usr/local/include -I/usr/local/include -I/usr/include -I/usr/include/libxml2 -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=1 -fno-common -pipe -fno-common -g -DXP_UNIX -O3 -Wall -Wcast-qual -Wwrite-strings -Wconversion -Wmissing-noreturn -Winline conftest.c -L. -L/usr/local/lib -L/opt/local/lib -L/usr/local/lib -L/usr/lib -L. -lruby-static -lpthread -ldl -lobjc "
Undefined symbols for architecture x86_64:
"_libiconv_open", referenced from:

Note the compile stage starts with /opt/local/include, but the link stage starts with /usr/local/lib, which is where the old version was

I'm using Lion and homebrew. For me, this did the trick:

brew install libiconv
brew link libiconv

In my case (Migration to OS X Lion), to fixe the problem, I cleaned macports :

port -qv installed > myports.txt
sudo port -fp uninstall --follow-dependents installed
sudo port clean all

More details : http://trac.macports.org/wiki/Migration

+1 to what jfahrenkrug said. The installation guide on http://nokogiri.org/tutorials/installing_nokogiri.html, homebrew section should probably be updated;

brew install libxml2 libxslt
brew link libxml2 libxslt

to

brew install libxml2 libxslt libiconv
brew link libxml2 libxslt libiconv

jfahrenkrug's fix also worked for me

+1 works for me

I've followed the instructions to build libxml2 and libxslt from brew. I can't build nokogiri.

I get an error when I try 'brew install libiconv':

$ brew install libiconv
Error: No available formula for libiconv
Apple distributes libiconv with OS X, you can find it in /usr/lib.
Some build scripts fail to detect it correctly, please check existing
formulae for solutions.

I'm on a fresh Snow Leopard install. I'd rather not use MacPorts.

Please help.

@jseigler I'm running brew 0.8.1 and I can install libiconv with no problem. Try upgrading brew?

I had the same issue as @jseigler. I'm running brew 0.9 on OS X 10.7.3.

Downloading libiconv and compiling manually worked.
http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.13.1.tar.gz

If you figure out how to install Nokogiri against brew 0.9 libraries, please let me know -- I'll update installation documentation. In the meantime, though, as a user primarily of Linux, I'm afraid I can't be of much help.

@tenderlove, any ideas? You <3 macs. :) :) :)

Using brew 0.9 on OSX 10.7.3 the following worked:

brew install libxml2 libxslt
brew link libxml2 libxslt

Install libiconv from source:
wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.13.1.tar.gz
tar xvfz libiconv-1.13.1.tar.gz
cd libiconv-1.13.1
./configure --prefix=/usr/local/Cellar/libiconv/1.13.1
make
sudo make install

Then install nokogiri:
gem install nokogiri -- --with-xml2-include=/usr/local/Cellar/libxml2/2.7.8/include/libxml2 --with-xml2-lib=/usr/local/Cellar/libxml2/2.7.8/lib --with-xslt-dir=/usr/local/Cellar/libxslt/1.1.26 --with-iconv-include=/usr/local/Cellar/libiconv/1.13.1/include --with-iconv-lib=/usr/local/Cellar/libiconv/1.13.1/lib

@ brennenf

Nice work. That did it for me too.

Instructions posted to nokogiri.org:

http://nokogiri.org/tutorials/installing_nokogiri.html#homebrew_09

Thanks, everybody.

@brennenf Thanks for this line
gem install nokogiri -- --with-xml2-include=/usr/local/Cellar/libxml2/2.7.8/include/libxml2 --with-xml2-lib=/usr/local/Cellar/libxml2/2.7.8/lib --with-xslt-dir=/usr/local/Cellar/libxslt/1.1.26 --with-iconv-include=/usr/local/Cellar/libiconv/1.13.1/include --with-iconv-lib=/usr/local/Cellar/libiconv/1.13.1/lib

@brennenf your fix gets me a nice "Successfully installed nokogiri-1.5.2" but when I run bundle install in my rails app it still fails on: "An error occured while installing nokogiri (1.5.2), and Bundler cannot continue.
Make sure that gem install nokogiri -v '1.5.2' succeeds before bundling." Note "gem install nokogiri -v '1.5.2" also fails. Anyone else running in to this?

@ttringas - please email nokogiri-talk. this issue is regarding nokogiri 1.4.4 and has been closed for months.

Hi any updates on Mountain Lion ?
I can't get the installation done.

I've updated the path mentioned on nokogiri installation page :

gem install nokogiri -- --with-xml2-include=/usr/local/Cellar/libxml2/2.8.0/include/libxml2 --with-xml2-lib=/usr/local/Cellar/libxml2/2.8.0/lib --with-xslt-dir=/usr/local/Cellar/libxslt/1.1.26 --with-iconv-include=/usr/local/Cellar/libiconv/1.13.1/include --with-iconv-lib=/usr/local/Cellar/libiconv/1.13.1/lib

this is what i've in return: https://gist.github.com/3186643

@polarblau look who has problems now :D (me)

It might help but it seems the RVM "ecosystem" seems buggy after updating to mountain lion. I remove the .rvm and install rvm using http://unfiniti.com/software/mac/jewelrybox/ and now it works but I have a warning that says:
WARNING: Nokogiri was built against LibXML version 2.8.0, but has dynamically loaded 2.7.8

Same here, reinstalling ruby did the trick (i'm not really surprised - that's pretty common pitfall)

@polarblau, to late! i've read that already ;)

To get the installation working on my newly upgraded Mountain Lion I had to remove my previously installed rvm packages (located in $rvm_path/usr), rebuild ruby 1.9.3, and then gem install nokogiri succeeded. See https://rvm.io/packages/

I'm on Mountain Lion. Reinstalling Ruby worked for me.

checking for libxml/parser.h... no

libxml2 is missing.

I found one extra step was required to solve the above for Homebrew 0.9 on Mac OS X 10.8 Mountain Lion and that is to create a "/usr/bin/gcc-4.2" link:

brew install libxml2 libxslt
brew link libxml2 libxslt
wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.13.1.tar.gz
tar xvfz libiconv-1.13.1.tar.gz
cd libiconv-1.13.1
./configure --prefix=/usr/local/Cellar/libiconv/1.13.1
make
sudo make install
sudo ln -s /usr/bin/gcc /usr/bin/gcc-4.2
gem install nokogiri -- --with-xml2-include=/usr/local/Cellar/libxml2/2.8.0/include/libxml2 --with-xml2-lib=/usr/local/Cellar/libxml2/2.8.0/lib --with-xslt-dir=/usr/local/Cellar/libxslt/1.1.26 --with-iconv-include=/usr/local/Cellar/libiconv/1.13.1/include --with-iconv-lib=/usr/local/Cellar/libiconv/1.13.1/lib

brax4444 thanks so much, you are a life saver!! Worked perfectly with the addition of a sudo on your last command because of the error:

ERROR: While executing gem ... (Errno::EPERM)
Operation not permitted - ~/.rvm/gems/ruby-1.9.3-p194/gems/nokogiri-1.5.5/.autotest

brax4444 is the man! thanks

Wondering if someone can help-out a noobie:

Running Mountain Lion 10.8.2 w/ Homebrew.

Started off with nokogiri 1.5.2 - was getting the typical libiconv missing.

Between this thread, specifically Brax4444 & the installation tutorial on nokogiri: http://nokogiri.org/tutorials/installing_nokogiri.html for Homebrew .9 I am left with the following FUBAR:

I now have nokogiri 1.5.5 where rails is looking for 1.5.2

Still been unable to rectify the libiconv issue despite countless hours.

Can no longer execute rails commands due to missing gem > nokogiri : 1.5.2

Try to install the gem (which would be 1.5.5) or rails new (I am trying anything at this point...come back with the following error now: ERROR: While executing gem ... (Errno::EPERM)
Operation not permitted - /usr/local/rvm/gems/ruby-1.9.3-p194/gems/nokogiri-1.5.5/.autotest

Can anyone save me? I have to turn in an app by tomorrow.

Hello!

Thanks for asking this question! Your request for assistance using Nokogiri will not go unanswered!

However, this issue is closed, and Nokogiri's Github Issues is reserved for reporting bugs or submitting patches. If you ask your question on the mailing list, Team Nokogiri promises someone will provide you with an answer in a timely manner.

If you'd like to read up on Team Nokogiri's rationale for this policy, please go to http://bit.ly/nokohelp.

Thank you so much for understanding! And thank you for using Nokogiri.

brax4444, thanks so much, you're steps helped me get through a simliar issue installing the pg gem.

Instructions on the page http://nokogiri.org/tutorials/installing_nokogiri.html are now out of date due to bundle installing newer versions (currently libxml2 at 2.9.0 and libxslt at 1.1.28).

@Winstons

Thanks for your comment, but this issue has been closed for quite a while now.

If you've got an update for the nokogiri installation docs, you can either send a pull request to:

https://github.com/sparklemotion/nokogiri.org-tutorials

or you can more clearly state the problem and solutions (bundler doesn't install libxml2 or libxslt ... do you mean brew? or mac ports? or ... ?) in a new issue and we'll address it when we can independently confirm.

Thanks for using Nokogiri!

@flavorjones Sorry! I'll fork away.

Thanks for co-authoring Nokogiri.

I also needed to actually install XCode in order to get this to work.

I needed to install command line developer tools.

@dentarg Thanks, I had the same problem, after restoring my Mac, the command line developer tools weren't installed.
Do it with this command 'xcode-select --install'

use https://github.com/Homebrew/homebrew-dupes/blob/master/libiconv.rb

brew tab homebrew/dupes
brew install libiconv

and maybe this:
bundle config build.nokogiri --use-system-libraries

but dunno if this works

Thanks @brax4444 , your advice worked for me too!

I used the native iconv. You don't need to link libxml2/libxslt.

brew install libxml2 libxslt
gem install nokogiri -- \
    --with-xml2-lib=/usr/local/Cellar/libxml2/2.7.8/lib \
    --with-xslt-dir=/usr/local/Cellar/libxslt/1.1.26 \
    --with-iconv-lib=/usr/lib \
    --with-iconv-include=/usr/local/include 

Thanks @patricksereno.

bundle config build.nokogiri --use-system-libraries
this worked for me

Thank you, brax4444! That one line (gcc link) did it for me too!

SEE UPDATED INSTRUCTIONS: https://github.com/sparklemotion/nokogiri/issues/442#issuecomment-53793550

NOTE: if you are going through all these posts one-by-one trying everything then YOU ARE DOING IT WRONG. All of the advice to 'brew link' (including those post) are bad advice. All of the advice to "--use-system-libraries" is also bad advice. STAAAHHHHP. Skip to the end.

i believe this worked for me:

brew tap homebrew/dupes
brew install libiconv
brew link libiconv --force
gem install nokogiri -v 1.6.3.1 -- --with-iconv-dir=/usr/local

The important point here is that libiconv needs to be 'brew link'd into /usr/local/include. With the 'dupes' tap it'll bitch about overriding a system lib so you need to --force it. Then you just need to tell nokogiri to search /usr/local to look for the iconv path to pass to the embeded libxml2 when it builds that. By using --use-system-libraries what people are doing is bypassing the need to find this library in order to pass it into the internal libxml2 build.

One way to fix this would be to have homebrew add a pkg-config config for libiconv, since then the nokogiri extconf.rb will be abel to find it.

Another way would be to hardcode /usr/local/include as a search path here:

https://github.com/sparklemotion/nokogiri/blob/983ff1f2962c7f9fd20a6fc4c18784eccfff21c0/ext/nokogiri/extconf.rb#L132-L133

I've tried to figure out how to extract /usr/local/include out of a path in ruby's mkmf and fuck if I can figure out how to do that. The layers of indirection in mkmf are too deep. The find_header() method in mkmf can certainly find iconv.h, but I don't know how to extract the information about which path it finds it in. That would actually be the correct fix for nokogiri in order to find the header file in the same way that mkmf can find the header file.

It does seem that the fact that nokogiri cannot find a simple header file in /usr/local/include without a requirement for an external pkg-config is broken. I can see how hardcoding the path is ugly, but I don't know how to ask mkmf which directory find_header is finding the header in...

It also looks like there's a bug here where dir_config() can return an array of ':'-separated paths instead of an array of two paths, but it doesn't seem to have any bearing on fixing this problem:

https://github.com/sparklemotion/nokogiri/blob/983ff1f2962c7f9fd20a6fc4c18784eccfff21c0/ext/nokogiri/extconf.rb#L126-L130

@lamont-granquist great instructions, even better if you change tab to tap :-)

@ericbrooke lol, i copypasta'd that from a previous poster... fixed now...

I think that I tried every command mentioned in this post and still running into this:

sudo gem install nokogiri -- --with-xml2-include=/usr/local/Cellar/libxml2/2.8.0/include/libxml2 --with-xml2-lib=/usr/local/Cellar/libxml2/2.8.0/lib --with-xslt-dir=/usr/local/Cellar/libxslt/1.1.26 --with-iconv-lib=/usr/lib --with-iconv-include=/usr/local/include/
Building native extensions with: '--with-xml2-include=/usr/local/Cellar/libxml2/2.8.0/include/libxml2 --with-xml2-lib=/usr/local/Cellar/libxml2/2.8.0/lib --with-xslt-dir=/usr/local/Cellar/libxslt/1.1.26 --with-iconv-lib=/usr/lib --with-iconv-include=/usr/local/include/'
This could take a while...
Building nokogiri using packaged libraries.
ERROR:  Error installing nokogiri:
    ERROR: Failed to build gem native extension.

    /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby extconf.rb --with-xml2-include=/usr/local/Cellar/libxml2/2.8.0/include/libxml2 --with-xml2-lib=/usr/local/Cellar/libxml2/2.8.0/lib --with-xslt-dir=/usr/local/Cellar/libxslt/1.1.26 --with-iconv-lib=/usr/lib --with-iconv-include=/usr/local/include/
Building nokogiri using packaged libraries.
checking for iconv.h... yes
checking for iconv_open() in iconv.h... no
checking for iconv_open() in -liconv... no
checking for libiconv_open() in iconv.h... no
checking for libiconv_open() in -liconv... no
checking for iconv.h... yes
checking for iconv_open() in iconv.h... no
checking for iconv_open() in -liconv... no
checking for libiconv_open() in iconv.h... no
checking for libiconv_open() in -liconv... no
-----
libiconv is missing.  please visit http://nokogiri.org/tutorials/installing_nokogiri.html for help with installing dependencies.

I've got no idea why iconv.h is discovered, but iconv_open() not (it's defined in that file). I saw that someone ran into exactly the same problem recently - http://stackoverflow.com/questions/24735534/nokogiri-install-fails-iconv-doesnt-work-osx

Have anyone seen this issue?

SEE UPDATED INSTRUCTIONS: https://github.com/sparklemotion/nokogiri/issues/442#issuecomment-53793550

@Reinmar can you try what I suggested. In particular:

  1. brew link libiconv --force and then validate that ls -la /usr/local/include/iconv.h is present
  2. also validate ls -la /usr/local/lib/libiconv.dylib is there
  3. then simply: gem install nokogiri -- --with-iconv-dir=/usr/local

All of the paths you are using to versioned directories deep inside homebrew's 'Cellar' are kind of awful and brittle and I'm not too surprised its not working correctly.

I used these instructions http://nokogiri.org/tutorials/installing_nokogiri.html so the paths are similar (versions differ).

piotrus@macus ~> brew link libiconv --force
Warning: Already linked: /usr/local/Cellar/libiconv/1.14
To relink: brew unlink libiconv && brew link libiconv
piotrus@macus ~> ls -la /usr/local/include/iconv.h
lrwxr-xr-x  1 piotrus  admin  39 Jul 29 19:47 /usr/local/include/iconv.h -> ../Cellar/libiconv/1.14/include/iconv.h
piotrus@macus ~> ls -la /usr/local/lib/libiconv.dylib
lrwxr-xr-x  1 piotrus  admin  42 Jul 29 19:47 /usr/local/lib/libiconv.dylib -> ../Cellar/libiconv/1.14/lib/libiconv.dylib
piotrus@macus ~> gem install nokogiri -- --with-iconv-dir=/usr/local
ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions for the /Library/Ruby/Gems/2.0.0 directory.
piotrus@macus ~> sudo gem install nokogiri -- --with-iconv-dir=/usr/local
Password:
Building native extensions with: '--with-iconv-dir=/usr/local'
This could take a while...
Building nokogiri using packaged libraries.
ERROR:  Error installing nokogiri:
    ERROR: Failed to build gem native extension.

    /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby extconf.rb --with-iconv-dir=/usr/local
Building nokogiri using packaged libraries.
checking for iconv.h... yes
checking for iconv_open() in iconv.h... no
checking for iconv_open() in -liconv... no
checking for libiconv_open() in iconv.h... no
checking for libiconv_open() in -liconv... no

Maybe I'll try to update Ruby/gem... :| It's a blind guess though.

Ok... Updating gem hasn't helped. Then, after installing rvm (but before I updated Ruby from 2.0.0 to 2.1.*), I decided to restart terminal and mistakenly I switched from iterm to the standard terminal and... installing nokogiri worked with the same command which hadn't worked before. So either I had to restart terminal (though I was using maany tabs in iterm previously) or it was some problem caused by xterm. No idea, but maybe it will help someone.

in addition to @lamont-granquist answer, you could skip the gem install if you're using bundle and run:

bundle config build.nokogiri --use-system-libraries
bundle install

After following @lamont-granquist 's instructions:

brew tap homebrew/dupes
brew install libiconv
brew link libiconv --force
gem install nokogiri -v 1.6.3.1 -- --with-iconv-dir=/usr/local

And running into the same issue as @Reinmar , I found a suggestion on StackOverflow, and combined it with the other suggestions around here, resulting in:

NOKOGIRI_USE_SYSTEM_LIBRARIES=1 sudo gem install nokogiri -- --use-system-libraries --with-iconv-dir="$(brew --prefix libiconv)" --with-xml2-include=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/libxml2

That finally worked for me, hopefully that will help you out @Reinmar . Why it works is beyond me, _[ restrained comments about Apple's closed ecosystem here]_. Thanks all for sharing your methods.

I hade to rip out home-brew and all of the brews reinstall it from scratch and then it worked

@doublerebel i think you might need to reinstall xcode and/or use xcode-select. what you did is all kinds of confused anyway...

It seems that with the current version of homebrew, I was able to make do with a slightly simplified version of what @lamont-granquist came up with:

brew tap homebrew/dupes
brew install libiconv
gem install nokogiri -v 1.6.3.1 -- --with-iconv-dir=/usr/local/opt/libiconv

No brew link --force libiconv...

that makes sense @jacobvosmaer, the docs should probably be updated to use that.

(probably drop the -v 1.6.3.1, though)

i'd update the docs but i'm distracted right now with other coding...

After a whole day working on this issue, worth mentioning that @doublerebel solution above is the first thing to work

SEE UPDATED INSTRUCTIONS: https://github.com/sparklemotion/nokogiri/issues/442#issuecomment-53793550

this might help for users with Xcode 5.1.1:

 sudo xcode-select -s /Applications/Xcode.app/Contents/Developer

SEE UPDATED INSTRUCTIONS: https://github.com/sparklemotion/nokogiri/issues/442#issuecomment-53793550

Also, I'd suggest ensuring that Xcode 5.1.1 is installed (and/or reinstalling it) and then re-installing the "Command Line Tools for Xcode" here: https://developer.apple.com/downloads/

Then do:

sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
brew tap homebrew/dupes
brew uninstall libiconv
brew install libiconv
gem install nokogiri -- --with-iconv-dir=/usr/local/opt/libiconv

Mac OSX updates and restores seem to trash Xcode and homebrew for some reason. I also like pointing it at /usr/local/opt/libiconv better than force linking it into /usr/local/include.

@lamont-granquist Thanks for your solutions. I applied every single one and your one is the thing that finally works for me.

Running 'brew uninstall libxml2 libxslt' and then 'sudo gem install nokogiri' fixed it for me

Whist using RVM and creating a new gem set I had to do the following:

brew uninstall libiconv
brew install libiconv
gem install nokogiri -- --with-iconv-dir=/usr/local/opt/libiconv

@lamont-granquist Thanks a bunch for the fix. Worked for me.

@lamont-granquist @ericbrooke thank you. that killed it.

EDIT: EVEN SIMPLER INSTRUCTIONS: https://github.com/sparklemotion/nokogiri/issues/442#issuecomment-54091434

So merging in the good bits from the stackoverflow question on this answer:

Run this to install the latest Xcode and Xcode Command Line Tools (select 'Get Xcode' from the dialog box):

sudo xcode-select --install

Then update libiconv in homebrew:

brew tap homebrew/dupes
brew update
brew uninstall libiconv
brew install libiconv

The do the nokogiri install:

gem install nokogiri -- --with-iconv-dir=/usr/local/opt/libiconv

That should work for everyone on 10.9 + homebrew

would love some feedback on ^^^ that

worked for me, thanks!

-Chris

On Thu, Aug 28, 2014 at 4:40 PM, lamont-granquist [email protected]
wrote:

would love some feedback on ^^^ that


Reply to this email directly or view it on GitHub
https://github.com/sparklemotion/nokogiri/issues/442#issuecomment-53794057
.

Out of curiosity, what is brew uninstall libiconv for? Does it imply you already have a libiconv installed somehow?

@knu yes it implies that you are using Mac OsX and have installed libiconv via Homebrew and not MacPorts. Both of these are Package Managers. If you are not sure if brew is installed just type brew in terminal, if you get this response:

Example usage:
  brew [info | home | options ] [FORMULA...]
  brew install FORMULA...
  brew uninstall FORMULA...
  brew search [foo]
  brew list [FORMULA...]
  brew update
  brew upgrade [FORMULA...]
  brew pin/unpin [FORMULA...]

Troubleshooting:
  brew doctor
  brew install -vd FORMULA
  brew [--env | config]

Brewing:
  brew create [URL [--no-fetch]]
  brew edit [FORMULA...]
  open https://github.com/Homebrew/homebrew/wiki/Formula-Cookbook

Further help:
  man brew
  brew home

you know that homebrew is installed.

brew list

will show all the packages you have installed including libiconv

@ericbrooke I mean, I didn't see the point in doing brew uninstall libiconv right after brew tap homebrew/dupes when libiconv would only be found after tapping dupes because it's not in the standard repo. Is it there just in case, like when your homebrew installation is somehow broken?

Also, as you know from the presence of libiconv in "dupes", iconv is part of OS X+Xcode you shouldn't need to add on via brew.

I don't use Homebrew, but Homebrew should not touch anything under /usr/{include,lib}, right? Or is reinstalling libiconv via brew an attempt to recover those that were broken?

On a fresh MacBook Pro with vanilla Mountain Lion+Xcode+Command Line Tools installed I see /usr/include/iconv.h and /usr/lib/libconv.so there, so I have been wondering what the cause for the lack of those files could be.

@lamont-granquist thanks.

I did what you have listed. I ran into permission issue on:
You don't have write permissions for the /Library/Ruby/Gems/2.0.0 directory.

Apparently I need to fix my rvm and after following step it got installed and continue on my bundle install process.
rvm install 2.1.2
rvm use 2.1.2
gem update --system

@knu yeah every time i upgrade the O/S or restore from time machine it seems to trash homebrew and i have to reinstall everything. those instructions are as pessimistic as possible. if someone has a sane clean xcode installation and libiconv is installed correctly in homebrew then the last gem install command is all that should be needed.

So, it turns out that /usr/local/include/iconv.h will #define iconv_open to libiconv_open and for some reason clang picks up the homebrew header file in /usr/local/include over /usr/include and then tries to link against /usr/lib/libiconv.dylib. The trick is to remove the iconv.h header file created by force linking libiconv.

TL;DR: prior instructions on how to 'fix' this problem (including mine) make it all worse.

So, the up-to-date instructions on how to repair your mac installation:

Make sure Xcode and the command line utilities are installed correctly. This fixes issues where I had no /usr/include/iconv.h header file. Select 'Get Xcode' from the dialog box:

sudo xcode-select --install

You may want to update homebrew and libiconv in case it has gotten way out of date (and in case your libiconv in homebrew was installed a long, long time ago). This is probably entirely optional since we won't be linking against homebrew libiconv at all now:

brew update
brew tap homebrew/dupes
brew uninstall libiconv
brew install libiconv

Do not link libiconv, so it does not override the system libiconv (if this gives you 'no such keg' that is fine, any other error and you probably want to go back and upgrade homebrew and libiconv):

brew unlink libiconv

Build nokogiri:

gem install nokogiri

@lamont-granquist Thanks a lot!!

@lamont-granquist Dunno why, but in your last comment, gem install nokogiri did not work for me.
Instead, after following all the steps like:

brew update
brew tap homebrew/dupes
brew uninstall libiconv
brew install libiconv
brew unlink libiconv

gem install nokogiri -- --with-iconv-dir=/usr/local/opt/libiconvworked for me.

I'm on 10.9, with rvm, if this helps.

@lamont-granquist Thanks for the great investigation! I think I'll try to reproduce the problem and work out a "fix" for this.

@SeanWangDev somewhere you've got another iconv.h which is getting picked up first and used. It might be the MacPorts one instead of the Homebrew one.

Yes, I believe so. There is one at

/usr/local/Cellar/libiconv/1.14/include/iconv.h

another one at

/usr/include/iconv.h

The later one is a previous version.
Thanks @lamont-granquist

/usr/include/iconv.h is fine, that's part of Xcode, the former is homebrew's which is hidden away in the 'Cellar' which is also fine. If there's nothing in /opt/local/include or whatever, then my guess would be that /usr/lib/libiconv.dylib is damaged somehow?

@lamont-granquist This is a bit out of my depth now. How can we know libiconv.dylib is damaged?
When I do ls -al /usr/lib/libiconv*, the result is this

xxxx  1 xxx  xxx       16 10  5  2013 /usr/lib/libiconv.2.4.0.dylib -> libiconv.2.dylib
xxxx  1 xxx  xxx  2088992 10  5  2013 /usr/lib/libiconv.2.dylib
xxxx  1 xxx  xxx       20 10  5  2013 /usr/lib/libiconv.dylib -> libiconv.2.4.0.dylib

Yeah, that looks fine, this is mine:

% ls -al /usr/lib/libiconv*
lrwxr-xr-x  1 root  wheel       16 Feb 26  2014 /usr/lib/libiconv.2.4.0.dylib@ -> libiconv.2.dylib
-r-xr-xr-x  1 root  wheel  2088992 Feb 26  2014 /usr/lib/libiconv.2.dylib*
lrwxr-xr-x  1 root  wheel       20 Feb 26  2014 /usr/lib/libiconv.dylib@ -> libiconv.2.4.0.dylib

What about /opt/local/lib/libiconv.dylib or /opt/local/include/iconv.h? Either of those could cause issues.

@lamont-granquist
Nope, I don't have /opt/ directory. Are these what we are looking for?

-r--r--r--+ 1 User  admin  1052128  9  4 11:30 /usr/local/Cellar/libiconv/1.14/lib/libiconv.2.dylib
-rw-r--r--+ 1 User  admin  1658  9  4 11:22 /usr/local/Library/Taps/homebrew/homebrew-dupes/libiconv.rb
-r-xr-xr-x  1 root  wheel  2088992 10  5  2013 /usr/lib/libiconv.2.dylib

iconv.h files are just these two:

/usr/local/Cellar/libiconv/1.14/include/iconv.h
/usr/include/iconv.h

Nope that's all cool. As far as I can see you should be able to 'gem install nokogiri' without any other arguments....

Just wanted to add that gem install nokogiri -v 1.6.3.1 -- --with-iconv-dir=/usr/local worked well for me. When using bundler I had to do bundle config build.nokogiril --with-iconv-dir=/usr/local

@lamont-granquist Yes, gem install nokogiri works fine now. But this condition is different. This time nokogiri was already installed previously by 'gem install nokogiri -- --with-iconv-dir=/usr/local/opt/libiconv'. Previously, when I didn't have nokogiri installed, 'gem install nokogiri' won't work.
Is it possible that running 'gem install nokogiri -- --with-iconv-dir=/usr/local/opt/libiconv' changed something files or added some files like libiconv.dylib oriconv.h?

@SeanWangDev :+1:

OS X 10.9.4 Mavericks.
Guys please help me. I tried every single way which is above. None of them working for me. I stacked on gem install nokogiri with/without arguments, im still getting error:

$ gem install nokogiri
Building native extensions. This could take a while...
Building nokogiri using packaged libraries.
ERROR: Error installing nokogiri:
ERROR: Failed to build gem native extension.

/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/ruby extconf.rb

Building nokogiri using packaged libraries.
checking for iconv.h... * extconf.rb failed *
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.

Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/ruby
--help
--clean
--use-system-libraries
--enable-static
--disable-static
--with-zlib-dir
--without-zlib-dir
--with-zlib-include
--without-zlib-include=${zlib-dir}/include
--with-zlib-lib
--without-zlib-lib=${zlib-dir}/lib
--enable-cross-build
--disable-cross-build
/usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/mkmf.rb:434:in try_do': The compiler failed to generate an executable file. (RuntimeError) You have to install development tools first. from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/mkmf.rb:565:intry_cpp'
from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/mkmf.rb:1044:in block in have_header' from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/mkmf.rb:895:inblock in checking_for'
from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/mkmf.rb:340:in block (2 levels) in postpone' from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/mkmf.rb:310:inopen'
from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/mkmf.rb:340:in block in postpone' from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/mkmf.rb:310:inopen'
from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/mkmf.rb:336:in postpone' from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/mkmf.rb:894:inchecking_for'
from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/mkmf.rb:1043:in have_header' from extconf.rb:103:inhave_iconv?'
from extconf.rb:148:in block (2 levels) in iconv_prefix' from extconf.rb:90:inpreserving_globals'
from extconf.rb:143:in block in iconv_prefix' from extconf.rb:120:ineach_iconv_idir'
from extconf.rb:137:in iconv_prefix' from extconf.rb:428:inblock in

'
from extconf.rb:161:in block in process_recipe' from extconf.rb:154:intap'
from extconf.rb:154:in process_recipe' from extconf.rb:423:in
'

Gem files will remain installed in /usr/local/rvm/gems/ruby-2.0.0-p353/gems/nokogiri-1.6.3.1 for inspection.
Results logged to /usr/local/rvm/gems/ruby-2.0.0-p353/gems/nokogiri-1.6.3.1/ext/nokogiri/gem_make.out

validate you have these files:

% ls -la /usr/include/iconv.h
-rw-r--r--  1 root  wheel  7981 Jul 29 15:32 /usr/include/iconv.h
% ls -la /usr/lib/libiconv.*
lrwxr-xr-x  1 root  wheel       16 Feb 26  2014 /usr/lib/libiconv.2.4.0.dylib@ -> libiconv.2.dylib
-r-xr-xr-x  1 root  wheel  2088992 Feb 26  2014 /usr/lib/libiconv.2.dylib*
lrwxr-xr-x  1 root  wheel       20 Feb 26  2014 /usr/lib/libiconv.dylib@ -> libiconv.2.4.0.dylib

if you don't, use sudo xcode-select --install and/or go to the apple developer download site and manually uninstall and reinstall xcode and the xcode command line utilities until you get those files installed.

then if you use homebrew, update your homebrew and unlink the libiconv

brew update
brew tap homebrew/dupes
brew uninstall libiconv
brew unlink libiconv
brew install libiconv

validate that you do not have /usr/local/include/iconv.h:

% ls -la /usr/local/include/iconv.h
ls: /usr/local/include/iconv.h: No such file or directory

also validate that you do not have /opt/local/include/iconv.h

% ls -la  /opt/local/include/iconv.h
ls: /opt/local/include/iconv.h: No such file or directory

if you do have that file then you've got iconv installed from macports, ideally you would get rid of this.

then try:

gem install nokogiri
gem install nokogiri -- --with-iconv-dir=/usr
gem install nokogiri -- --with-iconv-dir=/usr/local/opt/libiconv
gem install nokogiri -- --with-iconv-dir=/opt/local

(stop when you find the one that works)

if all else fails:

brew link --force libiconv
gem install nokogiri -- --with-iconv-dir=/usr/local

something in there should work.

@lamont-granquist :+1:

@lamont-granquist Thanks for your instruction, I'm getting what is going on and have started working on a fix. However I'm still unable to reproduce the problem on any of my Macs so there's no way to confirm if a fix works. Can you possibly provide the steps for reproduction?

Well, I've had this MBP for 3 years and after 2 OS upgrades and at least 3 TM restores, my /usr/include/iconv.h had disappeared somehow. So nuke'ing that file can cause the problem. The solution is to re-install xcode and the command line tools properly.

Also, while debugging this one thing people suggest doing is to 'brew link libiconv --force' so that it is installed into /usr/local/include/iconv.h -- even though there's huge warnings when you do 'brew install libiconv' and 'brew link libiconv' that this is a bad idea. The problem there is that you can pick up that iconv.h while picking up the system /usr/lib/libiconv.dylib and that has a conflict right away over iconv_open() vs libiconv_open() and fails to compile because they're different versions.

I'm not sure there's anything to fix in those cases. The problem is really with mangling your operating system and needing to back it out to a more pristine shape.

There is one thing I don't understand which is why nokogiri's build system will find MacPorts-installed /opt/local/include/iconv.h files. I don't use MacPorts so haven't debugged that, but its causing similar problems to when homebrew has iconv.h linked directly into /usr/local/include. If nokogiri could be made to stop doing that, it would help with people who have libiconv installed from MacPorts.

$ ls -la /usr/include/iconv.h

-rw-r--r-- 1 root wheel 7981 Sep 11 11:32 /usr/include/iconv.h

ls -la /usr/lib/libiconv.*
lrwxr-xr-x 1 root wheel 16 Sep 11 01:03 /usr/lib/libiconv.2.4.0.dylib -> libiconv.2.dylib
-r-xr-xr-x 1 root wheel 2088992 Sep 11 01:02 /usr/lib/libiconv.2.dylib
lrwxr-xr-x 1 root wheel 20 Sep 11 01:03 /usr/lib/libiconv.dylib -> libiconv.2.4.0.dylib

brew update
Updated Homebrew from 45e07c0e to e0dee399.
==> Updated Formulae
android-ndk go lockrun pazpar2
apache-spark h2 mpd pktanon
gearman juju-quickstart ossp-uuid

$ brew tap homebrew/dupes
Warning: Already tapped!

$ brew uninstall libiconv

Uninstalling /usr/local/Cellar/libiconv/1.13.1...

Error: Permission denied - /usr/local/Cellar/libiconv/1.13.1/bin/iconv

it's giving me Permission error on that step

you need to fix all your permissions under /usr/local

whatever your login is you need to fix the perms so root does not own homebrew files under /usr/local:

my username is 'lamont' which i can find with the id command:

% id -nu
lamont

to fix perms, i would do:

% sudo chown -R lamont /usr/local

you want to be real careful with that command, though, and double check that you've spelled everything correctly and gotten the right username.

@lamont-granquist Doesn't the libiconv package of Homebrew install /usr/local/lib/libiconv.dylib while putting /usr/local/include/iconv.h? What I don't understand is if /usr/local/include/iconv.h is picked up why /usr/local/lib/libiconv.dylib isn't when linking. Usually when -I/usr/local/include is in CPPFLAGS -L/usr/local/lib should be in LDFLAGS.

As for the environmental breakage, Nokogiri could at least suggest (re)installing Command Line Tools when /usr/include/iconv.h is missing on OS X.

brew unlink libiconv
Error: No such keg: /usr/local/Cellar/libiconv

I ignored message and followed next steps. No success

@knu yeah, i looked into that and i think i got lost in the twisty maze of mkmf.rb.

@Ilovebarca rm -f /usr/local/include/iconv.h might help

@lamont-granquist
nope didnt help. BTW thanks a lot for trying to help me.
That's crazy, im new on mac and im starting to hate it :))

I'm going to make some changes in the iconv detection process and added a check for missing /usr/include/iconv.h on OS X to extconf.rb. Here's a work in progress, and testers are welcome!

I've been going through github issues, stack exchange, google etc and the only thing that has worked so far has been @lamont-granquist suggestion. Setup using:

OSX 10.9.5
rbenv

I understand so many people somehow get their OS X environment broken and they don't even notice that. One problem is missing Xcode Command Line Tools and the other is Homebrew. The next release of nokogiri will try to detect such a breakage before entering the build and warn user of it.

Thanks for your support!

Thanks @lamont-granquist, I've been struggling this problem for a week now and finally your solution of reinstalling back libiconv works...that's a relief!!!

@jomz works on osx 10.10. thanks :+1:

From a fresh install of 10.10 just today:

xcode-select --install
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew doctor
brew install ruby
source /etc/profile
gem install nokogiri

This installed ruby version 2.1.5p273 and nokogiri 1.6.4.1. Presumably you could force the install of these versions via homebrew and rubygems.

From a 10.10 install upgraded from 10.9 with homebrew and ruby already installed:

brew install libxml2 libxslt
gem install nokogiri -- --use-system-libraries

I'm not sure it was necessary to re/install libxml2 and/or libxslt, but this did the trick for me.

I believe we've captured this in the updated installation tutorial here:

http://www.nokogiri.org/tutorials/installing_nokogiri.html

If not, please let me know and I'll reopen.

Thanks for the tutorial link! It worked for me when I specified the file paths, as described in "Using Nonstandard libxml2 / libxslt installations." (Seems /usr/bin in OS X 10.6.8 is considered a nonstandard location for system libraries.)

To find the file paths, I used the Terminal commands which xml2-config and which xslt-config. Maybe update the tutorial to suggest doing this?

Was this page helpful?
0 / 5 - 0 ratings