Django-compressor: Installation bug django-compressor (rcssmin)

Created on 29 Oct 2016  ·  20Comments  ·  Source: django-compressor/django-compressor

➜  byvshie ./build
+++ dirname ./build
++ cd .
++ pwd
+ DIR=/Users/lorddaedra/Projects/byvshie
+ docker build -t entropia/byvshie:latest -f /Users/lorddaedra/Projects/byvshie/Dockerfile /Users/lorddaedra/Projects/byvshie
Sending build context to Docker daemon 22.76 MB
Step 1 : FROM python:3.5-alpine
 ---> e57c79136a08
Step 2 : ENV PYTHONUNBUFFERED 1
 ---> Using cache
 ---> 22523b31cff4
Step 3 : ENV DJANGO_SETTINGS_MODULE byvshie.settings
 ---> Running in de1961075197
 ---> 145ec94af4da
Removing intermediate container de1961075197
Step 4 : ENV LIBRARY_PATH /lib:/usr/lib
 ---> Running in 4d3affe0e89e
 ---> a380f6806984
Removing intermediate container 4d3affe0e89e
Step 5 : WORKDIR /
 ---> Running in 1f46cac384aa
 ---> b81cbeaf84dd
Removing intermediate container 1f46cac384aa
Step 6 : RUN set -ex     && mkdir -p /usr/src/app
 ---> Running in e1e0a13099db
+ mkdir -p /usr/src/app
 ---> cd86298883ed
Removing intermediate container e1e0a13099db
Step 7 : COPY requirements.txt /usr/src/app
 ---> b21ad0d5135f
Removing intermediate container 43070bfb3213
Step 8 : COPY config/Lib/ctypes/util.py /usr/local/lib/python3.5/ctypes
 ---> 3ff35452a40d
Removing intermediate container 16ff555230a4
Step 9 : RUN set -ex    && addgroup -g 82 -S www-data   && adduser -u 82 -D -S -G www-data www-data     && apk update     && apk upgrade     && build_pkgs="build-base linux-headers openssl-dev pcre-dev wget postgresql-dev libxml2-dev libxslt-dev libffi-dev jpeg-dev freetype-dev  tiff-dev libwebp-dev zlib-dev git mercurial"     && runtime_pkgs="ca-certificates postgresql openssl libxml2 libxslt libffi jpeg freetype tiff libwebp    libmagic gettext pcre zlib bash bc"     && apk --update add --no-cache ${build_pkgs} ${runtime_pkgs}     && pip install --no-cache-dir  -U setuptools pip     && pip install --no-cache-dir  -r /usr/src/app/requirements.txt     && apk del ${build_pkgs}     && rm -rf /var/cache/apk/*
 ---> Running in 2b07b2bf8b7c
+ addgroup -g 82 -S www-data
+ adduser -u 82 -D -S -G www-data www-data
+ apk update
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz
v3.4.5 [http://dl-cdn.alpinelinux.org/alpine/v3.4/main]
v3.4.4-21-g75fc217 [http://dl-cdn.alpinelinux.org/alpine/v3.4/community]
OK: 5974 distinct packages available
+ apk upgrade
(1/2) Upgrading musl (1.1.14-r12 -> 1.1.14-r13)
(2/2) Upgrading musl-utils (1.1.14-r12 -> 1.1.14-r13)
Executing busybox-1.24.2-r11.trigger
OK: 31 MiB in 32 packages
+ build_pkgs=build-base linux-headers openssl-dev pcre-dev wget postgresql-dev libxml2-dev libxslt-dev libffi-dev jpeg-dev freetype-dev  tiff-dev libwebp-dev zlib-dev git mercurial
+ runtime_pkgs=ca-certificates postgresql openssl libxml2 libxslt libffi jpeg freetype tiff libwebp    libmagic gettext pcre zlib bash bc
+ apk --update add --no-cache build-base linux-headers openssl-dev pcre-dev wget postgresql-dev libxml2-dev libxslt-dev libffi-dev jpeg-dev freetype-dev tiff-dev libwebp-dev zlib-dev git mercurial ca-certificates postgresql openssl libxml2 libxslt libffi jpeg freetype tiff libwebp libmagic gettext pcre zlib bash bc
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz
(1/69) Installing bash (4.3.42-r3)
Executing bash-4.3.42-r3.post-install
(2/69) Installing bc (1.06.95-r2)
(3/69) Installing binutils-libs (2.26-r0)
(4/69) Installing binutils (2.26-r0)
(5/69) Installing gmp (6.1.0-r0)
(6/69) Installing isl (0.14.1-r0)
(7/69) Installing libgomp (5.3.0-r0)
(8/69) Installing libatomic (5.3.0-r0)
(9/69) Installing libgcc (5.3.0-r0)
(10/69) Installing pkgconf (0.9.12-r0)
(11/69) Installing pkgconfig (0.25-r1)
(12/69) Installing mpfr3 (3.1.2-r0)
(13/69) Installing mpc1 (1.0.3-r0)
(14/69) Installing libstdc++ (5.3.0-r0)
(15/69) Installing gcc (5.3.0-r0)
(16/69) Installing make (4.1-r1)
(17/69) Installing musl-dev (1.1.14-r13)
(18/69) Installing libc-dev (0.7-r0)
(19/69) Installing fortify-headers (0.8-r0)
(20/69) Installing g++ (5.3.0-r0)
(21/69) Installing build-base (0.4-r1)
(22/69) Installing zlib-dev (1.2.8-r2)
(23/69) Installing libpng-dev (1.6.21-r0)
(24/69) Installing freetype-dev (2.6.3-r0)
(25/69) Installing libintl (0.19.7-r3)
(26/69) Installing libunistring (0.9.6-r0)
(27/69) Installing libxml2 (2.9.4-r0)
(28/69) Installing gettext (0.19.7-r3)
(29/69) Installing libssh2 (1.7.0-r0)
(30/69) Installing libcurl (7.50.3-r0)
(31/69) Installing pcre (8.38-r1)
(32/69) Installing git (2.8.3-r0)
(33/69) Installing libjpeg-turbo (1.4.2-r0)
(34/69) Installing libjpeg-turbo-utils (1.4.2-r0)
(35/69) Installing jpeg (8-r6)
(36/69) Installing libjpeg-turbo-dev (1.4.2-r0)
(37/69) Installing jpeg-dev (8-r6)
(38/69) Installing libffi (3.2.1-r2)
(39/69) Installing libffi-dev (3.2.1-r2)
(40/69) Installing libmagic (5.27-r0)
(41/69) Installing libwebp (0.5.0-r0)
(42/69) Installing libwebp-dev (0.5.0-r0)
(43/69) Installing libxml2-dev (2.9.4-r0)
(44/69) Installing libgpg-error (1.23-r0)
(45/69) Installing libgcrypt (1.7.0-r1)
(46/69) Installing libxslt (1.1.29-r0)
(47/69) Installing libxslt-dev (1.1.29-r0)
(48/69) Installing linux-headers (4.4.6-r1)
(49/69) Installing gdbm (1.11-r1)
(50/69) Installing python (2.7.12-r0)
(51/69) Installing mercurial (3.8.2-r0)
(52/69) Installing openssl (1.0.2j-r0)
(53/69) Installing openssl-dev (1.0.2j-r0)
(54/69) Installing libpcre16 (8.38-r1)
(55/69) Installing libpcre32 (8.38-r1)
(56/69) Installing libpcrecpp (8.38-r1)
(57/69) Installing pcre-dev (8.38-r1)
(58/69) Installing libedit (20150325.3.1-r3)
(59/69) Installing db (5.3.28-r0)
(60/69) Installing libsasl (2.1.26-r7)
(61/69) Installing libldap (2.4.44-r1)
(62/69) Installing libpq (9.5.4-r0)
(63/69) Installing postgresql-client (9.5.4-r0)
(64/69) Installing postgresql (9.5.4-r0)
(65/69) Installing postgresql-libs (9.5.4-r0)
(66/69) Installing postgresql-dev (9.5.4-r0)
(67/69) Installing tiff (4.0.6-r3)
(68/69) Installing tiff-dev (4.0.6-r3)
(69/69) Installing wget (1.18-r0)
Executing busybox-1.24.2-r11.trigger
Executing ca-certificates-20160104-r4.trigger
OK: 304 MiB in 101 packages
+ pip install --no-cache-dir -U setuptools pip
Collecting setuptools
  Downloading setuptools-28.7.0-py2.py3-none-any.whl (472kB)
Requirement already up-to-date: pip in /usr/local/lib/python3.5/site-packages
Installing collected packages: setuptools
  Found existing installation: setuptools 20.10.1
    Uninstalling setuptools-20.10.1:
      Successfully uninstalled setuptools-20.10.1
Successfully installed setuptools-28.7.0
+ pip install --no-cache-dir -r /usr/src/app/requirements.txt
Collecting argon2_cffi==16.2.0 (from -r /usr/src/app/requirements.txt (line 1))
  Downloading argon2_cffi-16.2.0.tar.gz (1.2MB)
Collecting bcrypt==3.1.1 (from -r /usr/src/app/requirements.txt (line 2))
  Downloading bcrypt-3.1.1.tar.gz (40kB)
Collecting certifi==2016.9.26 (from -r /usr/src/app/requirements.txt (line 3))
  Downloading certifi-2016.9.26-py2.py3-none-any.whl (377kB)
Collecting cffi==1.8.3 (from -r /usr/src/app/requirements.txt (line 4))
  Downloading cffi-1.8.3.tar.gz (403kB)
Collecting defusedxml==0.4.1 (from -r /usr/src/app/requirements.txt (line 5))
  Downloading defusedxml-0.4.1.tar.gz (48kB)
Collecting Django==1.10.2 (from -r /usr/src/app/requirements.txt (line 6))
  Downloading Django-1.10.2-py2.py3-none-any.whl (6.8MB)
Collecting django-appconf==1.0.2 (from -r /usr/src/app/requirements.txt (line 7))
  Downloading django_appconf-1.0.2-py2.py3-none-any.whl
Collecting django-app-namespace-template-loader==0.4.1 (from -r /usr/src/app/requirements.txt (line 9))
  Downloading django_app_namespace_template_loader-0.4.1-py2.py3-none-any.whl
Obtaining django-backblazeb2-storage from git+https://github.com/royendgel/django-backblazeb2-storage.git@f017e4ce7564c852f55d2aa67e70983ccc7563ec#egg=django-backblazeb2-storage (from -r /usr/src/app/requirements.txt (line 10))
  Cloning https://github.com/royendgel/django-backblazeb2-storage.git (to f017e4ce7564c852f55d2aa67e70983ccc7563ec) to /src/django-backblazeb2-storage
  Could not find a tag or branch 'f017e4ce7564c852f55d2aa67e70983ccc7563ec', assuming commit.
Collecting django_compressor==2.1 (from -r /usr/src/app/requirements.txt (line 11))
  Downloading django_compressor-2.1-py2.py3-none-any.whl (130kB)
Collecting django-debug-toolbar==1.6 (from -r /usr/src/app/requirements.txt (line 12))
  Downloading django_debug_toolbar-1.6-py2.py3-none-any.whl (205kB)
Collecting django-el-pagination==3.0.1 (from -r /usr/src/app/requirements.txt (line 13))
  Downloading django-el-pagination-3.0.1.tar.gz (358kB)
Collecting django-formtools==1.0 (from -r /usr/src/app/requirements.txt (line 14))
  Downloading django_formtools-1.0-py2.py3-none-any.whl (132kB)
Collecting django-mailer==1.2.2 (from -r /usr/src/app/requirements.txt (line 15))
  Downloading django-mailer-1.2.2.tar.gz
Collecting django-user-accounts==2.0.0 (from -r /usr/src/app/requirements.txt (line 16))
  Downloading django_user_accounts-2.0.0-py2.py3-none-any.whl (104kB)
Collecting docutils==0.12 (from -r /usr/src/app/requirements.txt (line 17))
  Downloading docutils-0.12-py3-none-any.whl (508kB)
Collecting lxml==3.6.4 (from -r /usr/src/app/requirements.txt (line 18))
  Downloading lxml-3.6.4.tar.gz (3.7MB)
Collecting oauthlib==2.0.0 (from -r /usr/src/app/requirements.txt (line 19))
  Downloading oauthlib-2.0.0.tar.gz (122kB)
Collecting Pillow==3.4.2 (from -r /usr/src/app/requirements.txt (line 20))
  Downloading Pillow-3.4.2.tar.gz (10.8MB)
Collecting psycopg2==2.6.2 (from -r /usr/src/app/requirements.txt (line 21))
  Downloading psycopg2-2.6.2.tar.gz (376kB)
Collecting pycparser==2.16 (from -r /usr/src/app/requirements.txt (line 22))
  Downloading pycparser-2.16.tar.gz (230kB)
Collecting PyJWT==1.4.2 (from -r /usr/src/app/requirements.txt (line 23))
  Downloading PyJWT-1.4.2-py2.py3-none-any.whl
Collecting python-magic==0.4.12 (from -r /usr/src/app/requirements.txt (line 24))
  Downloading python-magic-0.4.12.tar.gz
Collecting python-simple-hipchat==0.4.0 (from -r /usr/src/app/requirements.txt (line 25))
  Downloading python-simple-hipchat-0.4.0.tar.gz
Collecting python-social-auth==0.2.21 (from -r /usr/src/app/requirements.txt (line 26))
  Downloading python_social_auth-0.2.21-py3-none-any.whl (302kB)
Collecting python3-openid==3.0.10 (from -r /usr/src/app/requirements.txt (line 27))
  Downloading python3-openid-3.0.10.zip (365kB)
Collecting pytz==2016.7 (from -r /usr/src/app/requirements.txt (line 28))
  Downloading pytz-2016.7-py2.py3-none-any.whl (480kB)
Collecting raven==5.30.0 (from -r /usr/src/app/requirements.txt (line 29))
  Downloading raven-5.30.0-py2.py3-none-any.whl (277kB)
Collecting requests==2.11.1 (from -r /usr/src/app/requirements.txt (line 30))
  Downloading requests-2.11.1-py2.py3-none-any.whl (514kB)
Collecting requests-oauthlib==0.7.0 (from -r /usr/src/app/requirements.txt (line 31))
  Downloading requests_oauthlib-0.7.0-py2.py3-none-any.whl
Collecting simplejson==3.8.2 (from -r /usr/src/app/requirements.txt (line 32))
  Downloading simplejson-3.8.2.tar.gz (76kB)
Collecting six==1.10.0 (from -r /usr/src/app/requirements.txt (line 33))
  Downloading six-1.10.0-py2.py3-none-any.whl
Collecting sqlparse==0.2.1 (from -r /usr/src/app/requirements.txt (line 34))
  Downloading sqlparse-0.2.1-py2.py3-none-any.whl
Collecting uwsgi==2.0.14 (from -r /usr/src/app/requirements.txt (line 35))
  Downloading uwsgi-2.0.14.tar.gz (788kB)
Collecting rcssmin==1.0.6 (from django_compressor==2.1->-r /usr/src/app/requirements.txt (line 11))
  Downloading rcssmin-1.0.6.tar.gz (582kB)
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info/rcssmin.egg-info
    writing pip-egg-info/rcssmin.egg-info/PKG-INFO
    writing dependency_links to pip-egg-info/rcssmin.egg-info/dependency_links.txt
    writing top-level names to pip-egg-info/rcssmin.egg-info/top_level.txt
    writing manifest file 'pip-egg-info/rcssmin.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-dzarxluf/rcssmin/setup.py", line 42, in <module>
        setup()
      File "/tmp/pip-build-dzarxluf/rcssmin/setup.py", line 33, in setup
        return run(script_args=args, ext=ext, manifest_only=_manifest)
      File "/tmp/pip-build-dzarxluf/rcssmin/_setup/py3/setup.py", line 426, in run
        return _core.setup(**kwargs)
      File "/usr/local/lib/python3.5/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/local/lib/python3.5/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/egg_info.py", line 279, in run
        self.find_sources()
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/egg_info.py", line 306, in find_sources
        mm.run()
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/egg_info.py", line 533, in run
        self.add_defaults()
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/egg_info.py", line 562, in add_defaults
        sdist.add_defaults(self)
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/py36compat.py", line 35, in add_defaults
        self._add_defaults_data_files()
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/py36compat.py", line 111, in _add_defaults_data_files
        dirname, filenames = item
    TypeError: 'Documentation' object is not iterable

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-dzarxluf/rcssmin/
The command '/bin/sh -c set -ex     && addgroup -g 82 -S www-data   && adduser -u 82 -D -S -G www-data www-data     && apk update     && apk upgrade     && build_pkgs="build-base linux-headers openssl-dev pcre-dev wget postgresql-dev libxml2-dev libxslt-dev libffi-dev jpeg-dev freetype-dev  tiff-dev libwebp-dev zlib-dev git mercurial"     && runtime_pkgs="ca-certificates postgresql openssl libxml2 libxslt libffi jpeg freetype tiff libwebp    libmagic gettext pcre zlib bash bc"     && apk --update add --no-cache ${build_pkgs} ${runtime_pkgs}     && pip install --no-cache-dir  -U setuptools pip     && pip install --no-cache-dir  -r /usr/src/app/requirements.txt     && apk del ${build_pkgs}     && rm -rf /var/cache/apk/*' returned a non-zero code: 1

Most helpful comment

All 20 comments

It's not related to that issue, /I have got the right headers installed and passing --install-option=without-c-extensions doesn't help. I think it's a problem with rcssmin being broken by setuptools.

I may suggest to use wheels ( http://pythonwheels.com/ ) , it should save from similar installation issues in future...

Yeah, except rcss is doing something weird and I can't seem to get it to build a wheel.

May I also suggest to do not require minification packages because of it may be useful even without them.

I use free CloudFlare, which provide CDN services and DNS. And I can enable JS/CSS/HTML minification from their side. But I need 1 .css file. It's main reason why do I need packages like django-compressor. Combine all .css to one during Docker image build step.

The same issue affects the rjsmin dependency.

let me try to understand: the problem in the original post is an installation issue of rcssmin/rjsmin and has nothing to do with django-compressor. however, if django-compressor wouldn't require these projects, their errors would be less of a pain for django-compressor users. additionally, it would be nice to distribute django-compressor as a wheel, since that might prevent some problems, but probably not this particular one.

if that is correct, @lorddaedra could you open new issues for the dependency and wheel thing and would you be fine with me closing this issue here? if that's not correct, please help me understand, i'm pretty much clueless when it comes to python packaging.

btw, could one of you guys post how you worked around this for other people landing here?

On the ndparker/rcssmin#6 issue, @dan-gittik reports manually calling python setup.py install to install the dependencies before leaving it to pip. That seems like a workable solution in a local environment while this is resolved, but might require a bit of fiddling depending on whatever deployment mechanism people are using to run production..

thanks!

@karyon, I posted this bug here because of, well, "vassal of my vassal is not my vassal" principle. I thought it was a good idea to write it here... At least you should know about this installation bug in one of your main components...

People that are after a quick (temporary) solution, you can downgrade setuptools using pip with these 2 commands:

  • pip uninstall setuptools
  • pip install -I setuptools==28.6.1

Then pip install django-compressor should work.

Setuptools has reverted the offending change in 28.7.1. I recommend closing this issue and letting setuptools resolve it.

agreed, thanks for the update.

@lorddaedra it absolutely was a good idea to post that problem here, i didn't mean to imply otherwise :) i'll close this issue, but again feel free to open new ones for the dependency and wheel issues.

..actually i'll leave this open for some days, in case any more people are stranding here

closing. thanks again for all your help!

For windows this requires the following library, follow the steps indicated here https://www.scivision.dev/python-windows-visual-c-14-required/

Was this page helpful?
0 / 5 - 0 ratings