Pip: Potential bug in "tests.lib.create_basic_wheel_for_package"
Environment
- pip version: 20.1.dev0
- Python version: 3.8.2
- OS: OSX 10.15.4
Description
When the helper tests.lib.create_basic_wheel_for_package is utilized in a unit test to create a wheel with name containing a -, e.g. simple-package, and then the wheel is installed, the package name is taken as simple and the version is taken as package
Expected behavior
Seems like according to PEP-491 file name convention, the package name cannot contain a -, so such package names shouldn't be allowed while creating wheels for testing purposes. (My understanding of the PEP might also be wrong here)
How to Reproduce
Execute the following unit test
def test_create_wheel_bug(script):
package = create_basic_wheel_for_package(script, 'simple-package', '1.0')
script.pip("install", "--no-cache-dir", "--no-index", package)
result = script.pip('list', '--format=json')
assert 'simple-package' in json.loads(result.stdout)
Output
The test fails with an assertion error because the output of pip list --format=json lists the package name as simple and version as package
assert 'simple-package' in [{'name': 'pip', 'version': '20.1.dev0'},
{'name': 'setuptools', 'version': '46.1.3'}, {'name': 'simple', 'version': 'package'}]
deveshks
All 7 comments
Ah, this makes sense. The culprit is this line:
archive_name = "{}-{}-py2.py3-none-any.whl".format(name, version)
which would give simple-package-1.0-py2.py3-none-any.whl, but the wheel spec expects simple_package-1.0-py2.py3-none-any.whl.
The fix should be straightforward as well:
archive_name = "{}-{}-py2.py3-none-any.whl".format(
canonicalize_name(name).replace('-', '_'),
version,
)
uranusjr
on 16 Apr 2020
I think that's the right approach, to canonicalize the name and replace - with _ before making the wheel file. Let me create a fix for it right away and use the same test which I wrote in the ticket to verify that it's been fixed :)
deveshks
on 16 Apr 2020
I just realised PEP 491 actually includes a regex to canonically transform a package name for a wheel’s file name. Maybe we should use that instead (with a link to the PEP); it would be much easier to understand for a future reader.
uranusjr
on 16 Apr 2020
I think you mean re.sub("[^\w\d.]+", "_", distribution, re.UNICODE) present at PEP-491 Escaping and Unicode ?
So we would create a correct wheel file name using that regex? Also do we want to do anything with the folder name of the package as well for this change?
Also will the unit test for this be similar to what we discussed in https://github.com/pypa/pip/pull/8054 ? (Note that the below might not work due to the quirk we found with pkg_resources.safe_name until we start using canonicalize_name as https://github.com/pypa/pip/pull/8054#discussion_r409654545)
@pytest.mark.parametrize(
'package_name',
['simple-package', 'simple_package', 'simple.package'],
)
def test_create_wheel_bug(script):
package = create_basic_wheel_for_package(script, package_name, '1.0')
script.pip("install", "--no-cache-dir", "--no-index", package)
result = script.pip('list', '--format=freeze')
assert package_name in result.stdout
Yes, that’s it. The same name can be used for the package as well (I don’t think the package is actually used by a lot of tests anyway).
@pradyunsg Does this need a test?
uranusjr
on 16 Apr 2020
Yea, I think adding a test in test_lib.py is a good idea.
pradyunsg
on 17 Apr 2020
Yea, I think adding a test in test_lib.py is a good idea.
So what will that test do ? Is it enough to just try out variants of package names with dot, dash and underscore, and verify that the wheel file name is correct according to the spec for all 3 cases?
deveshks
on 17 Apr 2020
Related issues
pradyunsg
·
3Comments
lululukas
·
3Comments
Zac-HD
·
3Comments
therefromhere
·
3Comments
dstufft
·
3Comments
Most helpful comment
Ah, this makes sense. The culprit is this line:
which would give
simple-package-1.0-py2.py3-none-any.whl, but the wheel spec expectssimple_package-1.0-py2.py3-none-any.whl.The fix should be straightforward as well: