pip freeze crashes with Invalid requirement, parse error at "'-lxc==0'" #3681
Comments
|
I was able to discover that the weird package is |
ssbarnea
changed the title
|
I have the same issue with debian-8.4, it happen with pip from the debian repository 1.5.6 and also with the last 8.1.2 from pypi. However 8.8.1 from pypi is working and |
|
I've not seen this happen myself yet, but some contributors to my project (using tox) saw this happen with |
|
As said user (contributor? not hardly), I just want to note that the OS is FreeBSD 10.3, and that I get the error on python versions 3.4 and 3.5. |
|
Same thing here, and I confirm that in pip 8.1.1 I can get normal output from It's pretty weird for a dependency to get listed starting with a dash, though. |
|
Hmm, a dependency starting with a |
|
For |
|
Probably should file a ticket with Debian on the lxc package. Names with a leading or trailing character that isn't an ascii alpha numeric are not valid. They worked previously due to setuptools attempting to accept whatever garbage was there (no matter how nonsensical) but things have gotten stricter in the past couple years in an attempt to better define behavior. Sent from my iPhone
|
|
That being said. We should probably just issue a warning and skip this package rather than bomb out completely. Sent from my iPhone
|
In any case, I'm not familiar with the motivations for adding these restrictions, so maybe there's a good reason behind it. I just wanted to share my reaction to the condition. |
|
@jaraco The reason for it is largely to make it harder to make names that appear to be one thing but are actually another thing and in this vein it's entirely unlike Python identifiers because you don't have to worry about people trying to attack you using a similar identifier. This also goes back to some history in the setuptools behavior too which is why we went a bit stricter overall. For instance, folks create create a guide telling people to copy/paste In addition to all of that, I'm not sure it makes sense to have this value there at all. Either |
|
I understand the motivation and support the decision of disallowing strange characters in package names, and I do find it ugly and not-pythonic to start a package with underscore; but I think underscores shouldn't be disallowed, since they're not ambiguous for the human eye. Not even in the start of the package name - though I personally find it ugly. |
|
Here is the bug report for Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=824669 |
|
The problem is very easy to replicate with any recent Ubuntu distribution (like 15.10). Just do While I totally agree that the naming is not good, I find unacceptable to for pip to crash. That's what we need to fix first, the pip failure. We will always have broken package names, as we cannot control the ability of people to make mistakes. What we can do is to improve our ability to deal with them. |
|
Agreed pip shouldn't crash. A warning and skipping the package seems like a reasonable fix (as @dstufft suggested). PRs, as usual, welcome :-) Also, help in making people more aware of the standards - either in the form of passing the word around, or PRs to documentation to clarify it, or as in this case bug reports to projects that don't conform - is hugely appreciated. The biggest problem we have in Python packaging is making people aware when we tighten up risks/loopholes (as in this case), or enforce long-standing but previously ignored rules. So getting the word out is a great help. |
|
There are a couple of problems with this issue that I'd like to untangle. Clearly pip shouldn't crash. I'd argue that it should never crash. There's really very little an end user can do when it does except get frustrated. In this case, there's no obvious repair the end user can guess to do based on the error. Where it finds bogus data, pip should complain, maybe try to offer a way out, and just continue on, ignoring the bogus data. It's also the case that the problem is very far removed from the source of the problem. What did the lxc developers do wrong to create this situation? Why did the tools allow them to create a bogus .egg-info file that can break end users without the lxc maintainers being aware of the problem? How do we stop them from creating the bogus data in the first place? I'm in the camp that disallowing _lxc as a package name seems very weird. @dstufft mentioned to me that package names and module names aren't the same thing, pointing to the fact that foo-bar is a valid package name but not a valid module name. And re: the point above about attempts to disguise malicious packages with almost correct looking package names. Fair points all, but I'm not sure the specific case of using a leading underscore should be disallowed, and if it must be then we have to make sure such packages can never be created, and won't break a system if they are specially crafted to be malicious. |
|
I'm OK with changing the PEP (can't see a reference to the number here, I presume @dstufft would know) to allow leading underscores. I also agree that pip shouldn't crash, IMO it should warn and skip any invalid package names (PRs welcome, as usual :-)) As far as not being allowed to build/upload/etc packages with invalid names, that's an entirely reasonable suggestion, but not one for the pip tracker. Someone would have to raise issues against setuptools, and distutils (at a minimum - what about tools like flit?) saying that they don't enforce PEP xxx. I don't know if PyPI allows registration of new non-compliant names, nor do I know what it does about existing ones (I suspect nothing, as the PyPI codebase is pretty much on life support). I assume that Warehouse enforces the rules. But I don't think pip should be forced to continue allowing legacy package names until everyone else changes. Someone has to be the first to implement new standards, why not pip? We've got a history of pip driving adoption of new standards which has so far worked well, no reason to alter that now. |
|
PEP 508 is the most relevant one to this particular behavior, though PEP 508 copied the restrictions from PEP 426. PyPI does not allow invalid names for a good long time now (I want to say a year+ but I could be misremembering) and when I made that change we also scrubbed the hand full of invalid names from the database. Changing the restriction would require updating the PEP (since we don't want to start going back down the path of implementation defined behavior) though I'm not sure if it's a good idea-- I definitely think we should not relax the restrictions on PyPI and I think it is simpler if the same restrictions exist everywhere instead of people having to remember which set of restrictions apply where (transitional issues with some pieces getting updated faster than others aside). In any case, discussions about whether or not to change the PEP belong on distutils-sig rather than the pip issue tracker. I also agree that pip should not relax the rules unless the PEP changes. Historically we've found that the only way we really push people forward is to force their hand through breaking the edge cases we no longer support. If we don't, people just never change their behavior and we're never able to deprecate anything. I also agree that setuptools should start to enforce not using an invalid name (though at this point, if people are going to try and get the PEP changed, we should wait until that gets decided if we will or not). |
|
Okay, if PEP 508/426 already define the name limitations, and PyPI already adheres to these, then I think you're right that we're down to two things that need to be done: make pip more robust in the face of invalid names, and make setuptools reject invalid names at the start. (Admittedly the latter isn't relevant for this project/issue.) |
|
Not to be dismissive or anything, but the quickest way to get this resolved would be if someone else can make a PR for it. I can commit to reviewing it if someone does, otherwise I'll try to get to it as quickly as I can but I have a lot of competing priorities at the moment. |
|
Fixed by #3704 |
|
I'm actually still getting this error on Arch Linux: |
|
The fix was committed to |
|
FWIW, I cherrypicked the fix into the Debian/Ubuntu version of the package. |
|
@diogobaeder That actually works for me. I guess I'll hang out at |
|
Me too, that's what I'm doing. Nice job, @warsaw :-) |
|
Just installed
uname: EDIT: Also: I expect that -- as I installed pip3 -- the proposed " EDIT2: Also funny to see that: |
|
@Mausy5043 That's exactly what you should see.
|
|
@Mausy5043 are you sure you tried this on an up-to-date Ubuntu 16.10? I just tried this in a Yakkety chroot and However, I noticed that the patch is not installed on the Ubuntu 16.04 LTS (Xenial) version, so you do get that crash. I've already cherry picked one fix into the Xenial SRU, but I missed this one. LP: #1611503 tracks the SRU for this problem. |
|
@warsaw : This is what I have: pip3 says: |
Now it makes sense! Thanks. A fix for Xenial is in the SRU pipeline. If you have xenial-updates enabled, you should get the fix soon. It has to be approved by the SRU team, but I expect there shouldn't be any hiccups with that. |
|
@warsaw: Thanks. Just to be sure: how do I make sure I have "updates enabled"? |
|
@Mausy5043 Take a look at I'm using the USA mirror, so YMMV. This page might also help. |
|
Also affects Trusty. Added note to LP: #1611503. |
|
Can we please reopen this bug and have it transformed into a warning ASAP? Using pip became a nightmare due to this and we all see it appearing more and more often. |
|
@ssbarnea its not very nice to copy&paste the same comment across a range of issues |
|
The issue was fixed more than 5 months ago, however there was no release for it yet. Why not release version 8.1.3 already? |
|
still happening on debian-stretch |
lock
bot
added
the
auto-locked
Description:
While trying to run
pip freezewe got a very long stacktrace that is ending with the line below:While I suspect that one of the packages installed from public pypi.python.org may have some weird version, i do find unacceptable to block pip freeze from working completely.
The text was updated successfully, but these errors were encountered: