Pip: Issue with MEO ISP in Portugal - 'ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

Are you using a proxy?

Hey @andmart!

This seems to be a network issue. Could you provide us with more details of how you're connecting, such as whether you're connecting through a proxy and whether your SSL version is new enough?

I am having the same issue. To reply to your question, @pradyunsg , I am not connecting through a proxy (not that I know of) and I don't know how to check if my SSL version is recent enough.

Update: Running the pip install <package> command via Powershell does not give this error.

@crlsmnzs So it sounds like you were using ubuntu 16.04, pip 10.0.0, and python 3.5 as well in your initial test. Is that accurate?

@sigmavirus24 no. I am not using a proxy. Using my home network.
Openssl version : 1.02g

This error would usually happen when there's something like a proxy or a firewall blocking connections.

I think you are right. It worked when connected in another network. But when @sigmavirus24 asked if I am using a proxy, I tried a VPN and it also did not wok.

@sigmavirus24 I'm using Windows 10, pip 10.0.0 and Python 3.6.5.

I'm getting the exact same issue. I'm using nextcloudpi on my RPi3 Model B. This issue started when I upgraded my internet plan and the ISP technitians switched my old router for a new one. That morning I had installed a package, but in the afternoon, after the router switch, I started getting this problem. I don't have a clue how to disable the router firewall or give it a whitelist. Guess I have some research to do.

Same issue here, not connected through a proxy, same issues are present if connected via VPN to another network. Besides the issue is intermittent.

On ubuntu 18.04
'OpenSSL 1.1.0g 2 Nov 2017'

Also having the same issues under others servers provided by fastly

vagrantup.com and python.org all of which have changes to the TLS forcing to TLS v1.2

The proxy is referred everywhere as possible issues, I'm not behind any corporate firewall. I'm using a simple business connection by a Portuguese ISP

Haven't tested under windows to check for issues on network yet, will update.

Curl also has issues with those even forcing it to tls , chrome also. So not a pip isolated problem, but rather a problem with network, just not proxy related.

eheheh. Is it MEO? Because I just upgraded my internet and got a new MEO router (they promised the FiberGateWay, but I got a crappy one. Still trying to solve that one out) and I started having this error on my RPi.

Yep. MEO here.

Yes, can confirm it's MEO, we've tried a mobile hotspot from NOS and we have been able to reach vagrantup.com rubygems.org python.org pypa.io , all fastly served at tls 1.2

Failure on MEO ISP network in Coimbra Area with the following OS's:

Network 1 (Business MEO ISP connection) :
Ubuntu 18.04 - Machine1
Ubuntu 16.04 - Machine2
Windows 10 Professional - Machine1

Network 2 (Home MEO ISP connection) :
Ubuntu 18.04 - Machine1

Success on NOS mobile hotspot:
Ubuntu 18.04 - Machine1
Ubuntu 16.04 - Machine2
Windows 10 Professional - Machine1

Haven't been able to test other ISP's.
However and this is the strange part:

I've been able to successfully use pip without failure using Docker NAT on MEO ISP:
docker run -it --rm python:alpine sh / # pip install requests Collecting requests Downloading https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl (91kB) 100% |████████████████████████████████| 92kB 776kB/s Collecting certifi>=2017.4.17 (from requests) Downloading https://files.pythonhosted.org/packages/df/f7/04fee6ac349e915b82171f8e23cee63644d83663b34c539f7a09aed18f9e/certifi-2018.8.24-py2.py3-none-any.whl (147kB) 100% |████████████████████████████████| 153kB 2.5MB/s Collecting idna<2.8,>=2.5 (from requests) Downloading https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl (58kB) 100% |████████████████████████████████| 61kB 5.4MB/s Collecting urllib3<1.24,>=1.21.1 (from requests) Downloading https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl (133kB) 100% |████████████████████████████████| 143kB 7.0MB/s Collecting chardet<3.1.0,>=3.0.2 (from requests) Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB) 100% |████████████████████████████████| 143kB 6.0MB/s Installing collected packages: certifi, idna, urllib3, chardet, requests Successfully installed certifi-2018.8.24 chardet-3.0.4 idna-2.7 requests-2.19.1 urllib3-1.23 / #
However unable to install while connected via wifi or ethernet
All dns cache has been flushed, nslookup returns the same IP address on both NOS and MEO network.
ilhicas@ilhicas-HP-Notebook:~$ pip install virtualenv Collecting virtualenv Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /simple/virtualenv/ Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /simple/virtualenv/ Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /simple/virtualenv/ Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /simple/virtualenv/

Yet another strange report.

When connected to MEO-WIFI ( using the same router ) i'm able to reach all above mentioned domains correctly as well as pip install.

I've also edited nameservers to google without any luck, flushing cache dns etc..on the private network.

I just moved to Coimbra. I can confirm I get the same error, only with MEO.

I ask that you please update this thread if you find a solution. Is there any point in contacting MEO? I wouldn't know what to say...

I've had this error only in Coimbra, been to Évora this past weekend without issues reaching the sites mentioned before under a Windows OS, I hadn't tried via Ubuntu (given other issues)

Regarding the contacting MEO, I haven't done it also due to the same reasons, If you have a technical issue with their hardware it's already a pain to get it fixed, so I can only imagine the terrors you would go through explaining this.. :D

The only NOT a real solution for now is to switch to their MEO-WIFI infra when you need it. And besides that, you can always use docker, as from what it seems under a docker container I have no issues.. That alone tilts me.. Can't figure out why yet. So you could create a volume from the container to the host to the place you wish to do the pip install, maybe some virtual environment creation with all the packages you wish to install via pip. When I get the time I'll post here that solution

@Ilhicas thanks for sharing your findings man.

I'm sticking to Vodafone for the time being, but I'll make sure to take a look at your Docker workaround and see if I can figure out the root of the issue.

I can confirm I am having the same issue. I am also with MEO as the service provider in Viseu, Portugal.
I'm using my mobile hotspot as a workaround in the meanwhile...

Confirming the same issue with MEO service provider in Porto, Portugal.
I also have been trouble to connect to https://www.python.org/ , https://docs.python.org/ and https://docs.python.org/ websites, with a "This site can’t be reached" error.
I've tried to change DNS provider, but that did not solve the issue...
I've have been having those issues for a long time, however, mysteriously there are times that pip and docs works....

Hello!!
I've found a solution! I've change not only the IPv4 DNS but also the IPv6 to another provider then the ISP one, and now it works!
Cheers!

@citylife4 and everyone else here.
I'm from MEO too and I'm having the same issue here.
I've changed the DNS on my MEO router to cloudflare's, google's, etc, but no luck on making pypi, pythonhosted or anything else work.
wget and curl give tls errors, and the browsers get stuck on error pages (tls errors too).
I'm assuming this is the same issue since I'm having the same errors.
Connecting to other networks, or strangely enough, using a VM ( Don't ask me how) seems to fix it.
What can I do for now besides using another network, since it's preferable to use my own home connection and it's a PIA to offload the .tar.gz python packages from my server to install them with python.

If you have disabled IPv6 (I didn't try to change the IPv6 DNS, but in theory, it should also work) and changed IPv4 DNS and it still doesn't work, I don't know what you can do...
For me has been working fine ever since!

UPDATE : Changing the IPv6 DNS provider does not resolve this, IPv6 needs to be disabled.

Tested on Windows 10 + Debian on Rasp Pi

Even removing MEO's DNS from their router via telnet and replacing them with my own (IPv4 and IPv6) doesn't work. Disabling IPv6 in the kernel would be even more unpratical for me than connecting to another network. Is this a problem on MEO's infrastructure(network) or router? I'm really puzzled since this doesn't work on my Linux distro or Windows 10, but works on Linux or BSD VMs...

I have faced the same problems after @citylife4 pointed out the DNS solution. What really worked for me was use nordvpn, but I will reinforce with MEO to fix this.

For now, my solution is temporarily disabling IPv6 on Linux with sudo sh -c 'echo 1 > /proc/sys/net/ipv6/conf/all/disable_ipv6' , that way it isn't anything permanent. But I still want this fixed on MEO's side, so I'll try to figure this out, since every time the router reboots, it keeps my DNS server settings (either google's or cloudflare's), but it creeps their IPv4 and IPv6 DNS back in on set 100.
Pip strangely works with IPv6 disabled, which is nothing short of bizarre, but I'll look onto this further.

This issue seems to be about specific ISPs (I'm not sure, as I don't recognise the names, they are presumably local to one particular region?). If so, an someone suggest a more useful title for the issue to help anyone with a similar problem locate it?

Also, I don't see anything actionable for pip here, so I suggest that the issue can be closed (which won't prevent further comments, if anyone has extra experience to add).

@pfmoore This seems to be a Portugal-only issue, with a specific ISP, so could be a good place to trade ideas and fixes for this? I've been having this issue for over an year, it's quite frustrating.

Can you visit http://www.fastly-debug.com/ (requires a browser and javascript, make sure adblockers are turned off too they sometimes mess with it) and paste the base64 blob it gives you? That'll help Fastly (our CDN provider) debug the issue.

Yes the issue is not directly related to pip but with isp and fastly cdn.
As i mentioned previously. Other fastly served servers present the same
issue. Vagrantup. Rubygems etc..

A sexta, 21 de set de 2018, 13:36, Donald Stufft notifications@github.com
escreveu:

Can you visit http://www.fastly-debug.com/ (requires a browser and
javascript, make sure adblockers are turned off too they sometimes miss
with it) and paste the base64 blob it gives you? That'll help Fastly (our
CDN provider) debug the issue.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pypa/pip/issues/5374#issuecomment-423516233, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AON64Uq9cAX0lYICxgvRFo_6xE5X6FDLks5udN0ygaJpZM4T0AdT
.

@dstufft Do I put it here or do I open a separated issue?

You can just add it to a comment here, ideally inside of a code block so that it's formatted nicely.

@osl21 Thanks - title updated.

ewogICJnZW9pcCI6IHsKICAgICJjaSI6ICJsaXNib24iLAogICAgInN0IjogIjExIiwKICAgICJjdCI6ICJwb3J0dWdhbCIsCiAgICAiY28iOiAiRVUiLAogICAgImNfYXNuIjogIjMyNDMiLAogICAgImNfYXNuX25hbWUiOiAibWVvIC0gc2Vydmljb3MgZGUgY29tdW5pY2Fjb2VzIGUgbXVsdGltZWRpYSBzLmEuIiwKICAgICJyX2lwIjogIjg0LjIwMC42OS44MCIsCiAgICAicl9hc24iOiAiMzE0MDAiLAogICAgInJfYXNuX25hbWUiOiAiYWNjZWxlcmF0ZWQgaXQgc2VydmljZXMgZ21iaCIsCiAgICAicl9jaSI6ICJiYW1iZXJnIiwKICAgICJyX3N0IjogIkJZIiwKICAgICJyX2N0IjogImdlcm1hbnkiLAogICAgInJfY28iOiAiRVUiCiAgfSwKICAicG9wTGF0ZW5jeSI6IHsKICAgICJsaHIiOiA0NCwKICAgICJsY3kiOiAzOSwKICAgICJjZGciOiA0NiwKICAgICJtYWQiOiA0OSwKICAgICJmcmEiOiA1NiwKICAgICJoaG4iOiA1NiwKICAgICJhbXMiOiA1MCwKICAgICJibWEiOiA3OSwKICAgICJoZWwiOiA4OCwKICAgICJqZmsiOiAxMjEsCiAgICAiaGtnIjogMzE2CiAgfSwKICAicG9wQXNzaWdubWVudHMiOiB7CiAgICAiYWMiOiAibGN5IiwKICAgICJhcyI6ICJoaG4iCiAgfSwKICAicmVxdWVzdCI6IHsKICAgICJyZXNvbHZlcl9pcCI6ICI4NC4yMDAuNjkuODAiLAogICAgInJlc29sdmVyX2FzX25hbWUiOiAiQUNDRUxFUkFURUQtSVQsIERFIiwKICAgICJyZXNvbHZlcl9hc19udW1iZXIiOiAiMzE0MDAiLAogICAgInJlc29sdmVyX2NvdW50cnlfY29kZSI6ICJERSIsCiAgICAiY2xpZW50X2lwIjogIlJFREFDVEVEIiwKICAgICJjbGllbnRfYXNfbmFtZSI6ICJNRU8tUkVTSURFTkNJQUwsIFBUIiwKICAgICJjbGllbnRfYXNfbnVtYmVyIjogIjMyNDMiLAogICAgInRpbWUiOiAiMjAxOC0wOS0yMVQxODowNzozMy4wMDBaIiwKICAgICJob3N0IjogInd3dy5mYXN0bHktZGVidWcuY29tIiwKICAgICJhY2NlcHQiOiAidGV4dC9odG1sLGFwcGxpY2F0aW9uL3hodG1sK3htbCxhcHBsaWNhdGlvbi94bWw7cT0wLjksKi8qO3E9MC44IiwKICAgICJ1c2VyYWdlbnQiOiAiTW96aWxsYS81LjAgKFgxMTsgTGludXggeDg2XzY0OyBydjo2MC4wKSBHZWNrby8yMDEwMDEwMSBGaXJlZm94LzYwLjAiLAogICAgImFjY2VwdGxhbmd1YWdlIjogImVuLVVTLGVuO3E9MC41IiwKICAgICJhY2NlcHRlbmNvZGluZyI6ICJnemlwIiwKICAgICJmYXN0bHlzZXJ2ZXJpcCI6ICIxNTEuMTAxLjEyLjY0IiwKICAgICJ4ZmYiOiAiIiwKICAgICJkYXRhY2VudGVyIjogIkZSQSIsCiAgICAiYmFuZHdpZHRoX21icHMiOiAiMTEuMzYiLAogICAgImN3bmQiOiA4NSwKICAgICJuZXh0aG9wIjogIjE0OS4xMS4yMC4xMTciLAogICAgInJ0dCI6IDUwLjEzLAogICAgImRlbHRhX3JldHJhbnMiOiAwLAogICAgInRvdGFsX3JldHJhbnMiOiAwCiAgfQp9
I'm currently using dns.watch DNS servers, but it fails with Cloudflare's (1.1.1.1) or Google's (8.8.8.8) .

@osl21 Just to be clear, that blob was made when using dns.watch servers, which do not exhibit the problem?

No, all of the DNS exhibit the problem, only disabling IPv6 fixes it. No matter what DNS I use, I can't go to pypi or use pip, I'm just using DNS.WATCH because of preference.

Ok thanks.

@osl21 Disabling IPv6 on my machine with sudo sh -c 'echo 1 > /proc/sys/net/ipv6/conf/all/disable_ipv6' worked for me too.

It works, but this is a really nasty issue on MEO's or Fastly's end (since I can't acess some of their services, specifically python-related) , disabling IPv6 isn't much of a solution.

For those experiencing this v6 issue it would be very helpful if you could provide a packet capture of it failing.
Please send them to [email protected] and mention this issue and our ticket ECO-2664

@supine Can you give us steps (tools or such) in order to provide you the said packet capture (like in a pip install or acessing pypi via browser or curl/wget)?

I have the same problem with Portuguese NOS provider

Guys just change the IpV4 DNS to 8.8.8.8 or 8.8.4.4 and the IpV6 DNS to 2001:4860:4860::8888 or 2001:4860:4860::8844.
This is for sure due to the websites they are blocking in Portugal to prevent piracy...

@SirRibeiro As previously mentioned, it fails with any DNS, be it google, cloudflare's or any other. The issue is with cloudflare's/fastly side, not with the piracy blacklist.

@osl21 oh ok, sorry, for me just changing the DNS was enough.

Just bumped into this, and even changing the DNS's didn't help.

I'm on MEO fiber and just opened an issue on their support.

Using the mobile hotspot from my mobile phone (also MEO) sorted the issue for now..

What kind of router does the ISP supply? Is anyone able to run a test bypassing the supplied router?

@osl21 you can take a capture with a tool like tcpdump. Command would look something like:

tcpdump -n -nn -N -s 0 -i INTERFACE -w /PATH/TO/FILE.pcap "net 2a04:4e40::/29"

where INTERFACE is your "external" interface where your default route would go out.

ok. Meo solution one was to ask me to reboot the router.....As expected is not a solution

The details are

Technicolor Gateway

Product Name: TG789vac v2

Serial Number: xxxxxxxx

Software Release: 10.5.8.I

Software Variant: BQ

Boot Loader Version: 2.0.85

Product Code: 3743534A

Board Name: VANT-6

@lvale thank you for that information.

There is a known issue with Thomson / Technicolor Gateway routers where they reset the IPv6 flow label in the middle of a TCP session when it is supposed to be static from the SYN to the FIN.

Large networks and service providers, Fastly included, use ECMP for load balancing. Ideally the path selection uses hashing that keeps a single TCP session on the same path.

Although Fastly mostly avoids using IPv6 flow label as an input into hashing:

• some equipment does not let you exclude it
• some other networks choose to configure it.

By reseting the IPv6 flow label in the middle of the TCP session the router is likely forcing ECMP rehashing somewhere in the path between yourself and Fastly. This could mean that subsequent packets land on the wrong cache or even in the wrong POP.

You can prove it's the router by connecting your computer directly to the ONT and retesting.

I don't have any straightforward solution for you. Your options appear to be:

• seek out the latest firmware for your device, see if that fixes the problem
• replace the supplied router with one of your own

Different equipment here, yet its the same brand, just an higher version from the technical details, just tested and it seemed to work now, but as mentioned previously the error occurrence is non-deterministic.

Technicolor Gateway

Product Name: TG784n v3

Software Release: 10.2.1.O

Software Variant: DS

Boot Loader Version: 1.1.2

Product Code: 3690227E

Board Name: DANT-U

With a little more searching found a tentative solution here https://www.reddit.com/r/portugal/comments/8cjj3s/developers_com_meo_em_casa_conexões_bloqueadas/

which is basically to disable ipv6 on the router/gateway

telnet to router/gateway

telnet _ipaddress_
dhcp serverv6 config state=disabled
ip ifconfig intf=LocalNetwork ipv6=disabled
saveall
exit

Not sure it will last the "periodic" updates by the ISP....

One workaround is to disable IPv6.

1. Disable IPv6
2. Run pip
3. Enable IPv6

I report that sudo sh -c 'echo 1 > /proc/sys/net/ipv6/conf/all/disable_ipv6' worked for me as well! Thanks a lot!! I wasnt able to install anything with pip

sudo sh -c 'echo 1 > /proc/sys/net/ipv6/conf/all/disable_ipv6'
worked great for me to!!!

Have this problem also, not only with pip but with many HTTPS websites mostly behind Fastly CDN. Also with MEO as my ISP. Yes disable ipv6 did the trick.

Still would like to understand why this happens.

I know that before disabling ipv6, if I checked my Public IP it would give an ipv6 one. Dont really Know if its an ISP problem or if many websites cant still handle well ipv6 addresses.

@brpaz see https://github.com/pypa/pip/issues/5374#issuecomment-428160285

Thank you so much guys, I can confirm that only disabling IPv6 works. I have tried many things before.

It's likely somewhat obvious, but will just note that this issue also affects https://pypi.org via browsers, not just pip.

Disabling ipv6 solved this issue for me. Still annoying though.

Same problem for me. I am using MEO from Aveiro.
Disabling IPv6 works fine for any package install. As a brazilian, I consider Portugal a weird network...
Thanks for support.

Guys,

I had same problem, using MEO ISP/Router.

I solve using this command, to disable ipv6 network: sudo sh -c 'echo 1 > /proc/sys/net/ipv6/conf/all/disable_ipv6'

Then, i run pip and it works!!!

I have the same issue. Since this problem is clearly related to MEO, is there a more appropriate forum to discuss this issue ? I have opened a support ticket with MEO, but I'd like to know if anyone else has done the same, and what answer did they receive.

@aaubry I posted on the Meo forum, some time ago but no answer.

I didn't opened a support ticket tough. May be I will do. Please keep me posted if you get an answer.

Hello guys,

I'm from Coimbra, Portugal and had the same issue for so I tried the following and worked

I just changed my IPv4 DNS to 8.8.8.8 and 8.8.4.4 (Google)
And disabled IPv6 and it worked.

If you want to keep IPv6 on, you can only disable it when you're installing something with pip.

Reporting from Almada (MEO):

Disabling IPv6 allowed me to use 'pip install package'.
So if u want to install something just disable IPv6 while you're doing it.
For reference I'm using, in the IPv4 DNS : 8.8.8.8 and 8.8.4.4

Same problem using MEO in Évora (Ubunbu 18.04).
Also solved it with:
sudo sh -c 'echo 1 > /proc/sys/net/ipv6/conf/all/disable_ipv6'

I am also behind MEO. My workaround was to temporarily disable IPv6 as explained How to disable IPv6 address on Ubuntu 18.04 Bionic Beaver Linux . This link also explains how to permanently disable IPv6.

Has anyone tried this with 3rd party routers and modems? Are we sure the problem is meo's infrastructure? Or could it be the devices?

@Zizico2 I think it was a problem with the devices because since the last firmware update, I never had any problems with IPv6. Or maybe they updated the infrastructure at the same time.

Is anyone still having a problem with MEO? What version of the device are you using? (You can access http://192.168.1.254/ to find out.)

My device: Technicolor TG784n v3
Firmware version: M19 10.5.8.U [FV]

Hi folks!

If you're still having issues connection to PyPI w/ MEO as your ISP, please open an issue in pypa/pypi-support using the 'Network Access Issue' template.

That is a much better place for further discussion on this issue + better visibility for PyPI admins, so that they may take a look if there's anything actionable on the PSF-infra side.

AFAICT, this is a case of an IPv6 issue with MEO; since disabling IPv6 seems to be working for some users.

@pradyunsg the problem was identified in https://github.com/pypa/pip/issues/5374#issuecomment-428160285

I believe as a result of firmware upgrades and/or changes to routing topology this particular problem should be resolved.

Folks should be encouraged to retest IPv6 to see if it remains an issue for them.