Openlibrary: Run on Both Python 2 and Python 3
Blocked by: internetarchive/infogami#56, internetarchive/infogami#50
As discussed in #846 there is a lot of code on the Internet Archive Code base that is currently written in Python 2 and deprecated for Python 3.
The following link tells about the key differences from Python 2 to Python 3.
EDIT: Updated as per @cclauss's suggestions
- [x] Get the codebase to be Python 3 syntax compatible (see __make lint__ when run on Py3 for our status)
- [x] #1466 Fix how exceptions are raised in Python 3
- [x] #1509 Add __six__ to our requirements (I find it less subtle than __future__ mentioned below).
- [x] #1517 Function parameters may not be explicit tuples in Python 3
- [x] #1603 Redo the __lambda__ fixes that were reverted out of #1466
- [x] #1468 Malformed \N character escape in catalog/marc/load.py
- [x] #1501 __print()__ is a function in Python 3 in plugins/upstream/acs4.py
- [x] Undefined names __python3 -m flake8 . --count --exclude=scripts/20,vendor/ --select=F821__
- [x] A PR for __basestring__ #1563
- [x] A PR for __cmp()__ #1643
- [x] A PR for __execfile()__ #1525
- [x] A PR for __file()__ #1564
- [x] A PR for __raw_input()__ #1526
- [x] A PR for __unicode__ (This is a tricky one and best saved until last.)
- [x] A PR for __xrange()__ #1524
- [x] Update our dependencies to ensure that we have Python 3 support (See #1454 for some hints.)
- [x] Fix the vendored in stuff like __infogami__
- [x] __More options__, __Trigger build__ at https://travis-ci.org/internetarchive/acs4_py
- [x] internetarchive/acs4_py#7
- [x] internetarchive/infogami#40
- [x] internetarchive/infogami#42
- [x] internetarchive/infogami#43
- [x] internetarchive/infogami#56 Cherrypick safe imports from #50
- [x] internetarchive/infogami#50 More Python 3 updates
- [x] internetarchive/infogami passes Travis CI tests on Python 3
- [x] Pass the Travis tests and turn off __allow_failures__ mode on Python 3
- [ ] Test, Test, Test
salman-bhai
All 20 comments
At the risk of stating the obvious, priority should likely go to identifying those Python 2 features that both a) will break in Python 3 and b) are used in OL.
LeadSongDog
on 19 Mar 2018
Thanks for creating a separate issue to track this. The task is definitely low priority, but neither easy, nor a good beginners project, so I've removed both of those labels.
Things like this are time sensitive, so 3-4 year old random blog post is not a good source of information. For planning purposes, things that are important are things like when required libraries will all support Python 3, when they'll drop Python 2 support, what the latest migration tooling looks like, etc.
BUT, since it's low priority, we've got more important things to think about for the immediate term.
tfmorris
on 19 Mar 2018
Full agreement w/ @tfmorris on all fronts.
Thanks for organizing an issue for this, as eventually we'll need to figure out the path to maintain infogami.
mekarpeles
on 19 Mar 2018
Hmm, there are "priorities" and then there are "priorities". What's important may not be urgent, and vice versa. I'd say this one is important but not urgent. Ignored long enough it will surely cause breakage.
LeadSongDog
on 19 Mar 2018
@LeadSongDog Who are you disagreeing with? What would you like to see changed vis à vis priorities?
tfmorris
on 20 Mar 2018
@tfmorris Not disagreeing with you exactly, but I'm saying that one-dimensional priorities lead to crisis management. Once something difficult is labeled as low priority it goes unattended or gets closed until it causes breakage and becomes urgent. Better to not priority it at all?
LeadSongDog
on 20 Mar 2018
There is a Travis CI job to track our progress on this now, thanks to PR #1273
tfmorris
on 8 Oct 2018
Hi, I'm new to OpenLibrary, but this looks like something I'd be interested in doing. Do you mind if I proceed with this, albeit slowly, even though this isn't a 'good first issue'?
hundredrab
on 11 Oct 2018
NOTE: This list was moved to the top of this issue:
Also, I would suggest that we change the title of this issue to "__Run on both Python 2 and Python 3__" because it is clearly best practice to have a codebase that operates on both _before_ removing Python 2 support. 423 days until Python 2 end of life.
cclauss
on 4 Nov 2018
@cclauss with your permission, can I update your message above now that the title has been changed and the checklist has been updated? (to avoid 2 checklists getting out of sync)
Other than Infogami, any additional followup steps you can recommend would be extremely appreciated :bowing_man:
Thank you for being such a driving force in making this happen!
mekarpeles
on 12 Apr 2019
Feel free to make the edits that make sense to you. Let’s focus on getting Infogami to a point where it no longer blocks the Travis test. My focus has been elsewhere but hopefully I can reengage in the weeks to come.
cclauss
on 12 Apr 2019
Updated our task list above to celebrate 100 days until Python 2 end of life.
cclauss
on 23 Sep 2019
From the chatter here and the slack, I'm labeling this as a work in progress. However, there is no assignee. I'm assuming @cclauss and @salman-bhai would be the best candidates as they are the team leads for Python3.
xayhewalo
on 20 Oct 2019
@cclauss Is that emoticon a yes to being assignee? It looks like you and @hornc are responsible for nearly all the PRs in the breakdown of tasks.
xayhewalo
on 21 Oct 2019
Do we want to rehash the specific remaining steps / blockers for Python3?
mekarpeles
on 22 Nov 2019
I'm not sure why the priority got reduced. This doesn't need to be done by Jan. 1, but it does need to be done soon.
The main blocker right now is infogami, although getting it ported might reveal issues remaining other parts of the system. Rather than trying to track each infogami issue separately here, it might make more sense to point to an infogami top level Python 3 epic issue.
tfmorris
on 11 Dec 2019
Looks like we got a few more months reprieve. At the end of December, the Python Software Foundation pushed the retirement date out (again!) to April 2020.
tfmorris
on 16 Jan 2020
Yes but this project is the last major project that I know of that has not yet already gotten over the line.
cclauss
on 17 Jan 2020
The final Python 2 release has been made: https://stackoverflow.blog/2020/04/23/the-final-python-2-release-marks-the-end-of-an-era/ and it will receive no more updates of any kind, including critical security bugs.
Every day from now on (since a few months ago really) makes for increased exposure to unpatched vulnerabilities compromising OpenLibrary production systems.
tfmorris
on 25 Apr 2020
Time to close this one! Thanks to everyone for helping to get us to Python 3!!
cclauss
on 23 Nov 2020
Related issues
BrittanyBunk
·
4Comments
dcapillae
·
4Comments
nonom
·
3Comments
cdrini
·
4Comments
cclauss
·
3Comments
Most helpful comment
Thanks for creating a separate issue to track this. The task is definitely low priority, but neither easy, nor a good beginners project, so I've removed both of those labels.
Things like this are time sensitive, so 3-4 year old random blog post is not a good source of information. For planning purposes, things that are important are things like when required libraries will all support Python 3, when they'll drop Python 2 support, what the latest migration tooling looks like, etc.
BUT, since it's low priority, we've got more important things to think about for the immediate term.