#150 Upgrade from Python2 to Python3
Closed: complete 5 years ago by bt0dotninja. Opened 5 years ago by shraddhaag.

Summary

Upgrade from Python2 to Python3

Background

Our project currently uses Python 2.7 and Django 1.11
Both of them will reach the end of support in 2020. It needs to be upgraded to Python3.7 and Django 2.1

Details

First steps for this ticket may be to search the differences between the two versions and look for way's how they can implemented in our current codebase without loosing any functionality.
Also, the dependencies for our project will also be needed to be upgraded to support Python3 and Djnago2

Outcome

Project works effeciently on Python3.7 and Django2.1


Metadata Update from @jflory7:
- Issue priority set to: waiting on assignee (was: awaiting triage)
- Issue tagged with: PASSED, difficulty - medium, help wanted, new change, type - backend, type - summer coding

5 years ago

@shraddhaag Thanks for opening this. I think this ticket could be a good one for someone to begin work on.

How to start on this ticket

Before assigning the ticket, I'd like to see some research / analysis into how much work is needed for this ticket. To begin, I suggest the following:

  1. Change Docker image to python:3-alpine in Dockerfile (see here)
  2. Rebuild development environment: docker-compose up --build
  3. See what happens and what breaks, try to figure out what parts of the code need updating

Report back here with your findings. Give us an estimate of how much work you think this needs and an approximate estimate of how much time you think is needed to resolve this.

Hi @jflory7 I'd like to work on this! :smile:
I edited the Dockerfile to build the container with python:3-alpine image and the errors I encountered are:

  1. While installing wsgiref. The traceback can be found here. On research I found that wsgiref has been made a standard library for Python 3. (Please see this) so simply removing this dependency will suffice.

  2. While installing opbeat. The traceback can be found here On changing it to the latest version opbeat==3.6 I encounter the same error.

[I'll keep this comment updated as I work on this further]

I think in a couple of days I should be able to figure out exactly what all needs to be changed for the project to be upgraded to Python3. By the end of this week I think the project will have both Python and Django updated to their latest version.

While installing opbeat. The traceback can be found here On changing it to the latest version opbeat==3.6 I encounter the same error.

For what it's worth, upstream removed opbeat in mxsasha/happinesspackets@b8e057de222926e73e27ffa467c40b7c32a5fc0a. I have no context for why upstream made this decision though. :disappointed_relieved: It might be worthwhile for us to do the same. (I haven't dug into what opbeat is doing in the project yet.)

I think in a couple of days I should be able to figure out exactly what all needs to be changed for the project to be upgraded to Python3. By the end of this week I think the project will have both Python and Django updated to their latest version.

Sounds good. I can assign this one to you. :thumbsup: By your analysis so far, it doesn't seem overly complex. @cverna gave an interesting example of an existing Fedora Infra application using Django here. It might help to use that as an example for changes to our container image.

Metadata Update from @jflory7:
- Issue untagged with: help wanted
- Issue assigned to shraddhaag

5 years ago

@jflory7 @shraddhaag in my #80 issue i figured the wsgiref was using python2 and i was running the project in python3 environment. I got an error similar to that. After research i figured out that in python3 this module is already there as a library by default and the solution is to simply remove it

@shraddhaag Are working on this? i was also interested in it. if yes we can collaborate

I think we should have a separate ticket for migrating to Django2.1
@shraddhaag @jflory7 @jonatoni

@sanjana11147 Maybe you're right. My first thought was, since the dependencies for the project are being changed to Python 3 it will only make sense to upgrade Django to 2.1 along with it since the current version Django 1.11.5 doesn't support Python 3.7 as stated here

@shraddhaag Are working on this? i was also interested in it. if yes we can collaborate

Hi @feisal Yes I'm working on it :)
Can you please look for some other issues to work on?

@sanjana11147 Maybe you're right. My first thought was, since the dependencies for the project are being changed to Python 3 it will only make sense to upgrade Django to 2.1 along with it since the current version Django 1.11.5 doesn't support Python 3.7 as stated here

I guess these two changes have to go hand-in-hand. It makes it harder to split this ticket up into smaller tasks. For now, let's try to combine these two changes together in one PR.

If upgrading Django also yields several new issues to address, we could split it up then. I haven't tested a new version of Django on fedora-happiness-packets yet to know how much work is required.

@jflory7 A quick update:
I did these further changes after removing opbeat as suggested.

  1. Made all the changes as given here to upgrade to Django 2.1
  2. The way media files are configured in urlpatterns is changed to static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
  3. Encountered syntax error: async is a reserved keyword thrown by celery. On further research I found that it's an issue with celery==4.2.1 and will have to downgrade from Python3.7 to Python3.6 until they release a new version. Thus changed the web image to Python:3.6.8-alpine

After the above changes, the docker-compose up finally executed completely but the web container, although it didn't log any error, didn't start. I'm trying to find the reason for the same.

PS: It's taking a little longer than estimated to get right :sweat_smile:

Encountered syntax error: async is a reserved keyword thrown by celery. On further research I found that it's an issue with celery==4.2.1 and will have to downgrade from Python3.7 to Python3.6 until they release a new version. Thus changed the web image to Python:3.6.8-alpine

A side note, it would be a good idea to document that with a comment in the Dockerfile.

After the above changes, the docker-compose up finally executed completely but the web container, although it didn't log any error, didn't start. I'm trying to find the reason for the same.

PS: It's taking a little longer than estimated to get right 😅

Hmm, that's strange. Let us know if you need help debugging or if you get stuck.

I figured this would be a complex ticket since it is a fairly large change. This ticket needs to happen before the end of the year, but it's not an essential one to knock out before the end of March. There's not a huge rush for this one yet. :smiley:

@jflory7 I was able to figure out the issue. The issue is with Middleware and how it's handled in Django 1.11 and Django 2. I have been able to debug all the additional Middlewares (surprisingly, among others, this is where opbeat is used, and I was able to configure it to work seemlessly)

The website is functioning as expected. The only roadblock is the custom Middleware at utils/middleware.py. I'm figuring out a way to upgrade it to Django 2 and then this will be good to go :)

@jflory7 I was able to figure out the issue. The issue is with Middleware and how it's handled in Django 1.11 and Django 2. I have been able to debug all the additional Middlewares (surprisingly, among others, this is where opbeat is used, and I was able to configure it to work seemlessly)

Awesome! :smile: Nice work.

The website is functioning as expected. The only roadblock is the custom Middleware at utils/middleware.py. I'm figuring out a way to upgrade it to Django 2 and then this will be good to go :)

Great! Do you think it is a change you will make a pull request with before the end of this week or sometime next week?

@jflory7 I've opened a PR #190 for the same. Could you please find time to review? :)

let me take the task of review the PR #190 :smile:

Many thanks for the migrations I merge the PR :smile:

Metadata Update from @bt0dotninja:
- Issue close_status updated to: complete
- Issue status updated to: Closed (was: Open)

5 years ago

Metadata Update from @jflory7:
- Issue set to the milestone: Summer Coding 2019: community bonding

5 years ago

Login to comment on this ticket.

Metadata