Drop support for manylinux1 images January 1st, 2022

[mayeut]

CentOS 5 support - which is used as the base image for manylinux1 - has been EOL for almost 4 years.
There's now very little need for manylinux1 compatible wheels (less than 0.1% of glibc systems downloads on PyPI in October 2020, c.f. #542 (comment))

manylinux1 will not receive any new features from now on (e.g. no CPython 3.10).
Support will end January 1st, 2022.

What this means on January 1st, 2022:

• the manylinux1 branch will be removed (a tag will be created to keep history)
• existing images will be kept in quay.io

Amend on January 22nd, 2022:

• as long as automated updates are working, the branch will be kept around
• once automated updates start failing, what was first planned on January 1st, 2022 will happen.

[henryiii]

Just to be complete, since the “very little need” is misleading, the reason 99% users get manylinux1 is not glibc version, but pip version. The version of pip in the most popular distro (Ubuntu), the most popular version (18.04) is pip 9. Even if you install Python 3.8 (it has 3.6 by default), you still get Pip 9. It’s quite unsupported to overwrite the system package manager’s pip, too. Making a new venv defaults to pip 9 even if you do update pip manually (such as a user install). So even though manylinux2014 is supported with a new enough pip, using the default tooling, only manylinux1 is.

This issue should be fixed from recurring in the future by the perennial manylinuxes, once distros ship that.

Edit: forgot about CentOS 7!

[mayeut]

@henryiii,

the reason 99% users get manylinux1 is not glibc version

Can you provide the source of this stat ?

The stats in the linked comment suggested less than 25% last October
Yes those stats are probably biased by the fact that CI accounts for a number of them

I'm hopping that by the end of the year, things will have evolved in the right direction. Some major packages like cryptography already moved to manylinux2010 only and I guess this will get more users using an upgraded pip.

You'll still be able to use manylinux1 after end of support. Everything will be in a frozen state, that's all. No new features and no more updates.

[mayeut]

I updated the stats by looking at February downloads on PyPI:

SELECT t0.cpu, t0.num_downloads, t0.pip_version, t0.glibc_version FROM (SELECT
  COUNT(*) AS num_downloads,
  details.installer.version AS pip_version,
  details.distro.libc.version AS glibc_version,
  details.cpu
FROM `the-psf.pypi.downloads*`
WHERE
  details.installer.name = 'pip' AND
  details.system.name = 'Linux' AND
  details.distro.libc.lib = 'glibc' AND 
  _TABLE_SUFFIX
    BETWEEN FORMAT_DATE(
      '%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 28 DAY))
    AND FORMAT_DATE('%Y%m%d', CURRENT_DATE())
GROUP BY details.installer.version, details.distro.libc.version, details.cpu
ORDER BY num_downloads DESC) AS t0 LIMIT 16000; 

When looking at the data for all x86 cpus, this gives:

When looking only at 32-bit x86 (which might lessen the CI bias a bit):

[henryiii]

I didn’t mean 99% of users can only get manylinux1, I meant that 99% of the times that manylinux1 is selected when manylinux2010 is available, it’s due to pip, not glibc limitations (the stat is easily computable from your stats above). I know there are a lot of (non-CI) Ubuntu 18.04 users because build broke under Ubuntu 18.04’s setup and we had a stream of reports. Trying to upgrade based on Python version assuming the embedded version was a minimum (as seemly indicated in the manylinux readme) didn’t fix the problem because python3-pip and python3-venv are not connected to the main Python version. That’s also how I found out about the python3.8 package and the fact it’s still tied to pip 9.

My point is this is a huge number of users on an OS with three years left to go, many of whom have no idea about virtual environments or other such things, and they will all suddenly be compiling when pip installing if a project drops manylinux1. I expect that’s why manylinux1 is still used by 75% of the cibuildwheel projects even though the default is manylinux2010.

manylinux1 does have to die, and the best way is for everyone to drop it at the same time. But wanted to clarify it’s not only the “less than 0.1% of glibc systems” stat that matters.

[henryiii]

In your data, you can see CentOS 7 and Ubuntu 18.04 clearly listed with pip 9; there’s also a mystery 2.26 distro with pip 9.

[mayeut]

Thanks for the explanation of your stat. Indeed 99% of systems not manylinux2010 compliant systems are not compliant only because of pip.

many of whom have no idea about virtual environments or other such things, and they will all suddenly be compiling when pip installing if a project drops manylinux1

Again, projects do not have to drop manylinux1 and would still be able to use the last. The image just won't evolve as there won't be use for it to evolve unless some CPython 3.10 appears with pip <= 19.x

Yes that means on packager side, there would be a need to split the build depending on CPython version being built.

I guess python3.8 was added after in ubuntu 18.04 and it shares pip with the default python 3.6 so 9.x

[mayeut]

In your data, you can see CentOS 7 and Ubuntu 18.04 clearly listed with pip 9

yes

there’s also a mystery 2.26 distro with pip 9.

glibc 2.26 with pip 9 is probably amazonlinux:2 (opensuse/leap:15.2 uses 10.0.1)

[henryiii]

Seeing Python version there could also be interesting.

[mayeut]

Seeing Python version there could also be interesting.

I dropped the patch version on everything not to get too many rows.

https://docs.google.com/spreadsheets/d/1ZV-xf-2qXkPUdPp8D8i4qRq6Jo1GSNlJQBGuLyVkiGQ/edit?usp=sharing

SELECT t0.cpu, t0.num_downloads, t0.python_version, t0.pip_version, t0.glibc_version FROM (SELECT
  COUNT(*) AS num_downloads,
  REGEXP_EXTRACT(details.python, r"^([^\.]+\.[^\.]+)") as python_version,
  REGEXP_EXTRACT(details.installer.version, r"^([^\.]+\.[^\.]+)") AS pip_version,
  REGEXP_EXTRACT(details.distro.libc.version, r"^([^\.]+\.[^\.]+)") AS glibc_version,
  details.cpu
FROM `the-psf.pypi.downloads*`
WHERE
  details.installer.name = 'pip' AND
  details.system.name = 'Linux' AND
  details.distro.libc.lib = 'glibc'
 AND 
  _TABLE_SUFFIX
    BETWEEN FORMAT_DATE(
      '%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 28 DAY))
    AND FORMAT_DATE('%Y%m%d', CURRENT_DATE())
GROUP BY pip_version, python_version, glibc_version, details.cpu
ORDER BY num_downloads DESC) AS t0 LIMIT 16000; 

[henryiii]

Didn't polish off the spacing and clean the really small segments, but here's a basic pip & manylinux version pie chart for each Python version. Stuck 2.6 in to make the grid nicer. :)

from hist import Hist
import pandas as pd
import matplotlib.pyplot as plt

data = pd.read_csv(
    "results-20210227-133657 - results-20210227-133657.csv",
    usecols=("cpu", "num_downloads", "python_version", "pip_version", "glibc_version", "policy"),
    converters={
        "python_version": str,
        "pip_version": lambda x: int(x.split(".")[0]),
        "glibc_version": lambda x: int(float(x.split("-")[0]) % 1 * 100),
    },
)

h = Hist.from_columns(
    data,
    ("cpu", "python_version", "pip_version", "policy"),
    weight="num_downloads",
)

fig, axs = plt.subplots(2, 3, figsize=(12, 8))
for i, py in enumerate(["2.6", "2.7", "3.6", "3.7", "3.8", "3.9"]):
    ax = axs.flatten()[i]
    ph = h.project("python_version", "pip_version")[py, :]
    ph.plot_pie(ax=ax, normalize=True, autopct='%1.0f%%', pctdistance=.8)
    ax.set_title(f"Python {py} {int(ph.sum()) // 1000000:,} M")

plt.tight_layout()
plt.show()

fig, axs = plt.subplots(2, 3, figsize=(12, 8))
for i, py in enumerate(["2.6", "2.7", "3.6", "3.7", "3.8", "3.9"]):
    ax = axs.flatten()[i]
    ph = h.project("python_version", "policy")[py, :]
    ph.plot_pie(ax=ax, normalize=True, autopct='%1.0f%%', pctdistance=.8)
    ax.set_title(f"Python {py} {int(ph.sum()) // 1000000:,} M")

plt.show()

Quick observation; Pip 9 is frighteningly large for 3.7 (6%) and even for Python 3.8 (3%). But even then, nothing like 3.6 (17%), so dropping Python 3.6 might be good point for most projects to drop manylinux1.

[mayeut]

Thanks @henryiii for those.
The end of support date planned is after CPython 3.6 EOL.
The plan might be revisited depending on the evolution of statistics (Edit: added those to https://mayeut.github.io/manylinux-timeline/)

c.f. also https://discuss.python.org/t/blog-post-about-manylinux-and-the-future-of-manylinux1/5734

My point is this is a huge number of users on an OS with three years left to go, many of whom have no idea about virtual environments or other such things, and they will all suddenly be compiling when pip installing if a project drops manylinux1.

I agree with you however, I think the can be guided into learning those and that would be beneficial for everybody.
Projects dropping manylinux1 should help their user understand the issue and that's what at least cryptography and PyArrow are doing for exemple by updating the docs, adding a message when building from source, communicating by other means...

Let's see how it evolve in the coming months.