Skip to content

3.9.rst

Latest commit

 

History

History
826 lines (613 loc) · 33.1 KB

3.9.rst

File metadata and controls

826 lines (613 loc) · 33.1 KB

What's New In Python 3.9

Release:|release|
Date: |today|

This article explains the new features in Python 3.9, compared to 3.8.

For full details, see the :ref:`changelog <changelog>`.

Note

Prerelease users should be aware that this document is currently in draft form. It will be updated substantially as Python 3.9 moves towards release, so it's worth checking back even after reading earlier versions.

Summary -- Release highlights

You should check for DeprecationWarning in your code

When Python 2.7 was still supported, many functions were kept for backward compatibility with Python 2.7. With the end of Python 2.7 support, these backward compatibility layers have been removed, or will be removed soon. Most of them emitted a :exc:`DeprecationWarning` warning for several years. For example, using collections.Mapping instead of collections.abc.Mapping emits a :exc:`DeprecationWarning` since Python 3.3, released in 2012.

Test your application with the :option:`-W` default command-line option to see :exc:`DeprecationWarning` and :exc:`PendingDeprecationWarning`, or even with :option:`-W` error to treat them as errors. :ref:`Warnings Filter <warning-filter>` can be used to ignore warnings from third-party code.

It has been decided to keep a few backward compatibility layers for one last release, to give more time to Python projects maintainers to organize the removal of the Python 2 support and add support for Python 3.9.

Aliases to :ref:`Abstract Base Classes <collections-abstract-base-classes>` in the :mod:`collections` module, like collections.Mapping alias to :class:`collections.abc.Mapping`, are kept for one last release for backward compatibility. They will be removed from Python 3.10.

More generally, try to run your tests in the :ref:`Python Development Mode <devmode>` which helps to prepare your code to make it compatible with the next Python version.

New Features

Dictionary Merge & Update Operators

Merge (|) and update (|=) operators have been added to the built-in :class:`dict` class. See PEP 584 for a full description. (Contributed by Brandt Bucher in :issue:`36144`.)

Other Language Changes

  • :func:`__import__` now raises :exc:`ImportError` instead of :exc:`ValueError`, which used to occur when a relative import went past its top-level package. (Contributed by Ngalim Siregar in :issue:`37444`.)

  • Python now gets the absolute path of the script filename specified on the command line (ex: python3 script.py): the __file__ attribute of the :mod:`__main__` module became an absolute path, rather than a relative path. These paths now remain valid after the current directory is changed by :func:`os.chdir`. As a side effect, the traceback also displays the absolute path for :mod:`__main__` module frames in this case. (Contributed by Victor Stinner in :issue:`20443`.)

  • In the :ref:`Python Development Mode <devmode>` and in debug build, the encoding and errors arguments are now checked for string encoding and decoding operations. Examples: :func:`open`, :meth:`str.encode` and :meth:`bytes.decode`.

    By default, for best performance, the errors argument is only checked at the first encoding/decoding error and the encoding argument is sometimes ignored for empty strings. (Contributed by Victor Stinner in :issue:`37388`.)

  • "".replace("", s, n) now returns s instead of an empty string for all non-zero n. It is now consistent with "".replace("", s). There are similar changes for :class:`bytes` and :class:`bytearray` objects. (Contributed by Serhiy Storchaka in :issue:`28029`.)

  • Any valid expression can now be used as a :term:`decorator`. Previously, the grammar was much more restrictive. See PEP 614 for details. (Contributed by Brandt Bucher in :issue:`39702`.)

New Modules

  • None yet.

Improved Modules

ast

Added the indent option to :func:`~ast.dump` which allows it to produce a multiline indented output. (Contributed by Serhiy Storchaka in :issue:`37995`.)

Added :func:`ast.unparse` as a function in the :mod:`ast` module that can be used to unparse an :class:`ast.AST` object and produce a string with code that would produce an equivalent :class:`ast.AST` object when parsed. (Contributed by Pablo Galindo and Batuhan Taskaya in :issue:`38870`.)

Added docstrings to AST nodes that contains the ASDL signature used to construct that node. (Contributed by Batuhan Taskaya in :issue:`39638`.)

asyncio

Due to significant security concerns, the reuse_address parameter of :meth:`asyncio.loop.create_datagram_endpoint` is no longer supported. This is because of the behavior of the socket option SO_REUSEADDR in UDP. For more details, see the documentation for loop.create_datagram_endpoint(). (Contributed by Kyle Stanley, Antoine Pitrou, and Yury Selivanov in :issue:`37228`.)

Added a new :term:`coroutine` :meth:`~asyncio.loop.shutdown_default_executor` that schedules a shutdown for the default executor that waits on the :class:`~concurrent.futures.ThreadPoolExecutor` to finish closing. Also, :func:`asyncio.run` has been updated to use the new :term:`coroutine`. (Contributed by Kyle Stanley in :issue:`34037`.)

Added :class:`asyncio.PidfdChildWatcher`, a Linux-specific child watcher implementation that polls process file descriptors. (:issue:`38692`)

concurrent.futures

Added a new cancel_futures parameter to :meth:`concurrent.futures.Executor.shutdown` that cancels all pending futures which have not started running, instead of waiting for them to complete before shutting down the executor. (Contributed by Kyle Stanley in :issue:`39349`.)

Removed daemon threads from :class:`~concurrent.futures.ThreadPoolExecutor` and :class:`~concurrent.futures.ProcessPoolExecutor`. This improves compatibility with subinterpreters and predictability in their shutdown processes. (Contributed by Kyle Stanley in :issue:`39812`.)

curses

Add :func:`curses.get_escdelay`, :func:`curses.set_escdelay`, :func:`curses.get_tabsize`, and :func:`curses.set_tabsize` functions. (Contributed by Anthony Sottile in :issue:`38312`.)

fcntl

Added constants :data:`~fcntl.F_OFD_GETLK`, :data:`~fcntl.F_OFD_SETLK` and :data:`~fcntl.F_OFD_SETLKW`. (Contributed by Dong-hee Na in :issue:`38602`.)

ftplib

:class:`~ftplib.FTP` and :class:`~ftplib.FTP_TLS` now raise a :class:`ValueError` if the given timeout for their constructor is zero to prevent the creation of a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.)

functools

Add the :class:`functools.TopologicalSorter` class to offer functionality to perform topological sorting of graphs. (Contributed by Pablo Galindo, Tim Peters and Larry Hastings in :issue:`17005`.)

gc

When the garbage collector makes a collection in which some objects resurrect (they are reachable from outside the isolated cycles after the finalizers have been executed), do not block the collection of all objects that are still unreachable. (Contributed by Pablo Galindo and Tim Peters in :issue:`38379`.)

Added a new function :func:`gc.is_finalized` to check if an object has been finalized by the garbage collector. (Contributed by Pablo Galindo in :issue:`39322`.)

http

HTTP status codes 103 EARLY_HINTS, 418 IM_A_TEAPOT and 425 TOO_EARLY are added to :class:`http.HTTPStatus`. (Contributed by Dong-hee Na in :issue:`39509` and Ross Rhodes in :issue:`39507`.)

imaplib

:class:`~imaplib.IMAP4` and :class:`~imaplib.IMAP4_SSL` now have an optional timeout parameter for their constructors. Also, the :meth:`~imaplib.IMAP4.open` method now has an optional timeout parameter with this change. The overridden methods of :class:`~imaplib.IMAP4_SSL` and :class:`~imaplib.IMAP4_stream` were applied to this change. (Contributed by Dong-hee Na in :issue:`38615`.)

importlib

To improve consistency with import statements, :func:`importlib.util.resolve_name` now raises :exc:`ImportError` instead of :exc:`ValueError` for invalid relative import attempts. (Contributed by Ngalim Siregar in :issue:`37444`.)

inspect

:attr:`inspect.BoundArguments.arguments` is changed from OrderedDict to regular dict. (Contributed by Inada Naoki in :issue:`36350` and :issue:`39775`.)

ipaddress

:mod:`ipaddress` now supports IPv6 Scoped Addresses (IPv6 address with suffix %<scope_id>).

Scoped IPv6 addresses can be parsed using :class:`ipaddress.IPv6Address`. If present, scope zone ID is available through the :attr:`~ipaddress.IPv6Address.scope_id` attribute. (Contributed by Oleksandr Pavliuk in :issue:`34788`.)

math

Expanded the :func:`math.gcd` function to handle multiple arguments. Formerly, it only supported two arguments. (Contributed by Serhiy Storchaka in :issue:`39648`.)

Add :func:`math.lcm`: return the least common multiple of specified arguments. (Contributed by Mark Dickinson, Ananthakrishnan and Serhiy Storchaka in :issue:`39479` and :issue:`39648`.)

Add :func:`math.nextafter`: return the next floating-point value after x towards y. (Contributed by Victor Stinner in :issue:`39288`.)

Add :func:`math.ulp`: return the value of the least significant bit of a float. (Contributed by Victor Stinner in :issue:`39310`.)

nntplib

:class:`~nntplib.NNTP` and :class:`~nntplib.NNTP_SSL` now raise a :class:`ValueError` if the given timeout for their constructor is zero to prevent the creation of a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.)

os

Added :data:`~os.CLD_KILLED` and :data:`~os.CLD_STOPPED` for :attr:`si_code`. (Contributed by Dong-hee Na in :issue:`38493`.)

Exposed the Linux-specific :func:`os.pidfd_open` (:issue:`38692`) and :data:`os.P_PIDFD` (:issue:`38713`) for process management with file descriptors.

The :func:`os.unsetenv` function is now also available on Windows. (Contributed by Victor Stinner in :issue:`39413`.)

The :func:`os.putenv` and :func:`os.unsetenv` functions are now always available. (Contributed by Victor Stinner in :issue:`39395`.)

Add :func:`os.waitstatus_to_exitcode` function: convert a wait status to an exit code. (Contributed by Victor Stinner in :issue:`40094`.)

pathlib

Added :meth:`pathlib.Path.readlink()` which acts similarly to :func:`os.readlink`. (Contributed by Girts Folkmanis in :issue:`30618`)

poplib

:class:`~poplib.POP3` and :class:`~poplib.POP3_SSL` now raise a :class:`ValueError` if the given timeout for their constructor is zero to prevent the creation of a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.)

pprint

:mod:`pprint` can now pretty-print :class:`types.SimpleNamespace`. (Contributed by Carl Bordum Hansen in :issue:`37376`.)

pydoc

The documentation string is now shown not only for class, function, method etc, but for any object that has its own __doc__ attribute. (Contributed by Serhiy Storchaka in :issue:`40257`.)

random

Add a new :attr:`random.Random.randbytes` method: generate random bytes. (Contributed by Victor Stinner in :issue:`40286`.)

signal

Exposed the Linux-specific :func:`signal.pidfd_send_signal` for sending to signals to a process using a file descriptor instead of a pid. (:issue:`38712`)

smtplib

:class:`~smtplib.SMTP` and :class:`~smtplib.SMTP_SSL` now raise a :class:`ValueError` if the given timeout for their constructor is zero to prevent the creation of a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.)

:class:`~smtplib.LMTP` constructor now has an optional timeout parameter. (Contributed by Dong-hee Na in :issue:`39329`.)

socket

The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_JOIN_FILTERS` constant on Linux 4.1 and greater. (Contributed by Stefan Tatschner and Zackery Spytz in :issue:`25780`.)

sys

Add a new :attr:`sys.platlibdir` attribute: name of the platform-specific library directory. It is used to build the path of platform-specific dynamic libraries and the path of the standard library. It is equal to "lib" on most platforms. On Fedora and SuSE, it is equal to "lib64" on 64-bit platforms. (Contributed by Jan Matějek, Matěj Cepl, Charalampos Stratakis and Victor Stinner in :issue:`1294959`.)

typing

PEP 593 introduced an :data:`typing.Annotated` type to decorate existing types with context-specific metadata and new include_extras parameter to :func:`typing.get_type_hints` to access the metadata at runtime. (Contributed by Till Varoquaux and Konstantin Kashin.)

unicodedata

The Unicode database has been updated to version 13.0.0. (:issue:`39926`).

venv

The activation scripts provided by :mod:`venv` now all specify their prompt customization consistently by always using the value specified by __VENV_PROMPT__. Previously some scripts unconditionally used __VENV_PROMPT__, others only if it happened to be set (which was the default case), and one used __VENV_NAME__ instead. (Contributed by Brett Cannon in :issue:`37663`.)

xml

White space characters within attributes are now preserved when serializing :mod:`xml.etree.ElementTree` to XML file. EOLNs are no longer normalized to "n". This is the result of discussion about how to interpret section 2.11 of XML spec. (Contributed by Mefistotelis in :issue:`39011`.)

Optimizations

  • Optimized the idiom for assignment a temporary variable in comprehensions. Now for y in [expr] in comprehensions is as fast as a simple assignment y = expr. For example:

    sums = [s for s in [0] for x in data for s in [s + x]]

    Unlike to the := operator this idiom does not leak a variable to the outer scope.

    (Contributed by Serhiy Storchaka in :issue:`32856`.)

  • Optimize signal handling in multithreaded applications. If a thread different than the main thread gets a signal, the bytecode evaluation loop is no longer interrupted at each bytecode instruction to check for pending signals which cannot be handled. Only the main thread of the main interpreter can handle signals.

    Previously, the bytecode evaluation loop was interrupted at each instruction until the main thread handles signals. (Contributed by Victor Stinner in :issue:`40010`.)

Build and C API Changes

  • New :c:func:`PyThreadState_GetInterpreter` and :c:func:`PyInterpreterState_Get` functions to get the interpreter. New :c:func:`PyThreadState_GetFrame` function to get the current frame of a Python thread state. New :c:func:`PyThreadState_GetID` function: get the unique identifier of a Python thread state. (Contributed by Victor Stinner in :issue:`39947`.)

  • Add --with-platlibdir option to the configure script: name of the platform-specific library directory, stored in the new :attr:`sys.platlibdir` attribute. See :attr:`sys.platlibdir` attribute for more information. (Contributed by Jan Matějek, Matěj Cepl, Charalampos Stratakis and Victor Stinner in :issue:`1294959`.)

  • Add a new public :c:func:`PyObject_CallNoArgs` function to the C API, which calls a callable Python object without any arguments. It is the most efficient way to call a callable Python object without any argument. (Contributed by Victor Stinner in :issue:`37194`.)

  • The global variable :c:data:`PyStructSequence_UnnamedField` is now a constant and refers to a constant string. (Contributed by Serhiy Storchaka in :issue:`38650`.)

  • Exclude PyFPE_START_PROTECT() and PyFPE_END_PROTECT() macros of pyfpe.h from Py_LIMITED_API (stable API). (Contributed by Victor Stinner in :issue:`38835`.)

  • Remove PyMethod_ClearFreeList() and PyCFunction_ClearFreeList() functions: the free lists of bound method objects have been removed. (Contributed by Inada Naoki and Victor Stinner in :issue:`37340`.)

  • Remove PyUnicode_ClearFreeList() function: the Unicode free list has been removed in Python 3.3. (Contributed by Victor Stinner in :issue:`38896`.)

  • The tp_print slot of :ref:`PyTypeObject <type-structs>` has been removed. It was used for printing objects to files in Python 2.7 and before. Since Python 3.0, it has been ignored and unused. (Contributed by Jeroen Demeyer in :issue:`36974`.)

  • On non-Windows platforms, the :c:func:`setenv` and :c:func:`unsetenv` functions are now required to build Python. (Contributed by Victor Stinner in :issue:`39395`.)

  • The COUNT_ALLOCS special build macro has been removed. (Contributed by Victor Stinner in :issue:`39489`.)

  • Changes in the limited C API (if Py_LIMITED_API macro is defined):

    • Provide :c:func:`Py_EnterRecursiveCall` and :c:func:`Py_LeaveRecursiveCall` as regular functions for the limited API. Previously, there were defined as macros, but these macros didn't compile with the limited C API which cannot access PyThreadState.recursion_depth field (the structure is opaque in the limited C API).
    • Exclude the following functions from the limited C API:
      • _Py_CheckRecursionLimit
      • _Py_NewReference()
      • _Py_ForgetReference()
      • _PyTraceMalloc_NewReference()
      • _Py_GetRefTotal()
      • The trashcan mechanism which never worked in the limited C API.
      • PyTrash_UNWIND_LEVEL
      • Py_TRASHCAN_BEGIN_CONDITION
      • Py_TRASHCAN_BEGIN
      • Py_TRASHCAN_END
      • Py_TRASHCAN_SAFE_BEGIN
      • Py_TRASHCAN_SAFE_END
    • The following static inline functions or macros become regular "opaque" function to hide implementation details:
      • _Py_NewReference()
      • PyObject_INIT() and PyObject_INIT_VAR() become aliases to :c:func:`PyObject_Init` and :c:func:`PyObject_InitVar` in the limited C API, but are overriden with static inline function otherwise. Thanks to that, it was possible to exclude _Py_NewReference() from the limited C API.
    • Move following functions and definitions to the internal C API:
      • _PyDebug_PrintTotalRefs()
      • _Py_PrintReferences()
      • _Py_PrintReferenceAddresses()
      • _Py_tracemalloc_config
      • _Py_AddToAllObjects() (specific to Py_TRACE_REFS build)

    (Contributed by Victor Stinner in :issue:`38644` and :issue:`39542`.)

  • PyInterpreterState.eval_frame (PEP 523) now requires a new mandatory tstate parameter (PyThreadState*). (Contributed by Victor Stinner in :issue:`38500`.)

  • Extension modules: :c:member:`~PyModuleDef.m_traverse`, :c:member:`~PyModuleDef.m_clear` and :c:member:`~PyModuleDef.m_free` functions of :c:type:`PyModuleDef` are no longer called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More precisely, these functions are not called if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is NULL.

    Extension modules without module state (m_size <= 0) are not affected.

  • If :c:func:`Py_AddPendingCall` is called in a subinterpreter, the function is now scheduled to be called from the subinterpreter, rather than being called from the main interpreter. Each subinterpreter now has its own list of scheduled calls. (Contributed by Victor Stinner in :issue:`39984`.)

  • Remove _PyRuntime.getframe hook and remove _PyThreadState_GetFrame macro which was an alias to _PyRuntime.getframe. They were only exposed by the internal C API. Remove also PyThreadFrameGetter type. (Contributed by Victor Stinner in :issue:`39946`.)

  • The :c:func:`PyModule_AddType` function is added to help adding a type to a module. (Contributed by Dong-hee Na in :issue:`40024`.)

  • The Windows registry is no longer used to initialize :data:`sys.path` when the -E option is used. This is significant when embedding Python on Windows. (Contributed by Zackery Spytz in :issue:`8901`.)

  • Add the functions :c:func:`PyObject_GC_IsTracked` and :c:func:`PyObject_GC_IsFinalized` to the public API to allow to query if Python objects are being currently tracked or have been already finalized by the garbage collector respectively. (Contributed by Pablo Galindo in :issue:`40241`.)

Deprecated

Removed

Porting to Python 3.9

This section lists previously described changes and other bugfixes that may require changes to your code.

Changes in the Python API

CPython bytecode changes