Misc/NEWS.d/3.9.1rc1.rst

.. bpo: 42103
.. date: 2020-10-23-19-19-30
.. nonce: cILT66
.. release date: 2020-11-24
.. section: Security

Prevented potential DoS attack via CPU and RAM exhaustion when processing
malformed Apple Property List files in binary format.

..

.. bpo: 42051
.. date: 2020-10-19-10-56-27
.. nonce: EU_B7u
.. section: Security

The :mod:`plistlib` module no longer accepts entity declarations in XML
plist files to avoid XML vulnerabilities. This should not affect users as
entity declarations are not used in regular plist files.

..

.. bpo: 40791
.. date: 2020-05-28-06-06-47
.. nonce: QGZClX
.. section: Security

Add ``volatile`` to the accumulator variable in ``hmac.compare_digest``,
making constant-time-defeating optimizations less likely.

..

.. bpo: 41686
.. date: 2020-11-17-16-25-50
.. nonce: hX77kL
.. section: Core and Builtins

On Windows, the ``SIGINT`` event, ``_PyOS_SigintEvent()``, is now created
even if Python is configured to not install signal handlers (if
:c:member:`PyConfig.install_signal_handlers` equals to 0, or
``Py_InitializeEx(0)``).

..

.. bpo: 42381
.. date: 2020-11-16-23-45-56
.. nonce: G4AWxL
.. section: Core and Builtins

Allow assignment expressions in set literals and set comprehensions as per
PEP 572. Patch by Pablo Galindo.

..

.. bpo: 42374
.. date: 2020-11-16-17-57-09
.. nonce: t7np1E
.. section: Core and Builtins

Fix a regression introduced by the new parser, where an unparenthesized
walrus operator was not allowed within generator expressions.

..

.. bpo: 42296
.. date: 2020-11-13-13-53-11
.. nonce: DuGrLJ
.. section: Core and Builtins

On Windows, fix a regression in signal handling which prevented to interrupt
a program using CTRL+C. The signal handler can be run in a thread different
than the Python thread, in which case the test deciding if the thread can
handle signals is wrong.

..

.. bpo: 42332
.. date: 2020-11-12-23-16-14
.. nonce: fEQIdk
.. section: Core and Builtins

:class:`types.GenericAlias` objects can now be the targets of weakrefs.

..

.. bpo: 42218
.. date: 2020-10-31-17-50-23
.. nonce: Dp_Z3v
.. section: Core and Builtins

Fixed a bug in the PEG parser that was causing crashes in debug mode. Now
errors are checked in left-recursive rules to avoid cases where such errors
do not get handled in time and appear as long-distance crashes in other
places.

..

.. bpo: 42214
.. date: 2020-10-30-22-16-30
.. nonce: lXskM_
.. section: Core and Builtins

Fixed a possible crash in the PEG parser when checking for the '!=' token in
the ``barry_as_flufl`` rule. Patch by Pablo Galindo.

..

.. bpo: 42143
.. date: 2020-10-27-21-34-05
.. nonce: N6KXUO
.. section: Core and Builtins

Fix handling of errors during creation of ``PyFunctionObject``, which
resulted in operations on uninitialized memory. Patch by Yonatan
Goldschmidt.

..

.. bpo: 41659
.. date: 2020-10-27-18-32-49
.. nonce: d4a-8o
.. section: Core and Builtins

Fix a bug in the parser, where a curly brace following a `primary` didn't
fail immediately. This led to invalid expressions like `a {b}` to throw a
:exc:`SyntaxError` with a wrong offset, or invalid expressions ending with a
curly brace like `a {` to not fail immediately in the REPL.

..

.. bpo: 42150
.. date: 2020-10-25-21-14-18
.. nonce: b70u_T
.. section: Core and Builtins

Fix possible buffer overflow in the new parser when checking for
continuation lines. Patch by Pablo Galindo.

..

.. bpo: 42123
.. date: 2020-10-23-02-43-24
.. nonce: 64gJWC
.. section: Core and Builtins

Run the parser two times. On the first run, disable all the rules that only
generate better error messages to gain performance. If there's a parse
failure, run the parser a second time with those enabled.

..

.. bpo: 41910
.. date: 2020-10-21-14-40-54
.. nonce: CzBMit
.. section: Core and Builtins

Document the default implementation of `object.__eq__`.

..

.. bpo: 42057
.. date: 2020-10-20-11-36-14
.. nonce: BI-OoV
.. section: Core and Builtins

Fix peephole optimizer misoptimize conditional jump + JUMP_IF_NOT_EXC_MATCH
pair.

..

.. bpo: 41984
.. date: 2020-10-14-16-19-43
.. nonce: SEtKMr
.. section: Core and Builtins

The garbage collector now tracks all user-defined classes. Patch by Brandt
Bucher.

..

.. bpo: 41993
.. date: 2020-10-10-13-53-52
.. nonce: YMzixQ
.. section: Core and Builtins

Fixed potential issues with removing not completely initialized module from
``sys.modules`` when import fails.

..

.. bpo: 41979
.. date: 2020-10-09-10-55-50
.. nonce: ImXIk2
.. section: Core and Builtins

Star-unpacking is now allowed for with item's targets in the PEG parser.

..

.. bpo: 41909
.. date: 2020-10-04-10-55-12
.. nonce: BqHPcm
.. section: Core and Builtins

Fixed stack overflow in :func:`issubclass` and :func:`isinstance` when
getting the ``__bases__`` attribute leads to infinite recursion.

..

.. bpo: 41894
.. date: 2020-10-02-11-35-33
.. nonce: ffmtOt
.. section: Core and Builtins

When loading a native module and a load failure occurs, prevent a possible
UnicodeDecodeError when not running in a UTF-8 locale by decoding the load
error message using the current locale's encoding.

..

.. bpo: 39934
.. date: 2020-09-24-12-15-45
.. nonce: YVHTCF
.. section: Core and Builtins

Correctly count control blocks in 'except' in compiler. Ensures that a
syntax error, rather a fatal error, occurs for deeply nested, named
exception handlers.

..

.. bpo: 42328
.. date: 2020-11-15-17-02-00
.. nonce: bqpPlR
.. section: Library

Fixed :meth:`tkinter.ttk.Style.map`. The function accepts now the
representation of the default state as empty sequence (as returned by
``Style.map()``). The structure of the result is now the same on all
platform and does not depend on the value of ``wantobjects``.

..

.. bpo: 42345
.. date: 2020-11-15-15-23-34
.. nonce: hiIR7x
.. section: Library

Fix various issues with ``typing.Literal`` parameter handling (flatten,
deduplicate, use type to cache key). Patch provided by Yurii Karabas.

..

.. bpo: 42350
.. date: 2020-11-13-18-53-50
.. nonce: rsql7V
.. section: Library

Fix the :class:`threading.Thread` class at fork: do nothing if the thread is
already stopped (ex: fork called at Python exit). Previously, an error was
logged in the child process.

..

.. bpo: 42014
.. date: 2020-11-10-15-40-56
.. nonce: ShM37l
.. section: Library

The ``onerror`` callback from ``shutil.rmtree`` now receives correct
function when ``os.open`` fails.

..

.. bpo: 42237
.. date: 2020-11-10-14-27-49
.. nonce: F363jO
.. section: Library

Fix `os.sendfile()` on illumos.

..

.. bpo: 42249
.. date: 2020-11-03-09-22-56
.. nonce: vfNO2u
.. section: Library

Fixed writing binary Plist files larger than 4 GiB.

..

.. bpo: 35455
.. date: 2020-11-02-14-10-48
.. nonce: Q1xTIo
.. section: Library

On Solaris, :func:`~time.thread_time` is now implemented with
``gethrvtime()`` because ``clock_gettime(CLOCK_THREAD_CPUTIME_ID)`` is not
always available. Patch by Jakub Kulik.

..

.. bpo: 42233
.. date: 2020-11-02-01-31-15
.. nonce: YxRj-h
.. section: Library

The :func:`repr` of :mod:`typing` types containing :ref:`Generic Alias Types
<types-genericalias>` previously did not show the parameterized types in the
``GenericAlias``.  They have now been changed to do so.

..

.. bpo: 41754
.. date: 2020-11-01-15-07-20
.. nonce: DraSZh
.. section: Library

webbrowser: Ignore *NotADirectoryError* when calling ``xdg-settings``.

..

.. bpo: 29566
.. date: 2020-10-31-13-28-36
.. nonce: 6aDbty
.. section: Library

``binhex.binhex()`` consisently writes macOS 9 line endings.

..

.. bpo: 42183
.. date: 2020-10-29-11-17-35
.. nonce: 50ZcIi
.. section: Library

Fix a stack overflow error for asyncio Task or Future repr().

The overflow occurs under some circumstances when a Task or Future
recursively returns itself.

..

.. bpo: 42146
.. date: 2020-10-25-19-25-02
.. nonce: 6A8uvS
.. section: Library

Fix memory leak in :func:`subprocess.Popen` in case an uid (gid) specified
in `user` (`group`, `extra_groups`) overflows `uid_t` (`gid_t`).

..

.. bpo: 42140
.. date: 2020-10-24-04-02-36
.. nonce: miLqvb
.. section: Library

Improve asyncio.wait function to create the futures set just one time.

..

.. bpo: 42103
.. date: 2020-10-23-19-20-14
.. nonce: C5obK2
.. section: Library

:exc:`~plistlib.InvalidFileException` and :exc:`RecursionError` are now the
only errors caused by loading malformed binary Plist file (previously
ValueError and TypeError could be raised in some specific cases).

..

.. bpo: 41052
.. date: 2020-10-21-23-45-02
.. nonce: 3N7J2J
.. section: Library

Pickling heap types implemented in C with protocols 0 and 1 raises now an
error instead of producing incorrect data.

..

.. bpo: 41491
.. date: 2020-10-19-14-02-09
.. nonce: d1BUWH
.. section: Library

plistlib: fix parsing XML plists with hexadecimal integer values

..

.. bpo: 42065
.. date: 2020-10-17-23-17-18
.. nonce: 85BsRA
.. section: Library

Fix an incorrectly formatted error from :meth:`_codecs.charmap_decode` when
called with a mapped value outside the range of valid Unicode code points.
PR by Max Bernstein.

..

.. bpo: 41966
.. date: 2020-10-17-07-52-53
.. nonce: gwEQRZ
.. section: Library

Fix pickling pure Python :class:`datetime.time` subclasses. Patch by Dean
Inwood.

..

.. bpo: 41976
.. date: 2020-10-08-18-22-28
.. nonce: Svm0wb
.. section: Library

Fixed a bug that was causing :func:`ctypes.util.find_library` to return
``None`` when triying to locate a library in an environment when gcc>=9 is
available and ``ldconfig`` is not. Patch by Pablo Galindo

..

.. bpo: 41900
.. date: 2020-10-01-10-50-12
.. nonce: Cho7oh
.. section: Library

C14N 2.0 serialisation in xml.etree.ElementTree failed for unprefixed
attributes when a default namespace was defined.

..

.. bpo: 41840
.. date: 2020-09-23-23-17-59
.. nonce: QRFr4L
.. section: Library

Fix a bug in the :mod:`symtable` module that was causing module-scope global
variables to not be reported as both local and global. Patch by Pablo
Galindo.

..

.. bpo: 41831
.. date: 2020-09-22-11-07-50
.. nonce: k-Eop_
.. section: Library

``str()`` for the ``type`` attribute of the ``tkinter.Event`` object always
returns now the numeric code returned by Tk instead of the name of the event
type.

..

.. bpo: 41817
.. date: 2020-09-22-00-23-30
.. nonce: bnh-VG
.. section: Library

fix `tkinter.EventType` Enum so all members are strings, and none are tuples

..

.. bpo: 41815
.. date: 2020-09-19-23-14-54
.. nonce: RNpuX3
.. section: Library

Fix SQLite3 segfault when backing up closed database. Patch contributed by
Peter David McCormick.

..

.. bpo: 41316
.. date: 2020-07-28-12-08-58
.. nonce: bSCbK4
.. section: Library

Fix the :mod:`tarfile` module to write only basename of TAR file to GZIP
compression header.

..

.. bpo: 16936
.. date: 2020-07-08-09-45-00
.. nonce: z8o8Pn
.. section: Library

Allow ``ctypes.wintypes`` to be imported on non-Windows systems.

..

.. bpo: 40592
.. date: 2020-05-14-16-01-34
.. nonce: Cmk855
.. section: Library

:func:`shutil.which` now ignores empty entries in :envvar:`PATHEXT` instead
of treating them as a match.

..

.. bpo: 40550
.. date: 2020-05-08-21-30-54
.. nonce: i7GWkb
.. section: Library

Fix time-of-check/time-of-action issue in subprocess.Popen.send_signal.

..

.. bpo: 40492
.. date: 2020-05-04-12-16-00
.. nonce: ONk9Na
.. section: Library

Fix ``--outfile`` for :mod:`cProfile` / :mod:`profile` not writing the
output file in the original directory when the program being profiled
changes the working directory.  PR by Anthony Sottile.

..

.. bpo: 40105
.. date: 2020-04-03-16-13-59
.. nonce: hfM2c0
.. section: Library

ZipFile truncates files to avoid corruption when a shorter comment is
provided in append ("a") mode. Patch by Jan Mazur.

..

.. bpo: 27321
.. date: 2020-01-19-18-40-26
.. nonce: 8e6SpM
.. section: Library

Fixed KeyError exception when flattening an email to a string attempts to
replace a non-existent Content-Transfer-Encoding header.

..

.. bpo: 42153
.. date: 2020-11-15-13-46-31
.. nonce: KjBhx3
.. section: Documentation

Fix the URL for the IMAP protocol documents.

..

.. bpo: 42061
.. date: 2020-10-28-21-39-45
.. nonce: _x-0sg
.. section: Documentation

Document __format__ functionality for IP addresses.

..

.. bpo: 42010
.. date: 2020-10-21-02-21-14
.. nonce: 76vJ0u
.. section: Documentation

Clarify that subscription expressions are also valid for certain
:term:`classes <class>` and :term:`types <type>` in the standard library,
and for user-defined classes and types if the classmethod
:meth:`__class_getitem__` is provided.

..

.. bpo: 41805
.. date: 2020-10-10-01-36-37
.. nonce: l-CGv5
.. section: Documentation

Documented :ref:`generic alias type <types-genericalias>` and
:data:`types.GenericAlias`. Also added an entry in glossary for
:term:`generic types <generic type>`.

..

.. bpo: 41774
.. date: 2020-09-24-15-35-13
.. nonce: 5IqdGP
.. section: Documentation

In Programming FAQ "Sequences (Tuples/Lists)" section, add "How do you
remove multiple items from a list".

..

.. bpo: 35293
.. date: 2020-09-12-17-37-13
.. nonce: _cOwPD
.. section: Documentation

Fix RemovedInSphinx40Warning when building the documentation. Patch by
Dong-hee Na.

..

.. bpo: 41726
.. date: 2020-09-08-16-57-09
.. nonce: g0UXrn
.. section: Documentation

Update the refcounts info of ``PyType_FromModuleAndSpec``.

..

.. bpo: 39693
.. date: 2020-02-24-09-02-05
.. nonce: QXw0Fm
.. section: Documentation

Fix tarfile's extractfile documentation

..

.. bpo: 39416
.. date: 2020-01-22-05-14-53
.. nonce: uYjhEm
.. section: Documentation

Document some restrictions on the default string representations of numeric
classes.

..

.. bpo: 40754
.. date: 2020-11-13-21-51-34
.. nonce: Ekoxkg
.. section: Tests

Include ``_testinternalcapi`` module in Windows installer for test suite

..

.. bpo: 41739
.. date: 2020-10-12-00-11-47
.. nonce: wSCc4K
.. section: Tests

Fix test_logging.test_race_between_set_target_and_flush(): the test now
waits until all threads complete to avoid leaking running threads.

..

.. bpo: 41970
.. date: 2020-10-08-14-00-17
.. nonce: aZ8QFf
.. section: Tests

Avoid a test failure in ``test_lib2to3`` if the module has already imported
at the time the test executes. Patch by Pablo Galindo.

..

.. bpo: 41944
.. date: 2020-10-05-17-43-46
.. nonce: rf1dYb
.. section: Tests

Tests for CJK codecs no longer call ``eval()`` on content received via HTTP.

..

.. bpo: 41939
.. date: 2020-10-05-09-37-43
.. nonce: P4OlbA
.. section: Tests

Fix test_site.test_license_exists_at_url(): call
``urllib.request.urlcleanup()`` to reset the global
``urllib.request._opener``. Patch by Victor Stinner.

..

.. bpo: 41561
.. date: 2020-09-18-16-14-03
.. nonce: uPnwrW
.. section: Tests

test_ssl: skip test_min_max_version_mismatch when TLS 1.0 is not available

..

.. bpo: 41602
.. date: 2020-08-25-19-25-36
.. nonce: Z64s0I
.. section: Tests

Add tests for SIGINT handling in the runpy module.

..

.. bpo: 41306
.. date: 2020-08-03-13-44-37
.. nonce: VDoWXI
.. section: Tests

Fixed a failure in ``test_tk.test_widgets.ScaleTest`` happening when
executing the test with Tk 8.6.10.

..

.. bpo: 42398
.. date: 2020-11-18-11-58-44
.. nonce: Yt5wO8
.. section: Build

Fix a race condition in "make regen-all" when make -jN option is used to run
jobs in parallel. The clinic.py script now only use atomic write to write
files. Moveover, generated files are now left unchanged if the content does
not change, to not change the file modification time.

..

.. bpo: 41617
.. date: 2020-11-13-15-04-53
.. nonce: 98_oaE
.. section: Build

Fix building ``pycore_bitutils.h`` internal header on old clang version
without ``__builtin_bswap16()`` (ex: Xcode 4.6.3 on Mac OS X 10.7). Patch by
Joshua Root and Victor Stinner.

..

.. bpo: 38249
.. date: 2020-09-28-21-56-51
.. nonce: uzMCaZ
.. section: Build

Update :c:macro:`Py_UNREACHABLE` to use __builtin_unreachable() if only the
compiler is able to use it. Patch by Dong-hee Na.

..

.. bpo: 40998
.. date: 2020-06-17-09-05-02
.. nonce: sgqmg9
.. section: Build

Addressed three compiler warnings found by undefined behavior sanitizer
(ubsan).

..

.. bpo: 42120
.. date: 2020-11-16-22-41-02
.. nonce: 9scgko
.. section: Windows

Remove macro definition of ``copysign`` (to ``_copysign``) in headers.

..

.. bpo: 38439
.. date: 2020-10-20-13-19-42
.. nonce: eMLi-t
.. section: Windows

Updates the icons for IDLE in the Windows Store package.

..

.. bpo: 41744
.. date: 2020-09-11-17-59-33
.. nonce: e_ugDQ
.. section: Windows

Fixes automatic import of props file when using the Nuget package.

..

.. bpo: 41557
.. date: 2020-08-26-09-35-06
.. nonce: vt00cQ
.. section: Windows

Update Windows installer to use SQLite 3.33.0.

..

.. bpo: 38324
.. date: 2020-05-30-02-46-43
.. nonce: 476M-5
.. section: Windows

Avoid Unicode errors when accessing certain locale data on Windows.

..

.. bpo: 41116
.. date: 2020-11-15-16-43-45
.. nonce: oCkbrF
.. section: macOS

Ensure distutils.unixxcompiler.find_library_file can find system provided
libraries on macOS 11.

..

.. bpo: 41100
.. date: 2020-11-01-16-40-23
.. nonce: BApztP
.. section: macOS

Add support for macOS 11 and Apple Silicon systems.

It is now possible to build "Universal 2" binaries using
"--enable-universalsdk --with-universal-archs=universal2".

Binaries build on later macOS versions can be deployed back to older
versions (tested up to macOS 10.9), when using the correct deployment
target. This is tested using Xcode 11 and later.

..

.. bpo: 38443
.. date: 2020-10-23-10-26-53
.. nonce: vu64tl
.. section: macOS

The ``--enable-universalsdk`` and ``--with-universal-archs`` options for the
configure script now check that the specified architectures can be used.

..

.. bpo: 41471
.. date: 2020-10-19-12-25-19
.. nonce: gwA7un
.. section: macOS

Ignore invalid prefix lengths in system proxy excludes.

..

.. bpo: 41557
.. date: 2020-08-26-09-31-37
.. nonce: mcQ75z
.. section: macOS

Update macOS installer to use SQLite 3.33.0.

..

.. bpo: 42426
.. date: 2020-11-21-17-21-21
.. nonce: kNnPoC
.. section: IDLE

Fix reporting offset of the RE error in searchengine.

..

.. bpo: 42415
.. date: 2020-11-20-01-30-27
.. nonce: CyD-va
.. section: IDLE

Get docstrings for IDLE calltips more often by using inspect.getdoc.

..

.. bpo: 33987
.. date: 2020-10-24-21-27-37
.. nonce: fIh9JL
.. section: IDLE

Mostly finish using ttk widgets, mainly for editor, settings, and searches.
Some patches by Mark Roseman.

..

.. bpo: 41775
.. date: 2020-09-24-14-31-16
.. nonce: sB8Vre
.. section: IDLE

Use 'IDLE Shell' as shell title

..

.. bpo: 35764
.. date: 2020-09-22-11-13-45
.. nonce: VoNa8y
.. section: IDLE

Rewrite the Calltips doc section.

..

.. bpo: 40181
.. date: 2020-09-22-00-45-40
.. nonce: hhQi3z
.. section: IDLE

In calltips, stop reminding that '/' marks the end of positional-only
arguments.

..

.. bpo: 40511
.. date: 2020-06-16-12-16-13
.. nonce: XkihpM
.. section: IDLE

Typing opening and closing parentheses inside the parentheses of a function
call will no longer cause unnecessary "flashing" off and on of an existing
open call-tip, e.g. when typed in a string literal.

..

.. bpo: 38439
.. date: 2020-04-22-09-37-40
.. nonce: ieXL-c
.. section: IDLE

Add a 256×256 pixel IDLE icon to the Windows .ico file. Created by Andrew
Clover. Remove the low-color gif variations from the .ico file.

..

.. bpo: 42015
.. date: 2020-10-12-20-13-58
.. nonce: X4H2_V
.. section: C API

Fix potential crash in deallocating method objects when dynamically
allocated `PyMethodDef`'s lifetime is managed through the ``self`` argument
of a `PyCFunction`.

..

.. bpo: 41986
.. date: 2020-10-09-22-50-46
.. nonce: JUPE59
.. section: C API

:c:data:`Py_FileSystemDefaultEncodeErrors` and :c:data:`Py_UTF8Mode` are
available again in limited API.