Misc/NEWS.d/3.9.0b4.rst

.. bpo: 41004
.. date: 2020-06-29-16-02-29
.. nonce: ovF0KZ
.. release date: 2020-07-02
.. section: Security

The __hash__() methods of  ipaddress.IPv4Interface and
ipaddress.IPv6Interface incorrectly generated constant hash values of 32 and
128 respectively. This resulted in always causing hash collisions. The fix
uses hash() to generate hash values for the tuple of (address, mask length,
network address).

..

.. bpo: 41094
.. date: 2020-06-23-23-26-42
.. nonce: zEIJse
.. section: Core and Builtins

Fix decoding errors with audit when open files with non-ASCII names on
non-UTF-8 locale.

..

.. bpo: 41084
.. date: 2020-06-23-15-10-19
.. nonce: pt3y7F
.. section: Core and Builtins

Prefix the error message with 'f-string: ', when parsing an f-string
expression which throws a :exc:`SyntaxError`.

..

.. bpo: 41076
.. date: 2020-06-22-13-22-30
.. nonce: eWYw2N
.. section: Core and Builtins

Pre-feed the parser with the location of the f-string expression, not the
f-string itself, which allows us to skip the shifting of the AST node
locations after the parsing is completed.

..

.. bpo: 40939
.. date: 2020-06-21-23-48-12
.. nonce: RjLl6C
.. section: Core and Builtins

Deprecate :c:func:`PyNode_Compile`.

..

.. bpo: 41056
.. date: 2020-06-21-19-53-33
.. nonce: IDu_EK
.. section: Core and Builtins

Fixes a reference to deallocated stack space during startup when
constructing sys.path involving a relative symlink when code was supplied
via -c.  (discovered via Coverity)

..

.. bpo: 41061
.. date: 2020-06-21-10-54-02
.. nonce: AHf9MU
.. section: Core and Builtins

Fix incorrect expressions and asserts in hashtable code and tests.

..

.. bpo: 41052
.. date: 2020-06-20-22-46-18
.. nonce: 46MPeF
.. section: Core and Builtins

Opt out serialization/deserialization for _random.Random

..

.. bpo: 40939
.. date: 2020-06-20-18-36-05
.. nonce: V3eiAZ
.. section: Core and Builtins

Deprecate :c:func:`PyParser_SimpleParseStringFlags`,
:c:func:`PyParser_SimpleParseStringFlagsFilename` and
:c:func:`PyParser_SimpleParseFileFlags`.

..

.. bpo: 35975
.. date: 2020-06-20-17-16-25
.. nonce: UDHCHp
.. section: Core and Builtins

Stefan Behnel reported that cf_feature_version is used even when
PyCF_ONLY_AST is not set. This is against the intention and against the
documented behavior, so it's been fixed.

..

.. bpo: 40985
.. date: 2020-06-15-16-29-55
.. nonce: IIN_xX
.. section: Core and Builtins

Fix a bug that caused the :exc:`SyntaxError` text to be empty when a file
ends with a line ending in a line continuation character (i.e. backslash).
The error text should contain the text of the last line.

..

.. bpo: 40958
.. date: 2020-06-15-01-20-44
.. nonce: 7O2Wh1
.. section: Core and Builtins

Fix a possible buffer overflow in the PEG parser when gathering information
for emitting syntax errors. Patch by Pablo Galindo.

..

.. bpo: 40957
.. date: 2020-06-12-12-21-54
.. nonce: Z8n6I6
.. section: Core and Builtins

Fix refleak in _Py_fopen_obj() when PySys_Audit() fails

..

.. bpo: 40947
.. date: 2020-06-11-16-06-49
.. nonce: 72cZcR
.. section: Core and Builtins

The Python :ref:`Path Configuration <init-path-config>` now takes
:c:member:`PyConfig.platlibdir` in account.

..

.. bpo: 40847
.. date: 2020-06-09-23-52-32
.. nonce: 4XAACw
.. section: Core and Builtins

Fix a bug where a line with only a line continuation character is not
considered a blank line at tokenizer level. In such cases, more than a
single `NEWLINE` token was emitted. The old parser was working around the
issue, but the new parser threw a :exc:`SyntaxError` for valid input due to
this. For example, an empty line following a line continuation character was
interpreted as a :exc:`SyntaxError`.

..

.. bpo: 40824
.. date: 2020-05-30-14-37-18
.. nonce: XR3V5s
.. section: Core and Builtins

Unexpected errors in calling the ``__iter__`` method are no longer masked by
``TypeError`` in the :keyword:`in` operator and functions
:func:`~operator.contains`, :func:`~operator.indexOf` and
:func:`~operator.countOf` of the :mod:`operator` module.

..

.. bpo: 19569
.. date: 2018-08-29-15-57-07
.. nonce: RGu2Kb
.. section: Core and Builtins

Add the private macros ``_Py_COMP_DIAG_PUSH``,
``_Py_COMP_DIAG_IGNORE_DEPR_DECLS``, and ``_Py_COMP_DIAG_POP``.

..

.. bpo: 41193
.. date: 2020-07-02-11-53-45
.. nonce: 8-Tnql
.. section: Library

The ``write_history()`` atexit function of the readline completer now
ignores any :exc:`OSError` to ignore error if the filesystem is read-only,
instead of only ignoring :exc:`FileNotFoundError` and
:exc:`PermissionError`.

..

.. bpo: 41161
.. date: 2020-06-30-20-50-51
.. nonce: QTdJjz
.. section: Library

The decimal module now requires libmpdec-2.5.0. Users of
--with-system-libmpdec should update their system library.

..

.. bpo: 40874
.. date: 2020-06-28-21-16-51
.. nonce: YImvzA
.. section: Library

The decimal module now requires libmpdec-2.5.0.

..

.. bpo: 41138
.. date: 2020-06-27-13-51-36
.. nonce: bIpf7g
.. section: Library

Fixed the :mod:`trace` module CLI for Python source files with non-UTF-8
encoding.

..

.. bpo: 31938
.. date: 2020-06-22-20-08-40
.. nonce: EVuko9
.. section: Library

Fix default-value signatures of several functions in the :mod:`select`
module - by Anthony Sottile.

..

.. bpo: 41068
.. date: 2020-06-22-10-25-39
.. nonce: _bX2BW
.. section: Library

Fixed reading files with non-ASCII names from ZIP archive directly after
writing them.

..

.. bpo: 41058
.. date: 2020-06-20-21-03-55
.. nonce: gztdZy
.. section: Library

:func:`pdb.find_function` now correctly determines the source file encoding.

..

.. bpo: 41056
.. date: 2020-06-20-18-35-43
.. nonce: Garcle
.. section: Library

Fix a NULL pointer dereference within the ssl module during a MemoryError in
the keylog callback. (discovered by Coverity)

..

.. bpo: 41056
.. date: 2020-06-20-18-33-03
.. nonce: gTH4Bq
.. section: Library

Fixed an instance where a MemoryError within the zoneinfo module might not
be reported or not reported at its source. (found by Coverity)

..

.. bpo: 41048
.. date: 2020-06-20-10-16-57
.. nonce: hEXB-B
.. section: Library

:func:`mimetypes.read_mime_types` function reads the rule file using UTF-8
encoding, not the locale encoding. Patch by Srinivas Reddy Thatiparthy.

..

.. bpo: 41043
.. date: 2020-06-20-00-19-30
.. nonce: p-Pk-H
.. section: Library

Fixed the use of :func:`~glob.glob` in the stdlib: literal part of the path
is now always correctly escaped.

..

.. bpo: 40448
.. date: 2020-06-15-12-22-53
.. nonce: 1dk8Bu
.. section: Library

:mod:`ensurepip` now disables the use of `pip` cache when installing the
bundled versions of `pip` and `setuptools`.  Patch by Krzysztof Konopko.

..

.. bpo: 40967
.. date: 2020-06-15-00-13-57
.. nonce: _dx3OO
.. section: Library

Removed :meth:`asyncio.Task.current_task` and
:meth:`asyncio.Task.all_tasks`. Patch contributed by Rémi Lapeyre.

..

.. bpo: 40955
.. date: 2020-06-12-11-55-30
.. nonce: huixCg
.. section: Library

Fix a minor memory leak in :mod:`subprocess` module when extra_groups was
specified.

..

.. bpo: 40855
.. date: 2020-06-12-10-44-15
.. nonce: jSot83
.. section: Library

The standard deviation and variance functions in the statistics module were
ignoring their mu and xbar arguments.

..

.. bpo: 40924
.. date: 2020-06-11-23-41-50
.. nonce: m17Fkm
.. section: Library

Removed support for loaders implementing .files and supplying
TraversableResources.

..

.. bpo: 40939
.. date: 2020-06-11-11-07-10
.. nonce: -D5Asl
.. section: Library

Use the new PEG parser when generating the stdlib :mod:`keyword` module.

..

.. bpo: 40834
.. date: 2020-05-31-15-52-18
.. nonce: MO9_hb
.. section: Library

Fix truncate when sending str object with_xxsubinterpreters.channel_send.

..

.. bpo: 26407
.. date: 2020-05-30-14-19-47
.. nonce: MjWLO1
.. section: Library

Unexpected errors in calling the ``__iter__`` method are no longer masked by
``TypeError`` in :func:`csv.reader`, :func:`csv.writer.writerow` and
:meth:`csv.writer.writerows`.

..

.. bpo: 38488
.. date: 2020-05-28-16-51-00
.. nonce: hFQNgA
.. section: Library

Update ensurepip to install pip 20.1.1 and setuptools 47.1.0.

..

.. bpo: 36543
.. date: 2020-05-15-21-14-45
.. nonce: Jt-eSX
.. section: Library

Restored the deprecated :mod:`xml.etree.cElementTree` module.

..

.. bpo: 34226
.. date: 2018-07-29-12-14-54
.. nonce: BE7zbu
.. section: Library

Fix `cgi.parse_multipart` without content_length. Patch by Roger Duran

..

.. bpo: 41085
.. date: 2020-06-23-12-02-45
.. nonce: JZKsyz
.. section: Tests

Fix integer overflow in the :meth:`array.array.index` method on 64-bit
Windows for index larger than ``2**31``.

..

.. bpo: 41069
.. date: 2020-06-22-00-21-12
.. nonce: bLZkX-
.. section: Tests

:data:`test.support.TESTFN` and the current directory for tests when run via
``test.regrtest`` contain now non-ascii characters if possible.

..

.. bpo: 38377
.. date: 2020-06-17-18-00-21
.. nonce: jfg4TH
.. section: Tests

On Linux, skip tests using multiprocessing if the current user cannot create
a file in ``/dev/shm/`` directory. Add the
:func:`~test.support.skip_if_broken_multiprocessing_synchronize` function to
the :mod:`test.support` module.

..

.. bpo: 41009
.. date: 2020-06-17-17-27-07
.. nonce: Rvn6OQ
.. section: Tests

Fix use of ``support.require_{linux|mac|freebsd}_version()`` decorators as
class decorator.

..

.. bpo: 41003
.. date: 2020-06-17-15-07-14
.. nonce: tiH_Fy
.. section: Tests

Fix ``test_copyreg`` when ``numpy`` is installed: ``test.pickletester`` now
saves/restores warnings filters when importing ``numpy``, to ignore filters
installed by ``numpy``.

..

.. bpo: 40964
.. date: 2020-06-12-20-46-23
.. nonce: OBzf2c
.. section: Tests

Disable remote :mod:`imaplib` tests, host cyrus.andrew.cmu.edu is blocking
incoming connections.

..

.. bpo: 40927
.. date: 2020-06-09-18-48-18
.. nonce: 67ylLg
.. section: Tests

Fix test_binhex when run twice: it now uses import_fresh_module() to ensure
that it raises DeprecationWarning each time.

..

.. bpo: 34401
.. date: 2018-08-20-09-38-52
.. nonce: eGxMPm
.. section: Tests

Make test_gdb properly run on HP-UX. Patch by Michael Osipov.

..

.. bpo: 40204
.. date: 2020-06-25-06-59-13
.. nonce: GpD04D
.. section: Build

Pin Sphinx version to 2.3.1 in ``Doc/Makefile``.

..

.. bpo: 41074
.. date: 2020-06-24-21-30-42
.. nonce: gaQc3C
.. section: Windows

Fixed support of non-ASCII names in functions :func:`msilib.OpenDatabase`
and :func:`msilib.init_database` and non-ASCII SQL in method
:meth:`msilib.Database.OpenView`.

..

.. bpo: 40164
.. date: 2020-06-12-13-13-44
.. nonce: SPrSn5
.. section: Windows

Updates Windows OpenSSL to 1.1.1g

..

.. bpo: 37556
.. date: 2019-07-11-06-11-09
.. nonce: sygMUU
.. section: Windows

Extend py.exe help to mention overrides via venv, shebang, environmental
variables & ini files.

..

.. bpo: 39580
.. date: 2020-06-25-06-09-00
.. nonce: N_vJ9h
.. section: macOS

Avoid opening Finder window if running installer from the command line.
Patch contributed by Rick Heil.

..

.. bpo: 41100
.. date: 2020-06-24-13-51-57
.. nonce: mcHdc5
.. section: macOS

Fix configure error when building on macOS 11. Note that the current Python
release was released shortly after the first developer preview of macOS 11
(Big Sur); there are other known issues with building and running on the
developer preview. Big Sur is expected to be fully supported in a future
bugfix release of Python 3.8.x and with 3.9.0.

..

.. bpo: 41005
.. date: 2020-06-17-13-45-15
.. nonce: zZegdV
.. section: macOS

fixed an XDG settings issue not allowing macos to open browser in
webbrowser.py

..

.. bpo: 41152
.. date: 2020-06-29-14-51-15
.. nonce: d6mV0C
.. section: IDLE

The encoding of ``stdin``, ``stdout`` and ``stderr`` in IDLE is now always
UTF-8.

..

.. bpo: 41144
.. date: 2020-06-27-17-02-00
.. nonce: JoFGIX
.. section: IDLE

Make Open Module open a special module such as os.path.

..

.. bpo: 36346
.. date: 2020-06-17-11-24-00
.. nonce: fTMr3S
.. section: C API

Mark ``Py_UNICODE_COPY``, ``Py_UNICODE_FILL``, ``PyUnicode_WSTR_LENGTH``,
``PyUnicode_FromUnicode``, ``PyUnicode_AsUnicode``,
``_PyUnicode_AsUnicode``, and ``PyUnicode_AsUnicodeAndSize`` as deprecated
in C. Remove ``Py_UNICODE_MATCH`` which was deprecated and broken since
Python 3.3.

..

.. bpo: 36020
.. date: 2020-06-15-16-46-01
.. nonce: djI6jw
.. section: C API

On Windows, ``#include "pyerrors.h"`` no longer defines ``snprintf`` and
``vsnprintf`` macros.

..

.. bpo: 40703
.. date: 2020-05-20-19-11-12
.. nonce: qQXfW8
.. section: C API

The PyType_FromSpec*() functions no longer overwrite the type's "__module__"
attribute if it is set via "Py_tp_members" or "Py_tp_getset".