This article explains the new features in Python 3.6, compared to 3.5.
Python 3.6 was released on December 23, 2016.
For full details, see the changelog.
New Features
PEP 526: Syntax for variable annotations
PEP 484 introduced the standard for type annotations of function
parameters, a.k.a. type hints. This PEP adds syntax to Python for annotating
the types of variables including class variables and instance variables:
primes: List[int] = []
captain: str # Note: no initial value!
class Starship:
stats: Dict[str, int] = {}
Just as for function annotations, the Python interpreter does not attach any
particular meaning to variable annotations and only stores them in the
__annotations__ attribute of a class or module.
In contrast to variable declarations in statically typed languages,
the goal of annotation syntax is to provide an easy way to specify structured
type metadata for third party tools and libraries via the abstract syntax tree
and the __annotations__ attribute.
See also
- PEP 526 – Syntax for variable annotations.
- PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach,
and Guido van Rossum. Implemented by Ivan Levkivskyi.
Tools that use or will use the new syntax:
mypy,
pytype, PyCharm, etc.
PEP 515: Underscores in Numeric Literals
PEP 515 adds the ability to use underscores in numeric literals for
improved readability. For example:
>>> 1_000_000_000_000_000
1000000000000000
>>> 0x_FF_FF_FF_FF
4294967295
Single underscores are allowed between digits and after any base
specifier. Leading, trailing, or multiple underscores in a row are not
allowed.
The string formatting language also now has support
for the '_' option to signal the use of an underscore for a thousands
separator for floating point presentation types and for integer
presentation type 'd'. For integer presentation types 'b',
'o', 'x', and 'X', underscores will be inserted every 4
digits:
>>> '{:_}'.format(1000000)
'1_000_000'
>>> '{:_x}'.format(0xFFFFFFFF)
'ffff_ffff'
See also
- PEP 515 – Underscores in Numeric Literals
- PEP written by Georg Brandl and Serhiy Storchaka.
PEP 525: Asynchronous Generators
PEP 492 introduced support for native coroutines and async / await
syntax to Python 3.5. A notable limitation of the Python 3.5 implementation
is that it was not possible to use await and yield in the same
function body. In Python 3.6 this restriction has been lifted, making it
possible to define asynchronous generators:
async def ticker(delay, to):
"""Yield numbers from 0 to *to* every *delay* seconds."""
for i in range(to):
yield i
await asyncio.sleep(delay)
The new syntax allows for faster and more concise code.
See also
- PEP 525 – Asynchronous Generators
- PEP written and implemented by Yury Selivanov.
PEP 530: Asynchronous Comprehensions
PEP 530 adds support for using async for in list, set, dict
comprehensions and generator expressions:
result = [i async for i in aiter() if i % 2]
Additionally, await expressions are supported in all kinds
of comprehensions:
result = [await fun() for fun in funcs if await condition()]
See also
- PEP 530 – Asynchronous Comprehensions
- PEP written and implemented by Yury Selivanov.
PEP 487: Simpler customization of class creation
It is now possible to customize subclass creation without using a metaclass.
The new __init_subclass__ classmethod will be called on the base class
whenever a new subclass is created:
class PluginBase:
subclasses = []
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
cls.subclasses.append(cls)
class Plugin1(PluginBase):
pass
class Plugin2(PluginBase):
pass
In order to allow zero-argument super() calls to work correctly from
__init_subclass__() implementations, custom metaclasses must
ensure that the new __classcell__ namespace entry is propagated to
type.__new__ (as described in Creating the class object).
PEP 487: Descriptor Protocol Enhancements
PEP 487 extends the descriptor protocol to include the new optional
__set_name__() method. Whenever a new class is defined, the new
method will be called on all descriptors included in the definition, providing
them with a reference to the class being defined and the name given to the
descriptor within the class namespace. In other words, instances of
descriptors can now know the attribute name of the descriptor in the
owner class:
class IntField:
def __get__(self, instance, owner):
return instance.__dict__[self.name]
def __set__(self, instance, value):
if not isinstance(value, int):
raise ValueError(f'expecting integer in {self.name}')
instance.__dict__[self.name] = value
# this is the new initializer:
def __set_name__(self, owner, name):
self.name = name
class Model:
int_field = IntField()
PEP 519: Adding a file system path protocol
File system paths have historically been represented as str
or bytes objects. This has led to people who write code which
operate on file system paths to assume that such objects are only one
of those two types (an int representing a file descriptor
does not count as that is not a file path). Unfortunately that
assumption prevents alternative object representations of file system
paths like pathlib from working with pre-existing code,
including Python’s standard library.
To fix this situation, a new interface represented by
os.PathLike has been defined. By implementing the
__fspath__() method, an object signals that it
represents a path. An object can then provide a low-level
representation of a file system path as a str or
bytes object. This means an object is considered
path-like if it implements
os.PathLike or is a str or bytes object
which represents a file system path. Code can use os.fspath(),
os.fsdecode(), or os.fsencode() to explicitly get a
str and/or bytes representation of a path-like
object.
The built-in open() function has been updated to accept
os.PathLike objects, as have all relevant functions in the
os and os.path modules, and most other functions and
classes in the standard library. The os.DirEntry class
and relevant classes in pathlib have also been updated to
implement os.PathLike.
The hope is that updating the fundamental functions for operating
on file system paths will lead to third-party code to implicitly
support all path-like objects without any
code changes, or at least very minimal ones (e.g. calling
os.fspath() at the beginning of code before operating on a
path-like object).
Here are some examples of how the new interface allows for
pathlib.Path to be used more easily and transparently with
pre-existing code:
>>> import pathlib
>>> with open(pathlib.Path("README")) as f:
... contents = f.read()
...
>>> import os.path
>>> os.path.splitext(pathlib.Path("some_file.txt"))
('some_file', '.txt')
>>> os.path.join("/a/b", pathlib.Path("c"))
'/a/b/c'
>>> import os
>>> os.fspath(pathlib.Path("some_file.txt"))
'some_file.txt'
(Implemented by Brett Cannon, Ethan Furman, Dusty Phillips, and Jelle Zijlstra.)
See also
- PEP 519 – Adding a file system path protocol
- PEP written by Brett Cannon and Koos Zevenhoven.
PEP 495: Local Time Disambiguation
In most world locations, there have been and will be times when local clocks
are moved back. In those times, intervals are introduced in which local
clocks show the same time twice in the same day. In these situations, the
information displayed on a local clock (or stored in a Python datetime
instance) is insufficient to identify a particular moment in time.
PEP 495 adds the new fold attribute to instances of
datetime.datetime and datetime.time classes to differentiate
between two moments in time for which local times are the same:
>>> u0 = datetime(2016, 11, 6, 4, tzinfo=timezone.utc)
>>> for i in range(4):
... u = u0 + i*HOUR
... t = u.astimezone(Eastern)
... print(u.time(), 'UTC =', t.time(), t.tzname(), t.fold)
...
04:00:00 UTC = 00:00:00 EDT 0
05:00:00 UTC = 01:00:00 EDT 0
06:00:00 UTC = 01:00:00 EST 1
07:00:00 UTC = 02:00:00 EST 0
The values of the fold attribute have the
value 0 for all instances except those that represent the second
(chronologically) moment in time in an ambiguous case.
See also
- PEP 495 – Local Time Disambiguation
- PEP written by Alexander Belopolsky and Tim Peters, implementation
by Alexander Belopolsky.
PEP 529: Change Windows filesystem encoding to UTF-8
Representing filesystem paths is best performed with str (Unicode) rather than
bytes. However, there are some situations where using bytes is sufficient and
correct.
Prior to Python 3.6, data loss could result when using bytes paths on Windows.
With this change, using bytes to represent paths is now supported on Windows,
provided those bytes are encoded with the encoding returned by
sys.getfilesystemencoding(), which now defaults to 'utf-8'.
Applications that do not use str to represent paths should use
os.fsencode() and os.fsdecode() to ensure their bytes are
correctly encoded. To revert to the previous behaviour, set
PYTHONLEGACYWINDOWSFSENCODING or call
sys._enablelegacywindowsfsencoding().
See PEP 529 for more information and discussion of code modifications that
may be required.
PEP 528: Change Windows console encoding to UTF-8
The default console on Windows will now accept all Unicode characters and
provide correctly read str objects to Python code. sys.stdin,
sys.stdout and sys.stderr now default to utf-8 encoding.
This change only applies when using an interactive console, and not when
redirecting files or pipes. To revert to the previous behaviour for interactive
console use, set PYTHONLEGACYWINDOWSSTDIO.
See also
- PEP 528 – Change Windows console encoding to UTF-8
- PEP written and implemented by Steve Dower.
PEP 520: Preserving Class Attribute Definition Order
Attributes in a class definition body have a natural ordering: the same
order in which the names appear in the source. This order is now
preserved in the new class’s __dict__ attribute.
Also, the effective default class execution namespace (returned from
type.__prepare__()) is now an insertion-order-preserving
mapping.
See also
- PEP 520 – Preserving Class Attribute Definition Order
- PEP written and implemented by Eric Snow.
PEP 468: Preserving Keyword Argument Order
**kwargs in a function signature is now guaranteed to be an
insertion-order-preserving mapping.
See also
- PEP 468 – Preserving Keyword Argument Order
- PEP written and implemented by Eric Snow.
New dict implementation
The dict type now uses a “compact” representation
based on a proposal by Raymond Hettinger
which was first implemented by PyPy.
The memory usage of the new dict() is between 20% and 25% smaller
compared to Python 3.5.
The order-preserving aspect of this new implementation is considered an
implementation detail and should not be relied upon (this may change in
the future, but it is desired to have this new dict implementation in
the language for a few releases before changing the language spec to mandate
order-preserving semantics for all current and future Python
implementations; this also helps preserve backwards-compatibility
with older versions of the language where random iteration order is
still in effect, e.g. Python 3.5).
(Contributed by INADA Naoki in bpo-27350. Idea
originally suggested by Raymond Hettinger.)
PEP 523: Adding a frame evaluation API to CPython
While Python provides extensive support to customize how code
executes, one place it has not done so is in the evaluation of frame
objects. If you wanted some way to intercept frame evaluation in
Python there really wasn’t any way without directly manipulating
function pointers for defined functions.
PEP 523 changes this by providing an API to make frame
evaluation pluggable at the C level. This will allow for tools such
as debuggers and JITs to intercept frame evaluation before the
execution of Python code begins. This enables the use of alternative
evaluation implementations for Python code, tracking frame
evaluation, etc.
This API is not part of the limited C API and is marked as private to
signal that usage of this API is expected to be limited and only
applicable to very select, low-level use-cases. Semantics of the
API will change with Python as necessary.
See also
- PEP 523 – Adding a frame evaluation API to CPython
- PEP written by Brett Cannon and Dino Viehland.
PYTHONMALLOC environment variable
The new PYTHONMALLOC environment variable allows setting the Python
memory allocators and installing debug hooks.
It is now possible to install debug hooks on Python memory allocators on Python
compiled in release mode using PYTHONMALLOC=debug. Effects of debug hooks:
- Newly allocated memory is filled with the byte
0xCB
- Freed memory is filled with the byte
0xDB
- Detect violations of the Python memory allocator API. For example,
PyObject_Free() called on a memory block allocated by
PyMem_Malloc().
- Detect writes before the start of a buffer (buffer underflows)
- Detect writes after the end of a buffer (buffer overflows)
- Check that the GIL is held when allocator
functions of
PYMEM_DOMAIN_OBJ (ex: PyObject_Malloc()) and
PYMEM_DOMAIN_MEM (ex: PyMem_Malloc()) domains are called.
Checking if the GIL is held is also a new feature of Python 3.6.
See the PyMem_SetupDebugHooks() function for debug hooks on Python
memory allocators.
It is now also possible to force the usage of the malloc() allocator of
the C library for all Python memory allocations using PYTHONMALLOC=malloc.
This is helpful when using external memory debuggers like Valgrind on
a Python compiled in release mode.
On error, the debug hooks on Python memory allocators now use the
tracemalloc module to get the traceback where a memory block was
allocated.
Example of fatal error on buffer overflow using
python3.6 -X tracemalloc=5 (store 5 frames in traces):
Debug memory block at address p=0x7fbcd41666f8: API 'o'
4 bytes originally requested
The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
The 8 pad bytes at tail=0x7fbcd41666fc are not all FORBIDDENBYTE (0xfb):
at tail+0: 0x02 *** OUCH
at tail+1: 0xfb
at tail+2: 0xfb
at tail+3: 0xfb
at tail+4: 0xfb
at tail+5: 0xfb
at tail+6: 0xfb
at tail+7: 0xfb
The block was made by call #1233329 to debug malloc/realloc.
Data at p: 1a 2b 30 00
Memory block allocated at (most recent call first):
File "test/test_bytes.py", line 323
File "unittest/case.py", line 600
File "unittest/case.py", line 648
File "unittest/suite.py", line 122
File "unittest/suite.py", line 84
Fatal Python error: bad trailing pad byte
Current thread 0x00007fbcdbd32700 (most recent call first):
File "test/test_bytes.py", line 323 in test_hex
File "unittest/case.py", line 600 in run
File "unittest/case.py", line 648 in __call__
File "unittest/suite.py", line 122 in run
File "unittest/suite.py", line 84 in __call__
File "unittest/suite.py", line 122 in run
File "unittest/suite.py", line 84 in __call__
...
(Contributed by Victor Stinner in bpo-26516 and bpo-26564.)
DTrace and SystemTap probing support
Python can now be built --with-dtrace which enables static markers
for the following events in the interpreter:
- function call/return
- garbage collection started/finished
- line of code executed.
This can be used to instrument running interpreters in production,
without the need to recompile specific debug builds or providing
application-specific profiling/debugging code.
More details in Instrumenting CPython with DTrace and SystemTap.
The current implementation is tested on Linux and macOS. Additional
markers may be added in the future.
(Contributed by Łukasz Langa in bpo-21590, based on patches by
Jesús Cea Avión, David Malcolm, and Nikhil Benesch.)
Improved Modules
array
Exhausted iterators of array.array will now stay exhausted even
if the iterated array is extended. This is consistent with the behavior
of other mutable sequences.
Contributed by Serhiy Storchaka in bpo-26492.
ast
The new ast.Constant AST node has been added. It can be used
by external AST optimizers for the purposes of constant folding.
Contributed by Victor Stinner in bpo-26146.
asyncio
Starting with Python 3.6 the asyncio module is no longer provisional and its
API is considered stable.
Notable changes in the asyncio module since Python 3.5.0
(all backported to 3.5.x due to the provisional status):
- The
get_event_loop() function has been changed to
always return the currently running loop when called from coroutines
and callbacks.
(Contributed by Yury Selivanov in bpo-28613.)
- The
ensure_future() function and all functions that
use it, such as loop.run_until_complete(),
now accept all kinds of awaitable objects.
(Contributed by Yury Selivanov.)
- New
run_coroutine_threadsafe() function to submit
coroutines to event loops from other threads.
(Contributed by Vincent Michel.)
- New
Transport.is_closing()
method to check if the transport is closing or closed.
(Contributed by Yury Selivanov.)
- The
loop.create_server()
method can now accept a list of hosts.
(Contributed by Yann Sionneau.)
- New
loop.create_future()
method to create Future objects. This allows alternative event
loop implementations, such as
uvloop, to provide a faster
asyncio.Future implementation.
(Contributed by Yury Selivanov in bpo-27041.)
- New
loop.get_exception_handler()
method to get the current exception handler.
(Contributed by Yury Selivanov in bpo-27040.)
- New
StreamReader.readuntil()
method to read data from the stream until a separator bytes
sequence appears.
(Contributed by Mark Korenberg.)
- The performance of
StreamReader.readexactly()
has been improved.
(Contributed by Mark Korenberg in bpo-28370.)
- The
loop.getaddrinfo()
method is optimized to avoid calling the system getaddrinfo
function if the address is already resolved.
(Contributed by A. Jesse Jiryu Davis.)
- The
loop.stop()
method has been changed to stop the loop immediately after
the current iteration. Any new callbacks scheduled as a result
of the last iteration will be discarded.
(Contributed by Guido van Rossum in bpo-25593.)
Future.set_exception
will now raise TypeError when passed an instance of
the StopIteration exception.
(Contributed by Chris Angelico in bpo-26221.)
- New
loop.connect_accepted_socket()
method to be used by servers that accept connections outside of asyncio,
but that use asyncio to handle them.
(Contributed by Jim Fulton in bpo-27392.)
TCP_NODELAY flag is now set for all TCP transports by default.
(Contributed by Yury Selivanov in bpo-27456.)
- New
loop.shutdown_asyncgens()
to properly close pending asynchronous generators before closing the
loop.
(Contributed by Yury Selivanov in bpo-28003.)
Future and Task
classes now have an optimized C implementation which makes asyncio
code up to 30% faster.
(Contributed by Yury Selivanov and INADA Naoki in bpo-26081
and bpo-28544.)
binascii
The b2a_base64() function now accepts an optional newline
keyword argument to control whether the newline character is appended to the
return value.
(Contributed by Victor Stinner in bpo-25357.)
collections
The new Collection abstract base class has been
added to represent sized iterable container classes.
(Contributed by Ivan Levkivskyi, docs by Neil Girdhar in bpo-27598.)
The new Reversible abstract base class represents
iterable classes that also provide the __reversed__() method.
(Contributed by Ivan Levkivskyi in bpo-25987.)
The new AsyncGenerator abstract base class represents
asynchronous generators.
(Contributed by Yury Selivanov in bpo-28720.)
The namedtuple() function now accepts an optional
keyword argument module, which, when specified, is used for
the __module__ attribute of the returned named tuple class.
(Contributed by Raymond Hettinger in bpo-17941.)
The verbose and rename arguments for
namedtuple() are now keyword-only.
(Contributed by Raymond Hettinger in bpo-25628.)
Recursive collections.deque instances can now be pickled.
(Contributed by Serhiy Storchaka in bpo-26482.)
concurrent.futures
The ThreadPoolExecutor
class constructor now accepts an optional thread_name_prefix argument
to make it possible to customize the names of the threads created by the
pool.
(Contributed by Gregory P. Smith in bpo-27664.)
contextlib
The contextlib.AbstractContextManager class has been added to
provide an abstract base class for context managers. It provides a
sensible default implementation for __enter__() which returns
self and leaves __exit__() an abstract method. A matching
class has been added to the typing module as
typing.ContextManager.
(Contributed by Brett Cannon in bpo-25609.)
datetime
The datetime and time classes have
the new fold attribute used to disambiguate local time
when necessary. Many functions in the datetime have been
updated to support local time disambiguation.
See Local Time Disambiguation section for more
information.
(Contributed by Alexander Belopolsky in bpo-24773.)
The datetime.strftime() and
date.strftime() methods now support
ISO 8601 date directives %G, %u and %V.
(Contributed by Ashley Anderson in bpo-12006.)
The datetime.isoformat() function
now accepts an optional timespec argument that specifies the number
of additional components of the time value to include.
(Contributed by Alessandro Cucci and Alexander Belopolsky in bpo-19475.)
The datetime.combine() now
accepts an optional tzinfo argument.
(Contributed by Alexander Belopolsky in bpo-27661.)
decimal
New Decimal.as_integer_ratio()
method that returns a pair (n, d) of integers that represent the given
Decimal instance as a fraction, in lowest terms and
with a positive denominator:
>>> Decimal('-3.14').as_integer_ratio()
(-157, 50)
(Contributed by Stefan Krah amd Mark Dickinson in bpo-25928.)
distutils
The default_format attribute has been removed from
distutils.command.sdist.sdist and the formats
attribute defaults to ['gztar']. Although not anticipated,
any code relying on the presence of default_format may
need to be adapted. See bpo-27819 for more details.
email
The new email API, enabled via the policy keyword to various constructors, is
no longer provisional. The email documentation has been reorganized and
rewritten to focus on the new API, while retaining the old documentation for
the legacy API. (Contributed by R. David Murray in bpo-24277.)
The email.mime classes now all accept an optional policy keyword.
(Contributed by Berker Peksag in bpo-27331.)
The DecodedGenerator now supports the policy
keyword.
There is a new policy attribute,
message_factory, that controls what class is used
by default when the parser creates new message objects. For the
email.policy.compat32 policy this is Message,
for the new policies it is EmailMessage.
(Contributed by R. David Murray in bpo-20476.)
encodings
On Windows, added the 'oem' encoding to use CP_OEMCP, and the 'ansi'
alias for the existing 'mbcs' encoding, which uses the CP_ACP code page.
(Contributed by Steve Dower in bpo-27959.)
enum
Two new enumeration base classes have been added to the enum module:
Flag and IntFlags. Both are used to define
constants that can be combined using the bitwise operators.
(Contributed by Ethan Furman in bpo-23591.)
Many standard library modules have been updated to use the
IntFlags class for their constants.
The new enum.auto value can be used to assign values to enum
members automatically:
>>> from enum import Enum, auto
>>> class Color(Enum):
... red = auto()
... blue = auto()
... green = auto()
...
>>> list(Color)
[<Color.red: 1>, <Color.blue: 2>, <Color.green: 3>]
hashlib
hashlib supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2.
(Contributed by Christian Heimes in bpo-26470.)
BLAKE2 hash functions were added to the module. blake2b()
and blake2s() are always available and support the full
feature set of BLAKE2.
(Contributed by Christian Heimes in bpo-26798 based on code by
Dmitry Chestnykh and Samuel Neves. Documentation written by Dmitry Chestnykh.)
The SHA-3 hash functions sha3_224(), sha3_256(),
sha3_384(), sha3_512(), and SHAKE hash functions
shake_128() and shake_256() were added.
(Contributed by Christian Heimes in bpo-16113. Keccak Code Package
by Guido Bertoni, Joan Daemen, Michaël Peeters, Gilles Van Assche, and
Ronny Van Keer.)
The password-based key derivation function scrypt() is now
available with OpenSSL 1.1.0 and newer.
(Contributed by Christian Heimes in bpo-27928.)
idlelib and IDLE
The idlelib package is being modernized and refactored to make IDLE look and
work better and to make the code easier to understand, test, and improve. Part
of making IDLE look better, especially on Linux and Mac, is using ttk widgets,
mostly in the dialogs. As a result, IDLE no longer runs with tcl/tk 8.4. It
now requires tcl/tk 8.5 or 8.6. We recommend running the latest release of
either.
‘Modernizing’ includes renaming and consolidation of idlelib modules. The
renaming of files with partial uppercase names is similar to the renaming of,
for instance, Tkinter and TkFont to tkinter and tkinter.font in 3.0. As a
result, imports of idlelib files that worked in 3.5 will usually not work in
3.6. At least a module name change will be needed (see idlelib/README.txt),
sometimes more. (Name changes contributed by Al Swiegart and Terry Reedy in
bpo-24225. Most idlelib patches since have been and will be part of the
process.)
In compensation, the eventual result with be that some idlelib classes will be
easier to use, with better APIs and docstrings explaining them. Additional
useful information will be added to idlelib when available.
New in 3.6.2:
Multiple fixes for autocompletion. (Contributed by Louie Lu in bpo-15786.)
New in 3.6.3:
Module Browser (on the File menu, formerly called Class Browser),
now displays nested functions and classes in addition to top-level
functions and classes.
(Contributed by Guilherme Polo, Cheryl Sabella, and Terry Jan Reedy
in bpo-1612262.)
The IDLE features formerly implemented as extensions have been reimplemented
as normal features. Their settings have been moved from the Extensions tab
to other dialog tabs.
(Contributed by Charles Wohlganger and Terry Jan Reedy in bpo-27099.)
The Settings dialog (Options, Configure IDLE) has been partly rewritten
to improve both appearance and function.
(Contributed by Cheryl Sabella and Terry Jan Reedy in multiple issues.)
New in 3.6.4:
The font sample now includes a selection of non-Latin characters so that
users can better see the effect of selecting a particular font.
(Contributed by Terry Jan Reedy in bpo-13802.)
The sample can be edited to include other characters.
(Contributed by Serhiy Storchaka in bpo-31860.)
New in 3.6.6:
Editor code context option revised. Box displays all context lines up to
maxlines. Clicking on a context line jumps the editor to that line. Context
colors for custom themes is added to Highlights tab of Settings dialog.
(Contributed by Cheryl Sabella and Terry Jan Reedy in bpo-33642,
bpo-33768, and bpo-33679.)
On Windows, a new API call tells Windows that tk scales for DPI. On Windows
8.1+ or 10, with DPI compatibility properties of the Python binary
unchanged, and a monitor resolution greater than 96 DPI, this should
make text and lines sharper. It should otherwise have no effect.
(Contributed by Terry Jan Reedy in bpo-33656.)
New in 3.6.7:
Output over N lines (50 by default) is squeezed down to a button.
N can be changed in the PyShell section of the General page of the
Settings dialog. Fewer, but possibly extra long, lines can be squeezed by
right clicking on the output. Squeezed output can be expanded in place
by double-clicking the button or into the clipboard or a separate window
by right-clicking the button. (Contributed by Tal Einat in bpo-1529353.)
inspect
The inspect.signature() function now reports the
implicit .0 parameters generated by the compiler for comprehension and
generator expression scopes as if they were positional-only parameters called
implicit0. (Contributed by Jelle Zijlstra in bpo-19611.)
To reduce code churn when upgrading from Python 2.7 and the legacy
inspect.getargspec() API, the previously documented deprecation of
inspect.getfullargspec() has been reversed. While this function is
convenient for single/source Python 2/3 code bases, the richer
inspect.signature() interface remains the recommended approach for new
code. (Contributed by Nick Coghlan in bpo-27172)
json
json.load() and json.loads() now support binary input. Encoded
JSON should be represented using either UTF-8, UTF-16, or UTF-32.
(Contributed by Serhiy Storchaka in bpo-17909.)
math
The tau (τ) constant has been added to the math and cmath
modules.
(Contributed by Lisa Roach in bpo-12345, see PEP 628 for details.)
multiprocessing
Proxy Objects returned by
multiprocessing.Manager() can now be nested.
(Contributed by Davin Potts in bpo-6766.)
os
See the summary of PEP 519 for details on how the
os and os.path modules now support
path-like objects.
scandir() now supports bytes paths on Windows.
A new close() method allows explicitly closing a
scandir() iterator. The scandir() iterator now
supports the context manager protocol. If a scandir()
iterator is neither exhausted nor explicitly closed a ResourceWarning
will be emitted in its destructor.
(Contributed by Serhiy Storchaka in bpo-25994.)
On Linux, os.urandom() now blocks until the system urandom entropy pool
is initialized to increase the security. See the PEP 524 for the rationale.
The Linux getrandom() syscall (get random bytes) is now exposed as the new
os.getrandom() function.
(Contributed by Victor Stinner, part of the PEP 524)
pdb
The Pdb class constructor has a new optional readrc argument
to control whether .pdbrc files should be read.
pickle
Objects that need __new__ called with keyword arguments can now be pickled
using pickle protocols older than protocol version 4.
Protocol version 4 already supports this case. (Contributed by Serhiy
Storchaka in bpo-24164.)
pydoc
The pydoc module has learned to respect the MANPAGER
environment variable.
(Contributed by Matthias Klose in bpo-8637.)
help() and pydoc can now list named tuple fields in the
order they were defined rather than alphabetically.
(Contributed by Raymond Hettinger in bpo-24879.)
random
The new choices() function returns a list of elements of
specified size from the given population with optional weights.
(Contributed by Raymond Hettinger in bpo-18844.)
re
Added support of modifier spans in regular expressions. Examples:
'(?i:p)ython' matches 'python' and 'Python', but not 'PYTHON';
'(?i)g(?-i:v)r' matches 'GvR' and 'gvr', but not 'GVR'.
(Contributed by Serhiy Storchaka in bpo-433028.)
Match object groups can be accessed by __getitem__, which is
equivalent to group(). So mo['name'] is now equivalent to
mo.group('name'). (Contributed by Eric Smith in bpo-24454.)
Match objects now support
index-like objects as group
indices.
(Contributed by Jeroen Demeyer and Xiang Zhang in bpo-27177.)
readline
Added set_auto_history() to enable or disable
automatic addition of input to the history list. (Contributed by
Tyler Crompton in bpo-26870.)
rlcompleter
Private and special attribute names now are omitted unless the prefix starts
with underscores. A space or a colon is added after some completed keywords.
(Contributed by Serhiy Storchaka in bpo-25011 and bpo-25209.)
site
When specifying paths to add to sys.path in a .pth file,
you may now specify file paths on top of directories (e.g. zip files).
(Contributed by Wolfgang Langner in bpo-26587).
socket
The ioctl() function now supports the
SIO_LOOPBACK_FAST_PATH control code.
(Contributed by Daniel Stokes in bpo-26536.)
The getsockopt() constants SO_DOMAIN,
SO_PROTOCOL, SO_PEERSEC, and SO_PASSSEC are now supported.
(Contributed by Christian Heimes in bpo-26907.)
The setsockopt() now supports the
setsockopt(level, optname, None, optlen: int) form.
(Contributed by Christian Heimes in bpo-27744.)
The socket module now supports the address family
AF_ALG to interface with Linux Kernel crypto API. ALG_*,
SOL_ALG and sendmsg_afalg() were added.
(Contributed by Christian Heimes in bpo-27744 with support from
Victor Stinner.)
New Linux constants TCP_USER_TIMEOUT and TCP_CONGESTION were added.
(Contributed by Omar Sandoval, issue:26273).
ssl
ssl supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2.
(Contributed by Christian Heimes in bpo-26470.)
3DES has been removed from the default cipher suites and ChaCha20 Poly1305
cipher suites have been added.
(Contributed by Christian Heimes in bpo-27850 and bpo-27766.)
SSLContext has better default configuration for options
and ciphers.
(Contributed by Christian Heimes in bpo-28043.)
SSL session can be copied from one client-side connection to another
with the new SSLSession class. TLS session resumption can
speed up the initial handshake, reduce latency and improve performance
(Contributed by Christian Heimes in bpo-19500 based on a draft by
Alex Warhawk.)
The new get_ciphers() method can be used to
get a list of enabled ciphers in order of cipher priority.
All constants and flags have been converted to IntEnum and
IntFlags.
(Contributed by Christian Heimes in bpo-28025.)
Server and client-side specific TLS protocols for SSLContext
were added.
(Contributed by Christian Heimes in bpo-28085.)
Added SSLContext.post_handshake_auth to enable and
ssl.SSLSocket.verify_client_post_handshake() to initiate TLS 1.3
post-handshake authentication.
(Contributed by Christian Heimes in bpo-34670.)
struct
struct now supports IEEE 754 half-precision floats via the 'e'
format specifier.
(Contributed by Eli Stevens, Mark Dickinson in bpo-11734.)
subprocess
subprocess.Popen destructor now emits a ResourceWarning warning
if the child process is still running. Use the context manager protocol (with
proc: ...) or explicitly call the wait() method to
read the exit status of the child process. (Contributed by Victor Stinner in
bpo-26741.)
The subprocess.Popen constructor and all functions that pass arguments
through to it now accept encoding and errors arguments. Specifying either
of these will enable text mode for the stdin, stdout and stderr streams.
(Contributed by Steve Dower in bpo-6135.)
sys
The new getfilesystemencodeerrors() function returns the name of
the error mode used to convert between Unicode filenames and bytes filenames.
(Contributed by Steve Dower in bpo-27781.)
On Windows the return value of the getwindowsversion() function
now includes the platform_version field which contains the accurate major
version, minor version and build number of the current operating system,
rather than the version that is being emulated for the process
(Contributed by Steve Dower in bpo-27932.)
telnetlib
Telnet is now a context manager (contributed by
Stéphane Wirtel in bpo-25485).
time
The struct_time attributes tm_gmtoff and
tm_zone are now available on all platforms.
timeit
The new Timer.autorange() convenience
method has been added to call Timer.timeit()
repeatedly so that the total run time is greater or equal to 200 milliseconds.
(Contributed by Steven D’Aprano in bpo-6422.)
timeit now warns when there is substantial (4x) variance
between best and worst times.
(Contributed by Serhiy Storchaka in bpo-23552.)
tkinter
Added methods trace_add(),
trace_remove() and trace_info()
in the tkinter.Variable class. They replace old methods
trace_variable(), trace(),
trace_vdelete() and
trace_vinfo() that use obsolete Tcl commands and might
not work in future versions of Tcl.
(Contributed by Serhiy Storchaka in bpo-22115).
traceback
Both the traceback module and the interpreter’s builtin exception display now
abbreviate long sequences of repeated lines in tracebacks as shown in the
following example:
>>> def f(): f()
...
>>> f()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in f
File "<stdin>", line 1, in f
File "<stdin>", line 1, in f
[Previous line repeated 995 more times]
RecursionError: maximum recursion depth exceeded
(Contributed by Emanuel Barry in bpo-26823.)
tracemalloc
The tracemalloc module now supports tracing memory allocations in
multiple different address spaces.
The new DomainFilter filter class has been added
to filter block traces by their address space (domain).
(Contributed by Victor Stinner in bpo-26588.)
typing
Since the typing module is provisional,
all changes introduced in Python 3.6 have also been
backported to Python 3.5.x.
The typing module has a much improved support for generic type
aliases. For example Dict[str, Tuple[S, T]] is now a valid
type annotation.
(Contributed by Guido van Rossum in Github #195.)
The typing.ContextManager class has been added for
representing contextlib.AbstractContextManager.
(Contributed by Brett Cannon in bpo-25609.)
The typing.Collection class has been added for
representing collections.abc.Collection.
(Contributed by Ivan Levkivskyi in bpo-27598.)
The typing.ClassVar type construct has been added to
mark class variables. As introduced in PEP 526, a variable annotation
wrapped in ClassVar indicates that a given attribute is intended to be used as
a class variable and should not be set on instances of that class.
(Contributed by Ivan Levkivskyi in Github #280.)
A new TYPE_CHECKING constant that is assumed to be
True by the static type chekers, but is False at runtime.
(Contributed by Guido van Rossum in Github #230.)
A new NewType() helper function has been added to create
lightweight distinct types for annotations:
from typing import NewType
UserId = NewType('UserId', int)
some_id = UserId(524313)
The static type checker will treat the new type as if it were a subclass
of the original type. (Contributed by Ivan Levkivskyi in Github #189.)
unittest.mock
The Mock class has the following improvements:
urllib.request
If a HTTP request has a file or iterable body (other than a
bytes object) but no Content-Length header, rather than
throwing an error, AbstractHTTPHandler now
falls back to use chunked transfer encoding.
(Contributed by Demian Brecht and Rolf Krahl in bpo-12319.)
urllib.robotparser
RobotFileParser now supports the Crawl-delay and
Request-rate extensions.
(Contributed by Nikolay Bogoychev in bpo-16099.)
venv
venv accepts a new parameter --prompt. This parameter provides an
alternative prefix for the virtual environment. (Proposed by Łukasz Balcerzak
and ported to 3.6 by Stéphane Wirtel in bpo-22829.)
warnings
A new optional source parameter has been added to the
warnings.warn_explicit() function: the destroyed object which emitted a
ResourceWarning. A source attribute has also been added to
warnings.WarningMessage (contributed by Victor Stinner in
bpo-26568 and bpo-26567).
When a ResourceWarning warning is logged, the tracemalloc module is now
used to try to retrieve the traceback where the destroyed object was allocated.
Example with the script example.py:
import warnings
def func():
return open(__file__)
f = func()
f = None
Output of the command python3.6 -Wd -X tracemalloc=5 example.py:
example.py:7: ResourceWarning: unclosed file <_io.TextIOWrapper name='example.py' mode='r' encoding='UTF-8'>
f = None
Object allocated at (most recent call first):
File "example.py", lineno 4
return open(__file__)
File "example.py", lineno 6
f = func()
The “Object allocated at” traceback is new and is only displayed if
tracemalloc is tracing Python memory allocations and if the
warnings module was already imported.
winreg
Added the 64-bit integer type REG_QWORD.
(Contributed by Clement Rouault in bpo-23026.)
xmlrpc.client
The xmlrpc.client module now supports unmarshalling
additional data types used by the Apache XML-RPC implementation
for numerics and None.
(Contributed by Serhiy Storchaka in bpo-26885.)