This article explains the new features in Python 3.4, compared to 3.3.
Python 3.4 was released on March 16, 2014. For full details, see the
changelog.
Improved Modules
abc
New function abc.get_cache_token() can be used to know when to invalidate
caches that are affected by changes in the object graph. (Contributed
by Łukasz Langa in bpo-16832.)
New class ABC has ABCMeta as its meta class.
Using ABC as a base class has essentially the same effect as specifying
metaclass=abc.ABCMeta, but is simpler to type and easier to read.
(Contributed by Bruno Dupuis in bpo-16049.)
argparse
The FileType class now accepts encoding and
errors arguments, which are passed through to open(). (Contributed
by Lucas Maystre in bpo-11175.)
audioop
audioop now supports 24-bit samples. (Contributed by Serhiy Storchaka
in bpo-12866.)
New byteswap() function converts big-endian samples to
little-endian and vice versa. (Contributed by Serhiy Storchaka in
bpo-19641.)
All audioop functions now accept any bytes-like object. Strings
are not accepted: they didn’t work before, now they raise an error right away.
(Contributed by Serhiy Storchaka in bpo-16685.)
base64
The encoding and decoding functions in base64 now accept any
bytes-like object in cases where it previously required a
bytes or bytearray instance. (Contributed by Nick Coghlan in
bpo-17839.)
New functions a85encode(), a85decode(),
b85encode(), and b85decode() provide the ability to
encode and decode binary data from and to Ascii85 and the git/mercurial
Base85 formats, respectively. The a85 functions have options that can
be used to make them compatible with the variants of the Ascii85 encoding,
including the Adobe variant. (Contributed by Martin Morrison, the Mercurial
project, Serhiy Storchaka, and Antoine Pitrou in bpo-17618.)
collections
The ChainMap.new_child() method now accepts an m argument specifying
the child map to add to the chain. This allows an existing mapping and/or a
custom mapping type to be used for the child. (Contributed by Vinay Sajip in
bpo-16613.)
colorsys
The number of digits in the coefficients for the RGB — YIQ conversions have
been expanded so that they match the FCC NTSC versions. The change in
results should be less than 1% and may better match results found elsewhere.
(Contributed by Brian Landers and Serhiy Storchaka in bpo-14323.)
contextlib
The new contextlib.suppress context manager helps to clarify the
intent of code that deliberately suppresses exceptions from a single
statement. (Contributed by Raymond Hettinger in bpo-15806 and
Zero Piraeus in bpo-19266.)
The new contextlib.redirect_stdout() context manager makes it easier
for utility scripts to handle inflexible APIs that write their output to
sys.stdout and don’t provide any options to redirect it. Using the
context manager, the sys.stdout output can be redirected to any
other stream or, in conjunction with io.StringIO, to a string.
The latter can be especially useful, for example, to capture output
from a function that was written to implement a command line interface.
It is recommended only for utility scripts because it affects the
global state of sys.stdout. (Contributed by Raymond Hettinger
in bpo-15805.)
The contextlib documentation has also been updated to include a
discussion of the
differences between single use, reusable and reentrant context managers.
dbm
dbm.open() objects now support the context management protocol. When
used in a with statement, the close method of the database
object will be called automatically at the end of the block. (Contributed by
Claudiu Popa and Nick Coghlan in bpo-19282.)
dis
Functions show_code(), dis(), distb(), and
disassemble() now accept a keyword-only file argument that
controls where they write their output.
The dis module is now built around an Instruction class
that provides object oriented access to the details of each individual bytecode
operation.
A new method, get_instructions(), provides an iterator that emits
the Instruction stream for a given piece of Python code. Thus it is now
possible to write a program that inspects and manipulates a bytecode
object in ways different from those provided by the dis module
itself. For example:
>>> import dis
>>> for instr in dis.get_instructions(lambda x: x + 1):
... print(instr.opname)
LOAD_FAST
LOAD_CONST
BINARY_ADD
RETURN_VALUE
The various display tools in the dis module have been rewritten to use
these new components.
In addition, a new application-friendly class Bytecode provides
an object-oriented API for inspecting bytecode in both in human-readable form
and for iterating over instructions. The Bytecode constructor
takes the same arguments that get_instruction() does (plus an
optional current_offset), and the resulting object can be iterated to produce
Instruction objects. But it also has a dis
method, equivalent to calling dis on the constructor argument, but
returned as a multi-line string:
>>> bytecode = dis.Bytecode(lambda x: x + 1, current_offset=3)
>>> for instr in bytecode:
... print('{} ({})'.format(instr.opname, instr.opcode))
LOAD_FAST (124)
LOAD_CONST (100)
BINARY_ADD (23)
RETURN_VALUE (83)
>>> bytecode.dis().splitlines()
[' 1 0 LOAD_FAST 0 (x)',
' --> 3 LOAD_CONST 1 (1)',
' 6 BINARY_ADD',
' 7 RETURN_VALUE']
Bytecode also has a class method,
from_traceback(), that provides the ability to manipulate a
traceback (that is, print(Bytecode.from_traceback(tb).dis()) is equivalent
to distb(tb)).
(Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in bpo-11816
and Claudiu Popa in bpo-17916.)
New function stack_effect() computes the effect on the Python stack
of a given opcode and argument, information that is not otherwise available.
(Contributed by Larry Hastings in bpo-19722.)
doctest
A new option flag, FAIL_FAST, halts
test running as soon as the first failure is detected. (Contributed by R.
David Murray and Daniel Urban in bpo-16522.)
The doctest command line interface now uses argparse, and has two
new options, -o and -f. -o allows doctest options to be specified on the command line, and -f is a
shorthand for -o FAIL_FAST (to parallel the similar option supported by the
unittest CLI). (Contributed by R. David Murray in bpo-11390.)
doctest will now find doctests in extension module __doc__ strings.
(Contributed by Zachary Ware in bpo-3158.)
email
as_string() now accepts a policy argument to
override the default policy of the message when generating a string
representation of it. This means that as_string can now be used in more
circumstances, instead of having to create and use a generator in
order to pass formatting parameters to its flatten method. (Contributed by
R. David Murray in bpo-18600.)
New method as_bytes() added to produce a bytes
representation of the message in a fashion similar to how as_string
produces a string representation. It does not accept the maxheaderlen
argument, but does accept the unixfrom and policy arguments. The
Message __bytes__() method
calls it, meaning that bytes(mymsg) will now produce the intuitive
result: a bytes object containing the fully formatted message. (Contributed
by R. David Murray in bpo-18600.)
The Message.set_param() message now accepts a replace keyword argument.
When specified, the associated header will be updated without changing
its location in the list of headers. For backward compatibility, the default
is False. (Contributed by R. David Murray in bpo-18891.)
A pair of new subclasses of Message have been added
(EmailMessage and MIMEPart), along with a new sub-module,
contentmanager and a new policy attribute
content_manager. All documentation is
currently in the new module, which is being added as part of email’s new
provisional API. These classes provide a number of new methods that
make extracting content from and inserting content into email messages much
easier. For details, see the contentmanager documentation and
the email: Examples. These API additions complete the
bulk of the work that was planned as part of the email6 project. The currently
provisional API is scheduled to become final in Python 3.5 (possibly with a few
minor additions in the area of error handling). (Contributed by R. David
Murray in bpo-18891.)
filecmp
A new clear_cache() function provides the ability to clear the
filecmp comparison cache, which uses os.stat() information to
determine if the file has changed since the last compare. This can be used,
for example, if the file might have been changed and re-checked in less time
than the resolution of a particular filesystem’s file modification time field.
(Contributed by Mark Levitt in bpo-18149.)
New module attribute DEFAULT_IGNORES provides the list of
directories that are used as the default value for the ignore parameter of
the dircmp() function. (Contributed by Eli Bendersky in
bpo-15442.)
gc
New function get_stats() returns a list of three per-generation
dictionaries containing the collections statistics since interpreter startup.
(Contributed by Antoine Pitrou in bpo-16351.)
glob
A new function escape() provides a way to escape special characters
in a filename so that they do not become part of the globbing expansion but are
instead matched literally. (Contributed by Serhiy Storchaka in bpo-8402.)
hmac
hmac now accepts bytearray as well as bytes for the key
argument to the new() function, and the msg parameter to both the
new() function and the update() method now
accepts any type supported by the hashlib module. (Contributed
by Jonas Borgström in bpo-18240.)
The digestmod argument to the hmac.new() function may now be any hash
digest name recognized by hashlib. In addition, the current behavior in
which the value of digestmod defaults to MD5 is deprecated: in a
future version of Python there will be no default value. (Contributed by
Christian Heimes in bpo-17276.)
With the addition of block_size and name
attributes (and the formal documentation of the digest_size
attribute), the hmac module now conforms fully to the PEP 247 API.
(Contributed by Christian Heimes in bpo-18775.)
html
New function unescape() function converts HTML5 character references to
the corresponding Unicode characters. (Contributed by Ezio Melotti in
bpo-2927.)
HTMLParser accepts a new keyword argument
convert_charrefs that, when True, automatically converts all character
references. For backward-compatibility, its value defaults to False, but
it will change to True in a future version of Python, so you are invited to
set it explicitly and update your code to use this new feature. (Contributed
by Ezio Melotti in bpo-13633.)
The strict argument of HTMLParser is now deprecated.
(Contributed by Ezio Melotti in bpo-15114.)
http
send_error() now accepts an
optional additional explain parameter which can be used to provide an
extended error description, overriding the hardcoded default if there is one.
This extended error description will be formatted using the
error_message_format attribute and sent as the body
of the error response. (Contributed by Karl Cow in bpo-12921.)
The http.server command line interface now has
a -b/--bind option that causes the server to listen on a specific address.
(Contributed by Malte Swart in bpo-17764.)
idlelib and IDLE
Since idlelib implements the IDLE shell and editor and is not intended for
import by other programs, it gets improvements with every release. See
Lib/idlelib/NEWS.txt for a cumulative list of changes since 3.3.0,
as well as changes made in future 3.4.x releases. This file is also available
from the IDLE dialog.
inspect
The inspect module now offers a basic command line interface to quickly display source code and other
information for modules, classes and functions. (Contributed by Claudiu Popa
and Nick Coghlan in bpo-18626.)
unwrap() makes it easy to unravel wrapper function chains
created by functools.wraps() (and any other API that sets the
__wrapped__ attribute on a wrapper function). (Contributed by
Daniel Urban, Aaron Iles and Nick Coghlan in bpo-13266.)
As part of the implementation of the new enum module, the
inspect module now has substantially better support for custom
__dir__ methods and dynamic class attributes provided through
metaclasses. (Contributed by Ethan Furman in bpo-18929 and
bpo-19030.)
getfullargspec() and getargspec()
now use the signature() API. This allows them to
support a much broader range of callables, including those with
__signature__ attributes, those with metadata provided by argument
clinic, functools.partial() objects and more. Note that, unlike
signature(), these functions still ignore __wrapped__
attributes, and report the already bound first argument for bound methods,
so it is still necessary to update your code to use
signature() directly if those features are desired.
(Contributed by Yury Selivanov in bpo-17481.)
signature() now supports duck types of CPython functions,
which adds support for functions compiled with Cython. (Contributed
by Stefan Behnel and Yury Selivanov in bpo-17159.)
ipaddress
ipaddress was added to the standard library in Python 3.3 as a
provisional API. With the release of Python 3.4, this qualification
has been removed: ipaddress is now considered a stable API, covered
by the normal standard library requirements to maintain backwards
compatibility.
A new is_global property is True if
an address is globally routeable. (Contributed by Peter Moody in
bpo-17400.)
logging
The TimedRotatingFileHandler has a new atTime
parameter that can be used to specify the time of day when rollover should
happen. (Contributed by Ronald Oussoren in bpo-9556.)
SocketHandler and
DatagramHandler now support Unix domain sockets (by
setting port to None). (Contributed by Vinay Sajip in commit
ce46195b56a9.)
fileConfig() now accepts a
configparser.RawConfigParser subclass instance for the fname
parameter. This facilitates using a configuration file when logging
configuration is just a part of the overall application configuration, or where
the application modifies the configuration before passing it to
fileConfig(). (Contributed by Vinay Sajip in
bpo-16110.)
Logging configuration data received from a socket via the
logging.config.listen() function can now be validated before being
processed by supplying a verification function as the argument to the new
verify keyword argument. (Contributed by Vinay Sajip in bpo-15452.)
marshal
The default marshal version has been bumped to 3. The code implementing
the new version restores the Python2 behavior of recording only one copy of
interned strings and preserving the interning on deserialization, and extends
this “one copy” ability to any object type (including handling recursive
references). This reduces both the size of .pyc files and the amount of
memory a module occupies in memory when it is loaded from a .pyc (or
.pyo) file. (Contributed by Kristján Valur Jónsson in bpo-16475,
with additional speedups by Antoine Pitrou in bpo-19219.)
mmap
mmap objects can now be weakrefed. (Contributed by Valerie Lambert in
bpo-4885.)
multiprocessing
On Unix two new start methods,
spawn and forkserver, have been added for starting processes using
multiprocessing. These make the mixing of processes with threads more
robust, and the spawn method matches the semantics that multiprocessing has
always used on Windows. New function
get_all_start_methods() reports all start methods
available on the platform, get_start_method() reports
the current start method, and set_start_method() sets
the start method. (Contributed by Richard Oudkerk in bpo-8713.)
multiprocessing also now has the concept of a context, which
determines how child processes are created. New function
get_context() returns a context that uses a specified
start method. It has the same API as the multiprocessing module itself,
so you can use it to create Pools and other
objects that will operate within that context. This allows a framework and an
application or different parts of the same application to use multiprocessing
without interfering with each other. (Contributed by Richard Oudkerk in
bpo-18999.)
Except when using the old fork start method, child processes no longer
inherit unneeded handles/file descriptors from their parents (part of
bpo-8713).
multiprocessing now relies on runpy (which implements the
-m switch) to initialise __main__ appropriately in child processes
when using the spawn or forkserver start methods. This resolves some
edge cases where combining multiprocessing, the -m command line switch,
and explicit relative imports could cause obscure failures in child
processes. (Contributed by Nick Coghlan in bpo-19946.)
operator
New function length_hint() provides an implementation of the
specification for how the __length_hint__() special method should
be used, as part of the PEP 424 formal specification of this language
feature. (Contributed by Armin Ronacher in bpo-16148.)
There is now a pure-python version of the operator module available for
reference and for use by alternate implementations of Python. (Contributed by
Zachary Ware in bpo-16694.)
pdb
pdb has been enhanced to handle generators, yield, and
yield from in a more useful fashion. This is especially helpful when
debugging asyncio based programs. (Contributed by Andrew Svetlov and
Xavier de Gaye in bpo-16596.)
The print command has been removed from pdb, restoring access to the
Python print() function from the pdb command line. Python2’s pdb did
not have a print command; instead, entering print executed the
print statement. In Python3 print was mistakenly made an alias for the
pdb p command. p, however, prints the repr of its argument,
not the str like the Python2 print command did. Worse, the Python3
pdb print command shadowed the Python3 print function, making it
inaccessible at the pdb prompt. (Contributed by Connor Osborn in
bpo-18764.)
pickle
pickle now supports (but does not use by default) a new pickle protocol,
protocol 4. This new protocol addresses a number of issues that were present
in previous protocols, such as the serialization of nested classes, very large
strings and containers, and classes whose __new__() method takes
keyword-only arguments. It also provides some efficiency improvements.
See also
- PEP 3154 – Pickle protocol 4
- PEP written by Antoine Pitrou and implemented by Alexandre Vassalotti.
plistlib
plistlib now has an API that is similar to the standard pattern for
stdlib serialization protocols, with new load(),
dump(), loads(), and dumps()
functions. (The older API is now deprecated.) In addition to the already
supported XML plist format (FMT_XML), it also now supports
the binary plist format (FMT_BINARY). (Contributed by Ronald
Oussoren and others in bpo-14455.)
poplib
Two new methods have been added to poplib: capa(),
which returns the list of capabilities advertised by the POP server, and
stls(), which switches a clear-text POP3 session into an
encrypted POP3 session if the POP server supports it. (Contributed by Lorenzo
Catucci in bpo-4473.)
pprint
The pprint module’s PrettyPrinter class and its
pformat(), and pprint() functions have a new
option, compact, that controls how the output is formatted. Currently
setting compact to True means that sequences will be printed with as many
sequence elements as will fit within width on each (indented) line.
(Contributed by Serhiy Storchaka in bpo-19132.)
Long strings are now wrapped using Python’s normal line continuation
syntax. (Contributed by Antoine Pitrou in bpo-17150.)
pty
pty.spawn() now returns the status value from os.waitpid() on
the child process, instead of None. (Contributed by Gregory P. Smith.)
pydoc
The pydoc module is now based directly on the inspect.signature()
introspection API, allowing it to provide signature information for a wider
variety of callable objects. This change also means that __wrapped__
attributes are now taken into account when displaying help information.
(Contributed by Larry Hastings in bpo-19674.)
The pydoc module no longer displays the self parameter for
already bound methods. Instead, it aims to always display the exact current
signature of the supplied callable. (Contributed by Larry Hastings in
bpo-20710.)
In addition to the changes that have been made to pydoc directly,
its handling of custom __dir__ methods and various descriptor
behaviours has also been improved substantially by the underlying changes in
the inspect module.
As the help() builtin is based on pydoc, the above changes also
affect the behaviour of help().
re
New fullmatch() function and regex.fullmatch() method anchor
the pattern at both ends of the string to match. This provides a way to be
explicit about the goal of the match, which avoids a class of subtle bugs where
$ characters get lost during code changes or the addition of alternatives
to an existing regular expression. (Contributed by Matthew Barnett in
bpo-16203.)
The repr of regex objects now includes the pattern
and the flags; the repr of match objects now
includes the start, end, and the part of the string that matched. (Contributed
by Hugo Lopes Tavares and Serhiy Storchaka in bpo-13592 and
bpo-17087.)
resource
New prlimit() function, available on Linux platforms with a
kernel version of 2.6.36 or later and glibc of 2.13 or later, provides the
ability to query or set the resource limits for processes other than the one
making the call. (Contributed by Christian Heimes in bpo-16595.)
On Linux kernel version 2.6.36 or later, there are also some new
Linux specific constants: RLIMIT_MSGQUEUE,
RLIMIT_NICE, RLIMIT_RTPRIO,
RLIMIT_RTTIME, and RLIMIT_SIGPENDING.
(Contributed by Christian Heimes in bpo-19324.)
On FreeBSD version 9 and later, there some new FreeBSD specific constants:
RLIMIT_SBSIZE, RLIMIT_SWAP, and
RLIMIT_NPTS. (Contributed by Claudiu Popa in
bpo-19343.)
select
epoll objects now support the context management protocol.
When used in a with statement, the close()
method will be called automatically at the end of the block. (Contributed
by Serhiy Storchaka in bpo-16488.)
devpoll objects now have fileno() and
close() methods, as well as a new attribute
closed. (Contributed by Victor Stinner in
bpo-18794.)
shelve
Shelf instances may now be used in with statements,
and will be automatically closed at the end of the with block.
(Contributed by Filip Gruszczyński in bpo-13896.)
shutil
copyfile() now raises a specific Error subclass,
SameFileError, when the source and destination are the same
file, which allows an application to take appropriate action on this specific
error. (Contributed by Atsuo Ishimoto and Hynek Schlawack in
bpo-1492704.)
smtpd
The SMTPServer and SMTPChannel classes now
accept a map keyword argument which, if specified, is passed in to
asynchat.async_chat as its map argument. This allows an application
to avoid affecting the global socket map. (Contributed by Vinay Sajip in
bpo-11959.)
smtplib
SMTPException is now a subclass of OSError, which allows
both socket level errors and SMTP protocol level errors to be caught in one
try/except statement by code that only cares whether or not an error occurred.
(Contributed by Ned Jackson Lovely in bpo-2118.)
socket
The socket module now supports the CAN_BCM protocol on
platforms that support it. (Contributed by Brian Thorne in bpo-15359.)
Socket objects have new methods to get or set their inheritable flag, get_inheritable() and
set_inheritable().
The socket.AF_* and socket.SOCK_* constants are now enumeration values
using the new enum module. This allows meaningful names to be printed
during debugging, instead of integer “magic numbers”.
The AF_LINK constant is now available on BSD and OSX.
inet_pton() and inet_ntop() are now supported
on Windows. (Contributed by Atsuo Ishimoto in bpo-7171.)
ssl
PROTOCOL_TLSv1_1 and PROTOCOL_TLSv1_2 (TLSv1.1 and
TLSv1.2 support) have been added; support for these protocols is only available if
Python is linked with OpenSSL 1.0.1 or later. (Contributed by Michele Orrù and
Antoine Pitrou in bpo-16692.)
New function create_default_context() provides a standard way to
obtain an SSLContext whose settings are intended to be a
reasonable balance between compatibility and security. These settings are
more stringent than the defaults provided by the SSLContext
constructor, and may be adjusted in the future, without prior deprecation, if
best-practice security requirements change. The new recommended best
practice for using stdlib libraries that support SSL is to use
create_default_context() to obtain an SSLContext
object, modify it if needed, and then pass it as the context argument
of the appropriate stdlib API. (Contributed by Christian Heimes
in bpo-19689.)
SSLContext method load_verify_locations()
accepts a new optional argument cadata, which can be used to provide PEM or
DER encoded certificates directly via strings or bytes, respectively.
(Contributed by Christian Heimes in bpo-18138.)
New function get_default_verify_paths() returns
a named tuple of the paths and environment variables that the
set_default_verify_paths() method uses to set
OpenSSL’s default cafile and capath. This can be an aid in
debugging default verification issues. (Contributed by Christian Heimes
in bpo-18143.)
SSLContext has a new method,
cert_store_stats(), that reports the number of loaded
X.509 certs, X.509 CA certs, and certificate revocation lists (crls), as well as a get_ca_certs() method that returns a
list of the loaded CA certificates. (Contributed by Christian Heimes in
bpo-18147.)
If OpenSSL 0.9.8 or later is available, SSLContext has a new
attribute verify_flags that can be used to control the
certificate verification process by setting it to some combination of the new
constants VERIFY_DEFAULT, VERIFY_CRL_CHECK_LEAF,
VERIFY_CRL_CHECK_CHAIN, or VERIFY_X509_STRICT.
OpenSSL does not do any CRL verification by default. (Contributed by
Christien Heimes in bpo-8813.)
New SSLContext method load_default_certs()
loads a set of default “certificate authority” (CA) certificates from default
locations, which vary according to the platform. It can be used to load both
TLS web server authentication certificates
(purpose=SERVER_AUTH) for a client to use to verify a
server, and certificates for a server to use in verifying client certificates
(purpose=CLIENT_AUTH). (Contributed by Christian
Heimes in bpo-19292.)
Two new windows-only functions, enum_certificates() and
enum_crls() provide the ability to retrieve certificates,
certificate information, and CRLs from the Windows cert store. (Contributed
by Christian Heimes in bpo-17134.)
Support for server-side SNI (Server Name Indication) using the new
ssl.SSLContext.set_servername_callback() method.
(Contributed by Daniel Black in bpo-8109.)
The dictionary returned by SSLSocket.getpeercert() contains additional
X509v3 extension items: crlDistributionPoints, calIssuers, and
OCSP URIs. (Contributed by Christian Heimes in bpo-18379.)
stat
The stat module is now backed by a C implementation in _stat. A C
implementation is required as most of the values aren’t standardized and
are platform-dependent. (Contributed by Christian Heimes in bpo-11016.)
The module supports new ST_MODE flags, S_IFDOOR,
S_IFPORT, and S_IFWHT. (Contributed by
Christian Hiemes in bpo-11016.)
struct
New function iter_unpack and a new
struct.Struct.iter_unpack() method on compiled formats provide streamed
unpacking of a buffer containing repeated instances of a given format of data.
(Contributed by Antoine Pitrou in bpo-17804.)
subprocess
check_output() now accepts an input argument that can
be used to provide the contents of stdin for the command that is run.
(Contributed by Zack Weinberg in bpo-16624.)
getstatus() and getstatusoutput() now
work on Windows. This change was actually inadvertently made in 3.3.4.
(Contributed by Tim Golden in bpo-10197.)
sunau
The getparams() method now returns a namedtuple rather than a
plain tuple. (Contributed by Claudiu Popa in bpo-18901.)
sunau.open() now supports the context management protocol: when used in a
with block, the close method of the returned object will be
called automatically at the end of the block. (Contributed by Serhiy Storchaka
in bpo-18878.)
AU_write.setsampwidth() now supports 24 bit samples, thus adding
support for writing 24 sample using the module. (Contributed by
Serhiy Storchaka in bpo-19261.)
The writeframesraw() and
writeframes() methods now accept any bytes-like
object. (Contributed by Serhiy Storchaka in bpo-8311.)
sys
New function sys.getallocatedblocks() returns the current number of
blocks allocated by the interpreter. (In CPython with the default
--with-pymalloc setting, this is allocations made through the
PyObject_Malloc() API.) This can be useful for tracking memory leaks,
especially if automated via a test suite. (Contributed by Antoine Pitrou
in bpo-13390.)
When the Python interpreter starts in interactive mode, it checks for an __interactivehook__ attribute
on the sys module. If the attribute exists, its value is called with no
arguments just before interactive mode is started. The check is made after the
PYTHONSTARTUP file is read, so it can be set there. The site
module sets it to a function that enables tab
completion and history saving (in ~/.python-history) if the platform
supports readline. If you do not want this (new) behavior, you can
override it in PYTHONSTARTUP, sitecustomize, or
usercustomize by deleting this attribute from sys (or setting it
to some other callable). (Contributed by Éric Araujo and Antoine Pitrou in
bpo-5845.)
tarfile
The tarfile module now supports a simple Command-Line Interface when
called as a script directly or via -m. This can be used to create and
extract tarfile archives. (Contributed by Berker Peksag in bpo-13477.)
textwrap
The TextWrapper class has two new attributes/constructor
arguments: max_lines, which limits the number of
lines in the output, and placeholder, which is a
string that will appear at the end of the output if it has been truncated
because of max_lines. Building on these capabilities, a new convenience
function shorten() collapses all of the whitespace in the input
to single spaces and produces a single line of a given width that ends with
the placeholder (by default, [...]). (Contributed by Antoine Pitrou and
Serhiy Storchaka in bpo-18585 and bpo-18725.)
threading
The Thread object representing the main thread can be
obtained from the new main_thread() function. In normal
conditions this will be the thread from which the Python interpreter was
started. (Contributed by Andrew Svetlov in bpo-18882.)
traceback
A new traceback.clear_frames() function takes a traceback object
and clears the local variables in all of the frames it references,
reducing the amount of memory consumed. (Contributed by Andrew Kuchling in
bpo-1565525.)
types
A new DynamicClassAttribute() descriptor provides a way to define
an attribute that acts normally when looked up through an instance object, but
which is routed to the class __getattr__ when looked up through the
class. This allows one to have properties active on a class, and have virtual
attributes on the class with the same name (see Enum for an example).
(Contributed by Ethan Furman in bpo-19030.)
urllib
urllib.request now supports data: URLs via the
DataHandler class. (Contributed by Mathias Panzenböck
in bpo-16423.)
The http method that will be used by a Request class
can now be specified by setting a method
class attribute on the subclass. (Contributed by Jason R Coombs in
bpo-18978.)
Request objects are now reusable: if the
full_url or data
attributes are modified, all relevant internal properties are updated. This
means, for example, that it is now possible to use the same
Request object in more than one
OpenerDirector.open() call with different data arguments, or to
modify a Request’s url rather than recomputing it
from scratch. There is also a new
remove_header() method that can be used to remove
headers from a Request. (Contributed by Alexey
Kachayev in bpo-16464, Daniel Wozniak in bpo-17485, and Damien Brecht
and Senthil Kumaran in bpo-17272.)
HTTPError objects now have a
headers attribute that provides access to the
HTTP response headers associated with the error. (Contributed by
Berker Peksag in bpo-15701.)
unittest
The TestCase class has a new method,
subTest(), that produces a context manager whose
with block becomes a “sub-test”. This context manager allows a test
method to dynamically generate subtests by, say, calling the subTest
context manager inside a loop. A single test method can thereby produce an
indefinite number of separately-identified and separately-counted tests, all of
which will run even if one or more of them fail. For example:
class NumbersTest(unittest.TestCase):
def test_even(self):
for i in range(6):
with self.subTest(i=i):
self.assertEqual(i % 2, 0)
will result in six subtests, each identified in the unittest verbose output
with a label consisting of the variable name i and a particular value for
that variable (i=0, i=1, etc). See Distinguishing test iterations using subtests for the full
version of this example. (Contributed by Antoine Pitrou in bpo-16997.)
unittest.main() now accepts an iterable of test names for
defaultTest, where previously it only accepted a single test name as a
string. (Contributed by Jyrki Pulliainen in bpo-15132.)
If SkipTest is raised during test discovery (that is, at the
module level in the test file), it is now reported as a skip instead of an
error. (Contributed by Zach Ware in bpo-16935.)
discover() now sorts the discovered files to provide
consistent test ordering. (Contributed by Martin Melin and Jeff Ramnani in
bpo-16709.)
TestSuite now drops references to tests as soon as the test
has been run, if the test is successful. On Python interpreters that do
garbage collection, this allows the tests to be garbage collected if nothing
else is holding a reference to the test. It is possible to override this
behavior by creating a TestSuite subclass that defines a
custom _removeTestAtIndex method. (Contributed by Tom Wardill, Matt
McClure, and Andrew Svetlov in bpo-11798.)
A new test assertion context-manager, assertLogs(),
will ensure that a given block of code emits a log message using the
logging module. By default the message can come from any logger and
have a priority of INFO or higher, but both the logger name and an
alternative minimum logging level may be specified. The object returned by the
context manager can be queried for the LogRecords and/or
formatted messages that were logged. (Contributed by Antoine Pitrou in
bpo-18937.)
Test discovery now works with namespace packages (Contributed by Claudiu Popa
in bpo-17457.)
unittest.mock objects now inspect their specification signatures when
matching calls, which means an argument can now be matched by either position
or name, instead of only by position. (Contributed by Antoine Pitrou in
bpo-17015.)
mock_open() objects now have readline and readlines
methods. (Contributed by Toshio Kuratomi in bpo-17467.)
venv
venv now includes activation scripts for the csh and fish
shells. (Contributed by Andrew Svetlov in bpo-15417.)
EnvBuilder and the create() convenience function
take a new keyword argument with_pip, which defaults to False, that
controls whether or not EnvBuilder ensures that pip is
installed in the virtual environment. (Contributed by Nick Coghlan in
bpo-19552 as part of the PEP 453 implementation.)
weakref
New WeakMethod class simulates weak references to bound
methods. (Contributed by Antoine Pitrou in bpo-14631.)
New finalize class makes it possible to register a callback
to be invoked when an object is garbage collected, without needing to
carefully manage the lifecycle of the weak reference itself. (Contributed by
Richard Oudkerk in bpo-15528.)
The callback, if any, associated with a ref is now
exposed via the __callback__ attribute. (Contributed
by Mark Dickinson in bpo-17643.)
zipfile
The writepy() method of the
PyZipFile class has a new filterfunc option that can be
used to control which directories and files are added to the archive. For
example, this could be used to exclude test files from the archive.
(Contributed by Christian Tismer in bpo-19274.)
The allowZip64 parameter to ZipFile and
PyZipfile is now True by default. (Contributed by
William Mallard in bpo-17201.)