This article explains the new features in Python 3.3, compared to 3.2.
Python 3.3 was released on September 29, 2012. For full details,
see the changelog.
PEP 393: Flexible String Representation
The Unicode string type is changed to support multiple internal
representations, depending on the character with the largest Unicode ordinal
(1, 2, or 4 bytes) in the represented string. This allows a space-efficient
representation in common cases, but gives access to full UCS-4 on all
systems. For compatibility with existing APIs, several representations may
exist in parallel; over time, this compatibility should be phased out.
On the Python side, there should be no downside to this change.
On the C API side, PEP 393 is fully backward compatible. The legacy API
should remain available at least five years. Applications using the legacy
API will not fully benefit of the memory reduction, or - worse - may use
a bit more memory, because Python may have to maintain two versions of each
string (in the legacy format and in the new efficient storage).
Functionality
Changes introduced by PEP 393 are the following:
- Python now always supports the full range of Unicode code points, including
non-BMP ones (i.e. from
U+0000 to U+10FFFF). The distinction between
narrow and wide builds no longer exists and Python now behaves like a wide
build, even under Windows.
- With the death of narrow builds, the problems specific to narrow builds have
also been fixed, for example:
len() now always returns 1 for non-BMP characters,
so len('\U0010FFFF') == 1;
- surrogate pairs are not recombined in string literals,
so
'\uDBFF\uDFFF' != '\U0010FFFF';
- indexing or slicing non-BMP characters returns the expected value,
so
'\U0010FFFF'[0] now returns '\U0010FFFF' and not '\uDBFF';
- all other functions in the standard library now correctly handle
non-BMP code points.
- The value of
sys.maxunicode is now always 1114111 (0x10FFFF
in hexadecimal). The PyUnicode_GetMax() function still returns
either 0xFFFF or 0x10FFFF for backward compatibility, and it should
not be used with the new Unicode API (see bpo-13054).
- The
./configure flag --with-wide-unicode has been removed.
PEP 409: Suppressing exception context
PEP 409 introduces new syntax that allows the display of the chained
exception context to be disabled. This allows cleaner error messages in
applications that convert between exception types:
>>> class D:
... def __init__(self, extra):
... self._extra_attributes = extra
... def __getattr__(self, attr):
... try:
... return self._extra_attributes[attr]
... except KeyError:
... raise AttributeError(attr) from None
...
>>> D({}).x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 8, in __getattr__
AttributeError: x
Without the from None suffix to suppress the cause, the original
exception would be displayed by default:
>>> class C:
... def __init__(self, extra):
... self._extra_attributes = extra
... def __getattr__(self, attr):
... try:
... return self._extra_attributes[attr]
... except KeyError:
... raise AttributeError(attr)
...
>>> C({}).x
Traceback (most recent call last):
File "<stdin>", line 6, in __getattr__
KeyError: 'x'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 8, in __getattr__
AttributeError: x
No debugging capability is lost, as the original exception context remains
available if needed (for example, if an intervening library has incorrectly
suppressed valuable underlying details):
>>> try:
... D({}).x
... except AttributeError as exc:
... print(repr(exc.__context__))
...
KeyError('x',)
See also
- PEP 409 - Suppressing exception context
- PEP written by Ethan Furman; implemented by Ethan Furman and Nick
Coghlan.
Using importlib as the Implementation of Import
bpo-2377 - Replace __import__ w/ importlib.__import__
bpo-13959 - Re-implement parts of imp in pure Python
bpo-14605 - Make import machinery explicit
bpo-14646 - Require loaders set __loader__ and __package__
The __import__() function is now powered by importlib.__import__().
This work leads to the completion of “phase 2” of PEP 302. There are
multiple benefits to this change. First, it has allowed for more of the
machinery powering import to be exposed instead of being implicit and hidden
within the C code. It also provides a single implementation for all Python VMs
supporting Python 3.3 to use, helping to end any VM-specific deviations in
import semantics. And finally it eases the maintenance of import, allowing for
future growth to occur.
For the common user, there should be no visible change in semantics. For
those whose code currently manipulates import or calls import
programmatically, the code changes that might possibly be required are covered
in the Porting Python code section of this document.
Visible Changes
For potential required changes to code, see the Porting Python code
section.
Beyond the expanse of what importlib now exposes, there are other
visible changes to import. The biggest is that sys.meta_path and
sys.path_hooks now store all of the meta path finders and path entry
hooks used by import. Previously the finders were implicit and hidden within
the C code of import instead of being directly exposed. This means that one can
now easily remove or change the order of the various finders to fit one’s needs.
Another change is that all modules have a __loader__ attribute, storing the
loader used to create the module. PEP 302 has been updated to make this
attribute mandatory for loaders to implement, so in the future once 3rd-party
loaders have been updated people will be able to rely on the existence of the
attribute. Until such time, though, import is setting the module post-load.
Loaders are also now expected to set the __package__ attribute from
PEP 366. Once again, import itself is already setting this on all loaders
from importlib and import itself is setting the attribute post-load.
None is now inserted into sys.path_importer_cache when no finder
can be found on sys.path_hooks. Since imp.NullImporter is not
directly exposed on sys.path_hooks it could no longer be relied upon to
always be available to use as a value representing no finder found.
All other changes relate to semantic changes which should be taken into
consideration when updating code for Python 3.3, and thus should be read about
in the Porting Python code section of this document.
(Implementation by Brett Cannon)
Improved Modules
abc
Improved support for abstract base classes containing descriptors composed with
abstract methods. The recommended approach to declaring abstract descriptors is
now to provide __isabstractmethod__ as a dynamically updated
property. The built-in descriptors have been updated accordingly.
(Contributed by Darren Dale in bpo-11610.)
abc.ABCMeta.register() now returns the registered subclass, which means
it can now be used as a class decorator (bpo-10868).
array
The array module supports the long long type using q and
Q type codes.
(Contributed by Oren Tirosh and Hirokazu Yamamoto in bpo-1172711.)
base64
ASCII-only Unicode strings are now accepted by the decoding functions of the
base64 modern interface. For example, base64.b64decode('YWJj')
returns b'abc'. (Contributed by Catalin Iacob in bpo-13641.)
binascii
In addition to the binary objects they normally accept, the a2b_ functions
now all also accept ASCII-only strings as input. (Contributed by Antoine
Pitrou in bpo-13637.)
bz2
The bz2 module has been rewritten from scratch. In the process, several
new features have been added:
New bz2.open() function: open a bzip2-compressed file in binary or
text mode.
bz2.BZ2File can now read from and write to arbitrary file-like
objects, by means of its constructor’s fileobj argument.
(Contributed by Nadeem Vawda in bpo-5863.)
bz2.BZ2File and bz2.decompress() can now decompress
multi-stream inputs (such as those produced by the pbzip2 tool).
bz2.BZ2File can now also be used to create this type of file, using
the 'a' (append) mode.
(Contributed by Nir Aides in bpo-1625.)
bz2.BZ2File now implements all of the io.BufferedIOBase API,
except for the detach() and truncate() methods.
codecs
The mbcs codec has been rewritten to handle correctly
replace and ignore error handlers on all Windows versions. The
mbcs codec now supports all error handlers, instead of only
replace to encode and ignore to decode.
A new Windows-only codec has been added: cp65001 (bpo-13216). It is the
Windows code page 65001 (Windows UTF-8, CP_UTF8). For example, it is used
by sys.stdout if the console output code page is set to cp65001 (e.g., using
chcp 65001 command).
Multibyte CJK decoders now resynchronize faster. They only ignore the first
byte of an invalid byte sequence. For example, b'\xff\n'.decode('gb2312',
'replace') now returns a \n after the replacement character.
(bpo-12016)
Incremental CJK codec encoders are no longer reset at each call to their
encode() methods. For example:
>>> import codecs
>>> encoder = codecs.getincrementalencoder('hz')('strict')
>>> b''.join(encoder.encode(x) for x in '\u52ff\u65bd\u65bc\u4eba\u3002 Bye.')
b'~{NpJ)l6HK!#~} Bye.'
This example gives b'~{Np~}~{J)~}~{l6~}~{HK~}~{!#~} Bye.' with older Python
versions.
(bpo-12100)
The unicode_internal codec has been deprecated.
collections
Addition of a new ChainMap class to allow treating a
number of mappings as a single unit. (Written by Raymond Hettinger for
bpo-11089, made public in bpo-11297.)
The abstract base classes have been moved in a new collections.abc
module, to better differentiate between the abstract and the concrete
collections classes. Aliases for ABCs are still present in the
collections module to preserve existing imports. (bpo-11085)
The Counter class now supports the unary + and -
operators, as well as the in-place operators +=, -=, |=, and
&=. (Contributed by Raymond Hettinger in bpo-13121.)
contextlib
ExitStack now provides a solid foundation for
programmatic manipulation of context managers and similar cleanup
functionality. Unlike the previous contextlib.nested API (which was
deprecated and removed), the new API is designed to work correctly
regardless of whether context managers acquire their resources in
their __init__ method (for example, file objects) or in their
__enter__ method (for example, synchronisation objects from the
threading module).
(bpo-13585)
crypt
Addition of salt and modular crypt format (hashing method) and the mksalt()
function to the crypt module.
(bpo-10924)
curses
- If the
curses module is linked to the ncursesw library, use Unicode
functions when Unicode strings or characters are passed (e.g.
waddwstr()), and bytes functions otherwise (e.g. waddstr()).
- Use the locale encoding instead of
utf-8 to encode Unicode strings.
curses.window has a new curses.window.encoding attribute.
- The
curses.window class has a new get_wch()
method to get a wide character
- The
curses module has a new unget_wch() function to
push a wide character so the next get_wch() will return
it
(Contributed by Iñigo Serna in bpo-6755.)
decimal
- bpo-7652 - integrate fast native decimal arithmetic.
- C-module and libmpdec written by Stefan Krah.
The new C version of the decimal module integrates the high speed libmpdec
library for arbitrary precision correctly-rounded decimal floating point
arithmetic. libmpdec conforms to IBM’s General Decimal Arithmetic Specification.
Performance gains range from 10x for database applications to 100x for
numerically intensive applications. These numbers are expected gains
for standard precisions used in decimal floating point arithmetic. Since
the precision is user configurable, the exact figures may vary. For example,
in integer bignum arithmetic the differences can be significantly higher.
The following table is meant as an illustration. Benchmarks are available
at http://www.bytereef.org/mpdecimal/quickstart.html.
| |
decimal.py |
_decimal |
speedup |
| pi |
42.02s |
0.345s |
120x |
| telco |
172.19s |
5.68s |
30x |
| psycopg |
3.57s |
0.29s |
12x |
Features
- The
FloatOperation signal optionally enables stricter
semantics for mixing floats and Decimals.
- If Python is compiled without threads, the C version automatically
disables the expensive thread local context machinery. In this case,
the variable
HAVE_THREADS is set to False.
API changes
The C module has the following context limits, depending on the machine
architecture:
| |
32-bit |
64-bit |
MAX_PREC |
425000000 |
999999999999999999 |
MAX_EMAX |
425000000 |
999999999999999999 |
MIN_EMIN |
-425000000 |
-999999999999999999 |
In the context templates (DefaultContext,
BasicContext and ExtendedContext)
the magnitude of Emax and
Emin has changed to 999999.
The Decimal constructor in decimal.py does not observe
the context limits and converts values with arbitrary exponents or precision
exactly. Since the C version has internal limits, the following scheme is
used: If possible, values are converted exactly, otherwise
InvalidOperation is raised and the result is NaN. In the
latter case it is always possible to use create_decimal()
in order to obtain a rounded or inexact value.
The power function in decimal.py is always correctly-rounded. In the
C version, it is defined in terms of the correctly-rounded
exp() and ln() functions,
but the final result is only “almost always correctly rounded”.
In the C version, the context dictionary containing the signals is a
MutableMapping. For speed reasons,
flags and traps always
refer to the same MutableMapping that the context
was initialized with. If a new signal dictionary is assigned,
flags and traps
are updated with the new values, but they do not reference the RHS
dictionary.
Pickling a Context produces a different output in order
to have a common interchange format for the Python and C versions.
The order of arguments in the Context constructor has been
changed to match the order displayed by repr().
The watchexp parameter in the quantize() method
is deprecated.
email
Policy Framework
The email package now has a policy framework. A
Policy is an object with several methods and properties
that control how the email package behaves. The primary policy for Python 3.3
is the Compat32 policy, which provides backward
compatibility with the email package in Python 3.2. A policy can be
specified when an email message is parsed by a parser, or when a
Message object is created, or when an email is
serialized using a generator. Unless overridden, a policy passed
to a parser is inherited by all the Message object and sub-objects
created by the parser. By default a generator will use the policy of
the Message object it is serializing. The default policy is
compat32.
The minimum set of controls implemented by all policy objects are:
| max_line_length |
The maximum length, excluding the linesep character(s),
individual lines may have when a Message is
serialized. Defaults to 78. |
| linesep |
The character used to separate individual lines when a
Message is serialized. Defaults to \n. |
| cte_type |
7bit or 8bit. 8bit applies only to a
Bytes generator, and means that non-ASCII may
be used where allowed by the protocol (or where it
exists in the original input). |
| raise_on_defect |
Causes a parser to raise error when defects are
encountered instead of adding them to the Message
object’s defects list. |
A new policy instance, with new settings, is created using the
clone() method of policy objects. clone takes
any of the above controls as keyword arguments. Any control not specified in
the call retains its default value. Thus you can create a policy that uses
\r\n linesep characters like this:
mypolicy = compat32.clone(linesep='\r\n')
Policies can be used to make the generation of messages in the format needed by
your application simpler. Instead of having to remember to specify
linesep='\r\n' in all the places you call a generator, you can specify
it once, when you set the policy used by the parser or the Message,
whichever your program uses to create Message objects. On the other hand,
if you need to generate messages in multiple forms, you can still specify the
parameters in the appropriate generator call. Or you can have custom
policy instances for your different cases, and pass those in when you create
the generator.
ftplib
ftplib.FTP now accepts a source_address keyword argument to
specify the (host, port) to use as the source address in the bind call
when creating the outgoing socket. (Contributed by Giampaolo Rodolà
in bpo-8594.)
- The
FTP_TLS class now provides a new
ccc() function to revert control channel back to
plaintext. This can be useful to take advantage of firewalls that know how
to handle NAT with non-secure FTP without opening fixed ports. (Contributed
by Giampaolo Rodolà in bpo-12139.)
- Added
ftplib.FTP.mlsd() method which provides a parsable directory
listing format and deprecates ftplib.FTP.nlst() and
ftplib.FTP.dir(). (Contributed by Giampaolo Rodolà in bpo-11072.)
gc
It is now possible to register callbacks invoked by the garbage collector
before and after collection using the new callbacks list.
hmac
A new compare_digest() function has been added to prevent side
channel attacks on digests through timing analysis. (Contributed by Nick
Coghlan and Christian Heimes in bpo-15061.)
html
html.parser.HTMLParser is now able to parse broken markup without
raising errors, therefore the strict argument of the constructor and the
HTMLParseError exception are now deprecated.
The ability to parse broken markup is the result of a number of bug fixes that
are also available on the latest bug fix releases of Python 2.7/3.2.
(Contributed by Ezio Melotti in bpo-15114, and bpo-14538,
bpo-13993, bpo-13960, bpo-13358, bpo-1745761,
bpo-755670, bpo-13357, bpo-12629, bpo-1200313,
bpo-670664, bpo-13273, bpo-12888, bpo-7311.)
A new html5 dictionary that maps HTML5 named character
references to the equivalent Unicode character(s) (e.g. html5['gt;'] ==
'>') has been added to the html.entities module. The dictionary is
now also used by HTMLParser. (Contributed by Ezio
Melotti in bpo-11113 and bpo-15156.)
imaplib
The IMAP4_SSL constructor now accepts an SSLContext
parameter to control parameters of the secure channel.
(Contributed by Sijin Joseph in bpo-8808.)
inspect
A new getclosurevars() function has been added. This function
reports the current binding of all names referenced from the function body and
where those names were resolved, making it easier to verify correct internal
state when testing code that relies on stateful closures.
(Contributed by Meador Inge and Nick Coghlan in bpo-13062.)
A new getgeneratorlocals() function has been added. This
function reports the current binding of local variables in the generator’s
stack frame, making it easier to verify correct internal state when testing
generators.
(Contributed by Meador Inge in bpo-15153.)
io
The open() function has a new 'x' mode that can be used to
exclusively create a new file, and raise a FileExistsError if the file
already exists. It is based on the C11 ‘x’ mode to fopen().
(Contributed by David Townshend in bpo-12760.)
The constructor of the TextIOWrapper class has a new
write_through optional argument. If write_through is True, calls to
write() are guaranteed not to be buffered: any data
written on the TextIOWrapper object is immediately handled to its
underlying binary buffer.
logging
The basicConfig() function now supports an optional handlers
argument taking an iterable of handlers to be added to the root logger.
A class level attribute append_nul has
been added to SysLogHandler to allow control of the
appending of the NUL (\000) byte to syslog records, since for some
deamons it is required while for others it is passed through to the log.
math
The math module has a new function, log2(), which returns
the base-2 logarithm of x.
(Written by Mark Dickinson in bpo-11888.)
mmap
The read() method is now more compatible with other file-like
objects: if the argument is omitted or specified as None, it returns the
bytes from the current file position to the end of the mapping. (Contributed
by Petri Lehtinen in bpo-12021.)
nntplib
The nntplib.NNTP class now supports the context management protocol to
unconditionally consume socket.error exceptions and to close the NNTP
connection when done:
>>> from nntplib import NNTP
>>> with NNTP('news.gmane.org') as n:
... n.group('gmane.comp.python.committers')
...
('211 1755 1 1755 gmane.comp.python.committers', 1755, 1, 1755, 'gmane.comp.python.committers')
>>>
(Contributed by Giampaolo Rodolà in bpo-9795.)
os
The os module has a new pipe2() function that makes it
possible to create a pipe with O_CLOEXEC or
O_NONBLOCK flags set atomically. This is especially useful to
avoid race conditions in multi-threaded programs.
The os module has a new sendfile() function which provides
an efficient “zero-copy” way for copying data from one file (or socket)
descriptor to another. The phrase “zero-copy” refers to the fact that all of
the copying of data between the two descriptors is done entirely by the
kernel, with no copying of data into userspace buffers. sendfile()
can be used to efficiently copy data from a file on disk to a network socket,
e.g. for downloading a file.
(Patch submitted by Ross Lagerwall and Giampaolo Rodolà in bpo-10882.)
To avoid race conditions like symlink attacks and issues with temporary
files and directories, it is more reliable (and also faster) to manipulate
file descriptors instead of file names. Python 3.3 enhances existing functions
and introduces new functions to work on file descriptors (bpo-4761,
bpo-10755 and bpo-14626).
- The
os module has a new fwalk() function similar to
walk() except that it also yields file descriptors referring to the
directories visited. This is especially useful to avoid symlink races.
- The following functions get new optional dir_fd (paths relative to
directory descriptors) and/or follow_symlinks (not
following symlinks):
access(), chflags(), chmod(), chown(),
link(), lstat(), mkdir(), mkfifo(),
mknod(), open(), readlink(), remove(),
rename(), replace(), rmdir(), stat(),
symlink(), unlink(), utime(). Platform
support for using these parameters can be checked via the sets
os.supports_dir_fd and os.supports_follows_symlinks.
- The following functions now support a file descriptor for their path argument:
chdir(), chmod(), chown(),
execve(), listdir(), pathconf(), exists(),
stat(), statvfs(), utime(). Platform support
for this can be checked via the os.supports_fd set.
access() accepts an effective_ids keyword argument to turn on
using the effective uid/gid rather than the real uid/gid in the access check.
Platform support for this can be checked via the
supports_effective_ids set.
The os module has two new functions: getpriority() and
setpriority(). They can be used to get or set process
niceness/priority in a fashion similar to os.nice() but extended to all
processes instead of just the current one.
(Patch submitted by Giampaolo Rodolà in bpo-10784.)
The new os.replace() function allows cross-platform renaming of a
file with overwriting the destination. With os.rename(), an existing
destination file is overwritten under POSIX, but raises an error under
Windows.
(Contributed by Antoine Pitrou in bpo-8828.)
The stat family of functions (stat(), fstat(),
and lstat()) now support reading a file’s timestamps
with nanosecond precision. Symmetrically, utime()
can now write file timestamps with nanosecond precision. (Contributed by
Larry Hastings in bpo-14127.)
The new os.get_terminal_size() function queries the size of the
terminal attached to a file descriptor. See also
shutil.get_terminal_size().
(Contributed by Zbigniew Jędrzejewski-Szmek in bpo-13609.)
- New functions to support Linux extended attributes (bpo-12720):
getxattr(), listxattr(), removexattr(),
setxattr().
- New interface to the scheduler. These functions
control how a process is allocated CPU time by the operating system. New
functions:
sched_get_priority_max(), sched_get_priority_min(),
sched_getaffinity(), sched_getparam(),
sched_getscheduler(), sched_rr_get_interval(),
sched_setaffinity(), sched_setparam(),
sched_setscheduler(), sched_yield(),
- New functions to control the file system:
posix_fadvise(): Announces an intention to access data in a
specific pattern thus allowing the kernel to make optimizations.
posix_fallocate(): Ensures that enough disk space is allocated
for a file.
sync(): Force write of everything to disk.
- Additional new posix functions:
lockf(): Apply, test or remove a POSIX lock on an open file descriptor.
pread(): Read from a file descriptor at an offset, the file
offset remains unchanged.
pwrite(): Write to a file descriptor from an offset, leaving
the file offset unchanged.
readv(): Read from a file descriptor into a number of writable buffers.
truncate(): Truncate the file corresponding to path, so that
it is at most length bytes in size.
waitid(): Wait for the completion of one or more child processes.
writev(): Write the contents of buffers to a file descriptor,
where buffers is an arbitrary sequence of buffers.
getgrouplist() (bpo-9344): Return list of group ids that
specified user belongs to.
times() and uname(): Return type changed from a tuple to
a tuple-like object with named attributes.
- Some platforms now support additional constants for the
lseek()
function, such as os.SEEK_HOLE and os.SEEK_DATA.
- New constants
RTLD_LAZY, RTLD_NOW,
RTLD_GLOBAL, RTLD_LOCAL, RTLD_NODELETE,
RTLD_NOLOAD, and RTLD_DEEPBIND are available on
platforms that support them. These are for use with the
sys.setdlopenflags() function, and supersede the similar constants
defined in ctypes and DLFCN. (Contributed by Victor Stinner
in bpo-13226.)
os.symlink() now accepts (and ignores) the target_is_directory
keyword argument on non-Windows platforms, to ease cross-platform support.
pdb
Tab-completion is now available not only for command names, but also their
arguments. For example, for the break command, function and file names
are completed.
(Contributed by Georg Brandl in bpo-14210)
pydoc
The Tk GUI and the serve() function have been removed from the
pydoc module: pydoc -g and serve() have been deprecated
in Python 3.2.
re
str regular expressions now support \u and \U escapes.
(Contributed by Serhiy Storchaka in bpo-3665.)
sched
run() now accepts a blocking parameter which when
set to false makes the method execute the scheduled events due to expire
soonest (if any) and then return immediately.
This is useful in case you want to use the scheduler in
non-blocking applications. (Contributed by Giampaolo Rodolà in bpo-13449.)
scheduler class can now be safely used in multi-threaded
environments. (Contributed by Josiah Carlson and Giampaolo Rodolà in
bpo-8684.)
- timefunc and delayfunct parameters of
scheduler class
constructor are now optional and defaults to time.time() and
time.sleep() respectively. (Contributed by Chris Clark in
bpo-13245.)
enter() and enterabs()
argument parameter is now optional. (Contributed by Chris Clark in
bpo-13245.)
enter() and enterabs()
now accept a kwargs parameter. (Contributed by Chris Clark in
bpo-13245.)
select
Solaris and derivative platforms have a new class select.devpoll
for high performance asynchronous sockets via /dev/poll.
(Contributed by Jesús Cea Avión in bpo-6397.)
shlex
The previously undocumented helper function quote from the
pipes modules has been moved to the shlex module and
documented. quote() properly escapes all characters in a string
that might be otherwise given special meaning by the shell.
shutil
- New functions:
disk_usage(): provides total, used and free disk space
statistics. (Contributed by Giampaolo Rodolà in bpo-12442.)
chown(): allows one to change user and/or group of the given
path also specifying the user/group names and not only their numeric
ids. (Contributed by Sandro Tosi in bpo-12191.)
shutil.get_terminal_size(): returns the size of the terminal window
to which the interpreter is attached. (Contributed by Zbigniew
Jędrzejewski-Szmek in bpo-13609.)
copy2() and copystat() now preserve file
timestamps with nanosecond precision on platforms that support it.
They also preserve file “extended attributes” on Linux. (Contributed
by Larry Hastings in bpo-14127 and bpo-15238.)
- Several functions now take an optional
symlinks argument: when that
parameter is true, symlinks aren’t dereferenced and the operation instead
acts on the symlink itself (or creates one, if relevant).
(Contributed by Hynek Schlawack in bpo-12715.)
- When copying files to a different file system,
move() now
handles symlinks the way the posix mv command does, recreating the
symlink rather than copying the target file contents. (Contributed by
Jonathan Niehof in bpo-9993.) move() now also returns
the dst argument as its result.
rmtree() is now resistant to symlink attacks on platforms
which support the new dir_fd parameter in os.open() and
os.unlink(). (Contributed by Martin von Löwis and Hynek Schlawack
in bpo-4489.)
signal
- The
signal module has new functions:
- The signal handler writes the signal number as a single byte instead of
a nul byte into the wakeup file descriptor. So it is possible to wait more
than one signal and know which signals were raised.
signal.signal() and signal.siginterrupt() raise an OSError,
instead of a RuntimeError: OSError has an errno attribute.
smtpd
The smtpd module now supports RFC 5321 (extended SMTP) and RFC 1870
(size extension). Per the standard, these extensions are enabled if and only
if the client initiates the session with an EHLO command.
(Initial ELHO support by Alberto Trevino. Size extension by Juhana
Jauhiainen. Substantial additional work on the patch contributed by Michele
Orrù and Dan Boswell. bpo-8739)
smtplib
The SMTP, SMTP_SSL, and
LMTP classes now accept a source_address keyword argument
to specify the (host, port) to use as the source address in the bind call
when creating the outgoing socket. (Contributed by Paulo Scardine in
bpo-11281.)
SMTP now supports the context management protocol, allowing an
SMTP instance to be used in a with statement. (Contributed
by Giampaolo Rodolà in bpo-11289.)
The SMTP_SSL constructor and the starttls()
method now accept an SSLContext parameter to control parameters of the secure
channel. (Contributed by Kasun Herath in bpo-8809.)
ssl
The ssl module has two new random generation functions:
(Contributed by Victor Stinner in bpo-12049.)
The ssl module now exposes a finer-grained exception hierarchy
in order to make it easier to inspect the various kinds of errors.
(Contributed by Antoine Pitrou in bpo-11183.)
load_cert_chain() now accepts a password argument
to be used if the private key is encrypted.
(Contributed by Adam Simpkins in bpo-12803.)
Diffie-Hellman key exchange, both regular and Elliptic Curve-based, is
now supported through the load_dh_params() and
set_ecdh_curve() methods.
(Contributed by Antoine Pitrou in bpo-13626 and bpo-13627.)
SSL sockets have a new get_channel_binding() method
allowing the implementation of certain authentication mechanisms such as
SCRAM-SHA-1-PLUS. (Contributed by Jacek Konieczny in bpo-12551.)
You can query the SSL compression algorithm used by an SSL socket, thanks
to its new compression() method. The new attribute
OP_NO_COMPRESSION can be used to disable compression.
(Contributed by Antoine Pitrou in bpo-13634.)
Support has been added for the Next Protocol Negotiation extension using
the ssl.SSLContext.set_npn_protocols() method.
(Contributed by Colin Marc in bpo-14204.)
SSL errors can now be introspected more easily thanks to
library and reason attributes.
(Contributed by Antoine Pitrou in bpo-14837.)
The get_server_certificate() function now supports IPv6.
(Contributed by Charles-François Natali in bpo-11811.)
New attribute OP_CIPHER_SERVER_PREFERENCE allows setting
SSLv3 server sockets to use the server’s cipher ordering preference rather
than the client’s (bpo-13635).
stat
The undocumented tarfile.filemode function has been moved to
stat.filemode(). It can be used to convert a file’s mode to a string of
the form ‘-rwxrwxrwx’.
(Contributed by Giampaolo Rodolà in bpo-14807.)
struct
The struct module now supports ssize_t and size_t via the
new codes n and N, respectively. (Contributed by Antoine Pitrou
in bpo-3163.)
subprocess
Command strings can now be bytes objects on posix platforms. (Contributed by
Victor Stinner in bpo-8513.)
A new constant DEVNULL allows suppressing output in a
platform-independent fashion. (Contributed by Ross Lagerwall in
bpo-5870.)
tarfile
tarfile now supports lzma encoding via the lzma module.
(Contributed by Lars Gustäbel in bpo-5689.)
textwrap
The textwrap module has a new indent() that makes
it straightforward to add a common prefix to selected lines in a block
of text (bpo-13857).
threading
threading.Condition, threading.Semaphore,
threading.BoundedSemaphore, threading.Event, and
threading.Timer, all of which used to be factory functions returning a
class instance, are now classes and may be subclassed. (Contributed by Éric
Araujo in bpo-10968.)
The threading.Thread constructor now accepts a daemon keyword
argument to override the default behavior of inheriting the deamon flag
value from the parent thread (bpo-6064).
The formerly private function _thread.get_ident is now available as the
public function threading.get_ident(). This eliminates several cases of
direct access to the _thread module in the stdlib. Third party code that
used _thread.get_ident should likewise be changed to use the new public
interface.
time
The PEP 418 added new functions to the time module:
get_clock_info(): Get information on a clock.
monotonic(): Monotonic clock (cannot go backward), not affected
by system clock updates.
perf_counter(): Performance counter with the highest available
resolution to measure a short duration.
process_time(): Sum of the system and user CPU time of the
current process.
Other new functions:
To improve cross platform consistency, sleep() now raises a
ValueError when passed a negative sleep value. Previously this was an
error on posix, but produced an infinite sleep on Windows.
urllib
The Request class, now accepts a method argument
used by get_method() to determine what HTTP method
should be used. For example, this will send a 'HEAD' request:
>>> urlopen(Request('https://www.python.org', method='HEAD'))
(bpo-1673007)
webbrowser
The webbrowser module supports more “browsers”: Google Chrome (named
chrome, chromium, chrome-browser or
chromium-browser depending on the version and operating system),
and the generic launchers xdg-open, from the FreeDesktop.org
project, and gvfs-open, which is the default URI handler for GNOME
3. (The former contributed by Arnaud Calmettes in bpo-13620, the latter
by Matthias Klose in bpo-14493.)
xml.etree.ElementTree
The xml.etree.ElementTree module now imports its C accelerator by
default; there is no longer a need to explicitly import
xml.etree.cElementTree (this module stays for backwards compatibility,
but is now deprecated). In addition, the iter family of methods of
Element has been optimized (rewritten in C).
The module’s documentation has also been greatly improved with added examples
and a more detailed reference.
zlib
New attribute zlib.Decompress.eof makes it possible to distinguish
between a properly-formed compressed stream and an incomplete or truncated one.
(Contributed by Nadeem Vawda in bpo-12646.)
New attribute zlib.ZLIB_RUNTIME_VERSION reports the version string of
the underlying zlib library that is loaded at runtime. (Contributed by
Torsten Landschoff in bpo-12306.)