Python is an open-source, general-purpose and object-oriented programming language created over 30 years ago. It is used for the development of many applications, most notably Youtube, Instagram, Uber, Reddit, and Spotify. Due to its flexibility, Python has been utilized across a wide range of sectors, with its applications ranging from web development and game development to machine learning, data science, and web scraping. According to a 2022 StackOverflow survey of over 70,000 developers, Python is the most popular backend programming language and the second most used tool when it comes to backend development. Moreover, Python is at the top of the TIOBE Index table, with a rating of 16.66%, meaning that over 16% of all programming-related searches contained the keyword Python.
New features in Python 3.11
The newest version of Python, 3.11, was released on the 24th of October 2022 and comes with various significant improvements. The 5 best new features in the latest version are:
- Better tracebacks message
- Speed improvements
- Exceptions enhancements
- Support for parsing TOML files
- Upgraded Type Variables
In this article we’ll go through what’s new in python 3.11 and why you should ditch the older python and get the latest python.
Better tracebacks message
In python 3.10, the new parser that was integrated into the interpreter already made some great improvements when it comes to the error reporting quality. However, Python 3.11 takes it a step further by providing fine-grained error locations within the tracebacks. This means that the developer would get more detailed feedback regarding the exact location inside of the expression that caused the error. This is a big improvement because before this update, the interpreter would just point to the line where the error happened, making it difficult to locate the issue, especially if the developer is dealing with a more complex code. Moreover, the tracebacks now have decorative annotations that help speed up error message interpretations. For example, look at the following code:
|
1 2 3 4 5 6 7 8 9 10 |
def division(): list1 = [1, 2, 3] list2 = [0, 5, 2] list3 = [3, 7] return [(a / b) + (a / c) for a in list1 for b in list2 for c in list3] if __name__ == "__main__": division() |
In the code above, the part where we’d expect to get an error is the first division, a/b, since b contains a zero element. As we can see in the traceback we get in Python 3.11, that exact part is highlighted so we know that that’s exactly where the error occurred. Running the code above will return the following tracback message:
|
1 2 3 4 5 6 7 8 9 10 |
Traceback (most recent call last): File "C:\NewFeatures\Division.py", line 10, in <module> division() File "C:\NewFeatures\Division.py", line 6, in division return [(a / b) + (a / c) for a in list1 for b in list2 for c in list3] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\NewFeatures\Division.py", line 6, in <listcomp> return [(a / b) + (a / c) for a in list1 for b in list2 for c in list3] ~~^~~ ZeroDivisionError: division by zero |
Speed Improvements
One of the biggest improvements of 3.11 is the improvement in speed. Python has always been considered a very slow language when compared to other programming languages, such as C, C++, or Java. According to pyperformance,the official Python benchmark suite, Python 3.11 runs around 25% faster than Python 3.10 on average. It can even run 60% faster in some instances. These speed improvements include faster startup and runtime. However, we should keep in mind that this speedup is an overall measurement and different parts of our code will have different execution time improvements.
Exceptions Enhancements
When it comes to the language’s error-handling mechanism – exceptions, Python 3.11 released some new big features. This version of Python introduced several new concepts such as exception groups, exception notes, and “zero-cost” exceptions.
Exception Groups
Firstly, Python 3.11 allows programs to raise and handle many unrelated exceptions at once by using the built-in types ExceptionGroup and BaseExceptionGroup. Anexception group may be viewed as a collection of regular exceptions that have been grouped into a single exception. While dealing with asynchronous or concurrent methods, managing numerous failures when retrying an action, or other situations where many exceptions might be raised concurrently, this feature can be seen as an absolute live-saver. Let’s look at a simple code example:
|
1 2 3 4 5 6 7 8 9 10 11 |
def division_by_two(x): if type(x) not in [int] or x == 0: raise ExceptionGroup("group", [ValueError(1), TypeError("Use integers only!")]) else: return x / 2 if __name__ == "__main__": print(division_by_two(1.2)) |
Since the value that’s provided is not an integer, the exception group is triggered and both sub-exception are raised, as seen below.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
+ Exception Group Traceback (most recent call last): | File "C:\NewFeatures\Division.py", line 11, in <module> | print(division_by_two(1.2)) | ^^^^^^^^^^^^^^^^^^^ | File "C:\NewFeatures\Division.py", line 3, in division_by_two | raise ExceptionGroup("group", | ExceptionGroup: group (2 sub-exceptions) +-+-----------------1----------------- | ValueError: 1 +-----------------2----------------- | TypeError: Use integers only! +------------------------------------ |
In order to better handle exception groups, 3.11 introduced a new keyword – except*. With the use of the except* syntax, the code and the execution output would look like this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
def division_by_two(x): if type(x) not in [int] or x == 0: try: raise ExceptionGroup("group", [ValueError(1), TypeError("Use integers only!")]) except* ValueError as value_error: print(f"ValueError {value_error.exceptions} is raised") except* TypeError as type_error: print(f"TypeError {type_error.exceptions} is raised") else: return x / 2 if __name__ == "__main__": print(division_by_two(1.2)) |
|
1 2 |
ValueErrror (ValueError(1),) is raised TypeError (TypeError('Use integers only!'),) is raised |
Exception Notes
The ability to add arbitrary exception notes in Python 3.11 is another addition to the regular exceptions. The python technical design doc (PEP) explains how these comments can be used to annotate an exception in a piece of code other than the one that first triggered it. The add_note() method should be used in order to add a note to an exception, while the .__notes__ attribute should be inspected for looking at notes that have already been added. The code syntax and traceback would be as follows:
|
1 2 3 4 5 6 7 8 9 |
def division(a, b): try: a / b except Exception as ex: ex.add_note(f"The error was raised at {datetime.now()}") raise if __name__ == "__main__": division(3,0) |
|
1 2 3 4 5 6 7 8 |
Traceback (most recent call last): File "C:\NewFeatures\Division.py", line 14, in <module> division(3, 0) File "C:\NewFeatures\Division.py", line 14, in divisions a / b ~~^~~ ZeroDivisionError: division by zero The error was raised at 2022-12-17 20:38:09.506993 |
The note stating the exact date and time when the exception was raised can be seen at the end of the exception output.
“Zero-cost” exceptions
With the new improvements in 3.11,exceptions now don’t have a cost except if they are indeed called. Therefore, the try/except block’s default path is quicker and requires less memory.
Support for Parsing TOML Files
Tom’s Obvious Minimal Language, or TOML, is a configuration file type that has gained great popularity over the past 10 years. The Python community has accepted TOML as the preferred standard for defining metadata for projects and packages. Up until now, Python hasn’t had built-in TOML support, despite the fact that this configuration type has been used by many different tools for years. So, in Python 3.11, they have finally added a package named tomllib to the standard library, a module that’s based on the well-known tomli third-party library and allows developers to parse TOML files.
However, it’s important to note that this package doesn’t support writing or creating TOML files and that a third-party module such as Tomli-W or TOML Kit is still required for such actions. Tomllib’s two main functions are load() and loads() for reading a TOML file and loading TOML from a string object, respectively. Let’s take a look at a practical code example of how they can be used:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
if __name__ == "__main__": with open("toml_file.toml", "rb") as toml_file: toml_data = tomllib.load(toml_file) toml_str = """ Animal = "cat" Fur = "black and white" Name = "Blacky" """ data = tomllib.loads(toml_str) print(data) |
In the example above, the first function tomllib.load() would just read the toml file, while the second one – tomllib.loads() parses a TOML string. The output of the second function would look like this:
|
1 |
{'Animal': 'cat', 'Fur': 'black and white', 'Name': 'Blacky'} |
Upgraded Type Variables
Python is a dynamically typed language, although it offers optional type hints to allow static typing. In PEP 484, published in 2015, laid the groundwork for Python’s static type system. Every Python release since version 3.5 has included a number of additional typing-related suggestions. Python 3.11 comes with a record-high number of type-hinting enhancements, out of which, we’ll talk about the three most important ones.
Arbitrary Literal String Type
Another nice improvement in 3.11 is the possibility of type annotations to indicate that the variable has to be a string literal, something that was not allowed in all other Python versions before. This can be done by using theLiteralString annotation. One scenario in which this annotation might deem useful is when we work with queries by making sure that the queries are generated from literal strings instead of constructed strings, thus preventing SQL injection.
|
1 2 3 4 5 6 7 |
def select_query(query: LiteralString): # do something here pass if __name__ == "__main__": select_query("SELECT * FROM table") |
The Self Type
When using type hints in the past, we had to explicitly specify a type variable if we wanted to refer to the current class itself. With the introduction of theSelf type in version 3.11, we can now refer to the encapsulating class by simply using the self keyword.
|
1 2 3 4 5 6 7 |
class Square: def __init__(self, area: int) -> None: self.area = area @classmethod def square_area(cls, size_len) -> Self: return cls(area=size_len * size_len) |
Variadic Generics
Python 3.11 introduces variadic generics, also known as TypeVarTuple, which allows developers to define a placeholder for a series of types given as a tuple. This would be especially handy in libraries like NumPy, where one could check for problems such as whether a provided array had the proper shape ahead of time.
Why you should upgrade to Python 3.11 release
Now that we’ve come to the end of this in-depth article on the most interesting new features Python 3.11 has to offer, the real question remains unanswered: should you switch to this new Python version? Yes, you should. There are many features in the 3.11 release – better speed, better error and exception handling, upgraded type hinting, and the addition of a variety of nice modules such as tomllib to name a few.
