There are some weird performance optimizations in Python, e.g.,
item = some_dict.get(key)
if item is None:
# key does not exist
Versus
try:
item = some_dict[key]
except KeyError:
# key does not exist
When I tested these (admittedly, a while ago), which one was faster depended on how often the key was missing. If “missing key” was an expected case, the first one was faster. If “missing key” was uncommon, the second was faster. It sounds like the fast path in the second case is getting faster, so this performance gap may be increasing.
Fun fact: all those approaches use multiple dict lookups, just of different dicts.
First approach is looking for `get` in `type(some_dict).__dict__` and then for `key` in `some_dict`.
Second approach is looking for `key` in `some_dict`, and then (only if missing) for `KeyError` in the module globals/builtins.
If the performance of hash lookups matters, Python is the wrong language for you.
> If the performance of hash lookups matters, Python is the wrong language for you.
Announcement to Python programmers: “Don’t bother trying to improve the performance of your Python code! If performance matters, just completely rewrite your code in a different language!”
I don’t know how to respond to that, except to disagree with the underlying assumptions that (1) there is a “right language”, (2) if performance matters, Python is not a suitable language, or (3) people are generally in a position to choose which language a project is written in.
Even if performance matters, it is not the only thing that matters. When you choose a language, there are necessarily tradeoffs... everything from the skillset of your team, to the ecosystem of libraries available affects that decision. Finally, there are projects already written in Python.
I am going to speculate here, so if I'm wrong please point it out.
Here, the number of steps directly affect the time.
In the first approach, the ".get()" method first analyses the type of "some_dict" and then uses an internal variable (the ones surrounded by double underscores) to try and fetch the value by using the provided key. If the key is present, then the value is returned, if not then a default value is returned. So if the key does not exist, the returning the default value saves 1 step (that of fetching the value from the map)
In the second approach, the exception raises the number of steps because the type of error has to be determined and the stack is traced every time an exception is raised. So the more exceptions are raised, the slower the code gets.
I tested this with 3.9.7 right now and in my testing, the runtime of first approach was virtually unchanged, while the second one was faster if exceptions were raised ~12% of the time or less. (I ran both 10 million times)
