Formatting strings in two steps with Python
<p>
This note may be a bit of an extreme case of how to format
<a class="wikilink" href="/strings/">
strings
</a>
with
<a class="wikilink" href="/python/">
Python
</a>
, but nonetheless it is useful to understand some inner workings of string formatting.
</p>
<p>
Because of some projects at work, I needed to be able to handle string formatting in two steps. Let's say, I have a string like this:
</p>
<pre class="codehilite"><code class="language-python">var = '{val1}_{val2}.dat'
</code></pre>
<p>
The thing is that I want to format the string using only
<code>
val1
</code>
and leaving the part concerning
<code>
val2
</code>
intact, because that will be passed down to some other code. Basically, I want this:
</p>
<pre class="codehilite"><code class="language-python">var.format(val1=123)
</code></pre>
<p>
to return this:
</p>
<pre class="codehilite"><code class="language-python">'123_{val2}.dat'
</code></pre>
<p>
But if we try, we get a
<code>
KeyError
</code>
because
<code>
val2
</code>
is missing. This means that
<a class="wikilink" href="/python/">
Python
</a>
is using a dictionary to handle the formatting, and if we could somehow get in between, we can actually skip the error and return the
<em>
unformatted
</em>
part of the string.
</p>
<p>
It takes a bit of black-magic googling, but the answer is actually in the
<a href="https://docs.python.org/3/library/stdtypes.html#str.format_map">
docs
</a>
. We can use a custom formatter, we just need to make look like a
<a class="wikilink" href="/dictionary/">
dictionary
</a>
that handles the missing key appropriately:
</p>
<pre class="codehilite"><code class="language-python">class FormatDict(dict):
def __missing__(self, key):
return '{' + str(key) + '}'
</code></pre>
<p>
If we try again, now it works and we get the expected outcome:
</p>
<pre class="codehilite"><code class="language-pycon">>>> var.format_map(FormatDict(val1=123))
'123_{val2}.dat'
</code></pre>
<p>
However, we can go one step further. What happens if we actually specify a format for
<code>
val2
</code>
, let's say:
</p>
<pre class="codehilite"><code class="language-pycon">>>> var = '{val1}_{val2:04}.dat'
>>> var.format_map(FormatDict(val1=123))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: '=' alignment not allowed in string format specifier
</code></pre>
<p>
So, we need to work around to get it to work. A good starting point to look for answers is the
<a href="https://www.python.org/dev/peps/pep-3101/#controlling-formatting-on-a-per-type-basis">
PEP 3101
</a>
where the
<code>
.format
</code>
notation is introduced. Note that, under the hood, each argument is passed to a formatter class, and the
<code>
__format__
</code>
method will be called. So, we want to define our own method only for the keys that are missing:
</p>
<pre class="codehilite"><code class="language-python">class FormatPlaceholder:
def __init__(self, key):
self.key = key
def __format__(self, spec):
result = self.key
if spec:
result += ":" + spec
return "{" + result + "}"
class FormatDict(dict):
def __missing__(self, key):
return FormatPlaceholder(key)
</code></pre>
<p>
In this case, if the key is missing, it get's a
<code>
FormatPlaceholder
</code>
, gets instantiated with the missing
<code>
key
</code>
and its
<code>
__format__
</code>
method is called. Then, we simply append the specification to the result in case it is provided. Now we get it to work:
</p>
<pre class="codehilite"><code class="language-pycon">>>> var.format_map(FormatDict(val1=123))
'123_{val2:04}.dat'
</code></pre>
<p>
If you want to see this pattern in the real-world: check
<a href="https://github.com/aquilesC/experimentor/blob/9d3320694223a1081c69a4081bed3aeb2ae6b2cd/experimentor/models/experiments/base_experiment.py#L54">
my project
</a>
one
<a href="https://github.com/aquilesC/DisperPy/blob/3589f36586261ac9f818f76ae47202b0e78e87a1/dispertech/models/experiment/dispertech/experiment.py#L108">
one use
</a>
. To see how the
<code>
__format__
</code>
method can be used on custom objects, you can check, for example, what
<a href="https://github.com/hgrecco/pint/blob/132a9fd5ef4737d82527354b26cce6859d631ab3/pint/unit.py#L75">
Pint
</a>
does to format quantities including their units.
</p>
Comment
Share your thoughts on this note. Comments are not public, they are
messages sent directly to my inbox.
Aquiles Carattino
This note you are reading is part
of my digital garden. Follow the
links to learn more, and remember that these notes evolve over time. After all,
this website is not a
blog.