Cell Tag Cheatsheet
These preprocessors allow you to make special comments to enable/disable them. Here is a list of all special comments:
All comments start with #meta or #cell_meta, which are both aliases for the same thing. For brevity, we will use #meta in this cheatsheet.
Black code formatting
#meta:tag=black will apply black code formatting.
Show/Hide Cells
- Remvoe entire cells:
#meta:tag=remove_cellor#meta:tag=hide - Remove output:
#meta:tag=remove_outputor#meta:tag=remove_outputor#meta:tag=hide_outputsor#meta:tag=hide_output - Remove input: same as above, except
inputinstead ofoutput.
Hiding specific lines of outptut
- Remove lines of output containing keywords:
#meta:filter_words=FutureWarning,MultiIndex - Show maximum number of lines of output:
#meta:limit=6, will show only the first 6 lines
Hiding specific lines of input (code):
Use the comment #meta_hide_line to hide a specific line of code:
def show():
a = 2
b = 3 #meta_hide_line
Selecting Metaflow Steps
You can selectively show meataflow steps in the output logs:
- Show one step:
#meta:show_steps=<step_name> - Show multiple steps:
#meta:show_steps=<step1_name>,<step2_name>
To inject metadata make a comment in a cell with the following pattern: #cell_meta:{key=value}. Note that #meta is an alias for #cell_meta
For example, consider the following code:
_test_file = 'test_files/hello_world.ipynb'
first_cell = read_nb(_test_file)['cells'][0]
print(first_cell['source'])
At the moment, this cell has no metadata:
print(first_cell['metadata'])
However, after we process this notebook with InjectMeta, the appropriate metadata will be injected:
c = Config()
c.NotebookExporter.preprocessors = [InjectMeta]
exp = NotebookExporter(config=c)
cells, _ = exp.from_filename(_test_file)
first_cell = json.loads(cells)['cells'][0]
assert first_cell['metadata'] == {'nbdoc': {'show_steps': 'start,train'}}
first_cell['metadata']
Gets rid of colors that are streamed from standard out, which can interfere with static site generators:
c, _ = run_preprocessor([StripAnsi], 'test_files/run_flow.ipynb')
assert not _re_ansi_escape.findall(c)
This preprocessor inserts a warning in the markdown destination that the file is autogenerated. This warning is inserted in the second cell so we do not interfere with front matter.
c, _ = run_preprocessor([InsertWarning], 'test_files/hello_world.ipynb', display_results=True)
assert "<!-- WARNING: THIS FILE WAS AUTOGENERATED!" in c
Notice how this notebook has an empty code cell at the end:
show_plain_md('test_files/hello_world.ipynb')
With RmEmptyCode these empty code cells are stripped from the markdown:
c, _ = run_preprocessor([RmEmptyCode], 'test_files/hello_world.ipynb', display_results=True)
assert len(re.findall('```python',c)) == 1
When you run a metaflow Flow, you are presented with a fair amount of boilerpalte before the job starts running that is not necesary to show in the documentation:
show_plain_md('test_files/run_flow.ipynb')
We don't need to see the beginning part that validates the graph, and we don't need the time-stamps either. We can remove these with the MetaflowTruncate preprocessor:
c, _ = run_preprocessor([MetaflowTruncate], 'test_files/run_flow.ipynb', display_results=True)
assert 'Validating your flow...' not in c
Consider this python notebook prior to processing. The comments can be used configure the visibility of cells.
#cell_meta:tags=remove_outputwill just remove the output#cell_meta:tags=remove_inputwill just remove the input#cell_meta:tags=remove_cellwill remove both the input and output
Note that you can use #cell_meta:tag or #cell_meta:tags as they are both aliases for the same thing. Here is a notebook before preprocessing:
show_plain_md('test_files/visibility.ipynb')
UpdateTags is meant to be used with InjectMeta and TagRemovePreprocessor to configure the visibility of cells in rendered docs. Here you can see what the notebook looks like after pre-processing:
_test_file = 'test_files/visibility.ipynb'
c = Config()
c.TagRemovePreprocessor.remove_cell_tags = ("remove_cell",)
c.TagRemovePreprocessor.remove_all_outputs_tags = ('remove_output',)
c.TagRemovePreprocessor.remove_input_tags = ('remove_input',)
c.MarkdownExporter.preprocessors = [InjectMeta, UpdateTags, TagRemovePreprocessor]
exp = MarkdownExporter(config=c)
result = exp.from_filename(_test_file)[0]
# show the results
assert 'you will not be able to see this cell at all either' not in result
print(result)
MetaflowSelectSteps is meant to be used with InjectMeta to only show specific steps in the output logs from Metaflow.
For example, if you want to only show the start and train steps in your flow, you would annotate your cell with the following pattern: #cell_meta:show_steps=<step_name>
Note that show_step and show_steps are aliases for convenience, so you don't need to worry about the s at the end.
In the below example, #cell_meta:show_steps=start,train shows the start and train steps, whereas #cell_meta:show_steps=train only shows the train step:
c, _ = run_preprocessor([InjectMeta, MetaflowSelectSteps],
'test_files/run_flow_showstep.ipynb',
display_results=True)
assert 'end' not in c
If we want to exclude output with certain keywords, we can use the #meta:filter_words comment. For example, if we wanted to ignore all output that contains the text FutureWarning or MultiIndex we can use the comment:
#meta:filter_words=FutureWarning,MultiIndex
Consider this output below:
show_plain_md('test_files/strip_out.ipynb')
Notice how the lines containing the terms FutureWarning or MultiIndex are stripped out:
c, _ = run_preprocessor([InjectMeta, FilterOutput],
'test_files/strip_out.ipynb',
display_results=True)
assert 'FutureWarning:' not in c and 'from pandas import MultiIndex, Int64Index' not in c
c, _ = run_preprocessor([InjectMeta, Limit],
'test_files/limit.ipynb',
display_results=True)
_res = """```
hello
hello
hello
hello
hello
...
```"""
assert _res in c
You can use the special comment #meta_hide_line to hide a specific line of code in a code cell. This is what the code looks like before:
show_plain_md('test_files/hide_lines.ipynb')
and after:
c, _ = run_preprocessor([InjectMeta, HideInputLines],
'test_files/hide_lines.ipynb',
display_results=True)
WriteTitle creates the proper code-fence with a title in the situation where the %%writefile magic is used.
For example, here are contents before pre-processing:
show_plain_md('test_files/writefile.ipynb')
When we use WriteTitle, you will see the code-fence will change appropriately:
c, _ = run_preprocessor([WriteTitle], 'test_files/writefile.ipynb', display_results=True)
assert '```py title="myflow.py"' in c and '```txt title="hello.txt"' in c
c, _ = run_preprocessor([CleanFlags], _gen_nb())
assert '#notest' not in c
CleanMagics strips magic cell commands %% so they do not appear in rendered markdown files:
c, _ = run_preprocessor([WriteTitle, CleanMagics], 'test_files/writefile.ipynb', display_results=True)
assert '%%' not in c
Here is how CleanMagics Works on the file with the Metaflow log outputs from earlier, we can see that the #cell_meta comments are gone:
c, _ = run_preprocessor([InjectMeta, MetaflowSelectSteps, CleanMagics],
'test_files/run_flow_showstep.ipynb', display_results=True)
Black is a preprocessor that will format cells that have the cell tag black with Python black code formatting. You can apply tags via the notebook interface or with a comment meta:tag=black.
This is how cell formatting looks before black formatting:
show_plain_md('test_files/black.ipynb')
After black is applied, the code looks like this:
c, _ = run_preprocessor([InjectMeta, UpdateTags, CleanMagics, Black], 'test_files/black.ipynb', display_results=True)
assert '[1, 2, 3]' in c
assert 'very_important_function(\n template: str,' in c
When we issue a shell command in a notebook with !, we need to change the code-fence from python to bash and remove the !:
c, _ = run_preprocessor([MetaflowTruncate, CleanMagics, BashIdentify], 'test_files/run_flow.ipynb', display_results=True)
assert "```bash" in c and '!python' not in c
_result, _ = run_preprocessor([CleanShowDoc], 'test_files/doc.ipynb')
assert '<HTMLRemove>' not in _result
print(_result)
get_mdx_exporter combines all of the previous preprocessors, along with the built in TagRemovePreprocessor to allow for hiding cell inputs/outputs based on cell tags. Here is an example of markdown generated from a notebook with the default preprocessing:
show_plain_md('test_files/example_input.ipynb')
Here is the same notebook, but with all of the preprocessors that we defined in this module. Additionally, we hide the input of the last cell which prints hello, you should not see the print statement... by using the built in TagRemovePreprocessor:
exp = get_mdx_exporter()
print(exp.from_filename('test_files/example_input.ipynb')[0])