Who am I¶

Marc Wouts

Researcher at Capital Fund Management
Mathematician by training
Author of Jupytext

Why Jupytext?¶

Jupyter Notebooks are great!

But .ipynb files have a few limitations:

You can only edit them in Jupyter (1)
They are not so well suited for version control (2), (3)

(1) Well, this is changing, see Jupyter Notebooks in the IDE: VS Code versus PyCharm
(2) Too large, mix user inputs with programmatic outputs
(3) JSON is hard to merge

The idea behind Jupytext

Keep using Jupyter Notebooks!
Store them in a format that makes sense outside of Jupyter

Formats supported by Jupytext:

Scripts (Julia, Python, R,...)
Markdown files

Installing Jupytext¶

pip install jupytext or conda install jupytext -c conda-forge

Restart Jupyter... and text files become notebooks!

Before		After

Using text notebooks¶

Demo

example.py has a notebook icon! Open it as a notebook.
Set the appropriate kernel and run the notebook
Modify the file in Jupyter, save, and see the change on the script
Modify the script in a text editor (1), and reload it in Jupyter:
- Inputs are updated
- Outputs are gone (2)
- Python variables are still there
Add the kernel to the .py file with the Include Metadata command

(1) If your edit takes more than 2 minutes, answer 'Reload' to the message in Jupyter telling you that the notebook has changed on disk, and consider a) turning the Jupyter autosave off or b) closing the notebook while you edit the text file
(2) To be solved at the next slide

Paired notebooks¶

Text notebooks with persistent outputs!

The notebook is saved to an .ipynb and a .py file.

Outputs are loaded from the .ipynb file, and inputs are
taken from the .py file if present.

Paired notebooks are robust. You can delete any file in the pair.
Share only one file, the other one will be recreated.

How to pair notebooks¶

Pairing an individual notebook in Jupyter¶

Jupyter Lab	Jupyter Notebook

Pairing notebooks globally using a configuration file ¶

Create a jupytext.toml file with this content:

default_jupytext_formats = ipynb,py:percent

You can also pair in different subfolders:

default_jupytext_formats = notebooks///ipynb,scripts///py:percent

Jupytext at the command line¶

Notebook to text¶

jupytext --to markdown notebook.ipynb
jupytext --to script notebook.ipynb

Text to notebook¶

jupytext --to notebook notebook.py
jupytext --to notebook notebook.md

Text to notebook & preserve outputs¶

jupytext --to notebook --update notebook.py
jupytext --to notebook --update notebook.md

Pair notebooks¶

jupytext --set-formats ipynb,py:percent notebook.ipynb

Synchronize paired notebooks¶

jupytext --sync notebook.ipynb

What can I do with Jupytext?¶

Version control¶

Jupytext formats are concise and have clean diffs

Merge notebooks¶

.py and .md files are easy to merge - unlike .ipynb (JSON)

Write notebooks¶

Copy/paste content from other notebooks
Edit your notebook in your favorite text editor
Or in Jupyter!

Search in notebooks¶

Easily find all the references to a function or a dataset
within your notebooks!

Refactor your notebooks into libraries¶

Access to your notebooks as scripts, and refactor them:

move your code into libraries
document & test it!

Code style and checks¶

You like to apply flake8, black, isort on scripts?
Jupytext let you do the same on notebooks:

jupytext --pipe black notebook.ipynb

Why so many Jupytext formats?¶

Because you can do different things with them!

Notebook content	You want?	Recommended format	Compatible with
Mostly code	A simple script	`py:light`	Everything
	A notebook-like script	`py:percent`	VS Code, PyCharm Pro, Spyder, Hydrogen, PTVS
Text & code	Render the `.md` on GitHub	`md:markdown`	VS Code, PyCharm, GitHub, ...
	Write a book (cross references, biblio)	`md:myst`	Jupyter Book
	Edit a notebook in Jupyter & RStudio	`Rmd`	RStudio, `knitr`, `bookdown`