Responsive, reproducible, collaborative: calculation notebooks are evolving
This year marks the tenth anniversary of the launch of the IPython notebook. The open-source tool, now known as Jupyter Notebook, has grown into a hugely popular data science kit, with millions of notebooks uploaded to the GitHub code-sharing site.
Calculation notebooks combine code, results, text, and images into a single document, resulting in what Stephen Wolfram, creator of the Mathematica software package, called a “calculation essay”. And whether they’re written using Jupyter, Mathematica, RStudio, or any other platform, researchers can use them for iterative data mining, communication, teaching, and more.
But math books can also be confusing and promote bad coding practices. And they’re hard to share, collaborate, and replicate. A 2019 study found that only 24% of the 863,878 Jupyter notebooks available to the public on GitHub could be successfully rerun, and only 4% produced the same results (JF Pimentel et al. in 2019 16th IEEE / ACM International Conference on Mining Software Repositories (MSR) 507–517; IEEE, 2019).
“Laptops are a mess,” says Anita Sarma, a computer scientist at Oregon State University at Corvallis, who studies human-machine interaction. “You write stuff, you keep the crisp old code behind, and it’s hard to know which cells to run in which order, because you were trying different things.”
But a growing suite of platforms and tools aim to smooth out those rough edges. Some make notebooks “responsive”, so the code runs again whenever the software variables change; others focus on collaboration and version control. But all offer researchers innovative ways to explore, document, and share their data with their colleagues and the world.
For Sergei Pond, the notebooks provided a way to document the genetics of the pandemic. Pond, a computer biologist at Temple University in Philadelphia, Pa., Has created some three dozen documents related to SARS-CoV-2, the virus that causes COVID-19. “My default setting,” he says, is to “write an interactive notebook and send it to my people so they can play with the data, [so] they can immediately see what’s there. “
Pond, for example, uses Observable to share colored maps, graphs, protein structures, and sequence alignments that represent data it generates in other software. The modular structure of Observable means that other programmers can easily apply these visualizations to their own data. But Pond’s notebooks also take advantage of another key feature of the Observable: responsiveness.
Suppose you have a Jupyter notebook that draws a line. In a code cell, you define the slope and y-intercept; in the next, you draw the graph. The notepad structure allows encoders to go back to the previous cell to change the slope after rendering the plot. But this change does not entail the automatic redefinition of the figure; the user must manually re-run the cell that traces it.
This workflow can lead to “state issues”, in which a notebook’s output does not reflect its code – as would happen, for example, if the user deletes the cell that defines a variable after its execution. In 2018, Joel Grus, then a software engineer at the Allen Institute for Artificial Intelligence in Seattle, Washington, highlighted this behavior and the confusion that resulted in a widely circulated speech titled “I Don’t Like Notebooks.” . But, “to a large extent having fully responsive laptops eliminates this functionality,” he says now.
Responsive notebooks are similar to spreadsheets, says Bostock. Just as Microsoft Excel knows how to recalculate a formula if the underlying cells change, Responsive Notebooks follow the relationship between code cells to ensure that a notebook’s output always reflects its variables.
Combined with visual widgets, such as sliders and drop-down lists, this behavior makes notebooks interactive, allowing readers to explore how changing variables or assumptions can affect results. Herb Susmann, a doctoral student in biostatistics at the University of Massachusetts at Amherst, for example, uses reactive papers to explain statistical concepts. “It really helps me get a more visceral idea of how these stats work,” he says. (That said, responsiveness isn’t always desirable, especially if cells take a long time to run or when the datasets are very large.)
React and collaborate
Fernando Pérez, co-founder of the Jupyter project at the University of California at Berkeley, notes that Jupyter himself “is agnostic on the subject of responsiveness.” Most cores so far have been unreactive, but they don’t have to be: Richa Gadgil, a former Jupyter intern at California Polytechnic State University in San Luis Obispo, for example, spent her internship at co- develop an experimental reactive kernel for Python. “It was a test of the Jupyter architecture and the Jupyter architecture passed that test,” says Brian Granger, who led his work.
Another open-source system, called Vizier, focuses on data-driven responsiveness, says Juliana Freire, a computer scientist at New York University, who co-led the project. With built-in data validation functions and a spreadsheet interface, Vizier users can massage their data to correct inconsistencies – such as those caused by a column that contains both “ Y / N ” responses. and “ yes / no ”. During this time, the notebook is rerun. “You analyze, you clean, you analyze, you clean,” says Freire. “And by doing that, you are recording where the whole process came from.” As a result, users can go back to an earlier cleaning step and try again, saving any changes they’ve made all the time. (Vizier notebooks can handle Python, SQL, and Scala code.)
Some commercial responsive systems, including JetBrains’ Observable, Deepnote, and Datalore (the latter two of which are based in the Czech Republic), also point to another pain point of laptops: collaboration. Observable, for example, enables real-time collaborative editing, like Google Docs does, as well as comments. There are two plan levels: Personal (free for up to 5 members in the same interactive document) and Teams (for 6 or more members: US $ 15 per publisher, per month; free for viewers).
Gábor Csányi, who studies molecular modeling at the University of Cambridge, UK, uses Deepnote (free for up to 3 contributors, then $ 12 per user per month) in his teaching. With his university’s old system, a student seeking help could share a copy of a notebook with Csányi, but it was not possible for both of them to see and edit the same document at the same time. “It was kind of a pain,” he says. But with Deepnote, it can help students debug their code in real time. “Just like you do with Google Docs, we see each other’s cursors. We edit the same notebook, and as they press shift-enter on a cell, I see the result. It was an amazing experience in how a personalized support could be provided effectively. ”
Real-time collaboration is also “a topic of massive activity” in the Jupyter project, Pérez says, and a prototype in development is available on GitHub. “I’m pretty optimistic that this will happen soon,” he said.
Another notebook challenge is also addressed by many commercial platforms: version control.
The Jupyter Notebooks file format includes code, metadata, and calculation output. Since these outputs are often binary images, version control – the process used by developers to track file progress, optimized for plain text files – can become difficult. To complicate matters, programmers may find it difficult to adapt standard version control workflows to the fast and iterative nature of data mining. As a result, crucial experimental details may be lost.
Commercial platforms tend to provide built-in notebook version management. For those who prefer to stick with Jupyter, two plug-ins are available: nbdime, which provides an intelligent and structured view of file changes, including graphical output; and Verdant, which offers a graphical interface that tracks how cells are edited, rearranged, and executed.
According to developer Mary Beth Kery, who studies human-computer interaction at Carnegie Mellon University in Pittsburgh, Pa., Verdant can facilitate interactions with collaborators and peer reviewers. “Someone will say, oh, have you tried that in the model or have you tried this analysis?” she says. Many times the answer is yes, but because the scan did not work, the code is deleted. “What you want to do in the meeting is just pull it up and say, oh, yeah, I did, and here’s why it didn’t work.” And our tool lets you do it. “
Such features can make an already user-friendly computing paradigm even more user-friendly – and easier to share. And that makes them even more powerful vectors of science communication. “If you do some really great science but no one understands it or no one has access to it, what’s the point?” Said Drake. “These types of notebooks can really turn people on and expose people to concepts that are otherwise impenetrable.”