This guide covers running parameterized Jupyter notebooks as batch report generators with papermill. The central idea is that a notebook carries hidden, mutable kernel state, so its saved outputs are not by themselves reproducible - the trustworthy artifact is the notebook re-executed top-to-bottom in a fresh kernel. papermill automates that clean-kernel execution and injects parameters; nbconvert renders the result to a code-hidden report. The decisions that matter are parameter handling, execution controls, how to aggregate results across samples, and pairing execution with a pinned environment so the report is reproducible in practice.
pip install papermill nbconvert scrapbook jupytext nbstripout
# Register the analysis environment as a kernel so papermill can target it
python3 -m ipykernel install --user --name analysis-env
Tell your AI agent what you want to do: - "Parameterize my DE-analysis notebook and run it on all 12 samples" - "Generate a code-hidden HTML report from my analysis notebook" - "Aggregate the per-sample QC metrics from my notebook runs into one table" - "Stop my notebooks from bloating git with their outputs" - "Make my notebook reproduce the same numbers on the cluster"
"Turn my QC notebook into a template that accepts sample_id and input_dir, and run it across the sample sheet"
"Run my analysis template with three FDR thresholds and keep each executed notebook"
"Convert my executed notebooks to HTML with the code hidden"
"Make a PDF report from my notebook without installing LaTeX"
"Collect the AUC and DE-gene count each notebook recorded into a cohort summary table"
"Set up nbstripout so notebook outputs do not get committed, and pin the environment so reruns match"
parameters-tagged cell with sensible defaultsexecution_timeout, preserving failed notebooks as artifacts--no-input, --to webpdf when LaTeX is unavailable)scrapbook-glued metrics across runs into one tablenbconvert --execute) rather than trusting them.-r name value (raw string) for parameters that look numeric but are identifiers (sample 007, chromosome 1); -p would coerce them.execution_timeout - the default is forever, which hangs a pipeline on one slow cell.