Lightbox cell option ignored when using Jupyter engine

[mcanouil]

Bug description

When a .qmd/.ipynb document uses engine: jupyter, the lightbox.group cell option set via #| YAML comments is silently ignored (passed on to "metadata" in Jupyter notebook).
Each figure ends up in its own auto-generated gallery instead of the shared gallery specified by group.
The same option works correctly with the knitr engine.

This was reported in Discussion #14392.

Steps to reproduce

---
title: "Lightbox Example"
format:
  html:
    lightbox:
      match: auto
engine: jupyter
---

## First plot

```{python}
#| label: fig-sine
#| fig-cap: "Sine plot"
#| lightbox:
#|   group: figs

import matplotlib.pyplot as plt
import numpy as np

x = np.linspace(0, 10, 100)
plt.figure()
plt.plot(x, np.sin(x))
plt.title("Sine")
plt.show()
```

## Second plot

```{python}
#| label: fig-cosine
#| fig-cap: "Cosine plot"
#| lightbox:
#|   group: figs

plt.figure()
plt.plot(x, np.cos(x))
plt.title("Cosine")
plt.show()
```

Actual behaviour

Clicking either figure opens the lightbox, but no gallery navigation arrows appear — each image is in its own isolated gallery.
The rendered <a> elements have data-gallery set to auto-generated values (quarto-lightbox-gallery-1, quarto-lightbox-gallery-2) rather than figs.

Expected behaviour

Both figures should be in the same gallery (data-gallery="figs"), so the lightbox shows navigation arrows that cycle between the two plots.

Your environment

• OS: macOS 25.3.0

Quarto check output

Quarto 99.9.9
[✓] Checking environment information...
      Quarto cache location: /Users/mcanouil/Library/Caches/quarto
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.8.3: OK
      Dart Sass version 1.87.0: OK
      Deno version 2.4.5: OK
      Typst version 0.14.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 99.9.9
      commit: 2355f229b2764eae02b1f9916ac6e208d0909d02
      Path: /Users/mcanouil/Projects/quarto-dev/quarto-cli/package/dist/bin

[✓] Checking tools....................OK
      TinyTeX: v2026.02
      Chrome Headless Shell: (not installed)
      VeraPDF: (not installed)

[✓] Checking LaTeX....................OK
      Using: TinyTex
      Path: /Users/mcanouil/Library/TinyTeX/bin/universal-darwin
      Version: 2025

[✓] Checking Chrome Headless....................OK
      Using: Chrome from QUARTO_CHROMIUM
      Path: /Applications/Brave Browser.app/Contents/MacOS/Brave Browser

[✓] Checking basic markdown render....OK

(/) Checking R installation...........ℹ R version 4.5.3 (2026-03-11)
! Config '~/.Rprofile' was loaded!
[✓] Checking R installation...........OK
      Version: 4.5.3
      Path: /Library/Frameworks/R.framework/Resources
      LibPaths:
        - /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/library
      knitr: 1.50
      rmarkdown: 2.30

[✓] Checking Knitr engine render......OK

[✓] Checking Python 3 installation....OK
      Version: 3.14.4
      Path: /Users/mcanouil/Projects/quarto-dev/quarto-cli/tests/.venv/bin/python3
      Jupyter: 5.9.1
      Kernels: quarto-brand, uv, julia-1.12, python3

[✓] Checking Jupyter engine render....OK

[✓] Checking Julia installation...

[mcanouil]

Root cause

The bug is a three-step serialisation gap in src/core/jupyter/jupyter.ts.

Step 1 — quartoMdToJupyter

• quarto-cli/src/core/jupyter/jupyter.ts

  Lines 401 to 414 in 2355f22

  	const yamlKeys = Object.keys(yaml);
  	yamlKeys.forEach((key) => {
  	if (key === kCellId) {
  	if (includeIds) {
  	cell.id = String(yaml[key]);
  	}
  	delete yaml[key];
  	} else {
  	if (!kJupyterCellOptionKeys.includes(key)) {
  	cell.metadata[key] = yaml[key];
  	delete yaml[key];
  	}
  	}
  	});

When a .qmd file is converted to a temporary .ipynb before execution, any cell option key not in kJupyterCellOptionKeys (which does not include lightbox) is moved from the #| YAML comments into cell.metadata as a raw JavaScript object, and deleted from the cell source.

Step 2 — jupyterCellWithOptions

• quarto-cli/src/core/jupyter/jupyter.ts

  Lines 937 to 966 in 2355f22

  	for (const key of Object.keys(cell.metadata)) {
  	const value = cell.metadata[key];
  	let jsonEncodedKeyIndex = 0;
  	if (value !== undefined) {
  	if (!value && typeof value === "object") {
  	validMetadata[key] = null;
  	} else if (value && typeof value === "object" && !Array.isArray(value)) {
  	// https://github.com/quarto-dev/quarto-cli/issues/9089
  	// we need to json-encode this and signal the encoding in the key
  	// we can't use the key as is since it may contain invalid characters
  	// and modifying the key might introduce collisions
  	// we ensure the key is unique with a counter, and assume
  	// "quarto-private-*" to be a private namespace for quarto.
  	// we'd prefer to use _quarto-* instead, but Pandoc doesn't allow keys to start
  	// with an underscore.
  	validMetadata[
  	`quarto-private-${++jsonEncodedKeyIndex}`
  	] = JSON.stringify({ key, value });
  	} else if (
  	typeof value === "string" || typeof value === "number" ||
  	typeof value === "boolean" || Array.isArray(value)
  	) {
  	validMetadata[key] = value;
  	} else {
  	error(
  	`Invalid metadata type for key ${key}: ${typeof value}. Entry will not be serialized.`,
  	);
  	}
  	}
  	}

Object-type values in cell.metadata are re-encoded under an obfuscated quarto-private-N key (introduced in #9089 for ipynb serialisation compatibility).
cell.metadata["lightbox"] = { group: "figs" } becomes cell.metadata["quarto-private-1"] = '{"key":"lightbox","value":{"group":"figs"}}'.

Step 3 — mdFromCodeCell

• quarto-cli/src/core/jupyter/jupyter.ts

  Lines 1428 to 1447 in 2355f22

  	const cellOptions = {
  	...cell.metadata,
  	...cell.options,
  	};
  	let forwardedAttrs = false;
  	for (const key of Object.keys(cellOptions)) {
  	if (!kCellOptionsFilter.includes(key.toLowerCase())) {
  	// deno-lint-ignore no-explicit-any
  	let value = (cellOptions as any)[key];
  	if (value !== undefined) {
  	if (typeof value !== "string") {
  	value = JSON.stringify(value);
  	}
  	value = value.replaceAll("'", `\\'`);
  	divMd.push(`${key}='${value}' `);
  	forwardedAttrs = true;
  	}
  	}
  	}

The div-attribute construction loop spreads cell.metadata and emits every key not in kCellOptionsFilter.
The original lightbox key is gone, so the output div contains quarto-private-1='...' instead of lightbox='{"group":"figs"}'.

The Lua lightbox filter checks for div.attributes["lightbox"]:

• quarto-cli/src/resources/filters/layout/lightbox.lua

  Lines 196 to 211 in 2355f22

  	if div.classes:includes("cell") and div.attributes["lightbox"] ~= nil then
  	meta = quarto.json.decode(div.attributes["lightbox"])
  	local codeImgCount = 0
  	div = _quarto.ast.walk(div, {
  	Image = function(imgEl)
  	codeImgCount = codeImgCount + 1
  	if (type(meta) == "table" and meta[kNoLightboxClass] == true) or meta == false then
  	imgEl.classes:insert(kNoLightboxClass)
  	else
  	if not auto and ((type(meta) == "table" and not meta[kNoLightboxClass]) or meta == true) then
  	imgEl.classes:insert(kLightboxClass)
  	end
  	if (type(meta) == "table") then
  	if meta.group then
  	imgEl.attr.attributes.group = meta.group or imgEl.attr.attributes.group
  	end

That attribute is now absent, so the group is never applied.

This does not affect the knitr engine because its cell options go through getDivAttributes() in base.ts directly from cell.options, bypassing the quartoMdToJupyter conversion entirely.

---

Suggested fix

Decode the quarto-private-N entries back to their original key names in mdFromCodeCell, before the div-attribute loop runs:

--- a/src/core/jupyter/jupyter.ts
+++ b/src/core/jupyter/jupyter.ts
@@ -1426,7 +1426,22 @@ async function mdFromCodeCell(
   // forward other attributes we don't know about (combine attributes
   // from options yaml and cell metadata)
-  const cellOptions = {
-    ...cell.metadata,
-    ...cell.options,
-  };
+  // Decode quarto-private-N entries (object-type metadata values that were
+  // re-encoded in jupyterCellWithOptions) back to their original key names,
+  // so that custom cell options such as `lightbox` are correctly forwarded
+  // to the Pandoc div as attributes.
+  const cellMetadataDecoded: Record<string, unknown> = {};
+  for (const [key, value] of Object.entries(cell.metadata)) {
+    if (key.startsWith("quarto-private-") && typeof value === "string") {
+      try {
+        const decoded = JSON.parse(value) as { key: string; value: unknown };
+        cellMetadataDecoded[decoded.key] = decoded.value;
+      } catch {
+        cellMetadataDecoded[key] = value;
+      }
+    } else {
+      cellMetadataDecoded[key] = value;
+    }
+  }
+  const cellOptions = {
+    ...cellMetadataDecoded,
+    ...cell.options, // cell.options takes priority over metadata
+  };

After this change the loop will find lightbox = { group: "figs" }, JSON-stringify it, and emit lightbox='{"group":"figs"}'.
The Lua filter then decodes that JSON string, reads meta.group, and sets the gallery attribute on the wrapping link as expected.