path: root/PLAN.md

# Recursive Markdown rendering templates

## Goal

Let site templates override the HTML rendering of every element-producing
CommonMark or GFM construct supported by Weft. Overrides must compose
recursively: a node template receives its already-rendered children as
`.Content`, and nested overrides continue to apply inside it.

The feature is complete when a site can put `class="weft"` on every HTML
element produced from Markdown without replacing page templates or disabling
Goldmark parsing.

## Constraints

- Keep Goldmark as the Markdown parser and default HTML renderer.
- Preserve current output when no Markdown node templates are defined.
- Preserve `.Sections` as `[]template.HTML`, with one fragment per top-level
  Markdown node in source order.
- Keep raw Markdown HTML disabled. Do not provide an override that can turn an
  omitted raw HTML node back on.
- Continue using `html/template`; do not mark destinations or other
  user-controlled strings as `template.URL`.
- Add exactly one production dependency, `github.com/alecthomas/chroma/v3`, for
  fenced-code syntax highlighting. Add no configuration or plugin layer.
- Do not expose Goldmark AST nodes to site templates.
- Implement production behavior before changing tests that currently assume
  sections are rendered during discovery.

## Template naming and fallback

Markdown rendering templates share the existing site template set and use the
`markdown/` prefix:

```gotemplate
{{define "markdown/paragraph"}}<p class="weft">{{.Content}}</p>{{end}}
{{define "markdown/strong"}}<strong class="weft">{{.Content}}</strong>{{end}}
```

For each node:

1. Render its children recursively.
2. Execute its `markdown/...` template when that template is defined.
3. Otherwise emit Goldmark's current default HTML for that node.

An override may intentionally omit `.Content` to discard the node's children.
One subtype override must not disable defaults for sibling subtypes. For
example, defining only `markdown/strong` must leave ordinary emphasis rendered
as `<em>` while still allowing the nested strong override to run.

Reject defined template names beginning with `markdown/` when they are not in
the supported-name table below. This makes misspelled overrides build errors
instead of silently unused templates. Other template names remain unaffected.

## Data conventions

- `.Content` is always `template.HTML` and always means the node's primary
  rendered content. For container nodes it is recursively rendered children;
  for code spans and code blocks it is escaped code text.
- `.Content` is omitted only for nodes with no child content, such as images,
  task checkboxes, thematic breaks, and hard breaks.
- `.Destination`, `.Title`, `.Alt`, `.Language`, and `.ID` are plain strings so
  `html/template` applies the correct contextual escaping.
- `markdown/fenced_code_block` also receives `.HighlightedContent`. It is
  `template.HTML` containing only Chroma-generated token markup, with no code
  block wrappers, and is empty when the fence has no known language or cannot
  be highlighted. Its `.Content` remains the escaped source-code fallback.
- Optional string values are the empty string when absent.
- No generic `.Node`, `.Attributes`, `.Text`, or metadata map is exposed.

## Supported template objects

### Core block constructs

| Template | Fields | Meaning |
| --- | --- | --- |
| `markdown/heading1` through `markdown/heading6` | `Content template.HTML`, `ID string` | Heading contents and Goldmark's generated heading ID. |
| `markdown/paragraph` | `Content template.HTML` | Rendered inline paragraph contents. |
| `markdown/blockquote` | `Content template.HTML` | Rendered block contents. |
| `markdown/code_block` | `Content template.HTML` | Escaped contents of an indented code block. The template owns both `<pre>` and `<code>` wrappers. |
| `markdown/fenced_code_block` | `Content template.HTML`, `HighlightedContent template.HTML`, `Language string` | Escaped code, optional Chroma token markup, and the first word of the fence info string. The template owns both wrappers. |
| `markdown/unordered_list` | `Content template.HTML`, `Tight bool` | Rendered list items. |
| `markdown/ordered_list` | `Content template.HTML`, `Start int`, `Tight bool` | Rendered list items and the first source marker's number. `Start` is populated even when it is `1`. |
| `markdown/list_item` | `Content template.HTML` | Rendered block contents, including nested lists. |
| `markdown/thematic_break` | no fields | A thematic break such as `---`; the template owns the `<hr>`. |

`Tight` follows CommonMark. Tight list items contain unwrapped inline content;
loose list item paragraphs pass through `markdown/paragraph` and normally
produce `<p>` elements.

Do not expose list marker, indentation offset, item index, or item number.
Goldmark keeps `Start` on the list, while CommonMark disregards numeric markers
after the first item. The ordered-list wrapper is responsible for `<ol
start="...">`; `<li>` elements need only `.Content`.

### Core inline constructs

| Template | Fields | Meaning |
| --- | --- | --- |
| `markdown/emphasis` | `Content template.HTML` | Contents of `*emphasis*`. |
| `markdown/strong` | `Content template.HTML` | Contents of `**strong emphasis**`. |
| `markdown/code_span` | `Content template.HTML` | Escaped, Goldmark-normalized inline code contents. |
| `markdown/link` | `Content template.HTML`, `Destination string`, `Title string` | Explicit link label, rewritten destination, and optional Markdown title. |
| `markdown/autolink` | `Content template.HTML`, `Destination string` | URL autolink label and destination. |
| `markdown/email_autolink` | `Content template.HTML`, `Destination string` | Email label and a destination including the `mailto:` prefix. |
| `markdown/image` | `Alt string`, `Destination string`, `Title string` | Flattened plain-text image description, destination, and optional title. Images have no rendered child content. |
| `markdown/hard_break` | no fields | The `<br>` produced by a Markdown hard line break. |

Image formatting is flattened for `.Alt`, matching Goldmark:

```markdown
![A **very** black cat](cat.jpg "Mochi")
```

provides `Alt: "A very black cat"`, not HTML containing `<strong>`.

Code-span `.Content` must preserve Goldmark's code-span whitespace
normalization before being HTML-escaped. Plain text and soft line breaks do not
produce elements and therefore have no override.

### GFM constructs

| Template | Fields | Meaning |
| --- | --- | --- |
| `markdown/strikethrough` | `Content template.HTML` | Contents of `~~strikethrough~~`. |
| `markdown/task_checkbox` | `Checked bool` | Disabled task-list checkbox; the template owns the `<input>`. |
| `markdown/table` | `Content template.HTML` | Rendered table header followed by an optional table body. |
| `markdown/table_header` | `Content template.HTML` | Rendered header cells. The template owns both `<thead>` and its `<tr>`. |
| `markdown/table_body` | `Content template.HTML` | Rendered body rows. This is a synthetic Weft node for Goldmark's generated `<tbody>`. It is omitted when there are no body rows. |
| `markdown/table_row` | `Content template.HTML` | Rendered body cells; the template owns `<tr>`. |
| `markdown/table_header_cell` | `Content template.HTML`, `Alignment string` | Header-cell contents and `left`, `right`, `center`, or `none`. |
| `markdown/table_cell` | `Content template.HTML`, `Alignment string` | Body-cell contents and `left`, `right`, `center`, or `none`. |

The synthetic `table_body` is required because Goldmark's AST has table,
header, row, and cell nodes but its HTML renderer creates `<tbody>` as a side
effect between those nodes. Modeling the body explicitly makes partial table
overrides composable and lets a site class the generated `<tbody>`.

Raw inline HTML and raw HTML blocks remain rendered as Goldmark's omission
comments and have no template names. Document, text-block, text, and string
nodes do not emit their own HTML elements and also have no templates.

## Representative usage

```gotemplate
{{define "markdown/ordered_list"}}
<ol class="weft"{{if ne .Start 1}} start="{{.Start}}"{{end}}>{{.Content}}</ol>
{{end}}

{{define "markdown/list_item"}}
<li class="weft">{{.Content}}</li>
{{end}}

{{define "markdown/fenced_code_block"}}
<pre class="weft chroma"><code class="weft language-{{.Language}}">{{if .HighlightedContent}}{{.HighlightedContent}}{{else}}{{.Content}}{{end}}</code></pre>
{{end}}

{{define "markdown/table_header"}}
<thead class="weft"><tr class="weft">{{.Content}}</tr></thead>
{{end}}

{{define "markdown/table_body"}}
<tbody class="weft">{{.Content}}</tbody>
{{end}}
```

## Build-time syntax highlighting

Defining `markdown/fenced_code_block` activates build-time highlighting through
`github.com/alecthomas/chroma/v3`. It emits static, class-based HTML; the site
supplies matching CSS for the fixed Chroma token-class prefix in its normal
stylesheet, including any light/dark theme rules. No JavaScript, client-side
highlighter, external executable, generated stylesheet, or new CLI option is
involved.

The Weft fenced-code handler owns the node when this template is defined:

1. Extract the raw fence contents and the existing first-word `.Language`.
2. Look up that exact language in Chroma. Do not guess a lexer for blank or
   unknown info strings.
3. For a known lexer, format the code with Chroma's class-based HTML formatter,
   configured not to emit its own code-block wrappers, and assign that trusted
   fragment to `.HighlightedContent`.
4. On an absent/unknown language or any Chroma tokenization/formatting failure,
   leave `.HighlightedContent` empty. `.Content` remains the escaped source in
   every case, so templates have a deterministic safe fallback.
5. Execute the fenced-code template. Indented `markdown/code_block` nodes are
   not highlighted by this feature.

Only Chroma's output may become `template.HTML`; the Markdown code source must
never be trusted directly. A fence containing text such as
`</span><script>…</script>` must remain escaped even when highlighting succeeds.

### Rejected integrations

Goldmark remains the renderer that walks every rendered AST subtree. Weft
renders one direct child of the document at a time solely to preserve the
existing `.Sections` contract—one `template.HTML` fragment for each top-level
Markdown node. Rendering the whole document once would lose those source-node
boundaries; moving the same per-child buffering into a document renderer would
only obscure the current `renderSections` responsibility.

Do not register `goldmark-highlighting` as a Goldmark extension. Like Weft's
planned handler, it registers a renderer for `ast.KindFencedCodeBlock`, but it
writes the complete highlighted block, including its own wrappers, directly to
Goldmark's output writer. It therefore cannot compose with a template that
owns the wrappers and needs a `.HighlightedContent` value. Its useful
integration is already just Chroma tokenization and HTML formatting, and its
fenced-block render method is not a public fragment-returning API; invoking it
and stripping wrappers would be brittle. It also depends on Chroma v2 rather
than the chosen v3. Call Chroma directly from Weft's fenced-code handler
instead.

## Build lifecycle refactor

Current discovery renders `.Sections` before site templates are loaded. Split
parsing from rendering:

1. Create one site-level Goldmark instance with GFM and automatic heading IDs.
2. During discovery, read frontmatter, parse each Markdown body, rewrite links,
   derive page metadata, and retain the source bytes and AST on unexported page
   fields. Leave `.Sections` empty.
3. Discover and load the complete site template set as today.
4. Validate `markdown/` template names and configure the Markdown template
   renderer, including the Chroma-backed fenced-code handler when that template
   is defined, before Goldmark's renderer is used for the first time.
5. Render every page's retained top-level AST nodes into `.Sections`.
6. Only after every page has sections, render Markdown pages and explicit
   `.html.tmpl` or `.xml.tmpl` outputs. This preserves feeds and other templates
   that consume another page's `.Sections`.
7. Keep validation and atomic output writing unchanged.

Retaining parse state should use unexported fields on `Page` rather than a new
public model. Site templates must continue to see only the documented exported
page fields.

## Renderer design

Add a small Goldmark `NodeRenderer` backed by the loaded `html/template` set.
It should register handlers only for node kinds affected by at least one
defined override, except for the table adapter described below.

For an overridden container node:

1. On the entering callback, recursively render each child through the same
   fully configured Goldmark renderer into a buffer.
2. Execute the selected site template with typed data containing that buffer as
   `.Content`.
3. Return `ast.WalkSkipChildren` so the outer traversal does not render the
   subtree twice.
4. Do nothing on the matching exiting callback.

Re-entering the configured renderer preserves custom templates at every nested
level while allowing an unhandled parent to continue through Goldmark's stock
renderer. Do not remove or reparent AST nodes while rendering.

Goldmark represents headings, emphasis/strong, ordered/unordered lists, and
URL/email autolinks with shared node kinds. If any subtype is overridden, the
handler must dispatch by subtype and emit Goldmark-equivalent wrapper HTML for
an unoverridden sibling subtype while still recursively rendering its children.

Leaf handlers extract their values directly:

- Escape and normalize code before assigning trusted `.Content`. The fenced-code
  handler additionally follows the build-time syntax-highlighting contract.
- Flatten image descendants to plain `.Alt` independently of inline override
  templates.
- Resolve email autolink destinations to include `mailto:`.
- Leave explicit link destinations as plain strings after existing `.md` to
  `.html` rewriting.

Hard breaks need a narrowly scoped text-node handler only when
`markdown/hard_break` is defined. It must preserve Goldmark's existing text
escaping and newline behavior and replace only the generated `<br>` fragment.

### Table adapter

If no table-related override exists, leave the complete table subtree to
Goldmark for exact current output. If any table-related override exists,
intercept the table root and render its semantic pieces recursively:

```text
table
├── table_header
│   └── table_header_cell...
└── table_body (only with body rows)
    └── table_row...
        └── table_cell...
```

For each undefined table template, emit the equivalent current Goldmark HTML
wrapper. This localized fallback is necessary because Goldmark opens and closes
`<tbody>` across separate AST callbacks, which makes independent partial
overrides invalid without the adapter. Preserve Goldmark's current cell
alignment output when a cell template is absent.

## Error handling

- Wrap Markdown template execution errors with the page source path and
  template name, for example `render notes/example.md markdown/link: ...`.
- Treat unknown `markdown/` definitions as template-loading errors and list the
  unsupported name.
- Propagate recursive child-render errors without continuing the build.
- Preserve the existing all-or-nothing output behavior: no files change after
  any Markdown rendering failure.

## Production changes

Keep the implementation to the existing files unless the renderer makes
`main.go` materially harder to follow:

1. Change page parsing in `main.go` to retain Markdown source and AST without
   rendering sections.
2. Add the post-template-load section-rendering phase.
3. Add the node-template lookup, typed template-data values, recursive
   renderer, hard-break handler, table adapter, and direct Chroma v3
   fenced-code highlighter.
4. Keep `renderSections` responsible for the top-level `.Sections` boundary,
   but have it use the newly configured renderer.
5. Update `README.md` with the lifecycle-neutral public contract, full template
   table, list-numbering rule, escaping/highlighting behavior, and
   representative examples.

No CLI flag, manifest change, default-template file, or public AST wrapper is
needed. Add only the Chroma v3 production dependency.

## Checks after production behavior exists

Use the existing `main_test.go`; add the smallest set of table-driven or
end-to-end checks that covers the contract:

1. **No-override compatibility:** Existing Markdown/GFM output and `.Sections`
   behavior remain unchanged when no `markdown/` definitions exist.
2. **Recursive composition:** A paragraph containing emphasis, strong text,
   code, and a link receives every nested override once and in source order.
3. **Subtype fallback:** Defining only `markdown/strong`, one heading level, or
   one list type leaves sibling subtypes at Goldmark defaults while nested
   overrides still apply.
4. **Block fields and highlighting:** Heading IDs, fenced-code language,
   escaped code content, Chroma token markup for a known language, an empty
   highlighted value for blank/unknown languages, and blockquote content reach
   templates correctly. Highlighted hostile source remains escaped.
5. **List contract:** Tight and loose lists report `.Tight`; ordered lists
   beginning with `1`, repeated `1`, repeated `3`, and arbitrary later numbers
   expose only the correct first-item `.Start`; nested lists remain recursive.
6. **Link and image fields:** Explicit link title and rewritten destination,
   URL/email autolinks, image title, and formatting-flattened `.Alt` are exact.
7. **GFM fields:** Strikethrough, checked and unchecked task boxes, all table
   pieces, empty table bodies, and each alignment value render through their
   templates.
8. **Generated elements:** A representative document defining every supported
   override can place `class="weft"` on every emitted Markdown HTML element,
   including `<br>`, `<input>`, and `<tbody>`.
9. **Partial table overrides:** Overriding only a cell or only `table_body`
   still yields valid, balanced table HTML with default wrappers elsewhere.
10. **Safety:** Code and alt text stay escaped, dangerous custom link
    destinations remain subject to `html/template` URL filtering, and raw HTML
    cannot be re-enabled.
11. **Errors and atomicity:** Unknown names and execution failures identify the
    page/template and preserve previously generated files.
12. **Cross-page consumers:** Feed and explicit output templates see populated
    `.Sections` for every page regardless of source sort order.

Update discovery-focused tests that currently inspect `.Sections` immediately
after `discover()` to invoke the new preparation phase or assert only discovery
state. Do not retain eager production rendering solely for those tests.

## Acceptance criteria

- A site can override every template listed above independently.
- Recursive `.Content` works through arbitrarily nested supported nodes.
- A site can mark every Markdown-generated HTML element with a class.
- Missing overrides preserve Goldmark behavior, including inside overridden
  ancestors and around overridden descendants.
- Existing sites with no `markdown/` templates produce the same pages.
- `.Sections`, page inventory, feed generation, link rewriting, validation,
  output ownership, and atomic writes retain their current public behavior.
- Fenced-code highlighting produces static Chroma markup only when its node
  template is defined; blank/unknown languages fall back to escaped source and
  no client-side JavaScript is required.
- Documentation defines every template name and every available field.