[RFC] Optional CommonMark/GFM rendering of README files with markdown-it-py

[橋本 憲一]

Hello,

README files written for GitHub, GitLab or other CommonMark based tools
often render poorly in Kallithea, because MarkupRenderer uses
Python-Markdown, which follows the original Markdown.pl dialect:

 - a list directly after a paragraph line is not recognized (a blank
   line is required)
 - nested lists need 4 spaces of indentation; 2 or 3 are flattened
 - ~~strikethrough~~, task lists ("- [ ] item") and bare URLs are not
   supported
 - table column alignment and <details>/<summary> are removed by the
   sanitizer

setup.py also still pins "Markdown >= 3.0, < 3.2" with a TODO to
upgrade.

I see two possible directions and would like to hear which one (if any)
would be acceptable before sending patches:

 A) Upgrade Python-Markdown and add pymdown-extensions (tilde, tasklist,
    magiclink). This keeps the current dialect, so existing READMEs
    render as before, but the first two points above remain.

 B) Add markdown-it-py (CommonMark, MIT licensed, pure Python) as an
    optional renderer, with the gfm-like preset plus mdit-py-plugins
    (task lists, footnotes, definition lists). Python-Markdown stays the
    default, so nothing changes unless an administrator opts in.

We have implemented B in an in-house fork and run it in production:

 - an ini setting "markdown_renderer = python-markdown | markdown-it"
   selects the server default
 - a select box on the repository settings page overrides it per
   repository (stored as a repository extra field, so no schema change)
 - the README cache key includes the renderer
 - fenced code blocks are highlighted with Pygments using the same
   markup as codehilite, so existing CSS applies
 - the sanitizer additionally allows del, s, details/summary, kbd,
   section (footnotes), text-align, and disabled task list checkboxes;
   any other input element is dropped by a bleach filter
 - markdown-it-py and plugins are an optional extra; if missing,
   rendering falls back to Python-Markdown
 - tests cover sanitizing with both renderers and the new syntax

The sanitizer part builds on the bleach >= 5 patch sent earlier
("[PATCH] markup_renderer: support bleach >= 5").

Would a patch series for B be welcome, or would you prefer A, or a
smaller first step (e.g. only the ini setting without the per
repository select)?

Thanks,
--
Hashimoto Kenichi