nijel/mistletoe
A fast, extensible and spec-compliant Markdown parser in pure Python.
mistletoe
mistletoe is a Markdown parser in pure Python,
designed to be fast, spec-compliant and fully customizable.
Apart from being the fastest
CommonMark-compliant Markdown parser implementation in pure Python,
mistletoe also supports easy definitions of custom tokens.
Parsing Markdown into an abstract syntax tree
also allows us to swap out renderers for different output formats,
without touching any of the core components.
Remember to spell mistletoe in lowercase!
Features
-
Fast:
mistletoe is the fastest implementation of CommonMark in Python.
See the performance section for details. -
Spec-compliant:
CommonMark is a useful, high-quality project.
mistletoe follows the CommonMark specification
to resolve ambiguities during parsing.
Outputs are predictable and well-defined. -
Extensible:
Strikethrough and tables are supported natively,
and custom block-level and span-level tokens can easily be added.
Writing a new renderer for mistletoe is a relatively
trivial task.You can even write a Lisp in it.
Output formats
Renderers for the following "core" output formats exist within the mistletoe
main package:
- HTML
- LaTeX
- AST (Abstract Syntax Tree; handy for debugging the parsing process)
- Markdown (Can be used to reflow the text, or make other types of automated
changes to Markdown documents)
Renderers for the following output formats can be found
in the contrib package:
- HTML with MathJax (mathjax.py)
- HTML with code highlighting (using Pygments) (pygments_renderer.py)
- HTML with TOC (for programmatical use) (toc_renderer.py)
- HTML with support for GitHub wiki links (github_wiki.py)
- Jira Markdown (jira_renderer.py)
- XWiki Syntax (xwiki20_renderer.py)
- Scheme (scheme.py)
Installation
mistletoe is tested for Python 3.5 and above. Install mistletoe with pip:
pip3 install mistletoeAlternatively, clone the repo:
git clone https://github.com/miyuchina/mistletoe.git
cd mistletoe
pip3 install -e .This installs mistletoe in "editable" mode (because of the -e option).
That means that any changes made to the source code will get visible
immediately - that's because Python only makes a link to the specified
directory (.) instead of copying the files to the standard packages
folder.
See the contributing doc for how to contribute to mistletoe.
Usage
Usage from Python
Here's how you can use mistletoe in a Python script:
import mistletoe
with open('foo.md', 'r') as fin:
rendered = mistletoe.markdown(fin)mistletoe.markdown() uses mistletoe's default settings: allowing HTML mixins
and rendering to HTML. The function also accepts an additional argument
renderer. To produce LaTeX output:
import mistletoe
from mistletoe.latex_renderer import LaTeXRenderer
with open('foo.md', 'r') as fin:
rendered = mistletoe.markdown(fin, LaTeXRenderer)To reflow the text in a Markdown document with a max line length of 20 characters:
import mistletoe
from mistletoe.markdown_renderer import MarkdownRenderer
with open('dev-guide.md', 'r') as fin:
with MarkdownRenderer(max_line_length=20) as renderer:
print(renderer.render(mistletoe.Document(fin)))Finally, here's how you would manually specify extra tokens via a renderer.
In the following example, we use HtmlRenderer to render
the AST. The renderer itself adds HtmlBlock and HtmlSpan tokens to the parsing
process. The result should be equal to the output obtained from
the first example above.
from mistletoe import Document, HtmlRenderer
with open('foo.md', 'r') as fin:
with HtmlRenderer() as renderer: # or: `with HtmlRenderer(AnotherToken1, AnotherToken2) as renderer:`
doc = Document(fin) # parse the lines into AST
rendered = renderer.render(doc) # render the AST
# internal lists of tokens to be parsed are automatically reset when exiting this `with` blockImportant: As can be seen from the example above,
the parsing phase is currently tightly connected with initiation
and closing of a renderer. Therefore, you should never call Document(...)
outside of a with ... as renderer block, unless you know what you are doing.
Usage from command-line
pip installation enables mistletoe's command-line utility. Type the following
directly into your shell:
mistletoe foo.mdThis will transpile foo.md into HTML, and dump the output to stdout. To save
the HTML, direct the output into a file:
mistletoe foo.md > out.htmlYou can use a different renderer by including the full path to the renderer
class after a -r or --renderer flag. For example, to transpile into
LaTeX:
mistletoe foo.md --renderer mistletoe.latex_renderer.LaTeXRendererand similarly for a renderer in the contrib package:
mistletoe foo.md --renderer mistletoe.contrib.jira_renderer.JiraRenderermistletoe interactive mode
Running mistletoe without specifying a file will land you in interactive
mode. Like Python's REPL, interactive mode allows you to test how your
Markdown will be interpreted by mistletoe:
mistletoe [version 0.7.2] (interactive)
Type Ctrl-D to complete input, or Ctrl-C to exit.
>>> some **bold** text
... and some *italics*
...
<p>some <strong>bold</strong> text
and some <em>italics</em></p>
>>>The interactive mode also accepts the --renderer flag:
mistletoe [version 0.7.2] (interactive)
Type Ctrl-D to complete input, or Ctrl-C to exit.
Using renderer: LaTeXRenderer
>>> some **bold** text
... and some *italics*
...
\documentclass{article}
\begin{document}
some \textbf{bold} text
and some \textit{italics}
\end{document}
>>>Who uses mistletoe?
mistletoe is used by projects of various target audience.
You can find some concrete projects in the "Used by" section
on Libraries.io, but this is definitely not a complete
list.
Also a list of Dependents is tracked by GitHub directly.
Run mistletoe from CopyQ
One notable example is running mistletoe as a Markdown converter from the
advanced clipboard manager called CopyQ. One just needs to install
the Convert Markdown to ... custom script command
and then run this command on any selected Markdown text.
Why mistletoe?
"For fun," says David Beazley.