log in

ANTLR: lightweight markup grammars

Luke Breuer
2008-12-18 02:34 UTC

ANTLR mailing list thread
Markdown PEG
notes from airplane
  • need url syntax for tags and searches so they can be embedded nicely
  • tokenize control characters and send everything else to HIDDEN
    • how is precedence among tokens handled? (or is it?)
  • how do we detect contiguous list items? perhaps by specially recognizing leading whitespace?
  • NEWLINE --> HIDDEN or not? I don't think so
  • what do we do when \ does not need to escape the next character?
    • examples:
      • a\b
      • \\computer\share
      • dir \*.*
    • it is as if we first want to treat all characters as unescaped, and then only treat \ as an escape character if it would have escaped a character that would otherwise have been used to generate markup (like *, _, ...)
      • this appears to be a tricky proposition, but perhaps it is easy to just revert the escaping under certain circumstances
  • *, _ should only be tokens under certain circumstances --> OPEN_STAR, CLOSE_STAR
    • we do this by lookahead and lookbehind, respectively
    • can we optionally send to HIDDEN if predicate fails?
  • what to do with unmatched OPEN_STAR and CLOSE_STAR?
    • would it work to put a filter on the token stream? This assumes that tokens are generated independently of rules. We would look for the last N - 1 CLOSE_STAR and convert to "literals" — if no CLOSE_STAR, convert all OPEN_STAR to literals as well. We allow OPEN_STAR (OPEN_STAR)* CLOSE_STAR.
  • use **, __ for multiline, or where a predicate for OPEN_*/CLOSE_* would fail
    • how to parse *** or \**?
  •  and 
    may only have leading whitespace
  • what can occur inside `code`?
  • specify language for code blocks (what about inline code blocks?)
    • rudimentary parsing and coloring for TSQL, regex, C#, JS, VHDL
  • hideable, taggable sections:
    ((tag1 tag2 "description to show"
      and here is the content
    • can phrases be tagged, perhaps like this? "my phrase"{tag1, tag2}
  • collapseable lists
    • precede by +/-, regex ^([ ][ ])*+/-\s*$
    • make parents collapseable, but only if no parent text is selected (so that attempts to select text don't inadvertently show/hide)
    • double click on parent expands/contracts all descendants