edit: tsunami parsing model

links

history

pointed to by

tsunami: symbolic quoting
tsunami: embedded data types

item nametags

==introduction
There are two pieces of core tsunami technology which need a multi-stage parsing strategy which preserves whitespace:
- [[symbolic quoting/558]]
- [[embedded data types/541]]
The simplest situation occurs when we only have a [[lightweight markup syntax/340]]: we simply convert the markup to an "AST"[1] and then we can render HTML text, tracking which fragment or fragments of plaintext markup were used to define each fragment of output HTML.

[1]http://en.wikipedia.org/wiki/Abstract_syntax_tree

==enter new data types
The principle behind [[embedded data types/541]] is that we can take the basic lightweight markup syntax and add to it.  Perhaps we want to recognize addresses: we want a modularized way to allow text nodes to be analyzed and possibly "enhanced" in the AST (in other words, a single text node would be split into multiple nodes).

==possible pipeline
1. plaintext
2. do standard markup parsing (text -> AST)
3. detect data types in text nodes and split those nodes into more nodes (AST -> AST)
4. render the AST, allowing modules to override how nodes or subtrees are rendered
5. HTML, with a perfect mapping from each fragment of HTML to one or more fragments of plaintext

=="Parsing Expression Grammar (PEG)"[1]
This is probably the best option for parsing when all text must be valid and tokenization really needs to depend on semantics.  (In PEGs, tokenization and parsing are not split up.)

=Links
- "C# CodeProject PEG"[2]

[1]http://en.wikipedia.org/wiki/Parsing_expression_grammar
[2]http://www.codeproject.com/KB/recipes/grammar_support_1.aspx

==V archived V

==dynamic parsing?
Macros like "Boo"[1] or "Nemerle"[2]?

[1]http://boo.codehaus.org/Part+17+-+Macros
[2]http://nemerle.org/Macros_tutorial

==links
- "LLVM"[1]
  "Low Level Virtual Machine"
[1]http://llvm.cs.uiuc.edu/

- "Phoenix"[1]
  "It enables teaching and collaborative research in code generation, optimization, program analysis, binary transformation, and software correctness. Phoenix is used as a research platform by Microsoft Research and will be the universal compiler backend for upcoming Microsoft languages and development tools."
[1]http://research.microsoft.com/en-us/collaboration/focus/cs/phoenix.aspx

- "GNU C Compiler Internals/Architecture"[1]
[1]http://en.wikibooks.org/wiki/GNU_C_Compiler_Internals/GNU_C_Compiler_Architecture

some permissive license goes here
contact