% Introduction.

The purpose of parsing is to transform a sequence of characters, or a sequence
of tokens, into a semantic value, which typically is some kind of syntax tree.
The purpose of \emph{unparsing} is to perform the reverse transformation, that
is, to transform a syntax tree back into a sequence of tokens or characters.

Although unparsing may not seem conceptually very challenging, writing an
unparser by hand can require a significant amount of work. Furthermore,
ensuring that an unparser is correct can be difficult. An unparser is correct
if its output can be parsed and if this gives rise to the same syntax tree
that one started with. In other words, it is correct if the composition
$\mathit{unparse}; \mathit{parse}$ is the identity. Intuitively, this means
that the unparser must produce syntactically correct output and must insert
enough parentheses (or other disambiguation symbols) to ensure that its output
is correctly interpreted. If the grammar involves priority or precedence
declarations (\sref{sec:assoc}, \sref{sec:prec}), enforcing this property can
be tricky.

% External references: the paper and the demo.

When the switch \ounparsing is supplied on the command line, \menhir generates
code that helps implement a correct unparser. In this reference manual, we
give a brief overview of the generated code; for more details, the reader is
referred to the paper ``Correct, Fast {LR(1)}
Unparsing''~\cite{pottier-unparsing-24}.
Furthermore, the directory \distrib{demos/calc-unparsing}
contains a demo that illustrates the use of the unparsing API.
In this directory,
the command \texttt{make api} shows the file \texttt{parser.mli}
generated by \menhir.

% Brief reference: data structures.

The unparsing API involves two distinct data structures, namely
\emph{disjunctive concrete syntax trees}
and
\emph{concrete syntax trees}.
%
\begin{itemize}
\item
A \emph{concrete syntax tree} (CST) is
a tree where every node is either
a \emph{terminal node},
labeled with a token,
or
a \emph{nonterminal node},
labeled with a production.
%
A~terminal node has no children.
%
A~nonterminal node has children
whose number and types must match
the right-hand side of the production.
%
\item
A \emph{disjunctive concrete syntax tree} (DCST)
contains terminal nodes, nonterminal nodes,
and \emph{disjunction nodes}.
% that is, choices between two subtrees
A disjunction node has two children.
It represents a choice between two possible descriptions of a subtree:
for example, without or with surrounding parentheses.
\end{itemize}
%
The unparsing API expects the unparsing process to be performed in three
steps, as follows:
%
\begin{enumerate}
\item \label{step:describe}
      an abstract syntax tree (or, more generally, a semantic value)
      is transformed into a DCST;
\item \label{step:settle}
      the DCST is transformed into a CST;
\item \label{step:format}
      the CST is transformed into text,
      either directly,
      or by going through some form of structured document.
\end{enumerate}

% Brief reference: API.

The facilities provided by the unparsing API are:
\begin{itemize}
\item
a DCST construction API (a set of constructor functions) ;
\item
a \emph{settlement} algorithm, which converts a DCST to a CST;
\item
a CST deconstruction API (a visitor class).
\end{itemize}

The settlement algorithm performs step~\ref{step:settle} automatically. It is
up to you, the user, to implement steps~\ref{step:describe} and
\ref{step:settle}.
In step~\ref{step:settle}, you use the DCST construction API.
In step~\ref{step:format}, you use the CST deconstruction API.

\subsubsection{DCST construction API}

The DST construction API is found in the sub-module \texttt{DCST}. It offers a
set of constructor functions that you can use to construct DCSTs. OCaml's type
discipline ensures that only well-formed DCSTs can be constructed; this rules
out a class of mistakes that would lead to syntactically incorrect output.

\begin{itemize}
\item
For each nonterminal symbol, say~\texttt{expr},
there is an abstract type of DCSTs, also named~\texttt{expr}.
\begin{verbatim}
  type expr
\end{verbatim}
%
% (There is no way of constructing DCSTs whose head symbol is a terminal symbol.)
\item
For each production,
there is a constructor function,
whose name is determined by the production's \atname attribute.
%
This constructor function
  takes parameters % expects arguments
  whose number and types
  match the right-hand side of the production.
%
If a parameter corresponds to a terminal symbol
  that does not carry a semantic value,
  then this parameter is omitted.
%
For instance, if the grammar includes a production of the form
\begin{verbatim}
  expr:
    LPAREN; e = expr; RPAREN { e } [@name paren]
\end{verbatim}
then the sub-module \texttt{DCST} includes a constructor function
\begin{verbatim}
  val paren: expr -> expr
\end{verbatim}

\item
For each nonterminal symbol, say~\texttt{expr},
there is a constructor function,
named \texttt{expr\_choice},
that constructs a disjunction node.
\begin{verbatim}
  val expr_choice: expr -> expr -> expr
\end{verbatim}
\end{itemize}

\subsubsection{Settlement}

For each start nonterminal symbol, say~\texttt{main},
there is a function,
named \texttt{Settle.main},
which transforms a DCST to a CST.
%
\begin{verbatim}
  val main: DCST.main -> CST.main option
\end{verbatim}
%
This transformation eliminates all disjunction nodes. In other words, whenever
there is a choice between several ways of displaying a subtree (for example,
without or with surrounding parentheses), this choice is resolved. The
settlement algorithm is left-biased: if the left-hand alternative leads to a
viable tree (that is, a tree that can be printed and parsed again, without
confusion) then this alternative is chosen; otherwise, the right-hand
alternative is chosen.

If the algorithm is unable to find a viable tree, then it fails: it returns
\texttt{None}. If you encounter this situation, first make sure that in the
previous step (step~\ref{step:describe}) you have inserted enough parentheses,
or constructed enough disjunction nodes that offer a possibility of inserting
parentheses. If you think that you have done so, and if the algorithm still
fails, please contact \menhir's developers. As explained in the
paper~\cite{pottier-unparsing-24}, the settlement algorithm is fast but
incomplete. When faced with a disjunction node, it commits early to the
left-hand alternative. If this alternative later turns out to fail, then the
algorithm does not backtrack; the right-hand alternative is never tried.

\subsubsection{CST deconstruction API}

The CST deconstruction API is found in the sub-module \texttt{CST}. It offers
a visitor class, named \texttt{reduce}, which allows traversing a CST and
transforming it into a printable form. This printable form could be text, or
could be a document in a a structured document description language; it is up
to the user to choose it.

For each nonterminal symbol, say~\texttt{expr},
there is an abstract type of CSTs, also named~\texttt{expr}.
\begin{verbatim}
  type expr
\end{verbatim}

Then, there is a single visitor class:
\begin{verbatim}
  class virtual ['r] reduce : object
    method virtual zero : 'r
    method virtual cat : 'r -> 'r -> 'r
    method virtual text : string -> 'r
    (* one method per terminal symbol *)
    (* one method per nonterminal symbol *)
    (* one method per production *)
  end
\end{verbatim}

This class is parameterized by a type~\texttt{'r},
which is the result type of every method:
it is the type of the printable form
to which every tree must be reduced.

The user is expected to provide implementations
for the three virtual methods \texttt{zero},
\texttt{cat}, and \texttt{text}.
In short,
\texttt{zero} is the empty printable thing;
\texttt{cat} concatenates two printable things;
and
\texttt{text} converts a string to a printable thing.

Furthermore, the following methods exist:
%
\begin{itemize}
\item
For each terminal symbol,
there is a visitor method.
%
For example, if the token \texttt{INT} has been declared by
\verb+%token<int> INT+
then the following method exists:
\begin{verbatim}
  method virtual visit_INT : int -> 'r
\end{verbatim}
%
This method is normally virtual.
However,
if the terminal symbol carries no semantic value
and if a~token alias (\sref{sec:tokens})
has been provided by the user for this symbol
then a default implementation of this method
is generated by \menhir;
the method is then non-virtual.
% This default implementation relies on the method \texttt{text}.
% Otherwise, this method is declared virtual.
For example, if the token \texttt{LPAREN} has been declared by
\verb+%token LPAREN "("+
then the following non-virtual method exists:
\begin{verbatim}
  method visit_LPAREN : 'r
\end{verbatim}
\item
For each nonterminal symbol, say~\texttt{expr},
there is a visitor method,
named \texttt{visit\_expr}.
This method is not virtual: \menhir generates code for it.
This method expects a CST of type \texttt{expr}
as an argument,
performs a~case analysis of the root node,
and applies a suitable \texttt{case} method
to the children of the root node.
\begin{verbatim}
  method visit_expr : expr -> 'r
\end{verbatim}
\item
For each production,
there is a visitor method.
This method is not virtual: \menhir generates code for it.
This method expects zero, one, or more arguments.
First, each argument is reduced to a printable thing
via recursive calls to suitable \texttt{visit} methods;
then these printable things are concatenated
using \texttt{zero} and \texttt{cat}.
%
For instance, if the grammar includes a production of the form
\begin{verbatim}
  expr:
    LPAREN; e = expr; RPAREN { e } [@name paren]
\end{verbatim}
then the following method exists:
\begin{verbatim}
  method case_paren : expr -> 'r
\end{verbatim}
\end{itemize}
