[The subject of this document and the source code is still under development.]
If input is a String presumed to be an Expression, then�
(1) Expression[Branch][input]
�is transformed into an ExpressionBranch Sequence. This is realized by applying the RuleList associated with the CharacterSet employed by the String input. This algorithm is implemented one-character-at-a-time using the subprocedure�
(2) ExpressionCharacterMachine[char_Character]
The above is designed to take elements, (�char�), in Sequence from the �input� CharacterStream and return True so long as char is being interpreted as a potentially legitimate Character in an Expression's string. False is returned when the �end bracket� of a completed Expression has been reached, (in other words, there is no Expression completion sequence still pending), and the last Character was the EndCharacter.� If neither True nor False is returned by ExpressionCharacterMachine[char], then an error is presumed, and processed. The ExpressionCharacterMachine[char] is considered to have successfully processed an Expression-String if it generates a Sequence of (True, � , True, False) elements whose number equals the originating String�s length.
Each invocation of ExpressionCharacterMachine[char] changes the CharacterMachineState.�
Furthermore, each invocation of ExpressionCharacterMachine[char] returns
(True | False | Noop[Expression[Branch]][�] | Noop[Noop])
curParsedExpression must be assigned immediately before ExpressionCharacterMachine[char] returns False.
ExpressionCharacterMachine[char] is called by Expression[Branch][input] with the following invocation:
Name[ExpressionState, ExpressionCharacterMachine[char]]
The point is that the algorithm is expected to return a description of the �ExpressionState�, which may find its way into an error message�� �ExpressionState�
Shortly after Expression[Branch]["input"] begins execution, it assigns "CurrentHead", which is the Name assigned to the current (Expression) Head parsed in the CharacterStream sequence.� The CurrentHead is one of several Names that define the CharacterMachineState.
The CharacterMachineState is defined by Named assignment (symbols) in the
Construct`String`Expression`Character`Machine` Context.
HeadStack is used to keep track of nested subExpressions.� The following is an excerpt from the documentation for operators�
If initFunc is not func, then�
If func has higher precedence then initFunc, then the previous Expression is Slot[1] to func.� HeadStack stores a reference to initFunc "on top", and func becomes initFunc.
If func has lower precedence then initFunc, then the previous Expression is completed with initFunc, which reference is removed from HeadStack, and the completed Expression becomes Slot[1] in func.
The CharacterExpressionMachine interprets each Character by the CharacterSet's RuleList, (which includes Operators that elicit different parsing rules (and algorithms)), conditioned by the ExpressionState.
Once a Name has been parsed, it is the "CurrentHead", and its arguments are parsed from the CharacterStream (and compared against the Name's assignment if the Name has the "Pattern" Attribute).
When a symbol's arguments must conform to a mandatory syntax imposed by its elicitation forms, it is assigned the "Pattern" Attribute.� This Attribute changes the way the� CharacterExpressionMachine works.
For some Name, sym, if�
�������������������������� Name[sym[Pattern]]
�is True, then sym enforces a syntax through its ElicitationForm. sym's ElicitationForm must be obeyed or the larger String containing "sym" will not be parsed as an Expression. Furthermore, the process of eliciting sym cannot change the environment by any residual Name assignment. Any sym, with the NamePattern Attribute, that reassigns a non-temporary Name during sym's elicitation, will generate an error.
In this case, the CurrentHead, and its corresponding ElicitationForm further pattern the character stream.
����������� Every recognized Character�s CharacterCode is an element in one of the following Sets�
{Noops, = {EscapeSequence, Unassigned, Unrecognized, StartComment, EndComment, �EmptySpace�},
Digits, (*� =�� {CC0, CC1, CC2, �} *)
Letters, (*� = {CCa, CCb, �} *)
ParaPuncs, �= {ContextMark, �{ExprStart, ExprEnd}�,
����������� {StandardStart, StandardSeperator, StandardEnd},
����������������������� �ParaPuncSpec�� }
}
Any Character (including any EscapeSequence) which is not recognized in one of the above 5 Character subsets is, by definition, an Unrecognized Character (which is one of the Noops-Characters).
Thus every unrecognized character/string is interpreted as an UnrecognizedString amongst the Noops.
The ExpressionCharacterMachine[�] is the principle iterative subroutine in Expression[Branch][input_String] (see below).� The iterative Loop[�] ends by calling either completeExpression or deliverExpression.
When ExpressionCharacterMachine[char] returns False, completeExpression is called.
When the input CharacterStream terminates with Stream[Noop], deliverExpression is called.� If the ExpressionStack is empty the Expression[Branch] structure is returned.� If the ExpressionStack is not empty, Noop[Expression[Branch]][Slot[characterStream], �Missing End Bracket�] is returned.
�before the ExpressionCharacterMachine[�] has detected an end bracket, .
completeExpression is called only when it is positively known that an Expression has been parsed and its last end bracket has been reached. completeExpression is expected to generate a Sequence with the ExpressionBranch structure.� The subContext containing completeExpression and its subprocedures, has kept an account (made the necessary assignments) from which the ExpressionBranch structure can be constructed.
(*
����������� ExpressionStack is a list of containing Heads.
����������� ExpressionBranch is the iteratively generated result of Expression[Branch][expr].
*)
(* The following LocalNames are declared in Expression[Branch][input_String]�s Cast.
Any procedure that uses ExpressionCharacterMachine[char_Character] will want to declare these�
Cast[{� NewExpression=True,
����������� ExpressionStack={},
����������� ExpressionBranch={}},
*)
Name[ExpressionCharacterMachine[char_Character],
If[NewExpression,
];
�����������
]
Name[
]
]
� 2004, 2005
by John Van Wie Bergamini
NOTICE: Distribute this document under the terms of the Lesser General Public License.
(You may copy and distribute verbatim copies of this source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty.)
If warranted, Grok32�s Less General Public License is applicable.
(Grok32Inc provides warranty protection to Grok32Subscribers.)