| |
- ElementToken
-
- ErrorOnFail
- Group
-
- FirstOfGroup
- SequentialGroup
-
- CILiteral
- LibraryElement
- Literal
- Name
- Prebuilt
- _Range
-
- Range
class CILiteral(SequentialGroup) |
|
Case-insensitive Literal values
The CILiteral is a sequence of literal and
character-range values, where each element is
positive and required. Literal values are
composed of those characters which are not
upper-case/lower-case pairs, while the ranges
are all two-character ranges with the upper
and lower forms.
CILiterals in the SimpleParse EBNF grammar are defined like so:
c"test", c"test"?, c"test"*, c"test"+
-c"test", -c"test"?, -c"test"*, -c"test"+
Attributes:
value -- a string storing the literal's value
Notes:
Currently we don't support Unicode literals
A CILiteral will be *much* slower than a
regular literal or character range |
|
- Method resolution order:
- CILiteral
- SequentialGroup
- Group
- ElementToken
Methods defined here:
- ciParse(self, value)
- Break value into set of case-dependent groups...
- toParser(self, generator=None, noReport=0)
Data and other attributes defined here:
- value = ''
Methods inherited from Group:
- terminal(self, generator)
- Determine if this element is terminal for the generator
Data and other attributes inherited from Group:
- children = ()
- terminalValue = None
Methods inherited from ElementToken:
- __init__(self, **namedarguments)
- Initialize the object with named attributes
This method simply takes the named attributes and
updates the object's dictionary with them
- __repr__(self)
- Return a readily recognisable version of ourself
- permute(self, basetable)
- Given a positive, required, non-repeating table, convert to appropriately configured table
This method applies generic logic for applying the
operational flags to a basic recipe for an element.
It is normally called from the elements-token's own
toParser method.
Data and other attributes inherited from ElementToken:
- errorOnFail = None
- expanded = 0
- lookahead = 0
- negative = 0
- optional = 0
- repeating = 0
- report = 1
|
class ElementToken |
|
Abstract base class for all ElementTokens
Common Attributes:
negative -- the element token should match
a character if the "base" definition
would not match at the current position
optional -- the element token will match even
if the base definition would not match
at the current position
repeating -- if the element is successfully
matched, attempt to match it again.
lookahead -- if true, the scanning position
of the engine will be reset after the
element matches
errorOnFail -- if true, the engine will call the
object stored in errorOnFail as a text-
matching object iff the element token fails
to match. This is used to signal
SyntaxErrors.
Attributes only used for top-level Productions:
report -- if true, the production's results
will be added to the result tree
expanded -- if true, the production's children's
results will be added to the result tree
but the production's own result will be ignored |
|
Methods defined here:
- __init__(self, **namedarguments)
- Initialize the object with named attributes
This method simply takes the named attributes and
updates the object's dictionary with them
- __repr__(self)
- Return a readily recognisable version of ourself
- permute(self, basetable)
- Given a positive, required, non-repeating table, convert to appropriately configured table
This method applies generic logic for applying the
operational flags to a basic recipe for an element.
It is normally called from the elements-token's own
toParser method.
- terminal(self, generator)
- Determine if this element is terminal for the generator
- toParser(self, generator, noReport=0)
- Abstract interface for implementing the conversion to a text-tools table
generator -- an instance of generator.Generator
which provides various facilities for discovering
other productions.
noReport -- if true, we're being called recursively
for a terminal grammar fragment where one of our
parents has explicitly suppressed all reporting.
This method is called by the generator or by
another element-token's toParser method.
Data and other attributes defined here:
- errorOnFail = None
- expanded = 0
- lookahead = 0
- negative = 0
- optional = 0
- repeating = 0
- report = 1
|
class ErrorOnFail(ElementToken) |
|
When called as a matching function, raises a SyntaxError
Attributes:
expected -- list of strings describing expected productions
production -- string name of the production that's failing to parse
message -- overrides default message generation if non-null
(something,something)+!
(something,something)!
(something,something)+!"Unable to parse somethings in my production"
(something,something)!"Unable to parse somethings in my production"
if string -> give an explicit message (with optional % values)
else -> use a default string |
|
Methods defined here:
- __call__(self, text, position, end)
- Method called by mxTextTools iff the base production fails
- copy(self)
Data and other attributes defined here:
- expected = ''
- message = ''
- production = ''
Methods inherited from ElementToken:
- __init__(self, **namedarguments)
- Initialize the object with named attributes
This method simply takes the named attributes and
updates the object's dictionary with them
- __repr__(self)
- Return a readily recognisable version of ourself
- permute(self, basetable)
- Given a positive, required, non-repeating table, convert to appropriately configured table
This method applies generic logic for applying the
operational flags to a basic recipe for an element.
It is normally called from the elements-token's own
toParser method.
- terminal(self, generator)
- Determine if this element is terminal for the generator
- toParser(self, generator, noReport=0)
- Abstract interface for implementing the conversion to a text-tools table
generator -- an instance of generator.Generator
which provides various facilities for discovering
other productions.
noReport -- if true, we're being called recursively
for a terminal grammar fragment where one of our
parents has explicitly suppressed all reporting.
This method is called by the generator or by
another element-token's toParser method.
Data and other attributes inherited from ElementToken:
- errorOnFail = None
- expanded = 0
- lookahead = 0
- negative = 0
- optional = 0
- repeating = 0
- report = 1
|
class FirstOfGroup(Group) |
|
Set of tokens that matches (and stops searching) with the first successful child
A FirstOf group attempts to match each child in turn,
declaring success with the first successful child,
or failure if none of the children match.
Within the simpleparsegrammar, the FirstOf group
is defined like so:
("a" / b / c / "d")
i.e. a series of slash-separated element token definitions. |
|
- Method resolution order:
- FirstOfGroup
- Group
- ElementToken
Methods defined here:
- toParser(self, generator=None, noReport=0)
Methods inherited from Group:
- terminal(self, generator)
- Determine if this element is terminal for the generator
Data and other attributes inherited from Group:
- children = ()
- terminalValue = None
Methods inherited from ElementToken:
- __init__(self, **namedarguments)
- Initialize the object with named attributes
This method simply takes the named attributes and
updates the object's dictionary with them
- __repr__(self)
- Return a readily recognisable version of ourself
- permute(self, basetable)
- Given a positive, required, non-repeating table, convert to appropriately configured table
This method applies generic logic for applying the
operational flags to a basic recipe for an element.
It is normally called from the elements-token's own
toParser method.
Data and other attributes inherited from ElementToken:
- errorOnFail = None
- expanded = 0
- lookahead = 0
- negative = 0
- optional = 0
- repeating = 0
- report = 1
|
class Group(ElementToken) |
|
Abstract base class for all group element tokens
The primary feature of a group is that it has a set
of element tokens stored in the attribute "children". |
|
Methods defined here:
- terminal(self, generator)
- Determine if this element is terminal for the generator
Data and other attributes defined here:
- children = ()
- terminalValue = None
Methods inherited from ElementToken:
- __init__(self, **namedarguments)
- Initialize the object with named attributes
This method simply takes the named attributes and
updates the object's dictionary with them
- __repr__(self)
- Return a readily recognisable version of ourself
- permute(self, basetable)
- Given a positive, required, non-repeating table, convert to appropriately configured table
This method applies generic logic for applying the
operational flags to a basic recipe for an element.
It is normally called from the elements-token's own
toParser method.
- toParser(self, generator, noReport=0)
- Abstract interface for implementing the conversion to a text-tools table
generator -- an instance of generator.Generator
which provides various facilities for discovering
other productions.
noReport -- if true, we're being called recursively
for a terminal grammar fragment where one of our
parents has explicitly suppressed all reporting.
This method is called by the generator or by
another element-token's toParser method.
Data and other attributes inherited from ElementToken:
- errorOnFail = None
- expanded = 0
- lookahead = 0
- negative = 0
- optional = 0
- repeating = 0
- report = 1
|
class LibraryElement(ElementToken) |
|
Holder for a prebuilt item with it's own generator |
|
Methods defined here:
- toParser(self, generator=None, noReport=0)
Data and other attributes defined here:
- generator = None
- methodSource = None
- production = ''
Methods inherited from ElementToken:
- __init__(self, **namedarguments)
- Initialize the object with named attributes
This method simply takes the named attributes and
updates the object's dictionary with them
- __repr__(self)
- Return a readily recognisable version of ourself
- permute(self, basetable)
- Given a positive, required, non-repeating table, convert to appropriately configured table
This method applies generic logic for applying the
operational flags to a basic recipe for an element.
It is normally called from the elements-token's own
toParser method.
- terminal(self, generator)
- Determine if this element is terminal for the generator
Data and other attributes inherited from ElementToken:
- errorOnFail = None
- expanded = 0
- lookahead = 0
- negative = 0
- optional = 0
- repeating = 0
- report = 1
|
class Literal(ElementToken) |
|
Literal string value to be matched
Literals are one of the most common elements within
any grammar. The implementation tries to use the
most efficient mechanism available for matching/searching
for a literal value, so the Literal class does not
use the permute method, instead defining explicit
parsing methodologies for each flag and value combination
Literals in the SimpleParse EBNF grammar are defined like so:
"test", "test"?, "test"*, "test"+
-"test", -"test"?, -"test"*, -"test"+
Attributes:
value -- a string storing the literal's value
Notes:
Currently we don't support Unicode literals
See also:
CILiteral -- case-insensitive Literal values |
|
Methods defined here:
- baseToParser(self, generator=None)
- Parser generation without considering flag settings
- terminal(self, generator)
- Determine if this element is terminal for the generator
- toParser(self, generator=None, noReport=0)
- Create the parser for the element token
Data and other attributes defined here:
- value = ''
Methods inherited from ElementToken:
- __init__(self, **namedarguments)
- Initialize the object with named attributes
This method simply takes the named attributes and
updates the object's dictionary with them
- __repr__(self)
- Return a readily recognisable version of ourself
- permute(self, basetable)
- Given a positive, required, non-repeating table, convert to appropriately configured table
This method applies generic logic for applying the
operational flags to a basic recipe for an element.
It is normally called from the elements-token's own
toParser method.
Data and other attributes inherited from ElementToken:
- errorOnFail = None
- expanded = 0
- lookahead = 0
- negative = 0
- optional = 0
- repeating = 0
- report = 1
|
class Name(ElementToken) |
|
Reference to another rule in the grammar
The Name element token allows you to reference another
production within the grammar. There are three major
sub-categories of reference depending on both the Name
element token and the referenced table's values.
if the Name token's report attribute is false,
or the target table's report attribute is false,
or the Name token negative attribute is true,
the Name reference will report nothing in the result tree
if the target's expand attribute is true, however,
the Name reference will report the children
of the target production without reporting the
target production's results (SubTable match)
finally:
if the target is not expanded and the Name token
should report something, the generator object is
asked to supply the tag object and flags for
processing the results of the target. See the
generator.MethodSource documentation for details.
Notes:
expanded and un-reported productions won't get any
methodsource methods called when
they are finished, that's just how I decided to
do it, not sure if there's some case where you'd
want it. As a result, it's possible to have a
method getting called for one instance (where a
name ref is reporting) and not for another (where
the name ref isn't reporting). |
|
Methods defined here:
- terminal(self, generator)
- Determine if this element is terminal for the generator
- toParser(self, generator, noReport=0)
- Create the table for parsing a name-reference
Note that currently most of the "compression" optimisations
occur here.
Data and other attributes defined here:
- report = 1
- terminalValue = None
- value = ''
Methods inherited from ElementToken:
- __init__(self, **namedarguments)
- Initialize the object with named attributes
This method simply takes the named attributes and
updates the object's dictionary with them
- __repr__(self)
- Return a readily recognisable version of ourself
- permute(self, basetable)
- Given a positive, required, non-repeating table, convert to appropriately configured table
This method applies generic logic for applying the
operational flags to a basic recipe for an element.
It is normally called from the elements-token's own
toParser method.
Data and other attributes inherited from ElementToken:
- errorOnFail = None
- expanded = 0
- lookahead = 0
- negative = 0
- optional = 0
- repeating = 0
|
class Prebuilt(ElementToken) |
|
Holder for pre-built TextTools tag tables
You can pass in a Pre-built tag table when
creating your grammar, doing so creates
Prebuilt element tokens which can be referenced
by the other element tokens in your grammar. |
|
Methods defined here:
- toParser(self, generator=None, noReport=0)
Data and other attributes defined here:
- value = ()
Methods inherited from ElementToken:
- __init__(self, **namedarguments)
- Initialize the object with named attributes
This method simply takes the named attributes and
updates the object's dictionary with them
- __repr__(self)
- Return a readily recognisable version of ourself
- permute(self, basetable)
- Given a positive, required, non-repeating table, convert to appropriately configured table
This method applies generic logic for applying the
operational flags to a basic recipe for an element.
It is normally called from the elements-token's own
toParser method.
- terminal(self, generator)
- Determine if this element is terminal for the generator
Data and other attributes inherited from ElementToken:
- errorOnFail = None
- expanded = 0
- lookahead = 0
- negative = 0
- optional = 0
- repeating = 0
- report = 1
|
class Range(_Range) |
|
Range type which doesn't use the CharSet features in mx.TextTools
This is likely to be much slower than the CharSet version (below), and
is unable to handle unicode character sets. However, it will work with
TextTools 2.0.3, which may be needed in some cases. |
|
- Method resolution order:
- Range
- _Range
- ElementToken
Methods defined here:
- baseToParser(self, generator=None)
- Parser generation without considering flag settings
- terminal(self, generator)
- Determine if this element is terminal for the generator
Methods inherited from _Range:
- toParser(self, generator=None, noReport=0)
- Create the parser for the element token
Data and other attributes inherited from _Range:
- requiresExpandedSet = 1
- value = ''
Methods inherited from ElementToken:
- __init__(self, **namedarguments)
- Initialize the object with named attributes
This method simply takes the named attributes and
updates the object's dictionary with them
- __repr__(self)
- Return a readily recognisable version of ourself
- permute(self, basetable)
- Given a positive, required, non-repeating table, convert to appropriately configured table
This method applies generic logic for applying the
operational flags to a basic recipe for an element.
It is normally called from the elements-token's own
toParser method.
Data and other attributes inherited from ElementToken:
- errorOnFail = None
- expanded = 0
- lookahead = 0
- negative = 0
- optional = 0
- repeating = 0
- report = 1
|
class SequentialGroup(Group) |
|
A sequence of element tokens which must match in a particular order
A sequential group must match each child in turn
and all children must be satisfied to consider the
group matched.
Within the simpleparsegrammar, the sequential group
is defined like so:
("a", b, c, "d")
i.e. a series of comma-separated element token definitions. |
|
- Method resolution order:
- SequentialGroup
- Group
- ElementToken
Methods defined here:
- toParser(self, generator=None, noReport=0)
Methods inherited from Group:
- terminal(self, generator)
- Determine if this element is terminal for the generator
Data and other attributes inherited from Group:
- children = ()
- terminalValue = None
Methods inherited from ElementToken:
- __init__(self, **namedarguments)
- Initialize the object with named attributes
This method simply takes the named attributes and
updates the object's dictionary with them
- __repr__(self)
- Return a readily recognisable version of ourself
- permute(self, basetable)
- Given a positive, required, non-repeating table, convert to appropriately configured table
This method applies generic logic for applying the
operational flags to a basic recipe for an element.
It is normally called from the elements-token's own
toParser method.
Data and other attributes inherited from ElementToken:
- errorOnFail = None
- expanded = 0
- lookahead = 0
- negative = 0
- optional = 0
- repeating = 0
- report = 1
|
class _Range(ElementToken) |
|
Range of character values where any one of the characters may match
The Range token allows you to define a set of characters
(using a mini-grammar) of which any one may match. By using
the repetition flags, it is possible to easily create such
common structures as "names" and "numbers". For example:
name := [a-zA-Z]+
number := [0-9.eE]+
(Note: those are not beautifully defined examples :) ).
The mini-grammar for the simpleparsegrammar is defined as follows:
'[',CHARBRACE?,CHARDASH?, (CHARRANGE/CHARNOBRACE)*, CHARDASH?,']'
that is, if a literal ']' character is wanted, you must
define the character as the first item in the range. A literal
'-' character must appear as the first character after any
literal ']' character (or the beginning of the range) or as the
last character in the range.
Note: The expansion from the mini-grammar occurs before the
Range token is created (the simpleparse grammar does the
expansion), so the value attribute of the token is actually
the expanded string of characters. |
|
Methods defined here:
- toParser(self, generator=None, noReport=0)
- Create the parser for the element token
Data and other attributes defined here:
- requiresExpandedSet = 1
- value = ''
Methods inherited from ElementToken:
- __init__(self, **namedarguments)
- Initialize the object with named attributes
This method simply takes the named attributes and
updates the object's dictionary with them
- __repr__(self)
- Return a readily recognisable version of ourself
- permute(self, basetable)
- Given a positive, required, non-repeating table, convert to appropriately configured table
This method applies generic logic for applying the
operational flags to a basic recipe for an element.
It is normally called from the elements-token's own
toParser method.
- terminal(self, generator)
- Determine if this element is terminal for the generator
Data and other attributes inherited from ElementToken:
- errorOnFail = None
- expanded = 0
- lookahead = 0
- negative = 0
- optional = 0
- repeating = 0
- report = 1
| |