lexer

package
v0.1.12 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 5, 2024 License: GPL-3.0 Imports: 14 Imported by: 1

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrNotFound occurs when the input stream is not empty but the current rune
	// is not as expected.
	//
	// Format:
	//
	// 	"not found"
	ErrNotFound error

	// ErrUnexpectedSuccess occurs when the matcher succeeds with utf8.RuneError
	// as its character match.
	//
	// Format:
	//
	// 	"matcher must not succeeded with utf8.RuneError"
	ErrUnexpectedSuccess error
)
View Source
var (
	// ErrExhausted occurs when an iterator is exhausted.
	// This can be checked with the == operator.
	//
	// Format:
	//
	// 	"iterator is exhausted"
	ErrExhausted error
)

Functions

func ExtractAt added in v0.1.11

func ExtractAt(err error) (int, bool)

ExtractAt extracts the position in the input stream where the error occurred.

Parameters:

  • err: The error to extract the position from.

Returns:

  • int: The position in the input stream where the error occurred.
  • bool: A boolean indicating if the position was successfully extracted.

func Greedy added in v0.1.11

func Greedy(inner emtch.Matcher[rune]) emtch.Matcher[rune]

Greedy returns a Matcher that matches a given inner Matcher if and only if the next character does not satisfy the inner Matcher.

Parameters:

  • inner: The inner Matcher.

Returns:

  • Matcher: The matcher. Nil if the inner matcher is nil.

It is equivalent to:

WithRightBound(inner, func(char rune) bool { err := inner.Match(char); return err != nil }).

func Group added in v0.1.11

func Group(chars ...rune) emtch.Matcher[rune]

Group returns a new matcher that matches a group of characters.

Parameters:

  • chars: The characters to match.

Returns:

  • Matcher: The matcher. If no characters are provided, then nil is returned.

func Literal added in v0.1.11

func Literal(word string) emtch.Matcher[rune]

Literal returns a Matcher that matches a given literal string. If the string has only one character, then the character is matched directly.

Parameters:

  • word: The literal string to match.

Returns:

  • Matcher: The matcher. Nil if the word is empty.
  • error: An error if converting the string to a UTF-8 array fails.

func Many added in v0.1.11

func Many(inner emtch.Matcher[rune]) emtch.Matcher[rune]

Many returns a Matcher that matches a given inner Matcher as many times as possible.

Parameters:

  • inner: The inner Matcher.

Returns:

  • Matcher: The matcher. Nil if the inner matcher is nil.

func NewErrAfter added in v0.1.11

func NewErrAfter(quote bool, previous *byte, inner error) error

NewErrAfter creates a new ErrAfter error.

Parameters:

  • quote: A flag indicating whether the previous value should be quoted.
  • previous: The previous value associated with the error. If not provided, "at the start" is used.
  • inner: The inner error that occurred. If not provided, "something went wrong" is used.

Returns:

  • error: The newly created ErrAfter error. Never returns nil.

Format:

"after <previous>: <inner>"

func NewErrLexing added in v0.1.11

func NewErrLexing(at int, reason error) error

NewErrLexing returns an error that occurs during lexing.

Parameters:

  • at: The position in the input stream where the error occurred.
  • reason: The reason for the error.

Returns:

  • error: An error that occurs during lexing. Never returns nil.

Format:

  • exactly as "reason"

func NewErrNotAsExpected added in v0.1.11

func NewErrNotAsExpected(quote bool, kind string, got *byte, expecteds ...byte) error

NewErrNotAsExpected is a convenience function that creates a new ErrNotAsExpected error with the specified kind, got value, and expected values.

See common.NewErrNotAsExpected for more information.

func Newline added in v0.1.11

func Newline() emtch.Matcher[rune]

Newline returns a new matcher for newline sequences.

Returns:

  • Matcher: A matcher that detects newline sequences. Never returns nil.

func One added in v0.1.11

func One(char rune) emtch.Matcher[rune]

One returns a new matcher that matches a single character.

Parameters:

  • char: The character to match.

Returns:

  • Matcher: The matcher.

func Predicate added in v0.1.11

func Predicate(group_name string, predicate func(char rune) bool) emtch.Matcher[rune]

Predicate returns a new matcher that matches according to a predicate.

Parameters:

  • group_name: The name of the group.
  • predicate: The function to match.

Returns:

  • Matcher: The matcher. Nil if the predicate is nil.

func Range added in v0.1.11

func Range(left, right rune) emtch.Matcher[rune]

Range returns a matcher that matches a group of characters between left and right.

If left and right are equal, the returned matcher will match exactly one character. Otherwise, the returned matcher will match any character in the range [left, right].

Parameters:

  • left: The left boundary of the group.
  • right: The right boundary of the group.

Returns:

  • Matcher: The matcher. Never returns nil.

func Sequence added in v0.1.11

func Sequence(seq ...emtch.Matcher[rune]) emtch.Matcher[rune]

Sequence returns a Matcher that matches a sequence of provided Matchers in the order they are given. The sequence will be processed by iterating through each Matcher and attempting to match the input character.

Parameters:

  • seq: A variadic number of Matcher instances. Matchers in the sequence are expected to be non-nil objects.

Returns:

  • Matcher: A Matcher that represents a sequence of Matchers. Returns nil if no non-nil Matchers are provided.

func WithRightBound added in v0.1.11

func WithRightBound(inner emtch.Matcher[rune], bound func(char rune) bool) emtch.Matcher[rune]

WithRightBound returns a Matcher that matches a given inner Matcher until a boundary character is encountered.

Parameters:

  • inner: The inner Matcher.
  • bound: The boundary function. It takes a rune and returns true if it is a boundary and false otherwise.

Returns:

  • Matcher: The matcher. Nil if the inner matcher is nil.

If the bound function is nil, the inner Matcher is returned as is.

Types

type Builder

type Builder struct {
	// contains filtered or unexported fields
}

Builder is a builder for a Lexer. An empty builder can be created with the `var b Builder` syntax or with the `b := new(Builder)` constructor.

func (Builder) Build

func (b Builder) Build() *Lexer

Build returns a Lexer with the given matchers and types. The returned Lexer's internal state is a copy of the Builder's internal state, so modifications to the Builder after calling Build will not affect the returned Lexer.

Returns:

  • Lexer: A Lexer with the given matchers and types. Never returns nil.

func (*Builder) Register

func (b *Builder) Register(m emtch.Matcher[rune], type_ string) error

Register associates the given matcher with the given type.

Parameters:

  • m: The matcher to associate with the given type.
  • type_: The type to associate with the given matcher.

Returns:

  • error: An error if the receiver is nil.

func (*Builder) RegisterLiteral added in v0.1.11

func (b *Builder) RegisterLiteral(literal, type_ string) error

RegisterLiteral is a convenience method for registering a literal. It is the same as calling `Register(common.Must(Literal(literal)), type_)`.

Parameters:

  • literal: The literal to register.
  • type_: The type to associate with the literal.

Returns:

  • error: An error if the receiver is nil.

Does nothing is the literal is empty.

func (*Builder) Reset

func (b *Builder) Reset()

Reset resets the builder's internal state for reuse. No-op if the receiver is nil.

type ErrAfter added in v0.1.11

type ErrAfter struct {
	// Quote is a flag that indicates that the error should be quoted.
	Quote bool

	// Previous is the previous value.
	Previous *byte

	// Inner is the inner error.
	Inner error
}

ErrAfter is an error that occurs after another error.

func (ErrAfter) Error added in v0.1.11

func (e ErrAfter) Error() string

Error implements the error interface.

type ErrLexing added in v0.1.11

type ErrLexing struct {
	// At is the position in the input stream where the error occurred.
	At int

	// Reason is the reason for the error.
	Reason error
}

ErrLexing is an error that occurs during lexing.

func (ErrLexing) Error added in v0.1.11

func (e ErrLexing) Error() string

Error implements the error interface.

type Iterator added in v0.1.11

type Iterator struct {
	// contains filtered or unexported fields
}

Iterator is an iterator over the results of an active lexer.

func NewIterator added in v0.1.11

func NewIterator(initFn func() (*baseActive, error)) (*Iterator, error)

NewIterator creates a new iterator over the results of an active lexer.

Parameters:

  • initFn: A function that initializes and returns an active lexer.

Returns:

  • *Iterator: An iterator over the results of the active lexer.
  • error: An error if the initFn returns an error.

func (Iterator) Next added in v0.1.11

func (itr Iterator) Next() (*Result, error)

Next returns the next result of the iterator.

Returns:

  • *Result: The next result.
  • error: An error if the iterator is exhausted or if the evaluation fails.

Errors:

  • ErrExhausted: The iterator is exhausted.

func (Iterator) Stop added in v0.1.11

func (itr Iterator) Stop()

Stop stops the iterator.

After calling Stop, the iterator is no longer valid.

type Lexer

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer is a implementation of a lexer that uses a function for lexing.

func (Lexer) GetCharAt added in v0.1.11

func (l Lexer) GetCharAt(idx int) (rune, bool)

GetCharAt returns the character at index idx in the lexer's data or 0 if it does not exist.

Parameters:

  • idx: The index of the character to retrieve.

Returns:

  • rune: The character at index idx.
  • bool: A boolean indicating whether the character exists or not.

func (Lexer) Lex

func (l Lexer) Lex() (*Iterator, error)

Lex lexes the input stream, if it was set.

Returns:

  • error: An error if lexing failed.

func (Lexer) Match added in v0.1.11

func (l Lexer) Match(from int) ([]internal.Event, error)

Match uses the lexer's matcher to find sequences in the lexer's data.

Parameters:

  • from: The index to start matching from.

Returns:

  • []MatchResult: A slice of MatchResult containing the types of the matched sequences.
  • error: An error if the matching process fails.

func (*Lexer) Reset

func (l *Lexer) Reset()

Reset resets the lexer's internal state for reuse. No-op if the receiver is nil.

func (*Lexer) Write added in v0.1.11

func (l *Lexer) Write(data []byte) (int, error)

Write adds the given data to the input stream.

Parameters:

  • data: The data to add to the input stream.

Returns:

  • int: The number of bytes written.
  • error: An error if the operation fails.

Always returns the length of data and a nil error, unless the receiver is nil; in which case it returns 0 and an error.

type Prettify added in v0.1.11

type Prettify struct {
	// contains filtered or unexported fields
}

Prettify is a pretty-printer.

func NewPrettify added in v0.1.11

func NewPrettify(tab_size int) (*Prettify, error)

NewPrettify returns a new instance of Prettify with the given tab size.

Parameters:

  • tab_size: The size of the tab.

Returns:

  • *Prettify: A new instance of Prettify. Never returns nil.
  • error: An error if the tab size is invalid.

func (Prettify) GetCoords added in v0.1.11

func (p Prettify) GetCoords(data []byte, pos int) ([2]int, error)

GetCoords returns the coordinates of the given position in the given data.

Parameters:

  • data: The data to get the coordinates from.
  • pos: The position in the data to get the coordinates for.

Returns:

  • [2]int: The coordinates of the given position in the given data. The coordinates are [x, y] where x is the column and y is the line.
  • error: An error if the position is invalid, or if the data cannot be processed.

func (Prettify) PrettifyData added in v0.1.11

func (p Prettify) PrettifyData() ([]byte, error)

PrettifyData prettifies the data by highlighting the position of the lexer error.

The prettified data is returned as a byte slice. The byte slice is created by wrapping the original data in a string and inserting a caret (^) at the position of the lexer error. The position of the caret is determined by the value of the at field, which is the index of the character in the original data where the lexer error occurred.

The prettified data is formatted as follows:

...<before>...
<faulty_line>
<caret>
...<after>...

Where:

  • <before> is the part of the original data before the lexer error.
  • <faulty_line> is the line of the original data containing the lexer error.
  • <caret> is a caret (^) indicating the position of the lexer error.
  • <after> is the part of the original data after the lexer error.

If the lexer error is at the beginning of the data, the caret is placed at the beginning of the line. If the lexer error is at the end of the data, the caret is placed at the end of the line.

Returns:

  • []byte: The prettified data.
  • error: An error if the receiver is nil, or if the data cannot be prettified.

func (Prettify) PrettifyLexError added in v0.1.11

func (p Prettify) PrettifyLexError() ([]byte, error)

PrettifyLexError returns a prettified lexer error.

Returns:

  • []byte: The prettified lexer error.
  • error: An error if the receiver is nil, or if the lexer error cannot be prettified.

func (*Prettify) SetData added in v0.1.11

func (p *Prettify) SetData(data []byte) error

SetData sets the data to be pretty-printed.

Parameters:

  • data: The data to be pretty-printed.

Returns:

  • error: An error if the receiver is nil.

func (*Prettify) SetError added in v0.1.11

func (p *Prettify) SetError(err error) error

SetError sets the error of the lexer.

Parameters:

  • err: The error to be set.

Returns:

  • error: An error if the receiver is nil, or if the error cannot be set.

The error is set by directly assigning the error to the receiver's err field. The receiver's at field is set to the value of the error if the error is an instance of ErrLexing, otherwise the receiver's at field is cleared.

type Result added in v0.1.11

type Result struct {
	// contains filtered or unexported fields
}

Result is the result of an active lexer.

func (Result) GetError added in v0.1.11

func (r Result) GetError() error

GetError returns the error of the active lexer.

Returns:

  • error: The error of the active lexer.

func (Result) Tokens added in v0.1.11

func (r Result) Tokens() []*slgr.Token

Tokens returns the tokens generated from the MatchResult of the active lexer.

Returns:

  • []*slgr.Token: The tokens generated from the MatchResult.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL