pip._vendor.pygments.token Namespace Reference

Data Structures
class	_TokenType

Functions
	is_token_subtype (ttype, other)
	string_to_tokentype (s)

Variables
	Token = _TokenType()
	Text = Token.Text
	Whitespace = Text.Whitespace
	Escape = Token.Escape
	Error = Token.Error
	Other = Token.Other
	Keyword = Token.Keyword
	Name = Token.Name
	Literal = Token.Literal
	String = Literal.String
	Number = Literal.Number
	Punctuation = Token.Punctuation
	Operator = Token.Operator
	Comment = Token.Comment
	Generic = Token.Generic
dict	STANDARD_TYPES

Detailed Description

    pygments.token
    ~~~~~~~~~~~~~~

    Basic token types and the standard tokens.

    :copyright: Copyright 2006-2023 by the Pygments team, see AUTHORS.
    :license: BSD, see LICENSE for details.

Function Documentation

is_token_subtype()

is_token_subtype	(		ttype,
			other
	)

Return True if ``ttype`` is a subtype of ``other``.

exists for backwards compatibility. use ``ttype in other`` now.

Definition at line 85 of file token.py.

def is_token_subtype(ttype, other):
    """
    Return True if ``ttype`` is a subtype of ``other``.
 
    exists for backwards compatibility. use ``ttype in other`` now.
    """
    return ttype in other
 
 

string_to_tokentype()

string_to_tokentype

(

s

)

Convert a string into a token type::

    >>> string_to_token('String.Double')
    Token.Literal.String.Double
    >>> string_to_token('Token.Literal.Number')
    Token.Literal.Number
    >>> string_to_token('')
    Token

Tokens that are already tokens are returned unchanged:

    >>> string_to_token(String)
    Token.Literal.String

Definition at line 94 of file token.py.

def string_to_tokentype(s):
    """
    Convert a string into a token type::
 
        >>> string_to_token('String.Double')
        Token.Literal.String.Double
        >>> string_to_token('Token.Literal.Number')
        Token.Literal.Number
        >>> string_to_token('')
        Token
 
    Tokens that are already tokens are returned unchanged:
 
        >>> string_to_token(String)
        Token.Literal.String
    """
    if isinstance(s, _TokenType):
        return s
    if not s:
        return Token
    node = Token
    for item in s.split('.'):
        node = getattr(node, item)
    return node
 
 
# Map standard token types to short names, used in CSS class naming.
# If you add a new item, please be sure to run this file to perform
# a consistency check for duplicate values.

References i.

Variable Documentation

Comment

Comment = Token.Comment

Definition at line 73 of file token.py.

Error

Error = Token.Error

Definition at line 61 of file token.py.

Escape

Escape = Token.Escape

Definition at line 60 of file token.py.

Generic

Generic = Token.Generic

Definition at line 76 of file token.py.

Keyword

Keyword = Token.Keyword

Definition at line 66 of file token.py.

Literal

Literal = Token.Literal

Definition at line 68 of file token.py.

Name

Name = Token.Name

Definition at line 67 of file token.py.

Number

Number = Literal.Number

Definition at line 70 of file token.py.

Operator

Operator = Token.Operator

Definition at line 72 of file token.py.

Other

Other = Token.Other

Definition at line 63 of file token.py.

Punctuation

Punctuation = Token.Punctuation

Definition at line 71 of file token.py.

STANDARD_TYPES

dict STANDARD_TYPES

Definition at line 123 of file token.py.

String

String = Literal.String

Definition at line 69 of file token.py.

Text

Text = Token.Text

Definition at line 58 of file token.py.

Token

Token = _TokenType()

Definition at line 55 of file token.py.

Whitespace

Whitespace = Text.Whitespace

Definition at line 59 of file token.py.