pip._vendor.pygments.util Namespace Reference

Data Structures
class	ClassNotFound
class	Future
class	OptionError
class	UnclosingTextIOWrapper

Functions
	get_choice_opt (options, optname, allowed, default=None, normcase=False)
	get_bool_opt (options, optname, default=None)
	get_int_opt (options, optname, default=None)
	get_list_opt (options, optname, default=None)
	docstring_headline (obj)
	make_analysator (f)
	shebang_matches (text, regex)
	doctype_matches (text, regex)
	html_doctype_matches (text)
	looks_like_xml (text)
	surrogatepair (c)
	format_lines (var_name, seq, raw=False, indent_level=0)
	duplicates_removed (it, already_seen=())
	guess_decode (text)
	guess_decode_from_terminal (text, term)
	terminal_encoding (term)

Variables
	split_path_re = re.compile(r'[/\\ ]')
	doctype_lookup_re = re.compile(, re.DOTALL | re.MULTILINE | re.VERBOSE)
	tag_re
	xml_decl_re = re.compile(r'\s*<\?xml[^>]*\?>', re.I)
dict	_looks_like_xml_cache = {}

Detailed Description

    pygments.util
    ~~~~~~~~~~~~~

    Utility functions.

    :copyright: Copyright 2006-2023 by the Pygments team, see AUTHORS.
    :license: BSD, see LICENSE for details.

Function Documentation

docstring_headline()

docstring_headline

(

obj

)

Definition at line 116 of file util.py.

def docstring_headline(obj):
    if not obj.__doc__:
        return ''
    res = []
    for line in obj.__doc__.strip().splitlines():
        if line.strip():
            res.append(" " + line.strip())
        else:
            break
    return ''.join(res).lstrip()
 
 

References i.

doctype_matches()

doctype_matches	(		text,
			regex
	)

Check if the doctype matches a regular expression (if present).

Note that this method only checks the first part of a DOCTYPE.
eg: 'html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"'

Definition at line 190 of file util.py.

def doctype_matches(text, regex):
    """Check if the doctype matches a regular expression (if present).
 
    Note that this method only checks the first part of a DOCTYPE.
    eg: 'html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"'
    """
    m = doctype_lookup_re.search(text)
    if m is None:
        return False
    doctype = m.group(1)
    return re.compile(regex, re.I).match(doctype.strip()) is not None
 
 

References i.

Referenced by pip._vendor.pygments.util.html_doctype_matches().

Here is the caller graph for this function:

duplicates_removed()

duplicates_removed	(		it,
			already_seen = ()
	)

Returns a list with duplicates removed from the iterable `it`.

Order is preserved.

Definition at line 255 of file util.py.

def duplicates_removed(it, already_seen=()):
    """
    Returns a list with duplicates removed from the iterable `it`.
 
    Order is preserved.
    """
    lst = []
    seen = set()
    for i in it:
        if i in seen or i in already_seen:
            continue
        lst.append(i)
        seen.add(i)
    return lst
 
 

References i.

format_lines()

format_lines	(		var_name,
			seq,
			raw = False,
			indent_level = 0
	)

Formats a sequence of strings for output.

Definition at line 236 of file util.py.

def format_lines(var_name, seq, raw=False, indent_level=0):
    """Formats a sequence of strings for output."""
    lines = []
    base_indent = ' ' * indent_level * 4
    inner_indent = ' ' * (indent_level + 1) * 4
    lines.append(base_indent + var_name + ' = (')
    if raw:
        # These should be preformatted reprs of, say, tuples.
        for i in seq:
            lines.append(inner_indent + i + ',')
    else:
        for i in seq:
            # Force use of single quotes
            r = repr(i + '"')
            lines.append(inner_indent + r[:-2] + r[-1] + ',')
    lines.append(base_indent + ')')
    return '\n'.join(lines)
 
 

References i.

get_bool_opt()

get_bool_opt	(		options,
			optname,
			default = None
	)

Intuitively, this is `options.get(optname, default)`, but restricted to
Boolean value. The Booleans can be represented as string, in order to accept
Boolean value from the command line arguments. If the key `optname` is
present in the dictionary `options` and is not associated with a Boolean,
raise an `OptionError`. If it is absent, `default` is returned instead.

The valid string values for ``True`` are ``1``, ``yes``, ``true`` and
``on``, the ones for ``False`` are ``0``, ``no``, ``false`` and ``off``
(matched case-insensitively).

Definition at line 54 of file util.py.

def get_bool_opt(options, optname, default=None):
    """
    Intuitively, this is `options.get(optname, default)`, but restricted to
    Boolean value. The Booleans can be represented as string, in order to accept
    Boolean value from the command line arguments. If the key `optname` is
    present in the dictionary `options` and is not associated with a Boolean,
    raise an `OptionError`. If it is absent, `default` is returned instead.
 
    The valid string values for ``True`` are ``1``, ``yes``, ``true`` and
    ``on``, the ones for ``False`` are ``0``, ``no``, ``false`` and ``off``
    (matched case-insensitively).
    """
    string = options.get(optname, default)
    if isinstance(string, bool):
        return string
    elif isinstance(string, int):
        return bool(string)
    elif not isinstance(string, str):
        raise OptionError('Invalid type %r for option %s; use '
                          '1/0, yes/no, true/false, on/off' % (
                              string, optname))
    elif string.lower() in ('1', 'yes', 'true', 'on'):
        return True
    elif string.lower() in ('0', 'no', 'false', 'off'):
        return False
    else:
        raise OptionError('Invalid value %r for option %s; use '
                          '1/0, yes/no, true/false, on/off' % (
                              string, optname))
 
 

References i.

get_choice_opt()

get_choice_opt	(		options,
			optname,
			allowed,
			default = None,
			normcase = False
	)

If the key `optname` from the dictionary is not in the sequence
`allowed`, raise an error, otherwise return it.

Definition at line 40 of file util.py.

def get_choice_opt(options, optname, allowed, default=None, normcase=False):
    """
    If the key `optname` from the dictionary is not in the sequence
    `allowed`, raise an error, otherwise return it.
    """
    string = options.get(optname, default)
    if normcase:
        string = string.lower()
    if string not in allowed:
        raise OptionError('Value for option %s must be one of %s' %
                          (optname, ', '.join(map(str, allowed))))
    return string
 
 

get_int_opt()

get_int_opt	(		options,
			optname,
			default = None
	)

As :func:`get_bool_opt`, but interpret the value as an integer.

Definition at line 85 of file util.py.

def get_int_opt(options, optname, default=None):
    """As :func:`get_bool_opt`, but interpret the value as an integer."""
    string = options.get(optname, default)
    try:
        return int(string)
    except TypeError:
        raise OptionError('Invalid type %r for option %s; you '
                          'must give an integer value' % (
                              string, optname))
    except ValueError:
        raise OptionError('Invalid value %r for option %s; you '
                          'must give an integer value' % (
                              string, optname))
 

References i.

get_list_opt()

get_list_opt	(		options,
			optname,
			default = None
	)

If the key `optname` from the dictionary `options` is a string,
split it at whitespace and return it. If it is already a list
or a tuple, it is returned as a list.

Definition at line 99 of file util.py.

def get_list_opt(options, optname, default=None):
    """
    If the key `optname` from the dictionary `options` is a string,
    split it at whitespace and return it. If it is already a list
    or a tuple, it is returned as a list.
    """
    val = options.get(optname, default)
    if isinstance(val, str):
        return val.split()
    elif isinstance(val, (list, tuple)):
        return list(val)
    else:
        raise OptionError('Invalid type %r for option %s; you '
                          'must give a list value' % (
                              val, optname))
 
 

References i.

guess_decode()

guess_decode

(

text

)

Decode *text* with guessed encoding.

First try UTF-8; this should fail for non-UTF-8 encodings.
Then try the preferred locale encoding.
Fall back to latin-1, which always works.

Definition at line 281 of file util.py.

def guess_decode(text):
    """Decode *text* with guessed encoding.
 
    First try UTF-8; this should fail for non-UTF-8 encodings.
    Then try the preferred locale encoding.
    Fall back to latin-1, which always works.
    """
    try:
        text = text.decode('utf-8')
        return text, 'utf-8'
    except UnicodeDecodeError:
        try:
            import locale
            prefencoding = locale.getpreferredencoding()
            text = text.decode()
            return text, prefencoding
        except (UnicodeDecodeError, LookupError):
            text = text.decode('latin1')
            return text, 'latin1'
 
 

References i.

guess_decode_from_terminal()

guess_decode_from_terminal	(		text,
			term
	)

Decode *text* coming from terminal *term*.

First try the terminal encoding, if given.
Then try UTF-8.  Then try the preferred locale encoding.
Fall back to latin-1, which always works.

Definition at line 302 of file util.py.

def guess_decode_from_terminal(text, term):
    """Decode *text* coming from terminal *term*.
 
    First try the terminal encoding, if given.
    Then try UTF-8.  Then try the preferred locale encoding.
    Fall back to latin-1, which always works.
    """
    if getattr(term, 'encoding', None):
        try:
            text = text.decode(term.encoding)
        except UnicodeDecodeError:
            pass
        else:
            return text, term.encoding
    return guess_decode(text)
 
 

References i.

html_doctype_matches()

html_doctype_matches

(

text

)

Check if the file looks like it has a html doctype.

Definition at line 203 of file util.py.

def html_doctype_matches(text):
    """Check if the file looks like it has a html doctype."""
    return doctype_matches(text, r'html')
 
 

References pip._vendor.pygments.util.doctype_matches().

Here is the call graph for this function:

looks_like_xml()

looks_like_xml

(

text

)

Check if a doctype exists or if we have some tags.

Definition at line 211 of file util.py.

def looks_like_xml(text):
    """Check if a doctype exists or if we have some tags."""
    if xml_decl_re.match(text):
        return True
    key = hash(text)
    try:
        return _looks_like_xml_cache[key]
    except KeyError:
        m = doctype_lookup_re.search(text)
        if m is not None:
            return True
        rv = tag_re.search(text[:1000]) is not None
        _looks_like_xml_cache[key] = rv
        return rv
 
 

References i.

make_analysator()

make_analysator

(

f

)

Return a static text analyser function that returns float values.

Definition at line 128 of file util.py.

def make_analysator(f):
    """Return a static text analyser function that returns float values."""
    def text_analyse(text):
        try:
            rv = f(text)
        except Exception:
            return 0.0
        if not rv:
            return 0.0
        try:
            return min(1.0, max(0.0, float(rv)))
        except (ValueError, TypeError):
            return 0.0
    text_analyse.__doc__ = f.__doc__
    return staticmethod(text_analyse)
 
 

References i.

shebang_matches()

shebang_matches	(		text,
			regex
	)

Check if the given regular expression matches the last part of the
shebang if one exists.

    >>> from pygments.util import shebang_matches
    >>> shebang_matches('#!/usr/bin/env python', r'python(2\.\d)?')
    True
    >>> shebang_matches('#!/usr/bin/python2.4', r'python(2\.\d)?')
    True
    >>> shebang_matches('#!/usr/bin/python-ruby', r'python(2\.\d)?')
    False
    >>> shebang_matches('#!/usr/bin/python/ruby', r'python(2\.\d)?')
    False
    >>> shebang_matches('#!/usr/bin/startsomethingwith python',
    ...                 r'python(2\.\d)?')
    True

It also checks for common windows executable file extensions::

    >>> shebang_matches('#!C:\\Python2.4\\Python.exe', r'python(2\.\d)?')
    True

Parameters (``'-f'`` or ``'--foo'`` are ignored so ``'perl'`` does
the same as ``'perl -e'``)

Note that this method automatically searches the whole string (eg:
the regular expression is wrapped in ``'^$'``)

Definition at line 145 of file util.py.

def shebang_matches(text, regex):
    r"""Check if the given regular expression matches the last part of the
    shebang if one exists.
 
        >>> from pygments.util import shebang_matches
        >>> shebang_matches('#!/usr/bin/env python', r'python(2\.\d)?')
        True
        >>> shebang_matches('#!/usr/bin/python2.4', r'python(2\.\d)?')
        True
        >>> shebang_matches('#!/usr/bin/python-ruby', r'python(2\.\d)?')
        False
        >>> shebang_matches('#!/usr/bin/python/ruby', r'python(2\.\d)?')
        False
        >>> shebang_matches('#!/usr/bin/startsomethingwith python',
        ...                 r'python(2\.\d)?')
        True
 
    It also checks for common windows executable file extensions::
 
        >>> shebang_matches('#!C:\\Python2.4\\Python.exe', r'python(2\.\d)?')
        True
 
    Parameters (``'-f'`` or ``'--foo'`` are ignored so ``'perl'`` does
    the same as ``'perl -e'``)
 
    Note that this method automatically searches the whole string (eg:
    the regular expression is wrapped in ``'^$'``)
    """
    index = text.find('\n')
    if index >= 0:
        first_line = text[:index].lower()
    else:
        first_line = text.lower()
    if first_line.startswith('#!'):
        try:
            found = [x for x in split_path_re.split(first_line[2:].strip())
                     if x and not x.startswith('-')][-1]
        except IndexError:
            return False
        regex = re.compile(r'^%s(\.(exe|cmd|bat|bin))?$' % regex, re.IGNORECASE)
        if regex.search(found) is not None:
            return True
    return False
 
 

References i.

surrogatepair()

surrogatepair

(

c

)

Given a unicode character code with length greater than 16 bits,
return the two 16 bit surrogate pair.

Definition at line 227 of file util.py.

def surrogatepair(c):
    """Given a unicode character code with length greater than 16 bits,
    return the two 16 bit surrogate pair.
    """
    # From example D28 of:
    # http://www.unicode.org/book/ch03.pdf
    return (0xd7c0 + (c >> 10), (0xdc00 + (c & 0x3ff)))
 
 

terminal_encoding()

terminal_encoding

(

term

)

Return our best guess of encoding for the given *term*.

Definition at line 319 of file util.py.

def terminal_encoding(term):
    """Return our best guess of encoding for the given *term*."""
    if getattr(term, 'encoding', None):
        return term.encoding
    import locale
    return locale.getpreferredencoding()
 
 

References i.

Variable Documentation

_looks_like_xml_cache

dict _looks_like_xml_cache = {}

protected

Definition at line 208 of file util.py.

doctype_lookup_re

doctype_lookup_re = re.compile(, re.DOTALL | re.MULTILINE | re.VERBOSE)

Definition at line 16 of file util.py.

split_path_re

split_path_re = re.compile(r'[/\\ ]')

Definition at line 15 of file util.py.

tag_re

tag_re

Initial value:

=  re.compile(r'<(.+?)(\s.*?)?>.*?</.+?>',
                    re.IGNORECASE | re.DOTALL | re.MULTILINE)

Definition at line 25 of file util.py.

xml_decl_re

xml_decl_re = re.compile(r'\s*<\?xml[^>]*\?>', re.I)

Definition at line 27 of file util.py.