optimism

A very simple testing library intended for use by students in an intro course. Includes capabilities for mocking input and setting up checks for the results and/or printed output of arbitrary expressions. Also includes basic AST checking tools (good for instructors to include in test suites but probably tricky for students to author) and a reverse tracing feature where comments specifying the evolution of program state can be checked.

optimism.py

Example usage

Basic test cases

See examples/basic.py for an extended example file showing basic usage of core functions. Here is a very short example:

>>> import optimism
>>> optimism.messagesAsErrors(False)
>>> optimism.colors(False)
>>> def f(x, y):
...     "Example function"
...     return x + y + 1
...
>>> # Simple test for that function
>>> tester = optimism.testFunction(f)
>>> case = tester.case(1, 2)
>>> case.checkReturnValue(4) # doctest: +ELLIPSIS
✓ ...
True
>>> case.checkReturnValue(5) # doctest: +ELLIPSIS
✗ ...
  Result:
    4
  was NOT equivalent to the expected value:
    5
  Called function 'f' with arguments:
    x = 1
    y = 2
False

For code in files, the filename and line number are shown instead of stdin:1 as shown above. Note that line numbers reported for function calls that span multiple lines may be different between Python versions before 3.8 and versions 3.8+.

Code Structure Checking

The examples/code.py file has a longer example of how to use the AST-checking mechanisms for checking code structures. Here is a simple one:

>>> import optimism
>>> def askNameAge():
...     "A function that uses input."
...     name = input("What is your name? ")
...     age = input("How old are you? ")
...     return (name, age)
...
>>> tester = optimism.testFunction(askNameAge)
>>> tester.checkCodeContains(optimism.Call('input')) # doctest: +ELLIPSIS
✓ ...
True
>>> tester.checkCodeContains(optimism.Loop()) # doctest: +ELLIPSIS
✗ ...
  Code does not contain the expected structure:
    at least 1 loop(s) or generator expression(s)
  checked code of function 'askNameAge'
False

The TestManager.checkCodeContains function and the various code-structure classes (see ASTRequirement) can be used to ensure that the Abstract Syntax Tree (AST) of the code associated with a test manager has certain structures.

Reverse Tracing

Note: This functionality is planned, but not implemented yet.

The examples/reverse_tracing.py file contains more thorough examples of this functionality. Here is a simple one:

>>> import optimism
>>> code = '''\
... x = 5
... ## x = 5
... y = x + 3
... ## y=8
... print(x, '\\n', y)
... ## prints:
... ## '5 '
... ## ' 8'
... x, y = y, x
... ## x = 8, y = 5
... z = x + y
... # this last trace assertion is incorrect
... ## z = 14
... '''
>>> tester = optimism.testBlock(code)
>>> tester.validateTrace() # doctest: +SKIP +ELLIPSIS
✗ ...
  Trace assertions were not completely valid:
    First failing assertion was on line 13:
      z = 14
    The actual value of z was:
      13
False
>>> functionBlock = '''\
... def f(x, y):
...     #: x = 3, y = 4
...     x += 1
...     ## x = 4
...     return x + y
...     ## returns: 8
...
... # These trace blocks replay the most-recently defined trace suite ('#:')
... # using different initial values.
... #> x = 1, y = 1
... ## x = 2
... ## returns: 3
...
... #> x=2, y=2
... ## x=3
... ## returns: 5
... '''
>>> tester2 = optimism.testBlock(functionBlock)
>>> tester2.validateTrace() # doctest: +SKIP +ELLIPSIS
✓ ...
True

Core functionality

The main functions you'll need are:

  • trace works like print, but shows some extra information, and you can use it as part of a larger expression. Use this for figuring out what's going wrong when your tests don't pass.
  • expect takes two arguments and prints a check-mark if they're equivalent or an x if not. If they aren't equivalent, it prints details about values that are part of the first expression. Use this for expectations-based debugging.
  • expectType works like expect, but the second argument is a type, and it checks whether the value of the first argument is an instance of that type or not. For more serious type-checking without having to run your program and without slowing it down when it does run, use MyPy or another actual type-checking system.
  • testFunction establishes a test manager object, which can be used to create test cases. When you call testFunction you specify the function that you want to test. The .case method of the resulting TestManager object can be used to set up individual test cases. The .checkCodeContains method can be used to check for certain structural properties of the code of the function.
  • testFile establishes a test manager object just like testFunction, but for running an entire file instead of for calling a single function.
  • testBlock establishes a test manager object just like testFunction, but for running a block of code (given as a string).
  • The TestManager.case method establishes a test case by specifying what arguments are going to be used with the associated function. It returns a TestCase object that can be used to establish expectations. For test managers based on files or blocks, no arguments are needed.
  • The TestCase.checkReturnValue, TestCase.checkVariableValue, TestCase.checkPrintedLines, and/or TestCase.checkCustom methods can be used to run checks for the return value and/or printed output and/or variables defined by a test case.
  • Normally, output printed during tests is hidden, but showPrintedLines can be used to show the text that's being captured instead.
  • TestCase.provideInputs sets up inputs for a test case, so that interactive code can be tested without pausing for real user input.
  • The TestManager.checkCodeContains method can be used to check the abstract syntax tree of of the function, file or block associated with a test manager. The argument must be an instance of the ASTRequirement class, which has multiple sub-classes for checking for the presence of various different code constructs.
  • The TestManager.validateTrace method can be used to check trace assertion comments within the code of a function, file, or code block. TODO: This function is not implemented yet.
  • detailLevel can be called to control the level of detail printed in the output. It affects all tracing, expectation, and testing output produced until it gets called again.
  • showSummary can be used to summarize the number of checks which passed or failed.
  • colors can be used to enable or disable color codes for printed text. Disable this if you're getting garbled printed output.

TODO: Workaround for tracing in interactive console?

Changelog

  • Version 2.7.10 adds Literal class for matching literals, and getLiteralValue function for extracting well-defined literal values from an AST. It also changes the type= keyword argument for Constant to be types= to avoid the name clash with the built-in type function (Literal also uses types=).
  • Version 2.7.9 fixes minor bugs from 2.7.7 and 2.7.8 (e.g., formatting). It also adds Reference for matching variable references.
  • Version 2.7.8 adds the type= argument to Constant for finding constants w/ unconstrained values but particular type(s).
  • Version 2.7.7 adds the Operator class for code checking.
  • Version 2.7.6 monkey-patches the input function in addition to overriding stdin during payload runs, to try to deal with notebook environments better where input doesn't read from stdin by default. This may cause more problems with other input-capturing solutions that also mock input...
  • Version 2.7.5 allows the code value for a function test manager to be None when the code for the function cannot be found using inspect.getsource, instead of letting the associated OSError bubble out.
  • Version 2.7.4 flips this changelog right-side-up (i.e., newest-first). Also introduces the code slot for TestManager objects, so that they store raw code in addition to a derived syntax tree. This change also means that BlockManager objects no longer store their code in the target slot, which is now just a fixed string. It also changes listAllCases to listAllTrials and changes 'cases' to 'trials' in a few other places since code checks are not associated with test cases but are trials. Common functionality is moved to the Trial class.
  • Version 2.7.3 introduces the mark function, and removes testCodeWithSuite, adding testMarkedCode instead. This helps prevent suites (previously required for marking code blocks) from being started when you need another suite to be active.
  • Version 2.7.2 introduces listOutcomesInSuite and registers outcomes from expect and expectType calls. These will also be included in showSummary results. It renames startTestSuite to just testSuite, and introduces freshTestSuite which deletes any old results instead of extending them.
  • Version 2.7.1 sets the SKIP_ON_FAILURE back to None, since default skipping is a potential issue for automated test reporting. It adds SUPPRESS_ON_FAILURE to compensate for this and enables it by default, suppressing error details from checks after one failure per manager. It also adds checkVariableValue for checking the values of variables set in files or code blocks (it does not work on function tests).
  • Version 2.7 introduces the ASTRequirement class and subclasses, along with the TestManager.checkCodeContains method for applying them. It reorganizes things a bit so that Trial is now a super-class of both TestCase and the new CodeChecks class.
  • Version 2.6.7 changes default SKIP_ON_FAILURE back to 'case', since 'all' makes interactive testing hard, and dedicated test files can call skipChecksAfterFail at the top. Also fixes an issue where comparing printed output correctly is lenient about the presence or absence of a final newline, but comparing file contents didn't do that. This change means that extra blank lines (including lines with whitespace on them) are ignored when comparing strings and IGNORE_TRAILING_WHITESPACE is on, and even when IGNORE_TRAILING_WHITESPACE is off, the presence or absence of a final newline in a file or in printed output will be copied over to a multi-line expectation (since otherwise there's no way to specify the lack of a final newline when giving multiple string arguments to checkPrintedLines or checkFileLines).
  • Version 2.6.6 changes from splitlines to split('\n') in a few places because the latter is more robust to extra carriage returns. This changes how some error messages look and it means that in some places the newline at the end of a file actually counts as having a blank line there in terms of output (behavior is mostly unchanged). Also escaped carriage returns in displayed strings so they're more visible. From this version we do NOT support files that use just '\r' as a newline as easily. But IGNORE_TRAILING_WHITESPACE will properly get rid of any extra '\r' before a newline, so when that's on (the default) this shouldn't change much. A test of this behavior was added to the file test example.
  • Version 2.6.5 fixes a bug with displaying filenames when a file does not exist and checkFileLines is used, and also sets the default length higher for printing first differing lines in findFirstDifference since those lines are displayed on their own line anyways. Fixes a bug where list differences were not displayed correctly, and improves the usefulness of first differences displayed for dictionaries w/ different lengths. Also fixes a bug where strings which were equivalent modulo trailing whitespace would not be treated as equivalent.
  • Version 2.6.4 immediately changes checkFileContainsLines to checkFileLines to avoid confusion about whether we're checking against the whole file (we are).
  • Version 2.6.3 introduces the checkFileContainsLines method, and also standardizes the difference-finding code and merges it with equality-checking code, removing checkEquality and introducing findFirstDifference instead (compare) remains but just calls findFirstDifference internally. Also fixes a bug w/ printing tracebacks for come checkers (need to standardize that!). Also adds global skip-on-failure and sets that as the default!
  • Version 2.6.2 fixes a bug with dictionary comparison which caused a crash when key sets weren't equal. It also adds unit tests for the compare function.
  • Version 2.4.0 adds a more object-oriented structure behind the scenes, without changing any core API functions. It also adds support for variable specifications in block tests.
  • Version 2.3.0 adds testFunctionMaybe for skippable tests if the target function hasn't been defined yet.
  • Version 2.2.0 changed the names of checkResult and checkOutputLines to checkReturnValue and checkPrintedLines
  • Version 2.0 introduced the TestManager and TestCase classes, and got rid of automatic tracking for test cases. The old test case functionality was moved over to the expect function. This helps make tests more stable and makes meta-reasoning easier.
   1# -*- coding: utf-8 -*-
   2"""
   3A very simple testing library intended for use by students in an intro
   4course. Includes capabilities for mocking input and setting up checks
   5for the results and/or printed output of arbitrary expressions. Also
   6includes basic AST checking tools (good for instructors to include in
   7test suites but probably tricky for students to author) and a
   8reverse tracing feature where comments specifying the evolution of
   9program state can be checked.
  10
  11optimism.py
  12
  13## Example usage
  14
  15### Basic test cases
  16
  17See `examples/basic.py` for an extended example file showing basic usage
  18of core functions. Here is a very short example:
  19
  20>>> import optimism
  21>>> optimism.messagesAsErrors(False)
  22>>> optimism.colors(False)
  23>>> def f(x, y):
  24...     "Example function"
  25...     return x + y + 1
  26...
  27>>> # Simple test for that function
  28>>> tester = optimism.testFunction(f)
  29>>> case = tester.case(1, 2)
  30>>> case.checkReturnValue(4) # doctest: +ELLIPSIS
  31✓ ...
  32True
  33>>> case.checkReturnValue(5) # doctest: +ELLIPSIS
  34✗ ...
  35  Result:
  36    4
  37  was NOT equivalent to the expected value:
  38    5
  39  Called function 'f' with arguments:
  40    x = 1
  41    y = 2
  42False
  43
  44For code in files, the filename and line number are shown instead of
  45stdin:1 as shown above. Note that line numbers reported for function
  46calls that span multiple lines may be different between Python versions
  47before 3.8 and versions 3.8+.
  48
  49
  50### Code Structure Checking
  51
  52The `examples/code.py` file has a longer example of how to use the
  53AST-checking mechanisms for checking code structures. Here is a simple
  54one:
  55
  56>>> import optimism
  57>>> def askNameAge():
  58...     "A function that uses input."
  59...     name = input("What is your name? ")
  60...     age = input("How old are you? ")
  61...     return (name, age)
  62...
  63>>> tester = optimism.testFunction(askNameAge)
  64>>> tester.checkCodeContains(optimism.Call('input')) # doctest: +ELLIPSIS
  65✓ ...
  66True
  67>>> tester.checkCodeContains(optimism.Loop()) # doctest: +ELLIPSIS
  68✗ ...
  69  Code does not contain the expected structure:
  70    at least 1 loop(s) or generator expression(s)
  71  checked code of function 'askNameAge'
  72False
  73
  74The `TestManager.checkCodeContains` function and the various
  75code-structure classes (see `ASTRequirement`) can be used to ensure that
  76the Abstract Syntax Tree (AST) of the code associated with a test manager
  77has certain structures.
  78
  79
  80### Reverse Tracing
  81
  82Note: This functionality is planned, but not implemented yet.
  83
  84The `examples/reverse_tracing.py` file contains more thorough examples of
  85this functionality. Here is a simple one:
  86
  87>>> import optimism
  88>>> code = '''\\
  89... x = 5
  90... ## x = 5
  91... y = x + 3
  92... ## y=8
  93... print(x, '\\\\n', y)
  94... ## prints:
  95... ## '5 '
  96... ## ' 8'
  97... x, y = y, x
  98... ## x = 8, y = 5
  99... z = x + y
 100... # this last trace assertion is incorrect
 101... ## z = 14
 102... '''
 103>>> tester = optimism.testBlock(code)
 104>>> tester.validateTrace() # doctest: +SKIP +ELLIPSIS
 105✗ ...
 106  Trace assertions were not completely valid:
 107    First failing assertion was on line 13:
 108      z = 14
 109    The actual value of z was:
 110      13
 111False
 112>>> functionBlock = '''\\
 113... def f(x, y):
 114...     #: x = 3, y = 4
 115...     x += 1
 116...     ## x = 4
 117...     return x + y
 118...     ## returns: 8
 119...
 120... # These trace blocks replay the most-recently defined trace suite ('#:')
 121... # using different initial values.
 122... #> x = 1, y = 1
 123... ## x = 2
 124... ## returns: 3
 125...
 126... #> x=2, y=2
 127... ## x=3
 128... ## returns: 5
 129... '''
 130>>> tester2 = optimism.testBlock(functionBlock)
 131>>> tester2.validateTrace() # doctest: +SKIP +ELLIPSIS
 132✓ ...
 133True
 134
 135
 136## Core functionality
 137
 138The main functions you'll need are:
 139
 140- `trace` works like `print`, but shows some extra information, and you
 141  can use it as part of a larger expression. Use this for figuring out
 142  what's going wrong when your tests don't pass.
 143- `expect` takes two arguments and prints a check-mark if they're
 144  equivalent or an x if not. If they aren't equivalent, it prints
 145  details about values that are part of the first expression. Use this
 146  for expectations-based debugging.
 147- `expectType` works like `expect`, but the second argument is a type,
 148  and it checks whether the value of the first argument is an instance
 149  of that type or not. For more serious type-checking without having to
 150  run your program and without slowing it down when it does run, use MyPy
 151  or another actual type-checking system.
 152- `testFunction` establishes a test manager object, which can be used to
 153  create test cases. When you call `testFunction` you specify the
 154  function that you want to test. The `.case` method of the resulting
 155  `TestManager` object can be used to set up individual test cases. The
 156  `.checkCodeContains` method can be used to check for certain structural
 157  properties of the code of the function.
 158- `testFile` establishes a test manager object just like `testFunction`,
 159  but for running an entire file instead of for calling a single
 160  function.
 161- `testBlock` establishes a test manager object just like `testFunction`,
 162  but for running a block of code (given as a string).
 163- The `TestManager.case` method establishes a test case by specifying
 164  what arguments are going to be used with the associated function. It
 165  returns a `TestCase` object that can be used to establish
 166  expectations. For test managers based on files or blocks, no arguments
 167  are needed.
 168- The `TestCase.checkReturnValue`, `TestCase.checkVariableValue`,
 169  `TestCase.checkPrintedLines`, and/or `TestCase.checkCustom` methods can
 170  be used to run checks for the return value and/or printed output and/or
 171  variables defined by a test case.
 172- Normally, output printed during tests is hidden, but `showPrintedLines`
 173  can be used to show the text that's being captured instead.
 174- `TestCase.provideInputs` sets up inputs for a test case, so that
 175  interactive code can be tested without pausing for real user input.
 176- The `TestManager.checkCodeContains` method can be used to check the
 177  abstract syntax tree of of the function, file or block associated with
 178  a test manager. The argument must be an instance of the
 179  `ASTRequirement` class, which has multiple sub-classes for checking for
 180  the presence of various different code constructs.
 181- The `TestManager.validateTrace` method can be used to check trace
 182  assertion comments within the code of a function, file, or code block.
 183  **TODO: This function is not implemented yet.**
 184- `detailLevel` can be called to control the level of detail printed in
 185  the output. It affects all tracing, expectation, and testing output
 186  produced until it gets called again.
 187- `showSummary` can be used to summarize the number of checks which
 188  passed or failed.
 189- `colors` can be used to enable or disable color codes for printed text.
 190  Disable this if you're getting garbled printed output.
 191
 192TODO: Workaround for tracing in interactive console?
 193
 194## Changelog
 195
 196- Version 2.7.10 adds `Literal` class for matching literals, and
 197  `getLiteralValue` function for extracting well-defined literal values
 198  from an AST. It also changes the type= keyword argument for `Constant`
 199  to be `types=` to avoid the name clash with the built-in `type`
 200  function (`Literal` also uses `types=`).
 201- Version 2.7.9 fixes minor bugs from 2.7.7 and 2.7.8 (e.g., formatting).
 202  It also adds `Reference` for matching variable references.
 203- Version 2.7.8 adds the `type=` argument to `Constant` for finding
 204  constants w/ unconstrained values but particular type(s).
 205- Version 2.7.7 adds the `Operator` class for code checking.
 206- Version 2.7.6 monkey-patches the `input` function in addition to
 207  overriding `stdin` during payload runs, to try to deal with notebook
 208  environments better where `input` doesn't read from `stdin` by
 209  default. This may cause more problems with other input-capturing
 210  solutions that also mock `input`...
 211- Version 2.7.5 allows the code value for a function test manager to be
 212  `None` when the code for the function cannot be found using
 213  `inspect.getsource`, instead of letting the associated `OSError`
 214  bubble out.
 215- Version 2.7.4 flips this changelog right-side-up (i.e., newest-first).
 216  Also introduces the `code` slot for `TestManager` objects, so that
 217  they store raw code in addition to a derived syntax tree. This change
 218  also means that `BlockManager` objects no longer store their code in
 219  the `target` slot, which is now just a fixed string. It also changes
 220  `listAllCases` to `listAllTrials` and changes 'cases' to 'trials' in a
 221  few other places since code checks are not associated with test cases
 222  but are trials. Common functionality is moved to the `Trial` class.
 223- Version 2.7.3 introduces the `mark` function, and removes
 224  `testCodeWithSuite`, adding `testMarkedCode` instead. This helps
 225  prevent suites (previously required for marking code blocks) from being
 226  started when you need another suite to be active.
 227- Version 2.7.2 introduces `listOutcomesInSuite` and registers outcomes
 228  from `expect` and `expectType` calls. These will also be included in
 229  `showSummary` results. It renames `startTestSuite` to just `testSuite`,
 230  and introduces `freshTestSuite` which deletes any old results instead
 231  of extending them.
 232- Version 2.7.1 sets the `SKIP_ON_FAILURE` back to `None`, since default
 233  skipping is a potential issue for automated test reporting. It adds
 234  `SUPPRESS_ON_FAILURE` to compensate for this and enables it by default,
 235  suppressing error details from checks after one failure per manager. It
 236  also adds `checkVariableValue` for checking the values of variables set
 237  in files or code blocks (it does not work on function tests).
 238- Version 2.7 introduces the `ASTRequirement` class and subclasses, along
 239  with the `TestManager.checkCodeContains` method for applying them. It
 240  reorganizes things a bit so that `Trial` is now a super-class of both
 241  `TestCase` and the new `CodeChecks` class.
 242- Version 2.6.7 changes default SKIP_ON_FAILURE back to 'case', since
 243  'all' makes interactive testing hard, and dedicated test files can call
 244  skipChecksAfterFail at the top. Also fixes an issue where comparing
 245  printed output correctly is lenient about the presence or absence of a
 246  final newline, but comparing file contents didn't do that. This change
 247  means that extra blank lines (including lines with whitespace on them)
 248  are ignored when comparing strings and IGNORE_TRAILING_WHITESPACE is
 249  on, and even when IGNORE_TRAILING_WHITESPACE is off, the presence or
 250  absence of a final newline in a file or in printed output will be
 251  copied over to a multi-line expectation (since otherwise there's no way
 252  to specify the lack of a final newline when giving multiple string
 253  arguments to `checkPrintedLines` or `checkFileLines`).
 254- Version 2.6.6 changes from splitlines to split('\\n') in a few places
 255  because the latter is more robust to extra carriage returns. This
 256  changes how some error messages look and it means that in some places
 257  the newline at the end of a file actually counts as having a blank line
 258  there in terms of output (behavior is mostly unchanged). Also escaped
 259  carriage returns in displayed strings so they're more visible. From
 260  this version we do NOT support files that use just '\\r' as a newline
 261  as easily. But `IGNORE_TRAILING_WHITESPACE` will properly get rid of
 262  any extra '\\r' before a newline, so when that's on (the default) this
 263  shouldn't change much. A test of this behavior was added to the file
 264  test example.
 265- Version 2.6.5 fixes a bug with displaying filenames when a file does
 266  not exist and `checkFileLines` is used, and also sets the default
 267  length higher for printing first differing lines in
 268  `findFirstDifference` since those lines are displayed on their own line
 269  anyways. Fixes a bug where list differences were not displayed
 270  correctly, and improves the usefulness of first differences displayed
 271  for dictionaries w/ different lengths. Also fixes a bug where strings
 272  which were equivalent modulo trailing whitespace would not be treated
 273  as equivalent.
 274- Version 2.6.4 immediately changes `checkFileContainsLines` to
 275  `checkFileLines` to avoid confusion about whether we're checking
 276  against the whole file (we are).
 277- Version 2.6.3 introduces the `checkFileContainsLines` method, and also
 278  standardizes the difference-finding code and merges it with
 279  equality-checking code, removing `checkEquality` and introducing
 280  `findFirstDifference` instead (`compare`) remains but just calls
 281  `findFirstDifference` internally. Also fixes a bug w/ printing
 282  tracebacks for come checkers (need to standardize that!). Also adds
 283  global skip-on-failure and sets that as the default!
 284- Version 2.6.2 fixes a bug with dictionary comparison which caused a
 285  crash when key sets weren't equal. It also adds unit tests for the
 286  `compare` function.
 287- Version 2.4.0 adds a more object-oriented structure behind the scenes,
 288  without changing any core API functions. It also adds support for
 289  variable specifications in block tests.
 290- Version 2.3.0 adds `testFunctionMaybe` for skippable tests if the
 291  target function hasn't been defined yet.
 292- Version 2.2.0 changed the names of `checkResult` and `checkOutputLines`
 293  to `checkReturnValue` and `checkPrintedLines`
 294- Version 2.0 introduced the `TestManager` and `TestCase` classes, and
 295  got rid of automatic tracking for test cases. The old test case
 296  functionality was moved over to the `expect` function. This helps make
 297  tests more stable and makes meta-reasoning easier.
 298"""
 299
 300# TODO: Cache compiled ASTs!
 301
 302__version__ = "2.7.10"
 303
 304import sys
 305import traceback
 306import inspect
 307import linecache
 308import ast
 309import copy
 310import io
 311import os
 312import re
 313import types
 314import builtins
 315import cmath
 316import textwrap
 317import warnings
 318
 319
 320# Flags for feature-detection on the ast module
 321HAS_WALRUS = hasattr(ast, "NamedExpr")
 322SPLIT_CONSTANTS = hasattr(ast, "Num")
 323
 324
 325#---------#
 326# Globals #
 327#---------#
 328
 329PRINT_TO = sys.stderr
 330"""
 331Where to print messages. Defaults to `sys.stderr` but you could set it to
 332`sys.stdout` (or another open file object) instead.
 333"""
 334
 335ALL_TRIALS = {}
 336"""
 337All test cases and code checks that have been created, organized by
 338test-suite names. By default all trials are added to the 'default' test
 339suite, but this can be changed using `testSuite`. Each entry has a test
 340suite name as the key and a list of `Trial` (i.e., `CodeChecks` and/or
 341`TestCase`) objects as the value.
 342"""
 343
 344ALL_OUTCOMES = {}
 345"""
 346The outcomes of all checks, including independent expectations (via
 347`expect` or `expectType`) and `Trial`-based expectations (via methods
 348like `TestManager.checkCodeContains`, `TestCase.checkReturnValue`, etc.).
 349
 350These are stored per test suite as lists with the suite name (see
 351`_CURRENT_SUITE_NAME`) as the key. They are ordered in the order that
 352checks happen in, but may be cleared if `resetTestSuite` is called.
 353
 354Each list entry is a 3-tuple with a boolean indicating success/failure, a
 355tag string indicating the file name and line number of the test, and a
 356string indicating the message that was displayed (which might have
 357depended on the current detail level or message suppression, etc.).
 358"""
 359
 360_CURRENT_SUITE_NAME = "default"
 361"""
 362The name of the current test suite, which organizes newly-created test
 363cases within the `ALL_TRIALS` variable. Use `testSuite` to begin/resume a
 364new test suite, and `currentTestSuite` to retrieve the value.
 365"""
 366
 367_MARKED_CODE_BLOCKS = {}
 368"""
 369A cache for notebook cells (or other code blocks) in which
 370`mark` has been called, for later retrieval in `testMarkedCode`.
 371"""
 372
 373COMPLETED_PER_LINE = {}
 374"""
 375A dictionary mapping function names to dictionaries mapping (filename,
 376line-number) pairs to counts. Each count represents the number of
 377functions of that name which have finished execution on the given line
 378of the given file already. This allows us to figure out which expression
 379belongs to which invocation if `get_my_context` is called multiple times
 380from the same line of code.
 381"""
 382
 383DETAIL_LEVEL = 0
 384"""
 385The current detail level, which controls how verbose our messages are.
 386See `detailLevel`.
 387"""
 388
 389SKIP_ON_FAILURE = None
 390"""
 391Controls which checks get skipped when a check fails. If set to `'all'`,
 392ALL checks will be skipped once one fails, until `clearFailure` is
 393called. If set to `'case'`, subsequent checks for the same test case will
 394be skipped when one fails. If set to `'manager'`, then all checks for any
 395case from a case manager will be skipped when any check for any case
 396derived from that manager fails. Any other value (including the default
 397`None`) will disable the skipping of checks based on failures.
 398"""
 399
 400SUPPRESS_ON_FAILURE = None
 401"""
 402Controls how failure messages are suppressed after a check fails. By
 403default, details from failures after the first failure for a given test
 404manager will printed as if the detail level were -1 as long as the
 405default level is 0. You can set this to `'case'` to only suppress details
 406on a per-case basis, or `'all'` to suppress all detail printing after any
 407failure. `clearFailure` can be used to reset the failure status, and
 408setting the base detail level above 1 will also undo the suppression.
 409
 410Set this to `None` or any other value that's not one of the strings
 411mentioned above to disable this functionality.
 412"""
 413
 414CHECK_FAILED = False
 415"""
 416Remembers whether we've failed a check yet or not. If True and
 417`SKIP_ON_FAILURE` is set to `'all'`, all checks will be skipped, or if
 418`SUPPRESS_ON_FAILURE` is `'all'` and the detail level is 0, failure
 419details will be suppressed. Use `clearFailure` to reset this and resume
 420checking without changing `SKIP_ON_FAILURE` if you need to.
 421"""
 422
 423COLORS = True
 424"""
 425Whether to print ANSI color control sequences to color the printed output
 426or not.
 427"""
 428
 429MSG_COLORS = {
 430    "succeeded": "34",  # blue
 431    "skipped": "33",  # yellow
 432    "failed": "1;31",  # bright red
 433    "reset": "0",  # resets color properties
 434}
 435
 436IGNORE_TRAILING_WHITESPACE = True
 437"""
 438Controls equality and inclusion tests on strings, including multiline
 439strings and strings within other data structures, causing them to ignore
 440trailing whitespace. True by default, since trailing whitespace is hard
 441to reason about because it's invisible.
 442
 443Trailing whitespace is any sequence whitespace characters before a
 444newline character (which is the only thing we count as a line break,
 445meaning \\r\\n breaks are only accepted if IGNORE_TRAILING_WHITESPACE is
 446on). Specifically, we use the `rstrip` method after splitting on \\n.
 447
 448Additionally, in multi-line scenarios, if there is a single extra line
 449containing just whitespace, that will be ignored, although that's not the
 450same as applying `rstrip` to the entire string, since if there are
 451multiple extra trailing newline characters, that still counts as a
 452difference.
 453"""
 454
 455_SHOW_OUTPUT = False
 456"""
 457Controls whether or not output printed during tests appears as normal or
 458is suppressed. Control this using the `showPrintedLines` function.
 459"""
 460
 461FLOAT_REL_TOLERANCE = 1e-8
 462"""
 463The relative tolerance for floating-point similarity (see
 464`cmath.isclose`).
 465"""
 466
 467FLOAT_ABS_TOLERANCE = 1e-8
 468"""
 469The absolute tolerance for floating-point similarity (see
 470`cmath.isclose`).
 471"""
 472
 473_RUNNING_TEST_CODE = False
 474"""
 475Will be set to `True` while testing code is running, allowing certain
 476functions to behave differently (usually to avoid infinite recursion).
 477"""
 478
 479
 480#--------#
 481# Errors #
 482#--------#
 483
 484class TestError(Exception):
 485    """
 486    An error with the testing mechanisms, as opposed to an error with
 487    the actual code being tested.
 488    """
 489    pass
 490
 491
 492#-------------#
 493# Trial Class #
 494#-------------#
 495
 496class Trial:
 497    """
 498    Base class for both code checks and test cases, delineating common
 499    functionality like having outcomes. All trials are derived from a
 500    manager.
 501    """
 502    def __init__(self, manager):
 503        """
 504        A manager must be specified, but that's it. This does extra
 505        things like registering the trial in the current test suite (see
 506        `testSuite`) and figuring out the location tag for the trial.
 507        """
 508        self.manager = manager
 509
 510        # Location and tag for trial creation
 511        self.location = get_my_location()
 512        self.tag = tag_for(self.location)
 513
 514        # List of outcomes of individual checks/tests based on this
 515        # trial. Each is a triple with a True/False indicator for
 516        # success/failure, a string tag for the expectation, and a full
 517        # result message.
 518        self.outcomes = []
 519
 520        # Whether or not a check has failed for this trial yet.
 521        self.any_failed = False
 522
 523        # How to describe this trial; should be overridden
 524        self.description = "an unknown trial"
 525
 526        # Register as a trial
 527        ALL_TRIALS.setdefault(_CURRENT_SUITE_NAME, []).append(self)
 528
 529    def trialDetails(self):
 530        """
 531        Returns a pair of strings containing base and extra details
 532        describing what was tested by this trial. If the base details
 533        capture all available information, the extra details value will
 534        be None.
 535
 536        This method is abstract and only sub-class implementations
 537        actually do anything.
 538        """
 539        raise NotImplementedError(
 540            "Cannot get trial details for a Trial or TestCase; you must"
 541            " create a specific kind of trial like a FunctionCase to be"
 542            " able to get trial details."
 543        )
 544
 545    def _create_success_message(
 546        self,
 547        tag,
 548        details,
 549        extra_details=None,
 550        include_test_details=True
 551    ):
 552        """
 553        Returns an expectation success message (a string) for an
 554        expectation with the given tag, using the given details and
 555        extra details. Unless `include_test_details` is set to False,
 556        details of the test expression/block will also be included (but
 557        only when the detail level is at least 1). The tag should be a
 558        filename:lineno string indicating where the expectation
 559        originated.
 560        """
 561        # Detail level 1 gives more output for successes
 562        if DETAIL_LEVEL < 1:
 563            result = f"✓ {tag}"
 564        else:  # Detail level is at least 1
 565            result = (
 566                f"✓ expectation from {tag} met for {self.description}"
 567            )
 568            detail_msg = indent(details, 2)
 569            if not detail_msg.startswith('\n'):
 570                detail_msg = '\n' + detail_msg
 571
 572            if DETAIL_LEVEL >= 2 and extra_details:
 573                extra_detail_msg = indent(extra_details, 2)
 574                if not extra_detail_msg.startswith('\n'):
 575                    extra_detail_msg = '\n' + extra_detail_msg
 576
 577                detail_msg += extra_detail_msg
 578
 579            # Test details unless suppressed
 580            if include_test_details:
 581                test_base, test_extra = self.trialDetails()
 582                detail_msg += '\n' + indent(test_base, 2)
 583                if DETAIL_LEVEL >= 2 and test_extra is not None:
 584                    detail_msg += '\n' + indent(test_extra, 2)
 585
 586            result += detail_msg
 587
 588        return result
 589
 590    def _create_failure_message(
 591        self,
 592        tag,
 593        details,
 594        extra_details=None,
 595        include_test_details=True
 596    ):
 597        """
 598        Creates a failure message string for an expectation with the
 599        given tag that includes the details and/or extra details
 600        depending on the current global detail level. Normally,
 601        information about the test that was run is included as well, but
 602        you can set `include_test_details` to False to prevent this.
 603        """
 604        # Detail level controls initial message
 605        if DETAIL_LEVEL < 1:
 606            result = f"✗ {tag}"
 607        else:
 608            result = (
 609                f"✗ expectation from {tag} NOT met for"
 610                f" {self.description}"
 611            )
 612
 613        # Assemble our details message
 614        detail_msg = ''
 615
 616        # Figure out if we should suppress details
 617        suppress = self._should_suppress()
 618
 619        # Detail level controls printing of detail messages
 620        if (DETAIL_LEVEL == 0 and not suppress) or DETAIL_LEVEL >= 1:
 621            detail_msg += '\n' + indent(details, 2)
 622        if DETAIL_LEVEL >= 1 and extra_details:
 623            detail_msg += '\n' + indent(extra_details, 2)
 624
 625        # Test details unless suppressed
 626        if include_test_details:
 627            test_base, test_extra = self.trialDetails()
 628            if (DETAIL_LEVEL == 0 and not suppress) or DETAIL_LEVEL >= 1:
 629                detail_msg += '\n' + indent(test_base, 2)
 630            if DETAIL_LEVEL >= 1 and test_extra is not None:
 631                detail_msg += '\n' + indent(test_extra, 2)
 632
 633        return result + detail_msg
 634
 635    def _print_skip_message(self, tag, reason):
 636        """
 637        Prints a standard message about the trial being skipped, using
 638        the given tag and a reason (shown only if detail level is 1+).
 639        """
 640        # Detail level controls initial message
 641        if DETAIL_LEVEL < 1:
 642            msg = f"~ {tag} (skipped)"
 643        else:
 644            msg = (
 645                f"~ expectation at {tag} for {self.description}"
 646                f" skipped ({reason})"
 647            )
 648        print_message(msg, color=msg_color("skipped"))
 649
 650    def _should_skip(self):
 651        """
 652        Returns True if this trial should be skipped based on a previous
 653        failure and the `SKIP_ON_FAILURE` mode.
 654        """
 655        return (
 656            (SKIP_ON_FAILURE == "all" and CHECK_FAILED)
 657         or (SKIP_ON_FAILURE == "case" and self.any_failed)
 658         or (SKIP_ON_FAILURE == "manager" and self.manager.any_failed)
 659        )
 660
 661    def _should_suppress(self):
 662        """
 663        Returns True if failure details for this trial should be
 664        suppressed based on a previous failure and the
 665        `SUPPRESS_ON_FAILURE` mode.
 666        """
 667        return (
 668            (SUPPRESS_ON_FAILURE == "all" and CHECK_FAILED)
 669         or (SUPPRESS_ON_FAILURE == "case" and self.any_failed)
 670         or (SUPPRESS_ON_FAILURE == "manager" and self.manager.any_failed)
 671        )
 672
 673    def _register_outcome(self, passed, tag, message):
 674        """
 675        Registers an outcome for this trial. `passed` should be either
 676        True or False indicating whether the check passed, `tag` is a
 677        string to label the outcome with, and `message` is the message
 678        displayed by the check. This appends an entry to `self.outcomes`
 679        with the passed boolean, the tag, and the message in a tuple, and
 680        it sets `self.any_failed` and `self.manager.any_failed` if the
 681        outcome is a failure.
 682        """
 683        global CHECK_FAILED
 684        self.outcomes.append((passed, tag, message))
 685        _register_outcome(passed, tag, message)
 686        if not passed:
 687            CHECK_FAILED = True
 688            self.any_failed = True
 689            self.manager.any_failed = True
 690
 691
 692#------------------#
 693# Code Check Class #
 694#------------------#
 695
 696class CodeChecks(Trial):
 697    """
 698    Represents one or more checks performed against code structure
 699    (without running that code) rather than against the behavior of code.
 700    Like a `TestCase`, it can have outcomes (one for each check
 701    performed) and is tracked globally.
 702    """
 703    def __init__(self, manager):
 704        """
 705        A manager must be specified, but that's it.
 706        """
 707        super().__init__(manager)
 708
 709        # How to describe this trial
 710        self.description = f"code checks for {self.manager.tag}"
 711
 712    def trialDetails(self):
 713        """
 714        The base details describe what kind of code was run; the full
 715        details include the AST dump.
 716        """
 717        baseDetails = self.manager.checkDetails()
 718
 719        # Get representation of the AST we checked:
 720        if self.manager.syntaxTree is not None:
 721            if sys.version_info < (3, 9):
 722                astRepr = ast.dump(self.manager.syntaxTree)
 723            else:
 724                astRepr = ast.dump(self.manager.syntaxTree, indent=2)
 725            return (
 726                baseDetails,
 727                "The code structure is:" + indent(astRepr, 2)
 728            )
 729        else:
 730            return (
 731                baseDetails,
 732                "No code was available for checking."
 733            )
 734
 735    def performCheck(self, checkFor):
 736        """
 737        Performs a check for the given `ASTRequirement` within the AST
 738        managed by this code check's manager. Prints a success/failure
 739        message, registers an outcome, and returns True on success and
 740        False on failure (including when there's a partial match).
 741        Returns `None` if the check is skipped (which can happen based on
 742        a previous failure depending on settings, or when the AST to
 743        check is not available.
 744        """
 745        tag = tag_for(get_my_location())
 746
 747        # Skip the check if there's nothing to test
 748        if self._should_skip() or self.manager.syntaxTree is None:
 749            self._print_skip_message(tag, "source code not available")
 750            return None
 751        else:
 752            # Perform the check
 753            matches = checkFor.allMatches(self.manager.syntaxTree)
 754            if not matches.isFull:
 755                passed = False
 756                if checkFor.maxMatches == 0:
 757                    contains = "contains a structure that it should not"
 758                elif checkFor.minMatches > 1:
 759                    contains = (
 760                        "does not contain enough of the expected"
 761                        " structures"
 762                    )
 763                else:
 764                    contains = "does not contain the expected structure"
 765            else:
 766                passed = True
 767                if checkFor.maxMatches == 0:
 768                    contains = (
 769                        "does not contain any structures it should not"
 770                    )
 771                elif checkFor.minMatches > 1:
 772                    contains = "contains enough expected structures"
 773                else:
 774                    contains = "contains the expected structure"
 775
 776            structureString = checkFor.fullStructure()
 777            base_msg = f"""\
 778Code {contains}:
 779{indent(structureString, 2)}"""
 780            if matches.isPartial:
 781                base_msg += f"""
 782Although it does partially satisfy the requirement:
 783{indent(str(matches), 2)}"""
 784
 785            # TODO: have partial/full structure strings?
 786            extra_msg = ""
 787
 788            if passed:
 789                msg = self._create_success_message(tag, base_msg, extra_msg)
 790                msg_cat = "succeeded"
 791            else:
 792                msg = self._create_failure_message(tag, base_msg, extra_msg)
 793                msg_cat = "failed"
 794
 795        # Print our message
 796        print_message(msg, color=msg_color(msg_cat))
 797
 798        # Record outcome
 799        self._register_outcome(passed, tag, msg)
 800        return passed
 801
 802
 803#-------------------#
 804# Test Case Classes #
 805#-------------------#
 806
 807class NoResult:
 808    """
 809    A special class used to indicate the absence of a result when None
 810    is a valid result value.
 811    """
 812    pass
 813
 814
 815def mimicInput(prompt):
 816    """
 817    A function which mimics the functionality of the default `input`
 818    function: it prints a prompt, reads input from stdin, and then
 819    returns that input. Unlike normal input, it prints what it reads
 820    from stdin to stdout, which in normal situations would result in
 821    that stuff showing up on the console twice, but when stdin is set to
 822    an alternate stream (as we do when capturing input/output) that
 823    doesn't happen.
 824    """
 825    print(prompt, end='')
 826    incomming = sys.stdin.readline()
 827    # Strip newline on incomming value
 828    incomming = incomming.rstrip('\n\r')
 829    print(incomming, end='\n')
 830    return incomming
 831
 832
 833class TestCase(Trial):
 834    """
 835    Represents a specific test to run, managing things like specific
 836    arguments, inputs or available variables that need to be in place.
 837    Derived from a `TestManager` using the `TestManager.case` method.
 838
 839    `TestCase` is abstract; subclasses should override a least the `run`
 840    and `trialDetails` functions.
 841    """
 842    def __init__(self, manager):
 843        """
 844        A manager must be specified, but that's it. This does extra
 845        things like registering the case in the current test suite (see
 846        `testSuite`) and figuring out the location tag for the case.
 847        """
 848        super().__init__(manager)
 849
 850        # How to describe this trial
 851        self.description = f"test case at {self.tag}"
 852
 853        # Inputs to provide on stdin
 854        self.inputs = None
 855
 856        # Results of running this case
 857        self.results = None
 858
 859        # Whether to echo captured printed outputs (overrides global)
 860        self.echo = None
 861
 862    def provideInputs(self, *inputLines):
 863        """
 864        Sets up fake inputs (each argument must be a string and is used
 865        for one line of input) for this test case. When information is
 866        read from stdin during the test, including via the `input`
 867        function, these values are the result. If you don't call
 868        `provideInputs`, then the test will pause and wait for real user
 869        input when `input` is called.
 870
 871        You must call this before the test actually runs (i.e., before
 872        `TestCase.run` or one of the `check` functions is called),
 873        otherwise you'll get an error.
 874        """
 875        if self.results is not None:
 876            raise TestError(
 877                "You cannot provide inputs because this test case has"
 878                " already been run."
 879            )
 880        self.inputs = inputLines
 881
 882    def showPrintedLines(self, show=True):
 883        """
 884        Overrides the global `showPrintedLines` setting for this test.
 885        Use None as the parameter to remove the override.
 886        """
 887        self.echo = show
 888
 889    def _run(self, payload):
 890        """
 891        Given a payload (a zero-argument function that returns a tuple
 892        with a result and a scope dictionary), runs the payload while
 893        managing things like output capturing and input mocking. Sets the
 894        `self.results` field to reflect the results of the run, which
 895        will be a dictionary that has the following slots:
 896
 897        - "result": The result value from a function call. This key
 898            will not be present for tests that don't have a result, like
 899            file or code block tests. To achieve this with a custom
 900            payload, have the payload return `NoResult` as the first part
 901            of the tuple it returns.
 902        - "output": The output printed during the test. Will be an empty
 903            string if nothing gets printed.
 904        - "error": An Exception object representing an error that
 905            occurred during the test, or None if no errors happened.
 906        - "traceback": If an exception occurred, this will be a string
 907            containing the traceback for that exception. Otherwise it
 908            will be None.
 909        - "scope": The second part of the tuple returned by the payload,
 910            which should be a dictionary representing the scope of the
 911            code run by the test. It may also be `None` in cases where no
 912            scope is available (e.g., function alls).
 913
 914        In addition to being added to the results slot, this dictionary
 915        is also returned.
 916        """
 917        # Set up the `input` function to echo what is typed, and to only
 918        # read from stdin (in case we're in a notebook where input would
 919        # do something else).
 920        original_input = builtins.input
 921        builtins.input = mimicInput
 922
 923        # Set up a capturing stream for output
 924        outputCapture = CapturingStream()
 925        outputCapture.install()
 926        if self.echo or (self.echo is None and _SHOW_OUTPUT):
 927            outputCapture.echo()
 928
 929        # Set up fake input contents, AND also monkey-patch the input
 930        # function since in some settings like notebooks input doesn't
 931        # just read from stdin
 932        if self.inputs is not None:
 933            fakeInput = io.StringIO('\n'.join(self.inputs))
 934            original_stdin = sys.stdin
 935            sys.stdin = fakeInput
 936
 937        # Set up default values before we run things
 938        error = None
 939        tb = None
 940        value = NoResult
 941        scope = None
 942
 943        # Actually run the test
 944        try:
 945            value, scope = payload()
 946        except Exception as e:
 947            # Catch any error that occurred
 948            error = e
 949            tb = traceback.format_exc()
 950        finally:
 951            # Release stream captures and reset the input function
 952            outputCapture.uninstall()
 953            builtins.input = original_input
 954            if self.inputs is not None:
 955                sys.stdin = original_stdin
 956
 957        # Grab captured output
 958        output = outputCapture.getvalue()
 959
 960        # Create self.results w/ output, error, and maybe result value
 961        self.results = {
 962            "output": output,
 963            "error": error,
 964            "traceback": tb,
 965            "scope": scope
 966        }
 967        if value is not NoResult:
 968            self.results["result"] = value
 969
 970        # Return new results object
 971        return self.results
 972
 973    def run(self):
 974        """
 975        Runs this test case, capturing printed output and supplying fake
 976        input if `TestCase.provideInputs` has been called. Stores the
 977        results in `self.results`. This will be called once
 978        automatically the first time an expectation method like
 979        `TestCase.checkReturnValue` is used, but the cached value will
 980        be re-used for subsequent expectations, unless you manually call
 981        this method again.
 982
 983        This method is overridden by specific test case types.
 984        """
 985        raise NotImplementedError(
 986            "Cannot run a TestCase; you must create a specific kind of"
 987            " test case like a FunctionCase to be able to run it."
 988        )
 989
 990    def fetchResults(self):
 991        """
 992        Fetches the results of the test, which will run the test if it
 993        hasn't already been run, but otherwise will just return the
 994        latest cached results.
 995
 996        `run` describes the format of the results.
 997        """
 998        if self.results is None:
 999            self.run()
1000        return self.results
1001
1002    def checkReturnValue(self, expectedValue):
1003        """
1004        Checks the result value for this test case, comparing it against
1005        the given expected value and printing a message about success or
1006        failure depending on whether they are considered different by
1007        the `findFirstDifference` function.
1008
1009        If this is the first check performed using this test case, the
1010        test case will run; otherwise a cached result will be used.
1011
1012        This method returns True if the expectation is met and False if
1013        it is not, in addition to printing a message indicating
1014        success/failure and recording that message along with the status
1015        and tag in `self.outcomes`. If the check is skipped, it returns
1016        None and does not add an entry to `self.outcomes`.
1017        """
1018        results = self.fetchResults()
1019
1020        # Figure out the tag for this expectation
1021        tag = tag_for(get_my_location())
1022
1023        # Skip this check if the case has failed already
1024        if self._should_skip():
1025            self._print_skip_message(tag, "prior test failed")
1026            # Note that we don't add an outcome here, and we return None
1027            # instead of True or False
1028            return None
1029
1030        # Figure out whether we've got an error or an actual result
1031        if results["error"] is not None:
1032            # An error during testing
1033            tb = results["traceback"]
1034            tblines = tb.splitlines()
1035            if len(tblines) < 12:
1036                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1037                extra_msg = None
1038            else:
1039                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1040                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1041                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1042
1043            msg = self._create_failure_message(
1044                tag,
1045                base_msg,
1046                extra_msg
1047            )
1048            print_message(msg, color=msg_color("failed"))
1049            self._register_outcome(False, tag, msg)
1050            return False
1051
1052        elif "result" not in results:
1053            # Likely impossible, since we verified the category above
1054            # and we're in a condition where no error was logged...
1055            msg = self._create_failure_message(
1056                tag,
1057                (
1058                    "This test case does not have a result value. (Did"
1059                    " you mean to use checkPrintedLines?)"
1060                )
1061            )
1062            print_message(msg, color=msg_color("failed"))
1063            self._register_outcome(False, tag, msg)
1064            return False
1065
1066        else:
1067            # We produced a result, so check equality
1068
1069            # Check equivalence
1070            passed = False
1071            firstDiff = findFirstDifference(results["result"], expectedValue)
1072            if firstDiff is None:
1073                equivalence = "equivalent to"
1074                passed = True
1075            else:
1076                equivalence = "NOT equivalent to"
1077
1078            # Get short/long versions of result/expected
1079            short_result = ellipsis(repr(results["result"]), 72)
1080            full_result = repr(results["result"])
1081            short_expected = ellipsis(repr(expectedValue), 72)
1082            full_expected = repr(expectedValue)
1083
1084            # Create base/extra messages
1085            if (
1086                short_result == full_result
1087            and short_expected == full_expected
1088            ):
1089                base_msg = (
1090                    f"Result:\n{indent(short_result, 2)}\nwas"
1091                    f" {equivalence} the expected value:\n"
1092                    f"{indent(short_expected, 2)}"
1093                )
1094                extra_msg = None
1095                if (
1096                    firstDiff is not None
1097                and differencesAreSubtle(short_result, short_expected)
1098                ):
1099                    base_msg += (
1100                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1101                    )
1102            else:
1103                base_msg = (
1104                    f"Result:\n{indent(short_result, 2)}\nwas"
1105                    f" {equivalence} the expected value:\n"
1106                    f"{indent(short_expected, 2)}"
1107                )
1108                extra_msg = ""
1109                if (
1110                    firstDiff is not None
1111                and differencesAreSubtle(short_result, short_expected)
1112                ):
1113                    base_msg += (
1114                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1115                    )
1116                if short_result != full_result:
1117                    extra_msg += (
1118                        f"Full result:\n{indent(full_result, 2)}\n"
1119                    )
1120                if short_expected != full_expected:
1121                    extra_msg += (
1122                        f"Full expected value:\n"
1123                        f"{indent(full_expected, 2)}\n"
1124                    )
1125
1126            if passed:
1127                msg = self._create_success_message(
1128                    tag,
1129                    base_msg,
1130                    extra_msg
1131                )
1132                print_message(msg, color=msg_color("succeeded"))
1133                self._register_outcome(True, tag, msg)
1134                return True
1135            else:
1136                msg = self._create_failure_message(
1137                    tag,
1138                    base_msg,
1139                    extra_msg
1140                )
1141                print_message(msg, color=msg_color("failed"))
1142                self._register_outcome(False, tag, msg)
1143                return False
1144
1145    def checkVariableValue(self, varName, expectedValue):
1146        """
1147        Checks the value of a variable established by this test case,
1148        which should be a code block or file test (use `checkReturnValue`
1149        instead for checking the result of a function test). It checks
1150        that a variable with a certain name (given as a string) has a
1151        certain expected value, and prints a message about success or
1152        failure depending on whether the actual value and expected value
1153        are considered different by the `findFirstDifference` function.
1154
1155        If this is the first check performed using this test case, the
1156        test case will run; otherwise a cached result will be used.
1157
1158        This method returns True if the expectation is met and False if
1159        it is not, in addition to printing a message indicating
1160        success/failure and recording that message along with the status
1161        and tag in `self.outcomes`. If the check is skipped, it returns
1162        None and does not add an entry to `self.outcomes`.
1163        """
1164        results = self.fetchResults()
1165
1166        # Figure out the tag for this expectation
1167        tag = tag_for(get_my_location())
1168
1169        # Skip this check if the case has failed already
1170        if self._should_skip():
1171            self._print_skip_message(tag, "prior test failed")
1172            # Note that we don't add an outcome here, and we return None
1173            # instead of True or False
1174            return None
1175
1176        # Figure out whether we've got an error or an actual result
1177        if results["error"] is not None:
1178            # An error during testing
1179            tb = results["traceback"]
1180            tblines = tb.splitlines()
1181            if len(tblines) < 12:
1182                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1183                extra_msg = None
1184            else:
1185                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1186                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1187                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1188
1189            msg = self._create_failure_message(
1190                tag,
1191                base_msg,
1192                extra_msg
1193            )
1194            print_message(msg, color=msg_color("failed"))
1195            self._register_outcome(False, tag, msg)
1196            return False
1197
1198        else:
1199            # No error, so look for our variable
1200            scope = results["scope"]
1201
1202            if varName not in scope:
1203                msg = self._create_failure_message(
1204                    tag,
1205                    f"No variable named '{varName}' was created.",
1206                    None
1207                )
1208                print_message(msg, color=msg_color("failed"))
1209                self._register_outcome(False, tag, msg)
1210                return False
1211
1212            # Check equivalence
1213            passed = False
1214            value = scope[varName]
1215            firstDiff = findFirstDifference(value, expectedValue)
1216            if firstDiff is None:
1217                equivalence = "equivalent to"
1218                passed = True
1219            else:
1220                equivalence = "NOT equivalent to"
1221
1222            # Get short/long versions of result/expected
1223            short_value = ellipsis(repr(value), 72)
1224            full_value = repr(value)
1225            short_expected = ellipsis(repr(expectedValue), 72)
1226            full_expected = repr(expectedValue)
1227
1228            # Create base/extra messages
1229            if (
1230                short_value == full_value
1231            and short_expected == full_expected
1232            ):
1233                base_msg = (
1234                    f"Variable '{varName}' with"
1235                    f" value:\n{indent(short_value, 2)}\nwas"
1236                    f" {equivalence} the expected value:\n"
1237                    f"{indent(short_expected, 2)}"
1238                )
1239                extra_msg = None
1240                if (
1241                    firstDiff is not None
1242                and differencesAreSubtle(short_value, short_expected)
1243                ):
1244                    base_msg += (
1245                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1246                    )
1247            else:
1248                base_msg = (
1249                    f"Variable '{varName}' with"
1250                    f" value:\n{indent(short_value, 2)}\nwas"
1251                    f" {equivalence} the expected value:\n"
1252                    f"{indent(short_expected, 2)}"
1253                )
1254                extra_msg = ""
1255                if (
1256                    firstDiff is not None
1257                and differencesAreSubtle(short_value, short_expected)
1258                ):
1259                    base_msg += (
1260                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1261                    )
1262                if short_value != full_value:
1263                    extra_msg += (
1264                        f"Full value:\n{indent(full_value, 2)}\n"
1265                    )
1266                if short_expected != full_expected:
1267                    extra_msg += (
1268                        f"Full expected value:\n"
1269                        f"{indent(full_expected, 2)}\n"
1270                    )
1271
1272            if passed:
1273                msg = self._create_success_message(
1274                    tag,
1275                    base_msg,
1276                    extra_msg
1277                )
1278                print_message(msg, color=msg_color("succeeded"))
1279                self._register_outcome(True, tag, msg)
1280                return True
1281            else:
1282                msg = self._create_failure_message(
1283                    tag,
1284                    base_msg,
1285                    extra_msg
1286                )
1287                print_message(msg, color=msg_color("failed"))
1288                self._register_outcome(False, tag, msg)
1289                return False
1290
1291    def checkPrintedLines(self, *expectedLines):
1292        """
1293        Checks that the exact printed output captured during the test
1294        matches a sequence of strings each specifying one line of the
1295        output. Note that the global `IGNORE_TRAILING_WHITESPACE`
1296        affects how this function treats line matches.
1297
1298        If this is the first check performed using this test case, the
1299        test case will run; otherwise a cached result will be used.
1300
1301        This method returns True if the check succeeds and False if it
1302        fails, in addition to printing a message indicating
1303        success/failure and recording that message along with the status
1304        and tag in `self.outcomes`. If the check is skipped, it returns
1305        None and does not add an entry to `self.outcomes`.
1306        """
1307        # Fetch captured output
1308        results = self.fetchResults()
1309        output = results["output"]
1310
1311        # Figure out the tag for this expectation
1312        tag = tag_for(get_my_location())
1313
1314        # Skip this check if the case has failed already
1315        if self._should_skip():
1316            self._print_skip_message(tag, "prior test failed")
1317            # Note that we don't add an outcome here, and we return None
1318            # instead of True or False
1319            return None
1320
1321        # Figure out whether we've got an error or an actual result
1322        if results["error"] is not None:
1323            # An error during testing
1324            tb = results["traceback"]
1325            tblines = tb.splitlines()
1326            if len(tblines) < 12:
1327                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1328                extra_msg = None
1329            else:
1330                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1331                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1332                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1333
1334            msg = self._create_failure_message(
1335                tag,
1336                base_msg,
1337                extra_msg
1338            )
1339            print_message(msg, color=msg_color("failed"))
1340            self._register_outcome(False, tag, msg)
1341            return False
1342
1343        else:
1344            # We produced printed output, so check it
1345
1346            # Get lines/single versions
1347            expected = '\n'.join(expectedLines) + '\n'
1348            # If the output doesn't end with a newline, don't add one to
1349            # our expectation either...
1350            if not output.endswith('\n'):
1351                expected = expected[:-1]
1352
1353            # Figure out equivalence category
1354            equivalence = None
1355            passed = False
1356            firstDiff = findFirstDifference(output, expected)
1357            if output == expected:
1358                equivalence = "exactly the same as"
1359                passed = True
1360            elif firstDiff is None:
1361                equivalence = "equivalent to"
1362                passed = True
1363            else:
1364                equivalence = "NOT the same as"
1365                # passed remains False
1366
1367            # Get short/long representations of our strings
1368            short, long = dual_string_repr(output)
1369            short_exp, long_exp = dual_string_repr(expected)
1370
1371            # Construct base and extra messages
1372            if short == long and short_exp == long_exp:
1373                base_msg = (
1374                    f"Printed lines:\n{indent(short, 2)}\nwere"
1375                    f" {equivalence} the expected printed"
1376                    f" lines:\n{indent(short_exp, 2)}"
1377                )
1378                if not passed:
1379                    base_msg += (
1380                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1381                    )
1382                extra_msg = None
1383            else:
1384                base_msg = (
1385                    f"Printed lines:\n{indent(short, 2)}\nwere"
1386                    f" {equivalence} the expected printed"
1387                    f" lines:\n{indent(short_exp, 2)}"
1388                )
1389                if not passed:
1390                    base_msg += (
1391                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1392                    )
1393                extra_msg = ""
1394                if short != long:
1395                    extra_msg += f"Full printed lines:\n{indent(long, 2)}\n"
1396                if short_exp != long_exp:
1397                    extra_msg += (
1398                        f"Full expected printed"
1399                        f" lines:\n{indent(long_exp, 2)}\n"
1400                    )
1401
1402            if passed:
1403                msg = self._create_success_message(
1404                    tag,
1405                    base_msg,
1406                    extra_msg
1407                )
1408                print_message(msg, color=msg_color("succeeded"))
1409                self._register_outcome(True, tag, msg)
1410                return True
1411            else:
1412                msg = self._create_failure_message(
1413                    tag,
1414                    base_msg,
1415                    extra_msg
1416                )
1417                print_message(msg, color="1;31" if COLORS else None)
1418                self._register_outcome(False, tag, msg)
1419                return False
1420
1421    def checkPrintedFragment(self, fragment, copies=1, allowExtra=False):
1422        """
1423        Works like checkPrintedLines, except instead of requiring that
1424        the printed output exactly match a set of lines, it requires that
1425        a certain fragment of text appears somewhere within the printed
1426        output (or perhaps that multiple non-overlapping copies appear,
1427        if the copies argument is set to a number higher than the
1428        default of 1).
1429
1430        If allowExtra is set to True, more than the specified number of
1431        copies will be ignored, but by default, extra copies are not
1432        allowed.
1433
1434        The fragment is matched against the entire output as a single
1435        string, so it may contain newlines and if it does these will
1436        only match newlines in the captured output. If
1437        `IGNORE_TRAILING_WHITESPACE` is active (it's on by default), the
1438        trailing whitespace in the output will be removed before
1439        matching, and trailing whitespace in the fragment will also be
1440        removed IF it has a newline after it (trailing whitespace at the
1441        end of the string with no final newline will be retained).
1442
1443        This function returns True if the check succeeds and False if it
1444        fails, and prints a message either way. If the check is skipped,
1445        it returns None and does not add an entry to `self.outcomes`.
1446        """
1447        # Fetch captured output
1448        results = self.fetchResults()
1449        output = results["output"]
1450
1451        # Figure out the tag for this expectation
1452        tag = tag_for(get_my_location())
1453
1454        # Skip this check if the case has failed already
1455        if self._should_skip():
1456            self._print_skip_message(tag, "prior test failed")
1457            # Note that we don't add an outcome here, and we return None
1458            # instead of True or False
1459            return None
1460
1461        # Figure out whether we've got an error or an actual result
1462        if results["error"] is not None:
1463            # An error during testing
1464            tb = results["traceback"]
1465            tblines = tb.splitlines()
1466            if len(tblines) < 12:
1467                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1468                extra_msg = None
1469            else:
1470                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1471                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1472                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1473
1474            msg = self._create_failure_message(
1475                tag,
1476                base_msg,
1477                extra_msg
1478            )
1479            print_message(msg, color=msg_color("failed"))
1480            self._register_outcome(False, tag, msg)
1481            return False
1482
1483        else:
1484            # We produced printed output, so check it
1485            if IGNORE_TRAILING_WHITESPACE:
1486                matches = re.findall(
1487                    re.escape(trimWhitespace(fragment, True)),
1488                    trimWhitespace(output)
1489                )
1490            else:
1491                matches = re.findall(re.escape(fragment), output)
1492            passed = False
1493            if copies == 1:
1494                copiesPhrase = ""
1495                exactly = ""
1496                atLeast = "at least "
1497            else:
1498                copiesPhrase = f"{copies} copies of "
1499                exactly = "exactly "
1500                atLeast = "at least "
1501
1502            fragShort, fragLong = dual_string_repr(fragment)
1503            outShort, outLong = dual_string_repr(output)
1504
1505            if len(matches) == copies:
1506                passed = True
1507                base_msg = (
1508                    f"Found {exactly}{copiesPhrase}the target"
1509                    f" fragment in the printed output."
1510                    f"\nFragment was:\n{indent(fragShort, 2)}"
1511                    f"\nOutput was:\n{indent(outShort, 2)}"
1512                )
1513            elif allowExtra and len(matches) > copies:
1514                passed = True
1515                base_msg = (
1516                    f"Found {atLeast}{copiesPhrase}the target"
1517                    f" fragment in the printed output (found"
1518                    f" {len(matches)})."
1519                    f"\nFragment was:\n{indent(fragShort, 2)}"
1520                    f"\nOutput was:\n{indent(outShort, 2)}"
1521                )
1522            else:
1523                passed = False
1524                base_msg = (
1525                    f"Did not find {copiesPhrase}the target fragment"
1526                    f" in the printed output (found {len(matches)})."
1527                    f"\nFragment was:\n{indent(fragShort, 2)}"
1528                    f"\nOutput was:\n{indent(outShort, 2)}"
1529                )
1530
1531            extra_msg = ""
1532            if fragLong != fragShort:
1533                extra_msg += f"Full fragment was:\n{indent(fragLong, 2)}"
1534
1535            if outLong != outShort:
1536                if not extra_msg.endswith('\n'):
1537                    extra_msg += '\n'
1538                extra_msg += f"Full output was:\n{indent(outLong, 2)}"
1539
1540            if passed:
1541                msg = self._create_success_message(
1542                    tag,
1543                    base_msg,
1544                    extra_msg
1545                )
1546                print_message(msg, color=msg_color("succeeded"))
1547                self._register_outcome(True, tag, msg)
1548                return True
1549            else:
1550                msg = self._create_failure_message(
1551                    tag,
1552                    base_msg,
1553                    extra_msg
1554                )
1555                print_message(msg, color="1;31" if COLORS else None)
1556                self._register_outcome(False, tag, msg)
1557                return False
1558
1559    def checkFileLines(self, filename, *lines):
1560        """
1561        Works like `checkPrintedLines`, but checks for lines in the
1562        specified file, rather than checking for printed lines.
1563        """
1564        # Figure out the tag for this expectation
1565        tag = tag_for(get_my_location())
1566
1567        # Skip this check if the case has failed already
1568        if self._should_skip():
1569            self._print_skip_message(tag, "prior test failed")
1570            # Note that we don't add an outcome here, and we return None
1571            # instead of True or False
1572            return None
1573
1574        # Fetch the results to actually run the test!
1575        expected = '\n'.join(lines) + '\n'
1576        results = self.fetchResults()
1577
1578        # Figure out whether we've got an error or an actual result
1579        if results["error"] is not None:
1580            # An error during testing
1581            tb = results["traceback"]
1582            tblines = tb.splitlines()
1583            if len(tblines) < 12:
1584                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1585                extra_msg = None
1586            else:
1587                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1588                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1589                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1590
1591            msg = self._create_failure_message(
1592                tag,
1593                base_msg,
1594                extra_msg
1595            )
1596            print_message(msg, color=msg_color("failed"))
1597            self._register_outcome(False, tag, msg)
1598            return False
1599
1600        else:
1601            # The test was able to run, so check the file contents
1602
1603            # Fetch file contents
1604            try:
1605                with open(filename, 'r', newline='') as fileInput:
1606                    fileContents = fileInput.read()
1607            except (OSError, FileNotFoundError, PermissionError):
1608                # We can't even read the file!
1609                msg = self._create_failure_message(
1610                    tag,
1611                    f"Expected file '{filename}' cannot be read.",
1612                    None
1613                )
1614                print_message(msg, color=msg_color("failed"))
1615                self._register_outcome(False, tag, msg)
1616                return False
1617
1618            # If the file doesn't end with a newline, don't add one to
1619            # our expectation either...
1620            if not fileContents.endswith('\n'):
1621                expected = expected[:-1]
1622
1623            # Get lines/single versions
1624            firstDiff = findFirstDifference(fileContents, expected)
1625            equivalence = None
1626            passed = False
1627            if fileContents == expected:
1628                equivalence = "exactly the same as"
1629                passed = True
1630            elif firstDiff is None:
1631                equivalence = "equivalent to"
1632                passed = True
1633            else:
1634                # Some other kind of difference
1635                equivalence = "NOT the same as"
1636                # passed remains False
1637
1638            # Get short/long representations of our strings
1639            short, long = dual_string_repr(fileContents)
1640            short_exp, long_exp = dual_string_repr(expected)
1641
1642            # Construct base and extra messages
1643            if short == long and short_exp == long_exp:
1644                base_msg = (
1645                    f"File contents:\n{indent(short, 2)}\nwere"
1646                    f" {equivalence} the expected file"
1647                    f" contents:\n{indent(short_exp, 2)}"
1648                )
1649                if not passed:
1650                    base_msg += (
1651                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1652                    )
1653                extra_msg = None
1654            else:
1655                base_msg = (
1656                    f"File contents:\n{indent(short, 2)}\nwere"
1657                    f" {equivalence} the expected file"
1658                    f" contents:\n{indent(short_exp, 2)}"
1659                )
1660                if not passed:
1661                    base_msg += (
1662                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1663                    )
1664                extra_msg = ""
1665                if short != long:
1666                    extra_msg += f"Full file contents:\n{indent(long, 2)}\n"
1667                if short_exp != long_exp:
1668                    extra_msg += (
1669                        f"Full expected file"
1670                        f" contents:\n{indent(long_exp, 2)}\n"
1671                    )
1672
1673            if passed:
1674                msg = self._create_success_message(
1675                    tag,
1676                    base_msg,
1677                    extra_msg
1678                )
1679                print_message(msg, color=msg_color("succeeded"))
1680                self._register_outcome(True, tag, msg)
1681                return True
1682            else:
1683                msg = self._create_failure_message(
1684                    tag,
1685                    base_msg,
1686                    extra_msg
1687                )
1688                print_message(msg, color="1;31" if COLORS else None)
1689                self._register_outcome(False, tag, msg)
1690                return False
1691
1692    def checkCustom(self, checker, *args, **kwargs):
1693        """
1694        Sets up a custom check using a testing function. The provided
1695        function will be given one argument, plus any additional
1696        arguments given to this function. The first and/or only argument
1697        to the checker function will be a dictionary with the following
1698        keys:
1699
1700        - "case": The test case object on which `checkCustom` was called.
1701            This could be used to do things like access arguments passed
1702            to the function being tested for a `FunctionCase` for
1703            example.
1704        - "output": Output printed by the test case, as a string.
1705        - "result": the result value (for function tests only, otherwise
1706            this key will not be present).
1707        - "error": the error that occurred (or None if no error
1708            occurred).
1709        - "traceback": the traceback (a string, or None if there was no
1710            error).
1711        - "scope": For file and code block cases, the variable dictionary
1712            created by the file/code block. `None` for function cases.
1713
1714        The testing function must return True to indicate success and
1715        False for failure. If it returns something other than True or
1716        False, it will be counted as a failure, that value will be shown
1717        as part of the test result if the `DETAIL_LEVEL` is 1 or higher,
1718        and this method will return False.
1719
1720        If this check is skipped (e.g., because of a previous failure),
1721        this method returns None and does not add an entry to
1722        `self.outcomes`; the custom checker function won't be called in
1723        that case.
1724        """
1725        results = self.fetchResults()
1726        # Add a 'case' entry
1727        checker_input = copy.copy(results)
1728        checker_input["case"] = self
1729
1730        # Figure out the tag for this expectation
1731        tag = tag_for(get_my_location())
1732
1733        # Skip this check if the case has failed already
1734        if self._should_skip():
1735            self._print_skip_message(tag, "prior test failed")
1736            # Note that we don't add an outcome here, and we return None
1737            # instead of True or False
1738            return None
1739
1740        # Only run the checker if we're not skipping the test
1741        test_result = checker(checker_input, *args, **kwargs)
1742
1743        if test_result is True:
1744            msg = self._create_success_message(tag, "Custom check passed.")
1745            print_message(msg, color=msg_color("succeeded"))
1746            self._register_outcome(True, tag, msg)
1747            return True
1748        elif test_result is False:
1749            msg = self._create_failure_message(tag, "Custom check failed")
1750            print_message(msg, color="1;31" if COLORS else None)
1751            self._register_outcome(False, tag, msg)
1752            return False
1753        else:
1754            msg = self._create_failure_message(
1755                tag,
1756                "Custom check failed:\n" + indent(str(test_result), 2),
1757            )
1758            print_message(msg, color="1;31" if COLORS else None)
1759            self._register_outcome(False, tag, msg)
1760            return False
1761
1762
1763class FileCase(TestCase):
1764    """
1765    Runs a particular file when executed. Its manager should be a
1766    `FileManager`.
1767    """
1768    # __init__ is inherited
1769
1770    def run(self):
1771        """
1772        Runs the code in the target file in an empty environment (except
1773        that `__name__` is set to `'__main__'`, to make the file behave
1774        as if it were run as the main file).
1775
1776        Note that the code is read and parsed when the `FileManager` is
1777        created, not when the test case is run.
1778        """
1779        def payload():
1780            "Payload function to run a file."
1781            global _RUNNING_TEST_CODE
1782
1783            # Fetch syntax tree from our manager
1784            node = self.manager.syntaxTree
1785
1786            if node is None:
1787                raise RuntimeError(
1788                    "Manager of a FileCase was missing a syntax tree!"
1789                )
1790
1791            # Compile the syntax tree
1792            code = compile(node, self.manager.target, 'exec')
1793
1794            # Run the code, setting __name__ to __main__ (this is
1795            # why we don't just import the file)
1796            env = {"__name__": "__main__"}
1797            try:
1798                _RUNNING_TEST_CODE = True
1799                exec(code, env)
1800            finally:
1801                _RUNNING_TEST_CODE = False
1802
1803            # Running a file doesn't have a result value, but it does
1804            # provide a module scope.
1805            return (NoResult, deepish_copy(env))
1806
1807        return self._run(payload)
1808
1809    def trialDetails(self):
1810        """
1811        Returns a pair of strings containing base and extra details
1812        describing what was tested by this test case. If the base
1813        details capture all available information, the extra details
1814        value will be None.
1815        """
1816        return (
1817            f"Ran file '{self.manager.target}'",
1818            None  # no further details to report
1819        )
1820
1821
1822class FunctionCase(TestCase):
1823    """
1824    Calls a particular function with specific arguments when run.
1825    """
1826    def __init__(self, manager, args=None, kwargs=None):
1827        """
1828        The arguments and/or keyword arguments to be used for the case
1829        are provided after the manager (as a list and a dictionary, NOT
1830        as actual arguments). If omitted, the function will be called
1831        with no arguments.
1832        """
1833        super().__init__(manager)
1834        self.args = args or ()
1835        self.kwargs = kwargs or {}
1836
1837    def run(self):
1838        """
1839        Runs the target function with the arguments specified for this
1840        case. The 'result' slot of the `self.results` dictionary that it
1841        creates holds the return value of the function.
1842        """
1843        def payload():
1844            "Payload for running a function with specific arguments."
1845            global _RUNNING_TEST_CODE
1846            try:
1847                _RUNNING_TEST_CODE = True
1848                result = (
1849                    self.manager.target(*self.args, **self.kwargs),
1850                    None  # no scope for a function TODO: Get locals?
1851                )
1852            finally:
1853                _RUNNING_TEST_CODE = False
1854            return result
1855
1856        return self._run(payload)
1857
1858    def trialDetails(self):
1859        """
1860        Returns a pair of strings containing base and extra details
1861        describing what was tested by this test case. If the base
1862        details capture all available information, the extra details
1863        value will be None.
1864        """
1865        # Show function name + args, possibly with some abbreviation
1866        fn = self.manager.target
1867        msg = f"Called function '{fn.__name__}'"
1868
1869        args = self.args if self.args is not None else []
1870        kwargs = self.kwargs if self.args is not None else {}
1871        all_args = len(args) + len(kwargs)
1872
1873        argnames = fn.__code__.co_varnames[:all_args]
1874        if len(args) > len(argnames):
1875            msg += " with too many arguments (!):"
1876        elif all_args > 0:
1877            msg += " with arguments:"
1878
1879        # TODO: Proper handling of *args and **kwargs entries!
1880
1881        # Create lists of full and maybe-abbreviated argument
1882        # strings
1883        argstrings = []
1884        short_argstrings = []
1885        for i, arg in enumerate(args):
1886            if i < len(argnames):
1887                name = argnames[i]
1888            else:
1889                name = f"extra argument #{i - len(argnames) + 1}"
1890            short_name = ellipsis(name, 20)
1891
1892            argstrings.append(f"{name} = {repr(arg)}")
1893            short_argstrings.append(
1894                f"{short_name} = {ellipsis(repr(arg), 60)}"
1895            )
1896
1897        # Order kwargs by original kwargs order and then by natural
1898        # order of kwargs dictionary
1899        keyset = set(kwargs)
1900        ordered = list(filter(lambda x: x in keyset, argnames))
1901        rest = [k for k in kwargs if k not in ordered]
1902        for k in ordered + rest:
1903            argstrings.append(f"{k} = {repr(kwargs[k])}")
1904            short_name = ellipsis(k, 20)
1905            short_argstrings.append(
1906                f"{short_name} = {ellipsis(repr(kwargs[k]), 60)}"
1907            )
1908
1909        full_args = '  ' + '\n  '.join(argstrings)
1910        # In case there are too many arguments
1911        if len(short_argstrings) < 20:
1912            short_args = '  ' + '\n  '.join(short_argstrings)
1913        else:
1914            short_args = (
1915                '  '
1916              + '\n  '.join(short_argstrings[:19])
1917              + f"...plus {len(argstrings) - 19} more arguments..."
1918            )
1919
1920        if short_args == full_args:
1921            return (
1922                msg + '\n' + short_args,
1923                None
1924            )
1925        else:
1926            return (
1927                msg + '\n' + short_args,
1928                "Full arguments were:\n" + full_args
1929            )
1930
1931
1932class BlockCase(TestCase):
1933    """
1934    Executes a block of code (provided as text) when run. Per-case
1935    variables may be defined for the execution environment, which
1936    otherwise just has builtins.
1937    """
1938    def __init__(self, manager, assignments=None):
1939        """
1940        A dictionary of variable name : value assignments may be
1941        provided and these will be inserted into the execution
1942        environment for the code block. If omitted, no extra variables
1943        will be defined (this means that global variables available when
1944        the test manager and/or code block is set up are NOT available to
1945        the code in the code block by default).
1946        """
1947        super().__init__(manager)
1948        self.assignments = assignments or {}
1949
1950    def run(self):
1951        """
1952        Compiles and runs the target code block in an environment which
1953        is empty except for the assignments specified in this case (and
1954        builtins).
1955        """
1956        def payload():
1957            "Payload for running a code block specific variables active."
1958            global _RUNNING_TEST_CODE
1959            env = dict(self.assignments)
1960            try:
1961                _RUNNING_TEST_CODE = True
1962                exec(self.manager.code, env)
1963            finally:
1964                _RUNNING_TEST_CODE = False
1965            return (NoResult, deepish_copy(env))
1966
1967        return self._run(payload)
1968
1969    def trialDetails(self):
1970        """
1971        Returns a pair of strings containing base and extra details
1972        describing what was tested by this test case. If the base
1973        details capture all available information, the extra details
1974        value will be None.
1975        """
1976        block = self.manager.code
1977        short = limited_repr(block)
1978        if block == short:
1979            # Short enough to show whole block
1980            return (
1981                "Ran code:\n" + indent(block, 2),
1982                None
1983            )
1984
1985        else:
1986            # Too long to show whole block in short view...
1987            return (
1988                "Ran code:\n" + indent(short, 2),
1989                "Full code was:\n" + indent(block, 2)
1990            )
1991
1992
1993class SkipCase(TestCase):
1994    """
1995    A type of test case which actually doesn't run checks, but instead
1996    prints a message that the check was skipped.
1997    """
1998    # __init__ is inherited
1999
2000    def run(self):
2001        """
2002        Since there is no real test, our results are fake. The keys
2003        "error" and "traceback" have None as their value, and "output"
2004        also has None. We add a key "skipped" with value True.
2005        """
2006        self.results = {
2007            "output": None,
2008            "error": None,
2009            "traceback": None,
2010            "skipped": True
2011        }
2012        return self.results
2013
2014    def trialDetails(self):
2015        """
2016        Provides a pair of topic/details strings about this test.
2017        """
2018        return (f"Skipped check of '{self.manager.target}'", None)
2019
2020    def checkReturnValue(self, _, **__):
2021        """
2022        Skips the check.
2023        """
2024        self._print_skip_message(
2025            tag_for(get_my_location()),
2026            "testing target not available"
2027        )
2028
2029    def checkVariableValue(self, *_, **__):
2030        """
2031        Skips the check.
2032        """
2033        self._print_skip_message(
2034            tag_for(get_my_location()),
2035            "testing target not available"
2036        )
2037
2038    def checkPrintedLines(self, *_, **__):
2039        """
2040        Skips the check.
2041        """
2042        self._print_skip_message(
2043            tag_for(get_my_location()),
2044            "testing target not available"
2045        )
2046
2047    def checkPrintedFragment(self, *_, **__):
2048        """
2049        Skips the check.
2050        """
2051        self._print_skip_message(
2052            tag_for(get_my_location()),
2053            "testing target not available"
2054        )
2055
2056    def checkFileLines(self, *_, **__):
2057        """
2058        Skips the check.
2059        """
2060        self._print_skip_message(
2061            tag_for(get_my_location()),
2062            "testing target not available"
2063        )
2064
2065    def checkCustom(self, _, **__):
2066        """
2067        Skips the check.
2068        """
2069        self._print_skip_message(
2070            tag_for(get_my_location()),
2071            "testing target not available"
2072        )
2073
2074
2075class SilentCase(TestCase):
2076    """
2077    A type of test case which actually doesn't run checks, and also
2078    prints nothing. Just exists so that errors won't be thrown when
2079    checks are attempted. Testing methods return `None` instead of `True`
2080    or `False`, although this is not counted as a test failure.
2081    """
2082    # __init__ is inherited
2083
2084    def run(self):
2085        "Returns fake empty results."
2086        self.results = {
2087            "output": None,
2088            "error": None,
2089            "traceback": None,
2090            "skipped": True
2091        }
2092        return self.results
2093
2094    def trialDetails(self):
2095        """
2096        Provides a pair of topic/details strings about this test.
2097        """
2098        return ("Silently skipped check", None)
2099
2100    def checkReturnValue(self, _, **__):
2101        "Returns `None`."
2102        return None
2103
2104    def checkVariableValue(self, *_, **__):
2105        "Returns `None`."
2106        return None
2107
2108    def checkPrintedLines(self, *_, **__):
2109        "Returns `None`."
2110        return None
2111
2112    def checkPrintedFragment(self, *_, **__):
2113        "Returns `None`."
2114        return None
2115
2116    def checkFileLines(self, *_, **__):
2117        "Returns `None`."
2118        return None
2119
2120    def checkCustom(self, _, **__):
2121        "Returns `None`."
2122        return None
2123
2124
2125#----------------------#
2126# Test Manager Classes #
2127#----------------------#
2128
2129class TestManager:
2130    """
2131    Abstract base class for managing tests for a certain function, file,
2132    or block of code. Create these using the `testFunction`, `testFile`,
2133    and/or `testBlock` factory functions. The `TestManager.case`
2134    function can be used to derive `TestCase` objects which can then be
2135    used to set up checks.
2136
2137    It can also be used to directly check structural properties of the
2138    function, file, or block it manages tests for TODO
2139    """
2140    case_type = TestCase
2141    """
2142    The case type determines what kind of test case will be constructed
2143    when calling the `TestManager.case` method. Subclasses override
2144    this.
2145    """
2146
2147    def __init__(self, target, code):
2148        """
2149        A testing target (a filename string, function object, code
2150        string, or test label string) must be provided. The relevant
2151        code text must also be provided, although this can be set to
2152        None in cases where it isn't available.
2153        """
2154        self.target = target
2155
2156        self.code = code
2157
2158        if code is not None:
2159            self.syntaxTree = ast.parse(code, filename=self.codeFilename())
2160        else:
2161            self.syntaxTree = None
2162
2163        # Keeps track of whether any cases derived from this manager have
2164        # failed so far
2165        self.any_failed = False
2166
2167        self.code_checks = None
2168
2169        self.tag = tag_for(get_my_location())
2170
2171    def codeFilename(self):
2172        """
2173        Returns the filename to be used when parsing the code for this
2174        test case.
2175        """
2176        return f"code specified at {self.tag}"
2177
2178    def checkDetails(self):
2179        """
2180        Returns base details string describing what code was checked for
2181        a `checkCodeContains` check.
2182        """
2183        return "checked unknown code"
2184
2185    def case(self):
2186        """
2187        Returns a `TestCase` object that will test the target
2188        file/function/block. Some manager types allow arguments to this
2189        function.
2190        """
2191        return self.case_type(self)
2192
2193    def checkCodeContains(self, checkFor):
2194        """
2195        Given an `ASTRequirement` object, ensures that some part of the
2196        code that this manager would run during a test case contains the
2197        structure specified by that check object. Immediately performs
2198        the check and prints a pass/fail message. The check's result will
2199        be added to the `CodeChecks` outcomes for this manager; a new
2200        `CodeChecks` trial will be created and registered if one hasn't
2201        been already.
2202
2203        Returns `True` if the check succeeds and `False` if it fails
2204        (including cases where there's a partial match). Returns `None`
2205        if the check was skipped.
2206        """
2207        # Create a code checks trial if we haven't already
2208        if self.code_checks is None:
2209            self.code_checks = CodeChecks(self)
2210        trial = self.code_checks
2211
2212        return trial.performCheck(checkFor)
2213
2214    def validateTrace(self):
2215        """
2216        Not implemented yet.
2217        """
2218        raise NotImplementedError(
2219            "validateTrace is a planned feature, but has not been"
2220            " implemented yet."
2221        )
2222
2223
2224class FileManager(TestManager):
2225    """
2226    Manages test cases for running an entire file. Unlike other
2227    managers, cases for a file cannot have parameters. Calling
2228    `TestCase.provideInputs` on a case to provide inputs still means
2229    that having multiple cases can be useful, however.
2230    """
2231    case_type = FileCase
2232
2233    def __init__(self, filename):
2234        """
2235        A FileManager needs a filename string that specifies which file
2236        we'll run when we run a test case.
2237        """
2238        if not isinstance(filename, str):
2239            raise TypeError(
2240                f"For a file test manager, the target must be a file"
2241                f" name string. (You provided a/an {type(filename)}.)"
2242            )
2243
2244        with open(filename, 'r') as inputFile:
2245            code = inputFile.read()
2246
2247        super().__init__(filename, code)
2248
2249    def codeFilename(self):
2250        return self.target
2251
2252    def checkDetails(self):
2253        return f"checked code in file '{self.target}'"
2254
2255    # case is inherited as-is
2256
2257
2258class FunctionManager(TestManager):
2259    """
2260    Manages test cases for running a specific function. Arguments to the
2261    `TestManager.case` function are passed to the function being tested
2262    for that case.
2263    """
2264    case_type = FunctionCase
2265
2266    def __init__(self, function):
2267        """
2268        A FunctionManager needs a function object as the target. Each
2269        case will call that function with arguments provided when the
2270        case is created.
2271        """
2272        if not isinstance(function, types.FunctionType):
2273            raise TypeError(
2274                f"For a function test manager, the target must be a"
2275                f" function. (You provided a/an {type(function)}.)"
2276            )
2277
2278        # We need to track down the source code for this function;
2279        # luckily the inspect module makes that easy :)
2280        try:
2281            sourceCode = inspect.getsource(function)
2282        except OSError:
2283            # In some cases code might not be available, for example
2284            # when testing a function that was defined using exec.
2285            sourceCode = None
2286
2287        super().__init__(function, sourceCode)
2288
2289    def codeFilename(self):
2290        return f"function {self.target.__name__}"
2291
2292    def checkDetails(self):
2293        return f"checked code of function '{self.target.__name__}'"
2294
2295    def case(self, *args, **kwargs):
2296        """
2297        Arguments supplied here are used when calling the function which
2298        is what happens when the case is run. Returns a `FunctionCase`
2299        object.
2300        """
2301        return self.case_type(self, args, kwargs)
2302
2303
2304class BlockManager(TestManager):
2305    """
2306    Manages test cases for running a block of code (from a string).
2307    Keyword arguments to the `TestManager.case` function are defined as
2308    variables before the block is executed in that case.
2309    """
2310    case_type = BlockCase
2311
2312    def __init__(self, code, includeGlobals=False):
2313        """
2314        A BlockManager needs a code string as the target (the actual
2315        target value will be set to `None`). Optionally, the
2316        `use_globals` argument (default `False`) can be set to `True` to
2317        make globals defined at case-creation time accessible to the
2318        code in the case.
2319        """
2320        if not isinstance(code, str):
2321            raise TypeError(
2322                f"For a 'block' test manager, the target must be a"
2323                f" string. (You provided a/an {type(code)}.)"
2324            )
2325
2326        # TODO: This check is good, but avoiding multiple parsing passes
2327        # might be nice for larger code blocks...
2328        try:
2329            ast.parse(code)
2330        except Exception:
2331            raise ValueError(
2332                "The code block you provided could not be parsed as Python"
2333                " code."
2334            )
2335
2336        self.includeGlobals = bool(includeGlobals)
2337
2338        super().__init__("a code block", code)
2339        # Now that we have a tag, update our target
2340        self.target = f"code block from {self.tag}"
2341
2342    def codeFilename(self):
2343        return self.target
2344
2345    def checkDetails(self):
2346        return f"checked code from block at {self.tag}"
2347
2348    def case(self, **assignments):
2349        """
2350        Keyword argument supplied here will be defined as variables
2351        in the environment used to run the code block, and will override
2352        any global variable values (which are only included if
2353        `includeGlobals` was set to true when the manager was created).
2354        Returns a `BlockCase` object.
2355        """
2356        if self.includeGlobals:
2357            provide = copy.copy(get_external_calling_frame().f_globals)
2358            provide.update(assignments)
2359        else:
2360            provide = assignments
2361
2362        return self.case_type(self, provide)
2363
2364
2365class SkipManager(TestManager):
2366    """
2367    Manages fake test cases for a file, function, or code block that
2368    needs to be skipped (perhaps for a function that doesn't yet exist,
2369    for example). Cases derived are `SkipCase` objects which just print
2370    skip messages for any checks requested.
2371    """
2372    case_type = SkipCase
2373
2374    def __init__(self, label):
2375        """
2376        Needs a label string to identify which tests are being skipped.
2377        """
2378        if not isinstance(label, str):
2379            raise TypeError(
2380                f"For a skip test manager, the target must be a string."
2381                f" (You provided a/an {type(label)}.)"
2382            )
2383        super().__init__(label, None)
2384
2385    def codeFilename(self):
2386        return "no code (cases skipped)"
2387
2388    def checkDetails(self):
2389        return "skipped check (no code available)"
2390
2391    def case(self, *_, **__):
2392        """
2393        Accepts (and ignores) any extra arguments.
2394        """
2395        return super().case()
2396
2397    def checkCodeContains(self, checkFor):
2398        """
2399        Skips checking the AST of the target; see
2400        `TestManager.checkCodeContains`.
2401        """
2402        tag = tag_for(get_my_location())
2403        # Detail level controls initial message
2404        if DETAIL_LEVEL < 1:
2405            msg = f"~ {tag} (skipped)"
2406        else:
2407            msg = (
2408                f"~ code check at {tag} skipped"
2409            )
2410        print_message(msg, color=msg_color("skipped"))
2411
2412
2413class QuietSkipManager(TestManager):
2414    """
2415    Manages fake test cases that should be skipped silently, without any
2416    notifications. Cases derived are `SilentCase` objects which don't
2417    print anything.
2418    """
2419    case_type = SilentCase
2420
2421    def __init__(self):
2422        """
2423        No arguments needed.
2424        """
2425        super().__init__("ignored", None)
2426
2427    def codeFilename(self):
2428        return "no code (cases skipped)"
2429
2430    def checkDetails(self):
2431        return "skipped check (no code available)"
2432
2433    def case(self, *_, **__):
2434        """
2435        Accepts (and ignores) any extra arguments.
2436        """
2437        return super().case()
2438
2439    def checkCodeContains(self, checkFor):
2440        """
2441        Skips checking the AST; returns `None`.
2442        """
2443        return None
2444
2445
2446#----------------#
2447# Test factories #
2448#----------------#
2449
2450def testFunction(fn):
2451    """
2452    Creates a test-manager for the given function.
2453    """
2454    if not isinstance(fn, types.FunctionType):
2455        raise TypeError(
2456            "Test target must be a function (use testFile or testBlock"
2457            " instead to test a file or block of code)."
2458        )
2459
2460    return FunctionManager(fn)
2461
2462
2463def testFunctionMaybe(module, fname):
2464    """
2465    This function creates a test-manager for a named function from a
2466    specific module, but displays an alternate message and returns a
2467    dummy manager if that module doesn't define any variable with the
2468    target name. Useful for defining tests for functions that will be
2469    skipped if the functions aren't done yet.
2470    """
2471    # Message if we can't find the function
2472    if not hasattr(module, fname):
2473        print_message(
2474            f"Did not find '{fname}' in module '{module.__name__}'...",
2475            color=msg_color("skipped")
2476        )
2477        return SkipManager(f"{module.__name__}.{fname}")
2478    else:
2479        target = getattr(module, fname)
2480        if not isinstance(target, types.FunctionType):
2481            print_message(
2482                (
2483                    f"'{fname}' in module '{module.__name__}' is not a"
2484                    f" function..."
2485                ),
2486                color=msg_color("skipped")
2487            )
2488            return SkipManager(f"{module.__name__}.{fname}")
2489        else:
2490            return FunctionManager(target)
2491
2492
2493def testFile(filename):
2494    """
2495    Creates a test-manager for running the named file.
2496    """
2497    if not isinstance(filename, str):
2498        raise TypeError(
2499            "Test target must be a file name (use testFunction instead"
2500            " to test a function)."
2501        )
2502
2503    if not os.path.exists(filename):
2504        raise FileNotFoundError(
2505            f"We cannot create a test for running '{filename}' because"
2506            f" that file does not exist."
2507        )
2508
2509    return FileManager(filename)
2510
2511
2512def testBlock(code, includeGlobals=False):
2513    """
2514    Creates a test-manager for running a block of code (provided as a
2515    string). If `includeGlobals` is set to true, global variables which
2516    are defined at the time a case is created from the manager will be
2517    available to the code in that test case; if not (the default) no
2518    variables defined outside of the test block are available to the code
2519    in the block, except for explicit definitions supplied when creating
2520    a test case (see `BlockManager.case`).
2521    """
2522    if not isinstance(code, str):
2523        raise TypeError(
2524            "Test target must be a code string (use testFunction instead"
2525            " to test a function)."
2526        )
2527
2528    return BlockManager(code, includeGlobals)
2529
2530
2531SKIP_NOTEBOOK_CELL_CHECKS = False
2532"""
2533If set to true, notebook cell checks will be skipped silently. This is
2534used to avoid recursive checking problems.
2535"""
2536
2537_SKIP_NOTEBOOK_CELL_CHECKS = True
2538"""
2539The value for `SKIP_NOTEBOOK_CELL_CHECKS` to restore to when
2540`endSkippingNotebookCellChecks` is called.
2541"""
2542
2543
2544def beginSkippingNotebookCellChecks():
2545    """
2546    Sets `SKIP_NOTEBOOK_CELL_CHECKS` to True, and saves the old value in
2547    `_SKIP_NOTEBOOK_CELL_CHECKS`.
2548    """
2549    global SKIP_NOTEBOOK_CELL_CHECKS, _SKIP_NOTEBOOK_CELL_CHECKS
2550    _SKIP_NOTEBOOK_CELL_CHECKS = SKIP_NOTEBOOK_CELL_CHECKS
2551    SKIP_NOTEBOOK_CELL_CHECKS = True
2552
2553
2554def endSkippingNotebookCellChecks():
2555    """
2556    Sets `SKIP_NOTEBOOK_CELL_CHECKS` back to whatever value was stored
2557    when `beginSkippingNotebookCellChecks` was called (might not actually
2558    end skipping, because of that).
2559    """
2560    global SKIP_NOTEBOOK_CELL_CHECKS, _SKIP_NOTEBOOK_CELL_CHECKS
2561    SKIP_NOTEBOOK_CELL_CHECKS = _SKIP_NOTEBOOK_CELL_CHECKS
2562
2563
2564def testThisNotebookCell(includeGlobals=True):
2565    """
2566    Creates a test manager for running code in an IPython (and by
2567    implication also Jupyter) notebook cell (without any
2568    other cells being run). The current cell that is executing when the
2569    function is called is captured as a string and a `BlockManager` is
2570    created for that string, with `includeGlobals` set to `True` (you
2571    can override that by providing `False` as an argument to this
2572    function).
2573
2574    This function will raise an error if it is called outside of an
2575    IPython context, although this will not happen if
2576    `SKIP_NOTEBOOK_CELL_CHECKS` is set (see below).
2577
2578    If the `SKIP_NOTEBOOK_CELL_CHECKS` global variable is `True`, the
2579    result will be a special silent `QuietSkipManager` instead of a
2580    `BlockManager`. The code block captured from the notebook cell is
2581    augmented to set that variable to True at the beginning and back to
2582    its original value at the end, to avoid infinite recursion.
2583    """
2584    if SKIP_NOTEBOOK_CELL_CHECKS:
2585        return QuietSkipManager()
2586
2587    try:
2588        hist = get_ipython().history_manager  # noqa F821
2589    except Exception:
2590        raise RuntimeError(
2591            "Failed to get IPython context; testThisNotebookCell will"
2592            " only work when run from within a notebook."
2593        )
2594
2595    sessionID = hist.get_last_session_id()
2596    thisCellCode = next(hist.get_range(sessionID, start=-1, stop=None))[2]
2597    return BlockManager(
2598        (
2599            "import optimism\n"
2600          + "optimism.beginSkippingNotebookCellChecks()\n"
2601          + "try:\n"
2602          + indent(thisCellCode, 4)
2603          + "\nfinally:\n"
2604          + "    optimism.endSkippingNotebookCellChecks()\n"
2605        ),
2606        includeGlobals
2607    )
2608
2609
2610def mark(name):
2611    """
2612    Collects the code of the file or notebook cell within which the
2613    function call occurs, and caches it for later testing using
2614    `testMarkedCode`. Note that all code in a file or notebook cell is
2615    collected: if you use `mark` multiple times in the same file each
2616    `testMarkedCode` call will still test the entire file.
2617
2618    Also, if this function is called during the run of another test and a
2619    code block is not available but an old code block was under the same
2620    name, that old code block will not be modified.
2621    """
2622    global _MARKED_CODE_BLOCKS
2623    block_filename = get_filename(get_external_calling_frame())
2624    contents = ''.join(linecache.getlines(block_filename))
2625    if contents or not _RUNNING_TEST_CODE:
2626        _MARKED_CODE_BLOCKS[name] = contents or None
2627
2628
2629def getMarkedCode(markName):
2630    """
2631    Gets the block of code (e.g., Python file; notebook cell; etc.)
2632    within which `mark` was called with the specified name. Returns
2633    `None` if that information isn't available. Reasons it isn't
2634    available include that `mark` was never called with that name, and
2635    that `mark` was called, but we weren't able to extract the source
2636    code of the block it was called in (e.g., because it was called in
2637    an interactive interpreter session).
2638    """
2639    return _MARKED_CODE_BLOCKS.get(markName)
2640
2641
2642def testMarkedCode(markName, includeGlobals=True):
2643    """
2644    Creates a test manager for running the code block (e.g., Python file;
2645    notebook cell; etc.) within which `mark` was called using the given
2646    mark name. `mark` must have already been called with the specified
2647    name, and changes to the code around it may or may not be picked up
2648    if they were made since the call happened. A `BlockManager` is
2649    created for that code, with `includeGlobals` set based on the value
2650    provided here (default `True`).
2651
2652    If no code is available, a `SkipManager` will be returned.
2653    """
2654    code = getMarkedCode(markName)
2655    if code is None:
2656        print(
2657            (
2658                f"Warning: unable to find code for test suite"
2659                f" '{markName}'. Have you called 'mark' already with"
2660                f" that name?"
2661            ),
2662            file=PRINT_TO
2663        )
2664        return SkipManager("Code around mark '{markName}'")
2665    else:
2666        return BlockManager(code, includeGlobals)
2667
2668
2669#----------------#
2670# Output capture #
2671#----------------#
2672
2673class CapturingStream(io.StringIO):
2674    """
2675    An output capture object which is an `io.StringIO` underneath, but
2676    which has an option to also write incoming text to normal
2677    `sys.stdout`. Call the install function to begin capture.
2678    """
2679    def __init__(self, *args, **kwargs):
2680        """
2681        Passes arguments through to `io.StringIO`'s constructor.
2682        """
2683        self.original_stdout = None
2684        self.tee = False
2685        super().__init__(*args, **kwargs)
2686
2687    def echo(self, doit=True):
2688        """
2689        Turn on echoing to stdout along with capture, or turn it off if
2690        False is given.
2691        """
2692        self.tee = doit
2693
2694    def install(self):
2695        """
2696        Replaces `sys.stdout` to begin capturing printed output.
2697        Remembers the old `sys.stdout` value so that `uninstall` can
2698        work. Note that if someone else changes `sys.stdout` after this
2699        is installed, uninstall will set `sys.stdout` back to what it was
2700        when `install` was called, which could cause issues. For example,
2701        if we have two capturing streams A and B, and we call:
2702
2703        ```py
2704        A.install()
2705        B.install()
2706        A.uninstall()
2707        B.uninstall()
2708        ```
2709
2710        The original `sys.stdout` will not be restored. In general, you
2711        must uninstall capturing streams in the reverse order that you
2712        installed them.
2713        """
2714        self.original_stdout = sys.stdout
2715        sys.stdout = self
2716
2717    def uninstall(self):
2718        """
2719        Returns `sys.stdout` to what it was before `install` was called,
2720        or does nothing if `install` was never called.
2721        """
2722        if self.original_stdout is not None:
2723            sys.stdout = self.original_stdout
2724
2725    def reset(self):
2726        """
2727        Resets the captured output.
2728        """
2729        self.seek(0)
2730        self.truncate(0)
2731
2732    def writelines(self, lines):
2733        """
2734        Override writelines to work through write.
2735        """
2736        for line in lines:
2737            self.write(line)
2738
2739    def write(self, stuff):
2740        """
2741        Accepts a string and writes to our capture buffer (and to
2742        original stdout if `echo` has been called). Returns the number
2743        of characters written.
2744        """
2745        if self.tee and self.original_stdout is not None:
2746            self.original_stdout.write(stuff)
2747        super().write(stuff)
2748
2749
2750def showPrintedLines(show=True):
2751    """
2752    Changes the testing mechanisms so that printed output produced during
2753    tests is shown as normal in addition to being captured. Call it with
2754    False as an argument to disable this.
2755    """
2756    global _SHOW_OUTPUT
2757    _SHOW_OUTPUT = show
2758
2759
2760#---------------------#
2761# Debugging functions #
2762#---------------------#
2763
2764def differencesAreSubtle(val, ref):
2765    """
2766    Judges whether differences between two strings are 'subtle' in which
2767    case the first difference details will be displayed. Returns true if
2768    either value is longer than a typical floating-point number, or if
2769    the representations are the same once all whitespace is stripped
2770    out.
2771    """
2772    # If either has non-trivial length, we'll include the first
2773    # difference report. 18 is the length of a floating-point number
2774    # with two digits before the decimal point and max digits afterwards
2775    if len(val) > 18 or len(ref) > 18:
2776        return True
2777
2778    valNoWS = re.sub(r'\s', '', val)
2779    refNoWS = re.sub(r'\s', '', ref)
2780    # If they're the same modulo whitespace, then it's probably useful
2781    # to report first difference, otherwise we won't
2782    return valNoWS == refNoWS
2783
2784
2785def expect(expr, value):
2786    """
2787    Establishes an immediate expectation that the values of the two
2788    arguments should be equivalent. The expression provided will be
2789    picked out of the source code of the module calling `expect` (see
2790    `get_my_context`). The expression and sub-values will be displayed
2791    if the expectation is not met, and either way a message indicating
2792    success or failure will be printed. Use `detailLevel` to control how
2793    detailed the messages are.
2794
2795    For `expect` to work properly, the following rules must be followed:
2796
2797    1. When multiple calls to `expect` appear on a single line of the
2798        source code (something you should probably avoid anyway), none of
2799        the calls should execute more times than another when that line
2800        is executed (it's difficult to violate this, but examples
2801        include the use of `expect` multiple times on one line within
2802        generator or if/else expressions)
2803    2. None of the following components of the expression passed to
2804        `expect` should have side effects when evaluated:
2805        - Attribute accesses
2806        - Subscripts (including expressions inside brackets)
2807        - Variable lookups
2808        (Note that those things don't normally have side effects!)
2809
2810    This function returns True if the expectation is met and False
2811    otherwise. It returns None if the check is skipped, which will
2812    happen when `SKIP_ON_FAILURE` is `'all'` and a previous check failed.
2813    If the values are not equivalent, this will count as a failed check
2814    and other checks may be skipped.
2815
2816    If not skipped, this function registers an outcome in `ALL_OUTCOMES`.
2817    """
2818    global CHECK_FAILED
2819    context = get_my_context(expect)
2820    tag = tag_for(context)
2821
2822    # Skip this expectation if necessary
2823    if SKIP_ON_FAILURE == 'all' and CHECK_FAILED:
2824        if DETAIL_LEVEL < 1:
2825            msg = f"~ {tag} (skipped)"
2826        else:
2827            msg = (
2828                f"~ direct expectation at {tag} for skipped because a"
2829                f" prior check failed"
2830            )
2831            print_message(msg, color=msg_color("skipped"))
2832            return None
2833
2834    # Figure out if we want to suppress any failure message
2835    suppress = SUPPRESS_ON_FAILURE == 'all' and CHECK_FAILED
2836
2837    short_result = ellipsis(repr(expr), 78)
2838    short_expected = ellipsis(repr(value), 78)
2839    full_result = repr(expr)
2840    full_expected = repr(value)
2841
2842    firstDiff = findFirstDifference(expr, value)
2843    if firstDiff is None:
2844        message = f"✓ {tag}"
2845        equivalent = "equivalent to"
2846        msg_cat = "succeeded"
2847        same = True
2848    else:
2849        message = f"✗ {tag}"
2850        equivalent = "NOT equivalent to"
2851        msg_cat = "failed"
2852        same = False
2853
2854    # At higher detail for success or any detail for unsuppressed
2855    # failure:
2856    if DETAIL_LEVEL >= 1 or (not same and not suppress):
2857        message += f"""
2858  Result:
2859{indent(short_result, 4)}
2860  was {equivalent} the expected value:
2861{indent(short_expected, 4)}"""
2862
2863    if (
2864        not same
2865    and not suppress
2866    and differencesAreSubtle(short_result, short_expected)
2867    ):
2868        message += f"\n  First difference was:\n{indent(firstDiff, 4)}"
2869
2870    # Report full values if detail level is turned up and the short
2871    # values were abbreviations
2872    if DETAIL_LEVEL >= 1:
2873        if short_result != full_result:
2874            message += f"\n  Full result:\n{indent(full_result, 4)}"
2875        if short_expected != full_expected:
2876            message += (
2877                f"\n  Full expected value:\n{indent(full_expected, 4)}"
2878            )
2879
2880    # Report info about the test expression
2881    base, extra = expr_details(context)
2882    if (
2883           (same and DETAIL_LEVEL >= 1)
2884        or (not same and not suppress and DETAIL_LEVEL >= 0)
2885    ):
2886        message += '\n' + indent(base, 2)
2887
2888    if DETAIL_LEVEL >= 1 and extra:
2889        message += '\n' + indent(extra, 2)
2890
2891    # Register a check failure if the expectation was not met
2892    if not same:
2893        CHECK_FAILED = True
2894
2895    # Print our message
2896    print_message(message, color=msg_color(msg_cat))
2897
2898    # Register our outcome
2899    _register_outcome(same, tag, message)
2900
2901    # Return our result
2902    return same
2903
2904
2905def expectType(expr, typ):
2906    """
2907    Works like `expect`, but establishes an expectation for the type of
2908    the result of the expression, not for the exact value. The same
2909    rules must be followed as for `expect` for this to work properly.
2910
2911    If the type of the expression's result is an instance of the target
2912    type, the expectation counts as met.
2913
2914    If not skipped, this function registers an outcome in `ALL_OUTCOMES`.
2915    """
2916    global CHECK_FAILED
2917    context = get_my_context(expectType)
2918    tag = tag_for(context)
2919
2920    # Skip this expectation if necessary
2921    if SKIP_ON_FAILURE == 'all' and CHECK_FAILED:
2922        if DETAIL_LEVEL < 1:
2923            msg = f"~ {tag} (skipped)"
2924        else:
2925            msg = (
2926                f"~ direct expectation at {tag} for skipped because a"
2927                f" prior check failed"
2928            )
2929            print_message(msg, color=msg_color("skipped"))
2930            return None
2931
2932    suppress = SUPPRESS_ON_FAILURE == 'all' and CHECK_FAILED
2933
2934    if type(expr) == typ:
2935        message = f"✓ {tag}"
2936        desc = "the expected type"
2937        msg_cat = "succeeded"
2938        same = True
2939    elif isinstance(expr, typ):
2940        message = f"✓ {tag}"
2941        desc = f"a kind of {typ}"
2942        msg_cat = "succeeded"
2943        same = True
2944    else:
2945        message = f"✗ {tag}"
2946        desc = f"NOT a kind of {typ}"
2947        msg_cat = "failed"
2948        same = False
2949
2950    # Note failed check
2951    if not same:
2952        CHECK_FAILED = True
2953
2954    # Report on the type if the detail level warrants it, and also about
2955    # the test expression
2956    base, extra = expr_details(context)
2957    if (
2958           (same and DETAIL_LEVEL >= 1)
2959        or (not same and not suppress and DETAIL_LEVEL >= 0)
2960    ):
2961        message += f"\n  The result type ({type(expr)}) was {desc}."
2962        message += '\n' + indent(base, 2)
2963
2964    if DETAIL_LEVEL >= 1 and extra:
2965        message += '\n' + indent(extra, 2)
2966
2967    # Print our message
2968    print_message(message, color=msg_color(msg_cat))
2969
2970    # Register our outcome
2971    _register_outcome(same, tag, message)
2972
2973    # Return our result
2974    return same
2975
2976
2977#--------------#
2978# AST Checking #
2979#--------------#
2980
2981class ASTMatch:
2982    """
2983    Represents a full, partial, or missing (i.e., non-) match of an
2984    `ASTRequirement` against an abstract syntax tree, ignoring
2985    sub-checks. The `isFull` and `isPartial` fields specify whether the
2986    match is a full match (values `True`, `False`), a partial match
2987    (`False`, `True`) or not a match at all (`False`, `False`).
2988    """
2989    def __init__(self, node, isPartial=None):
2990        """
2991        The matching AST node is required; use None for a non-match. If
2992        a node is given, `isPartial` will be stored to determine whether
2993        it's a partial or full match (when node is set to `None`,
2994        `isPartial` is ignored).
2995        """
2996        self.node = node
2997        if node is None:
2998            self.isFull = False
2999            self.isPartial = False
3000        else:
3001            self.isFull = not isPartial
3002            self.isPartial = isPartial
3003
3004    def __str__(self):
3005        """
3006        Represents the match using the name of the type of node matched,
3007        plus the line number of that node if available.
3008        """
3009        if self.isFull:
3010            result = "Full match: "
3011        elif self.isPartial:
3012            result = "Partial match: "
3013        else:
3014            return "No match found"
3015
3016        name = type(self.node).__name__
3017        if hasattr(self.node, "lineno") and self.node.lineno is not None:
3018            result += f"{name} on line {self.node.lineno}"
3019        else:
3020            result += f"a {name} (unknown location)"
3021
3022        return result
3023
3024
3025class RuleMatches:
3026    """
3027    Represents how an `ASTRequirement` matches against a syntax tree,
3028    including sub-checks. It can be a full, partial, or non-match, as
3029    dictated by the `isFull` and `isPartial` variables (`True`/`False` →
3030    full match, `False`/`True` → partial match, and `False`/`False` →
3031    non-match).
3032
3033    Stores a list of tuples each containing an `ASTMatch` object for the
3034    check itself, plus a list of `RuleMatches` objects for each
3035    sub-check.
3036
3037    The number of these tuples compared to the min/max match
3038    requirements of the check this `RuleMatches` was derived from
3039    determine if it's a full, partial, or non-match.
3040    """
3041    def __init__(self, check):
3042        """
3043        The check that we're deriving this `RuleMatches` from is
3044        required. An empty structure (set up as a non-match unless the
3045        check's maxMatches or minMatches is 0 in which case it's set up
3046        as a full match) will be created which can be populated using the
3047        `addMatch` method.
3048        """
3049        self.check = check
3050        self.nFull = 0
3051        self.matchPoints = []
3052        self.final = False
3053
3054        if self.check.minMatches == 0 or self.check.maxMatches == 0:
3055            self.isFull = True
3056            self.isPartial = False
3057        else:
3058            self.isFull = False
3059            self.isPartial = False
3060
3061    def __str__(self):
3062        """
3063        Represents the matches by listing them out over multiple lines,
3064        prefaced with a description of whether the whole rule is a
3065        full/partial/non- match.
3066        """
3067        if self.isFull:
3068            category = "fully"
3069        elif self.isPartial:
3070            category = "partially"
3071        else:
3072            category = "not"
3073
3074        # Separate full & partial matches (attending to sub-matches which
3075        # the match objects themselves don't)
3076        full = []
3077        partial = []
3078        for (match, subMatches) in self.matchPoints:
3079            if match.isFull and all(sub.isFull for sub in subMatches):
3080                full.append(str(match).split(':')[-1].strip())
3081            elif match.isFull or match.isPartial:
3082                partial.append(str(match).split(':')[-1].strip())
3083
3084        return (
3085            (
3086                f"Requirement {category} satisfied via {self.nFull} full"
3087                f" and {len(self.matchPoints) - self.nFull} partial"
3088                f" match(es):\n"
3089            )
3090          + '\n'.join(
3091                indent("Full match: " + matchStr, 2)
3092                for matchStr in full
3093            )
3094          + '\n'.join(
3095                indent("Partial match: " + matchStr, 2)
3096                for matchStr in partial
3097            )
3098        )
3099
3100    def addMatch(self, nodeMatch, subMatches):
3101        """
3102        Adds a single matching AST node to this matches suite. The node
3103        at which the match occurs is required (as an `ASTMatch` object),
3104        along with a list of sub-`RuleMatches` objects for each sub-check
3105        of the check. This list is not required if the `nodeMatch` is a
3106        non-match, but in that case the entry will be ignored.
3107
3108        This object's partial/full status will be updated according to
3109        whether or not the count of full matches falls within the
3110        min/max match range after adding the specified match point. Note
3111        that a match point only counts as a full match if the
3112        `nodeMatch` is a full match and each of the `subMatches` are
3113        full matches; if the `nodeMatch` is a non-match, then it doesn't
3114        count at all, and otherwise it's a partial match.
3115
3116        Note that each of the sub-matches provided will be marked as
3117        final, and any attempts to add new matches to them will fail
3118        with a `ValueError`.
3119        """
3120        if self.final:
3121            raise ValueError(
3122                "Attempted to add to a RuleMatches suite after it was"
3123                " used as a sub-suite for another RuleMatches suite"
3124                " (you may not call addMatch after using a RuleMatches"
3125                " as a sub-suite)."
3126            )
3127
3128        # Mark each sub-match as final now that it's being used to
3129        # substantiate a super-match.
3130        for sub in subMatches:
3131            sub.final = True
3132
3133        # IF this isn't actually a match at all, ignore it
3134        if not nodeMatch.isFull and not nodeMatch.isPartial:
3135            return
3136
3137        if len(subMatches) != len(self.check.subChecks):
3138            raise ValueError(
3139                f"One sub-matches object must be supplied for each"
3140                f" sub-check of the rule ({len(self.check.subChecks)}"
3141                f" were required but you supplied {len(subMatches)})."
3142            )
3143
3144        # Add to our list of match points, which includes all full and
3145        # partial matches.
3146        self.matchPoints.append((nodeMatch, subMatches))
3147
3148        # Check if the new match is a full match
3149        if nodeMatch.isFull and all(sub.isFull for sub in subMatches):
3150            self.nFull += 1
3151
3152        # Update our full/partial status depending on the new number
3153        # of full matches
3154        if (
3155            (
3156                self.check.minMatches is None
3157             or self.check.minMatches <= self.nFull
3158            )
3159        and (
3160                self.check.maxMatches is None
3161             or self.check.maxMatches >= self.nFull
3162            )
3163        ):
3164            self.isFull = True
3165            self.isPartial = False
3166        else:
3167            self.isFull = False
3168            self.isPartial = True
3169
3170    def explanation(self):
3171        """
3172        Produces a text explanation of whether or not the associated
3173        check succeeded, and if not, why.
3174        """
3175        # TODO
3176        if self.isFull:
3177            return "check succeeded"
3178        elif self.isPartial:
3179            return "check failed (partial match(es) found)"
3180        else:
3181            return "check failed (no matches)"
3182
3183
3184NoneType = type(None)
3185
3186
3187class DefaultMin:
3188    """
3189    Represents the default min value (to distinguish from an explicit
3190    value that's the same as the default).
3191    """
3192    pass
3193
3194
3195class ASTRequirement:
3196    """
3197    Represents a specific abstract syntax tree structure to check for
3198    within a file, function, or code block (see
3199    `TestManager.checkCodeContains`). This base class is abstract, the
3200    concrete subclasses each check for specific things.
3201    """
3202    def __init__(self, *, min=DefaultMin, max=None, n=None):
3203        """
3204        Creates basic common data structures. The `min`, `max`, and `n`
3205        keyword arguments can be used to specify the number of matches
3206        required: if `n` is set, it overrides both `min` and `max`;
3207        either of those can be set to `None` to eschew an upper/lower
3208        limit. Note that a lower limit of `None` or 0 will mean that the
3209        check isn't required to match, and an upper limit of 0 will mean
3210        that the check will only succeed if the specified structure is
3211        NOT present. If `min` is greater than `max`, the check will never
3212        succeed; a warning will be issued in that case.
3213        """
3214        self.subChecks = []
3215        if min is DefaultMin:
3216            if max == 0:
3217                self.minMatches = 0
3218            else:
3219                self.minMatches = 1
3220        else:
3221            self.minMatches = min
3222
3223        self.maxMatches = max
3224        if n is not None:
3225            self.minMatches = n
3226            self.maxMatches = n
3227
3228        if min is not DefaultMin and not isinstance(min, (int, NoneType)):
3229            raise TypeError(
3230                f"min argument must be an integer or None (got: '{min}'"
3231                f" which is a/an: {type(min)}."
3232            )
3233
3234        if not isinstance(max, (int, NoneType)):
3235            raise TypeError(
3236                f"max argument must be an integer or None (got: '{max}'"
3237                f" which is a/an: {type(max)}."
3238            )
3239
3240        if not isinstance(n, (int, NoneType)):
3241            raise TypeError(
3242                f"n argument must be an integer or None (got: '{n}'"
3243                f" which is a/an: {type(n)}."
3244            )
3245
3246        if (
3247            self.minMatches is not None
3248        and self.maxMatches is not None
3249        and self.minMatches > self.maxMatches
3250        ):
3251            warnings.warn(
3252                "Min matches is larger than max matches for"
3253                " ASTRequirement; it will always fail."
3254            )
3255
3256    def structureString(self):
3257        """
3258        Returns a string expressing the structure that this check is
3259        looking for.
3260        """
3261        raise NotImplementedError(
3262            "ASTRequirement base class is abstract."
3263        )
3264
3265    def howMany(self):
3266        """
3267        Returns a string describing how many are required based on min +
3268        max match values.
3269        """
3270        # Figure out numeric descriptor from min/max
3271        if self.maxMatches is None:
3272            if self.minMatches is None:
3273                return "any number of"
3274            else:
3275                return f"at least {self.minMatches}"
3276        else:
3277            if self.maxMatches == 0:
3278                return "no"
3279            elif self.minMatches is None:
3280                return f"at most {self.maxMatches}"
3281            elif self.minMatches == self.maxMatches:
3282                return str(self.minMatches)
3283            else:
3284                return f"{self.minMatches}-{self.maxMatches}"
3285
3286    def fullStructure(self):
3287        """
3288        The structure string (see `structureString`) plus a list of what
3289        sub-checks are used to constrain contents of those matches, and
3290        text describing how many matches are required.
3291        """
3292        result = f"{self.howMany()} {self.structureString()}"
3293        if len(self.subChecks) > 0:
3294            result += " containing:\n" + '\n'.join(
3295                indent(sub.fullStructure(), 2)
3296                for sub in self.subChecks
3297            )
3298
3299        return result
3300
3301    def _nodesToCheck(self, syntaxTree):
3302        """
3303        Given a syntax tree, yields each node from that tree that should
3304        be checked for subrule matches. These are yielded in tuples
3305        where the second element is True for a full match at that node
3306        and False for a partial match. This is used by `allMatches`.
3307        """
3308        raise NotImplementedError(
3309            "ASTRequirement base class is abstract."
3310        )
3311
3312    def allMatches(self, syntaxTree):
3313        """
3314        Returns a `RuleMatches` object representing all full and partial
3315        matches of this check within the given syntax tree.
3316
3317        Only matches which happen at distinct AST nodes are considered;
3318        this does NOT list out all of the ways a match could happen (per
3319        sub-rule possibilities) for each node that might match.
3320
3321        This object will be finalized and may be used for a sub-result in
3322        another check.
3323        """
3324        result = RuleMatches(self)
3325        for (node, isFull) in self._nodesToCheck(syntaxTree):
3326            subMatchSuites = self._subRuleMatches(node)
3327            result.addMatch(ASTMatch(node, not isFull), subMatchSuites)
3328
3329        return result
3330
3331    def _walkNodesOfType(self, root, nodeTypes):
3332        """
3333        A generator that yields all nodes within the given AST (including
3334        the root node) which match the given node type (or one of the
3335        types in the given node type tuple). The nodes are yielded in
3336        (an approximation of) execution order (see `walk_ast_in_order`).
3337        """
3338        for node in walk_ast_in_order(root):
3339            if isinstance(node, nodeTypes):
3340                yield node
3341
3342    def _subRuleMatches(self, withinNode):
3343        """
3344        Returns a list of one `RuleMatches` object for each sub-check of
3345        this check. These will be finalized and can safely be added as
3346        sub-rule-matches for a entry in a `RuleMatches` suite for this
3347        node.
3348        """
3349        return [
3350            check.allMatches(withinNode)
3351            for check in self.subChecks
3352        ]
3353
3354    def contains(self, *subChecks):
3355        """
3356        Enhances this check with one or more sub-check(s) which must
3357        match (anywhere) within the contents of a basic match for the
3358        whole check to have a full match.
3359
3360        Returns self for chaining.
3361
3362        For example:
3363
3364        >>> import optimism
3365        >>> optimism.messagesAsErrors(False)
3366        >>> optimism.colors(False)
3367        >>> manager = optimism.testBlock('''\\
3368        ... def f():
3369        ...     for i in range(3):
3370        ...         print('A' * i)
3371        ... ''')
3372        >>> manager.checkCodeContains(
3373        ...     optimism.Def().contains(
3374        ...         optimism.Loop().contains(
3375        ...             optimism.Call('print')
3376        ...         )
3377        ...     )
3378        ... ) # doctest: +ELLIPSIS
3379        ✓ ...
3380        True
3381        >>> manager.checkCodeContains(
3382        ...     optimism.Def().contains(
3383        ...         optimism.Call('print')
3384        ...     )
3385        ... ) # doctest: +ELLIPSIS
3386        ✓ ...
3387        True
3388        >>> manager.checkCodeContains(
3389        ...     optimism.Loop().contains(
3390        ...         optimism.Def()
3391        ...     )
3392        ... ) # doctest: +ELLIPSIS
3393        ✗ ...
3394          Code does not contain the expected structure:
3395            at least 1 loop(s) or generator expression(s) containing:
3396              at least 1 function definition(s)
3397          Although it does partially satisfy the requirement:
3398            Requirement partially satisfied via 0 full and 1 partial match(es):
3399              Partial match: For on line 2
3400          checked code from block at ...
3401        False
3402        """
3403        self.subChecks.extend(subChecks)
3404        return self
3405
3406
3407class MatchAny(ASTRequirement):
3408    """
3409    A special kind of `ASTRequirement` which matches when at least one of
3410    several other checks matches. Allows testing for one of several
3411    different acceptable code structures. For example, the following code
3412    shows how to check that either `with` was used with `open`, or
3413    `try/finally` was used with `open` in the try part and `close` in the
3414    finally part (and that either way, `read` was used):
3415
3416    >>> import optimism
3417    >>> optimism.messagesAsErrors(False)
3418    >>> optimism.colors(False)
3419    >>> manager1 = optimism.testBlock('''\\
3420    ... with open('f') as fileInput:
3421    ...     print(f.read())''')
3422    ...
3423    >>> manager2 = optimism.testBlock('''\\
3424    ... fileInput = None
3425    ... try:
3426    ...     fileInput = open('f')
3427    ...     print(f.read())
3428    ... finally:
3429    ...     close(fileInput)''')
3430    ...
3431    >>> check = optimism.MatchAny(
3432    ...     optimism.With().contains(optimism.Call('open')),
3433    ...     optimism.Try()
3434    ...         .contains(optimism.Call('open'))
3435    ...         .contains(optimism.Call('close'))
3436    ...     # TODO: Implement these
3437    ...     #    .tryContains(optimism.Call('open'))
3438    ...     #    .finallyContains(optimism.call('close'))
3439    ... ).contains(optimism.Call('read', isMethod=True))
3440    ...
3441    >>> manager1.checkCodeContains(check) # doctest: +ELLIPSIS
3442    ✓ ...
3443    True
3444    >>> manager2.checkCodeContains(check) # doctest: +ELLIPSIS
3445    ✓ ...
3446    True
3447    """
3448    def __init__(
3449        self,
3450        *checkers,
3451        min=1,
3452        max=None,
3453        n=None
3454    ):
3455        """
3456        Any number of sub-checks may be supplied. Note that `contains`
3457        will be broadcast to each of these sub-checks if called on the
3458        `MatchAny` object. `min`, `max`, and/or `n` may be specified as
3459        integers to place limits on the number of matches we look for;
3460        `min` must be at least 1, and the default is 1 minimum and no
3461        maximum. The min and max arguments are ignored if a specific
3462        number of required matches is provided.
3463        """
3464        super().__init__(min=min, max=max, n=n)
3465
3466        if self.minMatches <= 0:
3467            raise ValueError(
3468                f"minMatches for a matchAny must be > 0 (got"
3469                f" {self.minMatches})"
3470            )
3471
3472        if len(checkers) == 0:
3473            warnings.warn(
3474                "A MatchAny check without any sub-checks will always"
3475                " fail."
3476            )
3477        self.subChecks = checkers
3478
3479    def structureString(self):
3480        "Lists the full structures of each alternative."
3481        if len(self.subChecks) == 0:
3482            return "zero alternatives"
3483
3484        return "the following:\n" + (
3485            '\n...or...\n'.join(
3486              indent(check.fullStructure(), 2)
3487              for check in self.subChecks
3488          )
3489        )
3490
3491    def fullStructure(self):
3492        "Lists the alternatives."
3493        if len(self.subChecks) == 0:
3494            return "A MatchAny with no alternatives (always fails)"
3495
3496        # Special case 'no' -> 'none of'
3497        n = self.howMany()
3498        if n == "no":
3499            n = "none of"
3500
3501        return f"{n} {self.structureString()}"
3502
3503    def allMatches(self, syntaxTree):
3504        """
3505        Runs each sub-check and returns a `RuleMatches` with just one
3506        `ASTMatch` entry that targets the root of the syntax tree. The
3507        `RuleMatches` sub-entires for this single match point are full
3508        `RuleMatches` for each alternative listed in this `MatchAny`,
3509        which might contain match points on nodes also matched by other
3510        `RuleMatches` of different alternatives.
3511
3512        However, the overall `isFull`/`isPartial` status of the resulting
3513        `RuleMatches` is overridden to be based on the count of distinct
3514        node positions covered by full matches of any of the
3515        alternatives. So if you set min/max on the base `MatchAny`
3516        object, it will apply to the number of node points at which any
3517        sub-rule matches. For example:
3518
3519        >>> import optimism
3520        >>> optimism.messagesAsErrors(False)
3521        >>> optimism.colors(False)
3522        >>> manager = optimism.testBlock('''\\
3523        ... def f(x):
3524        ...     x = max(x, 0)
3525        ...     if x > 5:
3526        ...         print('big')
3527        ...     elif x > 1:
3528        ...         print('medium')
3529        ...     else:
3530        ...         print('small')
3531        ...     return x
3532        ... ''')
3533        ...
3534        >>> check = optimism.MatchAny(
3535        ...    optimism.IfElse(),
3536        ...    optimism.Call('print'),
3537        ...    n=5 # 2 matches for if/else, 3 for call to print
3538        ... )
3539        ...
3540        >>> manager.checkCodeContains(check) # doctest: +ELLIPSIS
3541        ✓ ...
3542        True
3543        >>> check = optimism.MatchAny(
3544        ...    optimism.Call(),
3545        ...    optimism.Call('print'),
3546        ...    n=4 # 4 nodes that match, 3 of which overlap
3547        ... )
3548        ...
3549        >>> manager.checkCodeContains(check) # doctest: +ELLIPSIS
3550        ✓ ...
3551        True
3552        """
3553        result = RuleMatches(self)
3554
3555        if len(self.subChecks) == 0:
3556            return result  # a failure, since minMatches >= 1
3557
3558        # Create a mapping from AST nodes to True/False/None for a
3559        # full/partial/no match at that node from ANY sub-check, since we
3560        # don't want to count multiple match points on the same node. At
3561        # the same time, build a list of sub-matches for each sub-check.
3562        nodeMap = {}
3563        subMatchList = []
3564        for i, check in enumerate(self.subChecks):
3565            # Find sub-matches for this alternative
3566            subMatches = check.allMatches(syntaxTree)
3567
3568            # Record in list + note first full/partial indices
3569            subMatchList.append(subMatches)
3570
3571            # Note per-node best matches
3572            for (match, subSubMatches) in subMatches.matchPoints:
3573                fullPos = (
3574                    match.isFull
3575                and all(subSub.isFull for subSub in subSubMatches)
3576                )
3577                prev = nodeMap.get(match.node, None)
3578                if prev is None or fullPos and prev is False:
3579                    nodeMap[match.node] = fullPos
3580
3581        # We have only a single result, containing the full sub-matches
3582        # for each alternative:
3583        result.addMatch(ASTMatch(syntaxTree), subMatchList)
3584
3585        # Set 'final' on the result so nobody adds more to it
3586        result.final = True
3587
3588        # But we override the counting logic: we don't want to count the
3589        # # of places where a match occurred (there's only ever 1); and
3590        # we don't want to count the # of sub-rules that matched (that
3591        # caps out at the # of subrules, even if they match multiple
3592        # nodes). Instead we want to count the # of distinct nodes
3593        # where full matches were found across all sub-rules.
3594        result.nFull = len([k for k in nodeMap if nodeMap[k]])
3595
3596        # Override isFull/isPartial on result based on new nFull
3597        if (
3598            (
3599                result.check.minMatches is None
3600             or result.check.minMatches <= result.nFull
3601            )
3602        and (
3603                result.check.maxMatches is None
3604             or result.check.maxMatches >= result.nFull
3605            )
3606        ):
3607            result.isFull = True
3608            result.isPartial = False
3609        else:
3610            result.isFull = False
3611            if (
3612                result.nFull == 0
3613            and (
3614                    result.check.maxMatches is None
3615                 or result.check.maxMatches > 0
3616                )
3617            ):
3618                # In this case we consider it to be not a match at all,
3619                # since we found 0 match points for any alternatives and
3620                # the requirement was a positive one where max was > 0.
3621                result.isPartial = False
3622            else:
3623                result.isPartial = True
3624
3625        # And we're done
3626        return result
3627
3628    def contains(self, *subChecks):
3629        """
3630        Broadcasts the call to each sub-check. Note that this can create
3631        a sharing situation where the same `ASTRequirement` object is a
3632        sub-check of multiple other checks.
3633
3634        This function returns the `MatchAny` object for chaining.
3635        """
3636        for check in self.subChecks:
3637            check.contains(*subChecks)
3638
3639        return self
3640
3641
3642class Import(ASTRequirement):
3643    """
3644    Checks for an `import` statement, possibly with a specific module
3645    name.
3646    """
3647    def __init__(self, moduleName=None, **kwargs):
3648        """
3649        The argument specifies the required module name; leave it as
3650        `None` (the default) to match any `import` statement.
3651        """
3652        super().__init__(**kwargs)
3653        self.name = moduleName
3654
3655    def structureString(self):
3656        if self.name is not None:
3657            return f"import(s) of {self.name}"
3658        else:
3659            return "import statement(s)"
3660
3661    def _nodesToCheck(self, syntaxTree):
3662        # Note that every import statement is a partial match
3663        for node in self._walkNodesOfType(
3664            syntaxTree,
3665            (ast.Import, ast.ImportFrom)
3666        ):
3667            if self.name is None:
3668                yield (node, True)
3669            else:
3670                if isinstance(node, ast.Import):
3671                    if any(
3672                        alias.name == self.name
3673                        for alias in node.names
3674                    ):
3675                        yield (node, True)
3676                    else:
3677                        yield (node, False)
3678                else:  # must be ImportFrom
3679                    if node.module == self.name:
3680                        yield (node, True)
3681                    else:
3682                        yield (node, False)
3683
3684
3685class Def(ASTRequirement):
3686    """
3687    Matches a function definition, possibly with a specific name and/or
3688    number of arguments.
3689    """
3690    def __init__(
3691        self,
3692        name=None,
3693        minArgs=0,
3694        maxArgs=None,
3695        nArgs=None,
3696        **kwargs
3697    ):
3698        """
3699        The first argument specifies the function name. Leave it as
3700        `None` (the default) to allow any function definition to match.
3701
3702        The `minArgs`, `maxArgs`, and `nArgs` arguments specify the
3703        number of arguments the function must accept. Min and max are
3704        ignored if `nArgs` is specified; min or max can be None to eschew
3705        an upper or lower limit. Default is any number of arguments.
3706
3707        A warning is issued if `minArgs` > `maxArgs`.
3708        """
3709        super().__init__(**kwargs)
3710        self.name = name
3711        self.minArgs = minArgs
3712        self.maxArgs = maxArgs
3713        if nArgs is not None:
3714            self.minArgs = nArgs
3715            self.maxArgs = nArgs
3716
3717        if (
3718            self.minArgs is not None
3719        and self.maxArgs is not None
3720        and self.minArgs > self.maxArgs
3721        ):
3722            warnings.warn(
3723                "A def node with minArgs > maxArgs cannot match."
3724            )
3725
3726    def structureString(self):
3727        if self.name is not None:
3728            result = f"definition(s) of {self.name}"
3729        else:
3730            result = "function definition(s)"
3731        if self.minArgs is not None and self.minArgs > 0:
3732            if self.maxArgs is None:
3733                result += f" (with at least {self.minArgs} arguments)"
3734            elif self.maxArgs == self.minArgs:
3735                result += f" (with {self.minArgs} arguments)"
3736            else:
3737                result += f" (with {self.minArgs}-{self.maxArgs} arguments)"
3738        elif self.maxArgs is not None:
3739            result += " (with at most {self.maxArgs} arguments)"
3740        # otherwise no parenthetical is necessary
3741        return result
3742
3743    def _nodesToCheck(self, syntaxTree):
3744        # Note that every def is considered a partial match, but
3745        # definitions whose name matches and whose arguments don't are
3746        # yielded before those whose names don't match.
3747        later = []
3748        for node in self._walkNodesOfType(
3749            syntaxTree,
3750            (ast.FunctionDef, ast.AsyncFunctionDef)
3751        ):
3752            nameMatch = self.name is None or node.name == self.name
3753            nArgs = (
3754                (
3755                    len(node.args.posonlyargs)
3756                    if hasattr(node.args, "posonlyargs")
3757                    else 0
3758                )
3759              + len(node.args.args)
3760              + len(node.args.kwonlyargs)
3761              + (1 if node.args.vararg is not None else 0)
3762              + (1 if node.args.kwarg is not None else 0)
3763            )
3764            argsMatch = (
3765                (self.minArgs is None or self.minArgs <= nArgs)
3766            and (self.maxArgs is None or self.maxArgs >= nArgs)
3767            )
3768            if nameMatch and argsMatch:
3769                yield (node, True)
3770            elif nameMatch:
3771                yield (node, False)
3772            else:
3773                # Order non-name-matched nodes last
3774                later.append(node)
3775
3776        for node in later:
3777            yield (node, False)
3778
3779
3780class Call(ASTRequirement):
3781    """
3782    Matches a function call, possibly with a specific name, and possibly
3783    restricted to only method calls or only non-method calls.
3784    """
3785    def __init__(
3786        self,
3787        name=None,
3788        isMethod=None,
3789        **kwargs
3790    ):
3791        """
3792        The first argument specifies the function name. Leave it as
3793        `None` (the default) to allow any function call to match.
3794
3795        The second argument `isMethod` specifies whether the call must be
3796        a method call, not a method call, or may be either. Note that any
3797        call to an attribute of an object is counted as a "method" call,
3798        including calls that use explicit module names, since it's not
3799        possible to know without running the code whether the attribute's
3800        object is a class or something else. Set this to `True` to
3801        match only method calls, `False` to match only non-method calls,
3802        and any other value (like the default `None`) to match either.
3803
3804        TODO: Support restrictions on arguments used?
3805        """
3806        super().__init__(**kwargs)
3807        self.name = name
3808        self.isMethod = isMethod
3809
3810    def structureString(self):
3811        if self.name is not None:
3812            if self.isMethod is True:
3813                result = f"call(s) to ?.{self.name}"
3814            else:
3815                result = f"call(s) to {self.name}"
3816        else:
3817            if self.isMethod is True:
3818                result = "method call(s)"
3819            elif self.isMethod is False:
3820                result = "non-method function call(s)"
3821            else:
3822                result = "function call(s)"
3823
3824        return result
3825
3826    def _nodesToCheck(self, syntaxTree):
3827        # Note that are calls whose name doesn't match are not considered
3828        # matches at all, while calls which are/aren't methods are still
3829        # considered partial matches even when isMethod indicates they
3830        # should be the opposite. Also note that only calls whose
3831        # function expression is either a Name or an Attribute will match
3832        # if isMethod is specified (one way or the other) or name is not
3833        # None. Things like lambdas, if/else expression results, or
3834        # subscripts won't match because they don't really have a name,
3835        # and they're not really specifically methods or not methods.
3836
3837        # If no specific requirements are present, then we can simply
3838        # yield all of the Call nodes
3839        if (
3840            self.isMethod is not True
3841        and self.isMethod is not False
3842        and self.name is None
3843        ):
3844            for node in self._walkNodesOfType(syntaxTree, ast.Call):
3845                yield (node, True)
3846        else:
3847            # Otherwise only call nodes w/ Name or Attribute expressions
3848            # can match
3849            for node in self._walkNodesOfType(syntaxTree, ast.Call):
3850                # Figure out the name and/or method status of the thing being
3851                # called:
3852                funcExpr = node.func
3853
3854                # Unwrap any := assignments to get at the actual function
3855                # object being used
3856                if HAS_WALRUS:
3857                    while isinstance(funcExpr, ast.NamedExpr):
3858                        funcExpr = funcExpr.value
3859
3860                # Handle name vs. attr nodes
3861                if isinstance(funcExpr, ast.Name):
3862                    name = funcExpr.id
3863                    method = False
3864                elif isinstance(funcExpr, ast.Attribute):
3865                    name = funcExpr.attr
3866                    method = True
3867                else:
3868                    # Only Name and Attribute nodes can actually be checked
3869                    # for details, so other matches are ignored
3870                    continue
3871
3872                if self.name is None or self.name == name:
3873                    # "is not not" is actually correct here...
3874                    yield (node, self.isMethod is not (not method))
3875
3876
3877def anyNameMatches(nameToMatch, targetsList):
3878    """
3879    Recursive function for matching assigned names within target
3880    tuple/list AST structures.
3881    """
3882    for target in targetsList:
3883        if isinstance(target, ast.Name) and target.id == nameToMatch:
3884            return True
3885        elif isinstance(target, (ast.List, ast.Tuple)):
3886            if anyNameMatches(nameToMatch, target.elts):
3887                return True
3888        # Any other kind of node is ignored
3889
3890    return False
3891
3892
3893class Assign(ASTRequirement):
3894    """
3895    Matches an assignment, possibly to a variable with a specific name.
3896    By default augmented assignments and assignments via named
3897    expressions are allowed, but these may be disallowed or required.
3898    Assignments of disallowed types are still counted as partial matches
3899    if their name matches or if no name was specified.
3900
3901    Assignments to things other than variables (like list slots) will not
3902    match when a variable name is specified.
3903
3904    Note that the entire assignment node is matched, so you can use
3905    `contains` to specify checks to apply to the expression (plus the
3906    target, but usually that's fine).
3907
3908    In cases where a tuple assignment is made, if any of the assigned
3909    names matches the required name, the entire tuple assignment is
3910    considered a match, since it may not be possible to pick apart the
3911    right-hand side to find a syntactic node that was assigned to just
3912    that variable. This can lead to some weird matches, for example,
3913
3914    >>> import optimism
3915    >>> optimism.messagesAsErrors(False)
3916    >>> optimism.colors(False)
3917    >>> tester = optimism.testBlock("x, (y, z) = 1, (3, 5)")
3918    >>> tester.checkCodeContains(
3919    ...     optimism.Assign('x').contains(optimism.Constant(5))
3920    ... ) # doctest: +ELLIPSIS
3921    ✓ ...
3922    True
3923    """
3924    def __init__(
3925        self,
3926        name=None,
3927        isAugmented=None,
3928        isNamedExpr=None,
3929        **kwargs
3930    ):
3931        """
3932        The first argument specifies the variable name. Leave it as
3933        `None` (the default) to allow any assignment to match.
3934
3935        `isAugmented` specifies whether augmented assignments (e.g., +=)
3936        are considered matches or not; `False` disallows them, `True`
3937        will only match them, and any other value (like the default
3938        `None`) will allow them and other assignment types.
3939
3940        `isNamedExpr` works the same way for controlling whether named
3941        expressions (:=) are permitted. A `ValueError` will be raised if
3942        both `isAugmented` and `isNamedExpr` are set to true, since named
3943        expressions can't be augmented.
3944
3945        TODO: Allow checking for assignment to fields?
3946        """
3947        if isAugmented is True and isNamedExpr is True:
3948            raise ValueError(
3949                "Both isAugmented and isNamedExpr cannot be set to True"
3950                " at once, since no assignments would match in that"
3951                " case."
3952            )
3953
3954        super().__init__(**kwargs)
3955        self.name = name
3956        self.isAugmented = isAugmented
3957        self.isNamedExpr = isNamedExpr
3958
3959    def structureString(self):
3960        if self.name is None:
3961            if self.isAugmented is True:
3962                result = "augmented assignment statement(s)"
3963            elif self.isNamedExpr is True:
3964                result = "assignment(s) via named expression(s)"
3965            else:
3966                result = "assignment(s)"
3967        else:
3968            if self.isAugmented is True:
3969                result = f"augmented assignment(s) to {self.name}"
3970            elif self.isNamedExpr is True:
3971                result = f"named assignment(s) to {self.name}"
3972            else:
3973                result = f"assignment(s) to {self.name}"
3974
3975        if self.isAugmented is False:
3976            result += " (not augmented)"
3977
3978        if self.isNamedExpr is False:
3979            result += " (not via named expression(s))"
3980
3981        return result
3982
3983    def _nodesToCheck(self, syntaxTree):
3984        # Consider all Assign, AugAssign, AnnAssign, and NamedExpr nodes
3985        matchTypes = (ast.Assign, ast.AugAssign, ast.AnnAssign)
3986        if HAS_WALRUS:
3987            matchTypes += (ast.NamedExpr,)
3988        for node in self._walkNodesOfType(
3989            syntaxTree,
3990            matchTypes
3991        ):
3992            # Figure out the name and/or method status of the thing being
3993            # called:
3994            if self.name is None:
3995                nameMatch = True
3996            else:
3997                if isinstance(node, ast.Assign):
3998                    nameMatch = anyNameMatches(self.name, node.targets)
3999                else:
4000                    nameExpr = node.target
4001                    nameMatch = (
4002                        isinstance(nameExpr, ast.Name)
4003                    and nameExpr.id == self.name
4004                    )
4005
4006            augmented = isinstance(node, ast.AugAssign)
4007            namedExpr = HAS_WALRUS and isinstance(node, ast.NamedExpr)
4008
4009            if (
4010                nameMatch
4011            and self.isAugmented is not (not augmented)
4012            and self.isNamedExpr is not (not namedExpr)
4013            ):
4014                yield (node, True)
4015            elif nameMatch:
4016                yield (node, False)
4017
4018
4019class Reference(ASTRequirement):
4020    """
4021    Matches a variable reference, possibly to a variable with a specific
4022    name. By default attribute accesses with the given name will also be
4023    matched (e.g., both 'pi' and 'math.pi' will match for the name 'pi').
4024    You may specify that only attributes should match or that attributes
4025    should not match; matches that violate that specification will still
4026    be partial matches.
4027
4028    >>> import optimism
4029    >>> optimism.messagesAsErrors(False)
4030    >>> optimism.colors(False)
4031    >>> tester = optimism.testBlock("x = 5\\ny = x * math.pi")
4032    >>> tester.checkCodeContains(
4033    ...     optimism.Reference('x')
4034    ... ) # doctest: +ELLIPSIS
4035    ✓ ...
4036    True
4037    >>> tester.checkCodeContains(
4038    ...     optimism.Reference('y')
4039    ... ) # doctest: +ELLIPSIS
4040    ✗ ...
4041    False
4042    >>> tester.checkCodeContains(
4043    ...     optimism.Reference('pi')
4044    ... ) # doctest: +ELLIPSIS
4045    ✓ ...
4046    True
4047    >>> tester.checkCodeContains(
4048    ...     optimism.Reference('x', attribute=True)
4049    ... ) # doctest: +ELLIPSIS
4050    ✗ ...
4051    False
4052    >>> tester.checkCodeContains(
4053    ...     optimism.Reference('pi', attribute=True)
4054    ... ) # doctest: +ELLIPSIS
4055    ✓ ...
4056    True
4057    >>> tester.checkCodeContains(
4058    ...     optimism.Reference('pi', attribute=False)
4059    ... ) # doctest: +ELLIPSIS
4060    ✗ ...
4061    False
4062    """
4063    def __init__(
4064        self,
4065        name=None,
4066        attribute=None,
4067        **kwargs
4068    ):
4069        """
4070        The first argument specifies the variable name. Leave it as
4071        `None` (the default) to allow any assignment to match.
4072
4073        The second argument specifies whether the reference must be to
4074        an attribute with that name (if `True`), or to a regular
4075        variable with that name (if `False`). Leave it as the default
4076        `None` to allow matches for either.
4077        """
4078        super().__init__(**kwargs)
4079        self.name = name
4080        self.attribute = attribute
4081
4082    def structureString(self):
4083        if self.name is None:
4084            if self.attribute is True:
4085                result = "attribute reference(s)"
4086            elif self.attribute is False:
4087                result = "non-attribute variable reference(s)"
4088            else:
4089                result = "variable reference(s)"
4090        else:
4091            if self.attribute is True:
4092                result = f"reference(s) to .{self.name}"
4093            else:
4094                result = f"reference(s) to {self.name}"
4095
4096        return result
4097
4098    def _nodesToCheck(self, syntaxTree):
4099        # Consider all Name and Attribute nodes
4100        for node in self._walkNodesOfType(
4101            syntaxTree,
4102            (ast.Name, ast.Attribute)
4103        ):
4104            # Only match references being loaded (use `Assign` for
4105            # variables being assigned).
4106            if not isinstance(node.ctx, ast.Load):
4107                continue
4108
4109            # Figure out whether the name matches:
4110            if self.name is None:
4111                nameMatch = True
4112            else:
4113                if isinstance(node, ast.Name):
4114                    nameMatch = node.id == self.name
4115                else:  # must be en Attribute
4116                    nameMatch = node.attr == self.name
4117
4118            if self.attribute is None:
4119                typeMatch = True
4120            else:
4121                if self.attribute is True:
4122                    typeMatch = isinstance(node, ast.Attribute)
4123                elif self.attribute is False:
4124                    typeMatch = isinstance(node, ast.Name)
4125
4126            if nameMatch and typeMatch:
4127                yield (node, True)
4128            elif nameMatch:
4129                yield (node, False)
4130
4131
4132class Class(ASTRequirement):
4133    """
4134    Matches a class definition, possibly with a specific name.
4135    """
4136    def __init__(self, name=None, **kwargs):
4137        """
4138        A name my be specified; `None` (the default) will match any class
4139        definition.
4140        """
4141        super().__init__(**kwargs)
4142        self.name = name
4143
4144    def structureString(self):
4145        if self.name is not None:
4146            return f"class definition(s) for {self.name}"
4147        else:
4148            return "class definition(s)"
4149
4150    def _nodesToCheck(self, syntaxTree):
4151        # Consider just ClassDef nodes; all such nodes are considered as
4152        # least partial matches.
4153        for node in self._walkNodesOfType(syntaxTree, ast.ClassDef):
4154            yield (node, self.name is None or node.name == self.name)
4155
4156
4157class IfElse(ASTRequirement):
4158    """
4159    Matches a single if or elif, possibly with an else attached. In an
4160    if/elif/else construction, it will match on the initial if plus on
4161    each elif, since Python treats them as nested if/else nodes. Also
4162    matches if/else expression nodes, although this can be disabled or
4163    required.
4164    """
4165    def __init__(self, onlyExpr=None, **kwargs):
4166        """
4167        Set `onlyExpr` to `False` to avoid matching if/else expression
4168        nodes; set it to `True` to only match those nodes; set it to
4169        anything else to match both normal and expression if/else.
4170        """
4171        super().__init__(**kwargs)
4172        self.onlyExpr = onlyExpr
4173
4174    def structureString(self):
4175        if self.onlyExpr is True:
4176            return "if/else expression(s)"
4177        elif self.onlyExpr is False:
4178            return "if/else statement(s)"
4179        else:
4180            return "if/else statement(s) or expression(s)"
4181
4182    def _nodesToCheck(self, syntaxTree):
4183        # Consider If and IfExp nodes
4184        for node in self._walkNodesOfType(syntaxTree, (ast.If, ast.IfExp)):
4185            if self.onlyExpr is False:
4186                full = isinstance(node, ast.If)
4187            elif self.onlyExpr is True:
4188                full = isinstance(node, ast.IfExp)
4189            else:
4190                full = True
4191
4192            yield (node, full)
4193
4194
4195class Loop(ASTRequirement):
4196    """
4197    Matches for and while loops, asynchronous versions of those loops,
4198    and also generator expressions and list/dict/set comprehensions. Can
4199    be restricted to match only some of those things, although all of
4200    them are always considered at least partial matches.
4201    """
4202    def __init__(self, only=None, **kwargs):
4203        """
4204        The `only` argument can be used to narrow what is matched, it
4205        should be a single string, or a set (or some other iterable) of
4206        strings, from the following list:
4207
4208            - "for" - for loops
4209            - "async for" - asynchronous for loops
4210            - "while" - while loops
4211            - "generator" - generator expressions (NOT in comprehensions)
4212            - "list comp" - list comprehensions
4213            - "dict comp" - dictionary comprehensions
4214            - "set comp" - set comprehensions
4215
4216        A few extra strings can be used as shortcuts for groups from the
4217        list above:
4218
4219            - "any generator" - generator expressions and list/dict/set
4220                comprehensions
4221            - "non-generator" - any non-generator non-comprehension
4222            - "non-async" - any kind except async for
4223
4224        A `ValueError` will be raised if an empty `only` set is provided;
4225        leave it as `None` (the default) to allow any kind of looping
4226        construct to match. A `ValueError` will also be raised if the
4227        `only` set contains any strings not listed above.
4228        """
4229        super().__init__(**kwargs)
4230
4231        if only is not None:
4232            if isinstance(only, str):
4233                only = { only }
4234            else:
4235                only = set(only)
4236
4237            if "any generator" in only:
4238                only.add("generator")
4239                only.add("list comp")
4240                only.add("dict comp")
4241                only.add("set comp")
4242                only.remove("any generator")
4243
4244            if "non-generator" in only:
4245                only.add("for")
4246                only.add("async for")
4247                only.add("while")
4248                only.remove("non-generator")
4249
4250            if "non-async" in only:
4251                only.add("for")
4252                only.add("while")
4253                only.add("generator")
4254                only.add("list comp")
4255                only.add("dict comp")
4256                only.add("set comp")
4257                only.remove("non-async")
4258
4259        self.only = only
4260
4261        if only is not None:
4262            invalid = only - {
4263                "for", "async for", "while", "generator", "list comp",
4264                "dict comp", "set comp"
4265            }
4266            if len(invalid) > 0:
4267                raise ValueError(
4268                    f"One or more invalid loop types was specified for"
4269                    f" 'only': {invalid}"
4270                )
4271
4272            if len(only) == 0:
4273                raise ValueError(
4274                    "At least one type of loop must be specified when"
4275                    " 'only' is used (leave it as None to allow all loop"
4276                    " types."
4277                )
4278
4279    def structureString(self):
4280        if self.only is None:
4281            return "loop(s) or generator expression(s)"
4282        elif self.only == {"for"} or self.only == {"for", "async for"}:
4283            return "for loop(s)"
4284        elif self.only == {"async for"}:
4285            return "async for loop(s)"
4286        elif self.only == {"while"}:
4287            return "while loop(s)"
4288        elif self.only == {"generator"}:
4289            return "generator expression(s)"
4290        elif self.only == {"list comp"}:
4291            return "list comprehension(s)"
4292        elif self.only == {"dict comp"}:
4293            return "dictionary comprehension(s)"
4294        elif self.only == {"set comp"}:
4295            return "set comprehension(s)"
4296        elif len(
4297            self.only - {"for", "async for", "while"}
4298        ) == 0:
4299            return "generator expression(s) or comprehension(s)"
4300        elif len(
4301            self.only - {"generator", "list comp", "dict comp", "set comp"}
4302        ) == 0:
4303            return (
4304                "for or while loop(s) (not generator expression(s) or"
4305                " comprehension(s))"
4306            )
4307
4308    def _nodesToCheck(self, syntaxTree):
4309        allIterationTypes = (
4310            ast.For,
4311            ast.AsyncFor,
4312            ast.While,
4313            ast.GeneratorExp,
4314            ast.ListComp,
4315            ast.DictComp,
4316            ast.SetComp
4317        )
4318        if self.only is not None:
4319            allowed = tuple([
4320                {
4321                    "for": ast.For,
4322                    "async for": ast.AsyncFor,
4323                    "while": ast.While,
4324                    "generator": ast.GeneratorExp,
4325                    "list comp": ast.ListComp,
4326                    "dict comp": ast.DictComp,
4327                    "set comp": ast.SetComp,
4328                }[item]
4329                for item in self.only
4330            ])
4331
4332        for node in self._walkNodesOfType(syntaxTree, allIterationTypes):
4333            if self.only is None or isinstance(node, allowed):
4334                yield (node, True)
4335            else:
4336                # If only some types are required, other types still
4337                # count as partial matches
4338                yield (node, False)
4339
4340
4341class Return(ASTRequirement):
4342    """
4343    Matches a return statement. An expression may be required or
4344    forbidden, but by default returns with or without expressions count.
4345    """
4346    def __init__(self, requireExpr=None, **kwargs):
4347        """
4348        `requireExpr` controls whether a return expression is
4349        allowed/required. Set to `True` to require one, or `False` to
4350        forbid one, and any other value (such as the default `None`) to
4351        match returns with or without an expression.
4352        """
4353        super().__init__(**kwargs)
4354        self.requireExpr = requireExpr
4355
4356    def structureString(self):
4357        if self.requireExpr is False:
4358            return "return statement(s) (without expression(s))"
4359        else:
4360            return "return statement(s)"
4361
4362    def _nodesToCheck(self, syntaxTree):
4363        for node in self._walkNodesOfType(syntaxTree, ast.Return):
4364            if self.requireExpr is True:
4365                full = node.value is not None
4366            elif self.requireExpr is False:
4367                full = node.value is None
4368            else:
4369                full = True
4370
4371            yield (node, full)
4372
4373
4374class Try(ASTRequirement):
4375    """
4376    Matches try/except/finally nodes. The presence of except, finally,
4377    and/or else clauses may be required or forbidden, although all
4378    try/except/finally nodes are counted as at least partial matches.
4379    """
4380    def __init__(
4381        self,
4382        requireExcept=None,
4383        requireFinally=None,
4384        requireElse=None,
4385        **kwargs
4386    ):
4387        """
4388        `requireExcept`, `requireFinally`, and `requireElse` are used to
4389        specify whether those blocks must be present, must not be
4390        present, or are neither required nor forbidden. Use `False` for
4391        to forbid matches with that block and `True` to only match
4392        constructs with that block. Any other value (like the default
4393        `None` will ignore the presence or absence of that block. A
4394        `ValueError` will be raised if both `requireExcept` and
4395        `requireFinally` are set to `False`, as a `try` block must have
4396        at least one or the other to be syntactically valid. Similarly,
4397        if `requireElse` is set to `True`, `requireExcept` must not be
4398        `False` (and syntactically, `else` can only be used when `except`
4399        is present).
4400        """
4401        super().__init__(**kwargs)
4402        if requireExcept is False and requireFinally is False:
4403            raise ValueError(
4404                "Cannot require that neither 'except' nor 'finally' is"
4405                " present on a 'try' statement, as one or the other will"
4406                " always be present."
4407            )
4408
4409        if requireElse is True and requireExcept is False:
4410            raise ValueError(
4411                "Cannot require that 'else' be present on a 'try'"
4412                " statement while also requiring that 'except' not be"
4413                " present, since 'else' cannot be used without 'except'."
4414            )
4415
4416        self.requireExcept = requireExcept
4417        self.requireFinally = requireFinally
4418        self.requireElse = requireElse
4419
4420    def structureString(self):
4421        result = "try statement(s)"
4422        if self.requireExcept is not False:
4423            result += " (with except block(s))"
4424        if self.requireElse is True:
4425            result += " (with else block(s))"
4426        if self.requireFinally is True:
4427            result += " (with finally block(s))"
4428        return result
4429
4430    def _nodesToCheck(self, syntaxTree):
4431        # All try/except/finally statements count as matches, but ones
4432        # missing required clauses or which have forbidden clauses count
4433        # as partial matches.
4434        for node in self._walkNodesOfType(syntaxTree, ast.Try):
4435            full = True
4436            if self.requireExcept is True and len(node.handlers) == 0:
4437                full = False
4438            if self.requireExcept is False and len(node.handlers) > 0:
4439                full = False
4440            if self.requireElse is True and len(node.orelse) == 0:
4441                full = False
4442            if self.requireElse is False and len(node.orelse) > 0:
4443                full = False
4444            if self.requireFinally is True and len(node.finalbody) == 0:
4445                full = False
4446            if self.requireFinally is False and len(node.finalbody) > 0:
4447                full = False
4448
4449            yield (node, full)
4450
4451
4452class With(ASTRequirement):
4453    """
4454    Matches a `with` or `async with` block. Async may be required or
4455    forbidden, although either form will always be considered at least a
4456    partial match.
4457    """
4458    def __init__(self, onlyAsync=None, **kwargs):
4459        """
4460        `onlyAsync` should be set to `False` to disallow `async with`
4461        blocks, `True` to match only async blocks, and any other value
4462        (like the default `None`) to match both normal and async blocks.
4463        """
4464        super().__init__(**kwargs)
4465        self.onlyAsync = onlyAsync
4466
4467    def structureString(self):
4468        if self.onlyAsync is True:
4469            return "async with statement(s)"
4470        else:
4471            return "with statement(s)"
4472
4473    def _nodesToCheck(self, syntaxTree):
4474        for node in self._walkNodesOfType(
4475            syntaxTree,
4476            (ast.With, ast.AsyncWith)
4477        ):
4478            yield (
4479                node,
4480                self.onlyAsync is not (not isinstance(node, ast.AsyncWith))
4481                # 'not not' is intentional here
4482            )
4483
4484
4485class AnyValue:
4486    """
4487    Represents the situation where any value can be accepted for a
4488    node in a `Constant` or `Literal` `ASTRequirement`. Also used to
4489    represent a `getLiteralValue` where we don't know the value.
4490    """
4491    pass
4492
4493
4494class AnyType:
4495    """
4496    Represents the situation where any type can be accepted for a
4497    node in a `Constant` or `Literal` `ASTRequirement`.
4498    """
4499
4500
4501class Constant(ASTRequirement):
4502    """
4503    A check for a constant, possibly with a specific value and/or of a
4504    specific type. All constants are considered partial matches.
4505
4506    Note that this cannot match literal tuples, lists, sets,
4507    dictionaries, etc.; only simple constants. Use `Literal` instead for
4508    literal lists, tuples, sets, or dictionaries.
4509    """
4510    def __init__(self, value=AnyValue, types=AnyType, **kwargs):
4511        """
4512        A specific value may be supplied (including `None`) or else any
4513        value will be accepted if the `AnyValue` class (not an instance
4514        of it) is used as the argument (this is the default).
4515
4516        If the value is `AnyValue`, `types` may be specified, and only
4517        constants with that type will match. `type` may be a tuple (but
4518        not list) of types or a single type, as with `isinstance`.
4519
4520        Even if a specific value is specified, the type check is still
4521        applied, since it's possible to create a value that checks equal
4522        to values from more than one type. For example, specifying
4523        `Constant(6)` will match both 6 and 6.0 but `Constant(6, float)`
4524        will only match the latter.
4525        """
4526        super().__init__(**kwargs)
4527        self.value = value
4528        self.types = types
4529
4530        # Allowed types for constants (ignoring doc which claims tuples
4531        # or frozensets can be Constant values)
4532        allowed = (int, float, complex, bool, NoneType, str)
4533
4534        # value-type and type-type checking
4535        if value is not AnyValue and type(value) not in allowed:
4536            raise TypeError(
4537                f"Value {value!r} has type {type(value)} which is not a"
4538                f" type that a Constant can be (did you mean to use a"
4539                f" Literal instead?)."
4540            )
4541
4542        if self.types is not AnyType:
4543            if isinstance(self.types, tuple):
4544                for typ in self.types:
4545                    if typ not in allowed:
4546                        raise TypeError(
4547                            f"Type {typ} has is not a type that a"
4548                            f" Constant can be (did you mean to use a"
4549                            f" Literal instead?)."
4550                        )
4551            else:
4552                if self.types not in allowed:
4553                    raise TypeError(
4554                        f"Type {self.types} has is not a type that a"
4555                        f" Constant can be (did you mean to use a"
4556                        f" Literal instead?)."
4557                    )
4558
4559    def structureString(self):
4560        if self.value == AnyValue:
4561            if self.types == AnyType:
4562                return "constant(s)"
4563            else:
4564                if isinstance(self.types, tuple):
4565                    types = (
4566                        ', '.join(t.__name__ for t in self.types[:-1])
4567                      + ' or ' + self.types[-1].__name__
4568                    )
4569                    return f"{types} constant(s)"
4570                else:
4571                    return f"{self.types.__name__} constant(s)"
4572        else:
4573            return f"constant {repr(self.value)}"
4574
4575    def _nodesToCheck(self, syntaxTree):
4576        # ALL Constants w/ values/types other than what was expected are
4577        # considered partial matches.
4578        if SPLIT_CONSTANTS:
4579            for node in self._walkNodesOfType(
4580                syntaxTree,
4581                (ast.Num, ast.Str, ast.Bytes, ast.NameConstant, ast.Constant)
4582            ):
4583                if isinstance(node, ast.Num):
4584                    val = node.n
4585                elif isinstance(node, (ast.Str, ast.Bytes)):
4586                    val = node.s
4587                elif isinstance(node, (ast.NameConstant, ast.Constant)):
4588                    val = node.value
4589
4590                valMatch = (
4591                    self.value == AnyValue
4592                 or val == self.value
4593                )
4594
4595                typeMatch = (
4596                    self.types == AnyType
4597                 or isinstance(val, self.types)
4598                )
4599
4600                yield (node, valMatch and typeMatch)
4601        else:
4602            for node in self._walkNodesOfType(syntaxTree, ast.Constant):
4603                valMatch = (
4604                    self.value == AnyValue
4605                 or node.value == self.value
4606                )
4607
4608                typeMatch = (
4609                    self.types == AnyType
4610                 or isinstance(node.value, self.types)
4611                )
4612
4613                yield (node, valMatch and typeMatch)
4614
4615
4616def getLiteralValue(astNode):
4617    """
4618    For an AST node that's entirely made up of `Constant` and/or
4619    `Literal` nodes, extracts the value of that node from the AST. For
4620    nodes which have things like variable references in them whose
4621    values are not determined by the AST alone, returns `AnyValue` (the
4622    class itself, not an instance).
4623
4624    Examples:
4625
4626    >>> node = ast.parse('[1, 2, 3]').body[0].value
4627    >>> type(node).__name__
4628    'List'
4629    >>> getLiteralValue(node)
4630    [1, 2, 3]
4631    >>> node = ast.parse("('string', 4, {5: (6, 7)})").body[0].value
4632    >>> getLiteralValue(node)
4633    ('string', 4, {5: (6, 7)})
4634    >>> node = ast.parse("(variable, 4, {5: (6, 7)})").body[0].value
4635    >>> getLiteralValue(node) # can't determine value from AST
4636    <class 'optimism.AnyValue'>
4637    >>> node = ast.parse("[x for x in range(3)]").body[0].value
4638    >>> getLiteralValue(node) # not a literal or constant
4639    <class 'optimism.AnyValue'>
4640    >>> node = ast.parse("[1, 2, 3][0]").body[0].value
4641    >>> getLiteralValue(node) # not a literal or constant
4642    <class 'optimism.AnyValue'>
4643    >>> getLiteralValue(node.value) # the value part is though
4644    [1, 2, 3]
4645    """
4646    # Handle constant node types depending on SPLIT_CONSTANTS
4647    if SPLIT_CONSTANTS:
4648        if isinstance(astNode, ast.Num):
4649            return astNode.n
4650        elif isinstance(astNode, (ast.Str, ast.Bytes)):
4651            return astNode.s
4652        elif isinstance(astNode, (ast.NameConstant, ast.Constant)):
4653            return astNode.value
4654        # Else check literal types below
4655    else:
4656        if isinstance(astNode, ast.Constant):
4657            return astNode.value
4658        # Else check literal types below
4659
4660    if isinstance(astNode, (ast.List, ast.Tuple, ast.Set)):
4661        result = []
4662        for elem in astNode.elts:
4663            subValue = getLiteralValue(elem)
4664            if subValue is AnyValue:
4665                return AnyValue
4666            result.append(subValue)
4667        return {
4668            ast.List: list,
4669            ast.Tuple: tuple,
4670            ast.Set: set
4671        }[type(astNode)](result)
4672
4673    elif isinstance(astNode, ast.Dict):
4674        result = {}
4675        for index in range(len(astNode.keys)):
4676            kv = getLiteralValue(astNode.keys[index])
4677            vv = getLiteralValue(astNode.values[index])
4678            if kv is AnyValue or vv is AnyValue:
4679                return AnyValue
4680            result[kv] = vv
4681        return result
4682
4683    else:
4684        return AnyValue
4685
4686
4687class Literal(ASTRequirement):
4688    """
4689    A check for a complex literal possibly with a specific value and/or
4690    of a specific type. All literals of the appropriate type(s) are
4691    considered partial matches even when a specific value is supplied,
4692    and list/tuple literals are both considered together for these
4693    partial matches.
4694
4695    Note that this cannot match string, number, or other constants, use
4696    `Constant` for that.
4697    """
4698    def __init__(self, value=AnyValue, types=AnyType, **kwargs):
4699        """
4700        A specific value may be supplied (it must be a list, tuple, set,
4701        or dictionary) or else any value will be accepted if the
4702        `AnyValue` class (not an instance of it) is used as the argument
4703        (that is the default).
4704
4705        If the value is `AnyValue`, one or more `types` may be
4706        specified, and only literals with that type will match. `types`
4707        may be a tuple (but not list) of types or a single type, as with
4708        `isinstance`. Matched nodes will always have a value which is one
4709        of the following types: `list`, `tuple`, `set`, or `dict`.
4710
4711        If both a specific value and a type or tuple of types is
4712        specified, any collection whose members match the members of the
4713        specific value supplied and whose type is one of the listed types
4714        will match. For example, `Literal([1, 2], types=(list, tuple,
4715        set))` will match any of `[1, 2]`, `(1, 2)`, or `{2, 1}` but will
4716        NOT match `[2, 1]`, `(2, 1)`, or any dictionary.
4717
4718        Specifically, the value is converted to match the type of the
4719        node being considered and then a match is checked, so for
4720        example, `Literal([1, 2, 2], types=set)` will match the set `{1,
4721        2}` and the equivalent sets `{2, 1}` and `{1, 1, 2}`.
4722
4723        If a node has elements which aren't constants or literals, it
4724        will never match when a value is provided because we don't
4725        evaluate code during matching. It might still match if only
4726        type(s) are provided, of course.
4727        """
4728        super().__init__(**kwargs)
4729        self.value = value
4730        self.types = types
4731
4732        # Allowed types for literals
4733        allowed = (list, tuple, set, dict)
4734
4735        # value-type and type-type checking
4736        if value is not AnyValue and type(value) not in allowed:
4737            raise TypeError(
4738                f"Value {value!r} has type {type(value)} which is not a"
4739                f" type that a Literal can be (did you mean to use a"
4740                f" Constant instead?)."
4741            )
4742
4743        if self.types is not AnyType:
4744            if isinstance(self.types, tuple):
4745                for typ in self.types:
4746                    if typ not in allowed:
4747                        raise TypeError(
4748                            f"Type {typ} has is not a type that a"
4749                            f" Literal can be (did you mean to use a"
4750                            f" Constant instead?)."
4751                        )
4752            else:
4753                if self.types not in allowed:
4754                    raise TypeError(
4755                        f"Type {self.types} has is not a type that a"
4756                        f" Literal can be (did you mean to use a"
4757                        f" Constant instead?)."
4758                    )
4759
4760    def structureString(self):
4761        if self.value == AnyValue:
4762            if self.types == AnyType:
4763                return "literal(s)"
4764            else:
4765                if isinstance(self.types, tuple):
4766                    types = (
4767                        ', '.join(t.__name__ for t in self.types[:-1])
4768                      + ' or ' + self.types[-1].__name__
4769                    )
4770                    return f"{types} literal(s)"
4771                else:
4772                    return f"{self.types.__name__} literal(s)"
4773        else:
4774            return f"literal {repr(self.value)}"
4775
4776    def _nodesToCheck(self, syntaxTree):
4777        # Some literals might be considered partial matches
4778        for node in self._walkNodesOfType(
4779            syntaxTree,
4780            (ast.List, ast.Tuple, ast.Set, ast.Dict)
4781        ):
4782            # First, get the value of the node. This will be None if
4783            # it's not computable from the AST alone.
4784            value = getLiteralValue(node)
4785
4786            valType = type(value)
4787            if value is None:
4788                valType = {
4789                    ast.List: list,
4790                    ast.Tuple: tuple,
4791                    ast.Set: set,
4792                    ast.Dict: dict,
4793                }[type(node)]
4794
4795            # Next, determine whether we have something that counts as a
4796            # partial match, and if we don't, continue to the next
4797            # potential match.
4798            partial = False
4799            partialTypes = self.types
4800            if partialTypes is AnyType:
4801                if self.value is not AnyValue:
4802                    partialTypes = (type(self.value),)
4803                else:
4804                    partial = True
4805
4806            # Only keep checking if we aren't already sure it's a
4807            # partial match
4808            if not partial:
4809                if not isinstance(partialTypes, tuple):
4810                    partialTypes = (partialTypes,)
4811
4812                # List and tuple imply each other for partials
4813                if list in partialTypes and tuple not in partialTypes:
4814                    partialTypes = partialTypes + (tuple,)
4815                if tuple in partialTypes and list not in partialTypes:
4816                    partialTypes = partialTypes + (list,)
4817
4818                partial = issubclass(valType, partialTypes)
4819
4820            # Skip this match entirely if it doesn't qualify as at least
4821            # a partial match.
4822            if not partial:
4823                continue
4824
4825            # Now check for a value match
4826            if self.value is AnyValue:
4827                valMatch = True
4828            elif value is None:
4829                valMatch = False
4830            elif self.types is AnyType:
4831                valMatch = value == self.value
4832            else:
4833                check = self.types
4834                if not isinstance(check, tuple):
4835                    check = (check,)
4836
4837                valMatch = (
4838                    isinstance(value, check)
4839                and type(value)(self.value) == value
4840                )
4841
4842            typeMatch = (
4843                self.types == AnyType
4844             or issubclass(valType, self.types)
4845            )
4846
4847            # Won't get here unless it's a partial match
4848            yield (node, valMatch and typeMatch)
4849
4850
4851class Operator(ASTRequirement):
4852    """
4853    A check for a unary operator, binary operator, boolean operator, or
4854    comparator. 'Similar' operations will count as partial matches. Note
4855    that 'is' and 'is not' are categorized as the same operator, as are
4856    'in' and 'not in'.
4857    """
4858    def __init__(self, op='+', **kwargs):
4859        """
4860        A specific operator must be specified. Use the text you'd write
4861        in Python to perform that operation (e.g., '//', '<=', or
4862        'and'). The two ambiguous cases are + and - which have both
4863        binary and unary forms. Add a 'u' beforehand to get their unary
4864        forms. Note that 'not in' and 'is not' are both allowed, but they
4865        are treated the same as 'in' and 'is'.
4866        """
4867        super().__init__(**kwargs)
4868        self.op = op
4869        # Determine correct + similar types
4870        typesToMatch = {
4871            'u+': ((ast.UAdd,), ()),
4872            'u-': ((ast.USub,), ()),
4873            'not': ((ast.Not,), ()),
4874            '~': ((ast.Invert,), ()),
4875            '+': ((ast.Add,), (ast.Sub,)),
4876            '-': ((ast.Sub,), (ast.Add,)),
4877            '*': ((ast.Mult,), (ast.Div,)),
4878            '/': ((ast.Div,), (ast.Mult)),
4879            '//': ((ast.FloorDiv,), (ast.Mod, ast.Div,)),
4880            '%': ((ast.Mod,), (ast.Div, ast.FloorDiv,)),
4881            '**': ((ast.Pow,), (ast.Mult,)),
4882            '<<': ((ast.LShift,), (ast.RShift,)),
4883            '>>': ((ast.RShift,), (ast.LShift,)),
4884            '|': ((ast.BitOr,), (ast.BitXor, ast.BitAnd)),
4885            '^': ((ast.BitXor,), (ast.BitOr, ast.BitAnd)),
4886            '&': ((ast.BitAnd,), (ast.BitXor, ast.BitOr)),
4887            '@': ((ast.MatMult,), (ast.Mult,)),
4888            'and': ((ast.And,), (ast.Or,)),
4889            'or': ((ast.Or,), (ast.And,)),
4890            '==': ((ast.Eq,), (ast.NotEq, ast.Is, ast.IsNot)),
4891            '!=': ((ast.NotEq,), (ast.Eq, ast.Is, ast.IsNot)),
4892            '<': ((ast.Lt,), (ast.LtE, ast.Gt, ast.GtE)),
4893            '<=': ((ast.LtE,), (ast.Lt, ast.Gt, ast.GtE)),
4894            '>': ((ast.Gt,), (ast.Lt, ast.LtE, ast.GtE)),
4895            '>=': ((ast.GtE,), (ast.Lt, ast.LtE, ast.Gt)),
4896            'is': ((ast.Is, ast.IsNot), (ast.Eq, ast.NotEq)),
4897            'is not': ((ast.IsNot, ast.Is), (ast.Eq, ast.NotEq)),
4898            'in': ((ast.In, ast.NotIn), ()),
4899            'not in': ((ast.NotIn, ast.In), ()),
4900        }.get(op)
4901
4902        if typesToMatch is None:
4903            raise ValueError(f"Unrecognized operator '{op}'.")
4904
4905        self.opTypes, self.partialTypes = typesToMatch
4906
4907    def structureString(self):
4908        return f"operator '{self.op}'"
4909
4910    def _nodesToCheck(self, syntaxTree):
4911        for node in self._walkNodesOfType(
4912            syntaxTree,
4913            (ast.UnaryOp, ast.BinOp, ast.BoolOp, ast.Compare)
4914        ):
4915            # Determine not/partial/full status of match...
4916            match = False
4917            if isinstance(node, ast.Compare):
4918                if any(
4919                    isinstance(op, self.opTypes)
4920                    for op in node.ops
4921                ):
4922                    match = True
4923                elif match is False and any(
4924                    isinstance(op, self.partialTypes)
4925                    for op in node.ops
4926                ):
4927                    match = "partial"
4928            else:
4929                if isinstance(node.op, self.opTypes):
4930                    match = True
4931                elif (
4932                    match is False
4933                and isinstance(node.op, self.partialTypes)
4934                ):
4935                    match = "partial"
4936
4937            # Yield node if it's a partial or full match
4938            if match:
4939                yield (node, match is True)
4940
4941
4942class SpecificNode(ASTRequirement):
4943    """
4944    A flexible check where you can simply specify the AST node class(es)
4945    that you're looking for, plus a filter function to determine which
4946    matches are full/partial/non-matches. This does not perform any
4947    complicated sub-checks and doesn't have the cleanest structure
4948    string, so other `ASTRequirement` sub-classes are preferable if one
4949    of them can match what you want.
4950    """
4951    def __init__(self, nodeTypes, filterFunction=None, **kwargs):
4952        """
4953        Either a single AST node class (from the `ast` module, for
4954        example `ast.Break`) or a sequence of such classes is required to
4955        specify what counts as a match. If a sequence is provided, any of
4956        those node types will match; a `ValueError` will be raised if an
4957        empty sequence is provided.
4958
4959        If a filter function is provided, it will be called with an AST
4960        node as the sole argument for each node that has one of the
4961        specified types. If it returns exactly `True`, that node will be
4962        counted as a full match, if it returns exactly `False` that node
4963        will be counted as a partial match, and if it returns any other
4964        value (e.g., `None`) then that node will not be counted as a
4965        match at all.
4966        """
4967        super().__init__(**kwargs)
4968        if issubclass(nodeTypes, ast.AST):
4969            nodeTypes = (nodeTypes,)
4970        else:
4971            nodeTypes = tuple(nodeTypes)
4972            if len(nodeTypes) == 0:
4973                raise ValueError(
4974                    "Cannot specify an empty sequence of node types."
4975                )
4976            wrongTypes = tuple(
4977                [nt for nt in nodeTypes if not issubclass(nt, ast.AST)]
4978            )
4979            if len(wrongTypes) > 0:
4980                raise TypeError(
4981                    (
4982                        "All specified node types must be ast.AST"
4983                        " subclasses, but you provided some node types"
4984                        " that weren't:\n  "
4985                    ) + '\n  '.join(repr(nt) for nt in wrongTypes)
4986                )
4987
4988        self.nodeTypes = nodeTypes
4989        self.filterFunction = filterFunction
4990
4991    def structureString(self):
4992        if isinstance(self.nodeTypes, ast.AST):
4993            result = f"{self.nodeTypes.__name__} node(s)"
4994        elif len(self.nodeTypes) == 1:
4995            result = f"{self.nodeTypes[0].__name__} node(s)"
4996        elif len(self.nodeTypes) == 2:
4997            result = (
4998                f"either {self.nodeTypes[0].__name__} or"
4999                f" {self.nodeTypes[1].__name__} node(s)"
5000            )
5001        elif len(self.nodeTypes) > 2:
5002            result = (
5003                "node(s) that is/are:"
5004              + ', '.join(nt.__name__ for nt in self.nodeTypes[:-1])
5005              + ', or ' + self.nodeTypes[-1].__name__
5006            )
5007
5008        if self.filterFunction is not None:
5009            result += " (with additional criteria)"
5010
5011        return result
5012
5013    def _nodesToCheck(self, syntaxTree):
5014        for node in self._walkNodesOfType(syntaxTree, self.nodeTypes):
5015            if self.filterFunction is None:
5016                yield (node, True)
5017            else:
5018                matchStatus = self.filterFunction(node)
5019                if matchStatus in (True, False):
5020                    yield (node, matchStatus)
5021                # Otherwise (e.g., None) it's a non-match
5022
5023
5024# TODO: custom classes could be set up for:
5025# ast.Assert, ast.Delete, ast.Match, ast.Raise, ast.Global, ast.Nonlocal,
5026# ast.Pass, ast.Break, ast.Continue, ast.JoinedStr/ast.FormattedValue
5027
5028#------------------#
5029# Message Handling #
5030#------------------#
5031
5032def indent(msg, level=2):
5033    """
5034    Indents every line of the given message (a string).
5035    """
5036    indent = ' ' * level
5037    return indent + ('\n' + indent).join(msg.splitlines())
5038
5039
5040def ellipsis(string, maxlen=40):
5041    """
5042    Returns the provided string as-is, or if it's longer than the given
5043    maximum length, returns the string, truncated, with '...' at the
5044    end, which will, including the ellipsis, be exactly the given
5045    maximum length. The maximum length must be 4 or more.
5046    """
5047    if len(string) > maxlen:
5048        return string[:maxlen - 3] + "..."
5049    else:
5050        return string
5051
5052
5053def dual_string_repr(string):
5054    """
5055    Returns a pair containing full and truncated representations of the
5056    given string. The formatting of even the full representation depends
5057    on whether it's a multi-line string or not and how long it is.
5058    """
5059    lines = string.split('\n')
5060    if len(repr(string)) < 80 and len(lines) == 1:
5061        full = repr(string)
5062        short = repr(string)
5063    else:
5064        full = '"""\\\n' + string.replace('\r', '\\r') + '"""'
5065        if len(string) < 240 and len(lines) <= 7:
5066            short = full
5067        elif len(lines) > 7:
5068            head = '\n'.join(lines[:7])
5069            short = (
5070                '"""\\\n' + ellipsis(head.replace('\r', '\\r'), 240) + '"""'
5071            )
5072        else:
5073            short = (
5074                '"""\\\n' + ellipsis(string.replace('\r', '\\r'), 240) + '"""'
5075            )
5076
5077    return (full, short)
5078
5079
5080def limited_repr(string):
5081    """
5082    Given a string that might include multiple lines and/or lots of
5083    characters (regardless of lines), returns version cut off by
5084    ellipses either after 5 or so lines, or after 240 characters.
5085    Returns the full string if it's both less than 240 characters and
5086    less than 5 lines.
5087    """
5088    # Split by lines
5089    lines = string.split('\n')
5090
5091    # Already short enough
5092    if len(string) < 240 and len(lines) < 5:
5093        return string
5094
5095    # Try up to 5 lines, cutting them off until we've got a
5096    # short-enough head string
5097    for n in range(min(5, len(lines)), 0, -1):
5098        head = '\n'.join(lines[:n])
5099        if n < len(lines):
5100            head += '\n...'
5101        if len(head) < 240:
5102            break
5103    else:
5104        # If we didn't break, just use first 240 characters
5105        # of the string
5106        head = string[:240] + '...'
5107
5108    # If we cut things too short (e.g., because of initial
5109    # empty lines) use first 240 characters of the string
5110    if len(head) < 12:
5111        head = string[:240] + '...'
5112
5113    return head
5114
5115
5116def msg_color(category):
5117    """
5118    Returns an ANSI color code for the given category of message (one of
5119    "succeeded", "failed", "skipped", or "reset"), or returns None if
5120    COLORS is disabled or an invalid category is provided.
5121    """
5122    if not COLORS:
5123        return None
5124    else:
5125        return MSG_COLORS.get(category)
5126
5127
5128def print_message(msg, color=None):
5129    """
5130    Prints a test result message to `PRINT_TO`, but also flushes stdout,
5131    stderr, and the `PRINT_TO` file beforehand and afterwards to improve
5132    message ordering.
5133
5134    If a color is given, it should be an ANSI terminal color code string
5135    (just the digits, for example '34' for blue or '1;31' for bright red).
5136    """
5137    sys.stdout.flush()
5138    sys.stderr.flush()
5139    try:
5140        PRINT_TO.flush()
5141    except Exception:
5142        pass
5143
5144    # Make the whole message blue
5145    if color:
5146        print(f"\x1b[{color}m", end="", file=PRINT_TO)
5147        suffix = "\x1b[0m"
5148    else:
5149        suffix = ""
5150
5151    print(msg + suffix, file=PRINT_TO)
5152
5153    sys.stdout.flush()
5154    sys.stderr.flush()
5155    try:
5156        PRINT_TO.flush()
5157    except Exception:
5158        pass
5159
5160
5161def expr_details(context):
5162    """
5163    Returns a pair of strings containing base and extra details for an
5164    expression as represented by a dictionary returned from
5165    `get_my_context`. The extra message may be an empty string if the
5166    base message contains all relevant information.
5167    """
5168    # Expression that was evaluated
5169    expr = context.get("expr_src", "???")
5170    short_expr = ellipsis(expr, 78)
5171    # Results
5172    msg = ""
5173    extra_msg = ""
5174
5175    # Base message
5176    msg += f"Test expression was:\n{indent(short_expr, 2)}"
5177
5178    # Figure out values to display
5179    vdict = context.get("values", {})
5180    if context.get("relevant") is not None:
5181        show = sorted(
5182            context["relevant"],
5183            key=lambda fragment: (expr.index(fragment), len(fragment))
5184        )
5185    else:
5186        show = sorted(
5187            vdict.keys(),
5188            key=lambda fragment: (expr.index(fragment), len(fragment))
5189        )
5190
5191    if len(show) > 0:
5192        msg += "\nValues were:"
5193
5194    longs = []
5195    for key in show:
5196        if key in vdict:
5197            val = repr(vdict[key])
5198        else:
5199            val = "???"
5200
5201        entry = f"  {key} = {val}"
5202        fits = ellipsis(entry)
5203        msg += '\n' + fits
5204        if fits != entry:
5205            longs.append(entry)
5206
5207    # Extra message
5208    if short_expr != expr:
5209        if extra_msg != "" and not extra_msg.endswith('\n'):
5210            extra_msg += '\n'
5211        extra_msg += f"Full expression:\n{indent(expr, 2)}"
5212    extra_values = sorted(
5213        [
5214            key
5215            for key in vdict.keys()
5216            if key not in context.get("relevant", [])
5217        ],
5218        key=lambda fragment: (expr.index(fragment), len(fragment))
5219    )
5220    if context.get("relevant") is not None and extra_values:
5221        if extra_msg != "" and not extra_msg.endswith('\n'):
5222            extra_msg += '\n'
5223        extra_msg += "Extra values:"
5224        for ev in extra_values:
5225            if ev in vdict:
5226                val = repr(vdict[ev])
5227            else:
5228                val = "???"
5229
5230            entry = f"  {ev} = {val}"
5231            fits = ellipsis(entry, 78)
5232            extra_msg += '\n' + fits
5233            if fits != entry:
5234                longs.append(entry)
5235
5236    if longs:
5237        if extra_msg != "" and not extra_msg.endswith('\n'):
5238            extra_msg += '\n'
5239        extra_msg += "Full values:"
5240        for entry in longs:
5241            extra_msg += '\n' + entry
5242
5243    return msg, extra_msg
5244
5245
5246#------------#
5247# Comparison #
5248#------------#
5249
5250def findFirstDifference(val, ref, comparing=None):
5251    """
5252    Returns a string describing the first point of difference between
5253    `val` and `ref`, or None if the two values are equivalent. If
5254    IGNORE_TRAILING_WHITESPACE is True, trailing whitespace will be
5255    trimmed from each string before looking for differences.
5256
5257    A small amount of difference is ignored between floating point
5258    numbers, including those found in complex structures.
5259
5260    Works for recursive data structures; the `comparing` argument serves
5261    as a memo to avoid infinite recursion, and the `within` argument
5262    indicates where in a complex structure we are; both should normally
5263    be left as their defaults.
5264    """
5265    if comparing is None:
5266        comparing = set()
5267
5268    cmpkey = (id(val), id(ref))
5269    if cmpkey in comparing:
5270        # Either they differ somewhere else, or they're functionally
5271        # identical
5272        # TODO: Does this really ward off all infinite recursion on
5273        # finite structures?
5274        return None
5275
5276    comparing.add(cmpkey)
5277
5278    try:
5279        simple = val == ref
5280    except RecursionError:
5281        simple = False
5282
5283    if simple:
5284        return None
5285
5286    else:  # let's hunt for differences
5287        if (
5288            isinstance(val, (int, float, complex))
5289        and isinstance(ref, (int, float, complex))
5290        ):  # what if they're both numbers?
5291            if cmath.isclose(
5292                val,
5293                ref,
5294                rel_tol=FLOAT_REL_TOLERANCE,
5295                abs_tol=FLOAT_ABS_TOLERANCE
5296            ):
5297                return None
5298            else:
5299                if isinstance(val, complex) and isinstance(ref, complex):
5300                    return f"complex numbers {val} and {ref} are different"
5301                elif isinstance(val, complex) or isinstance(ref, complex):
5302                    return f"numbers {val} and {ref} are different"
5303                elif val > 0 and ref < 0:
5304                    return f"numbers {val} and {ref} have different signs"
5305                else:
5306                    return f"numbers {val} and {ref} are different"
5307
5308        elif type(val) != type(ref):  # different types; not both numbers
5309            svr = ellipsis(repr(val), 8)
5310            srr = ellipsis(repr(ref), 8)
5311            return (
5312                f"values {svr} and {srr} have different types"
5313                f" ({type(val)} and {type(ref)})"
5314            )
5315
5316        elif isinstance(val, str):  # both strings
5317            if '\n' in val or '\n' in ref:
5318                # multi-line strings; look for first different line
5319                # Note: we *don't* use splitlines here because it will
5320                # give multiple line breaks in a \r\r\n situation like
5321                # those caused by csv.DictWriter on windows when opening
5322                # a file without newlines=''. We'd like to instead ignore
5323                # '\r' as a line break (we're not going to work on early
5324                # Macs) and strip it if IGNORE_TRAILING_WHITESPACE is on.
5325                valLines = val.split('\n')
5326                refLines = ref.split('\n')
5327
5328                # First line # where they differ (1-indexed)
5329                firstDiff = None
5330
5331                # Compute point of first difference
5332                i = None
5333                for i in range(min(len(valLines), len(refLines))):
5334                    valLine = valLines[i]
5335                    refLine = refLines[i]
5336                    if IGNORE_TRAILING_WHITESPACE:
5337                        valLine = valLine.rstrip()
5338                        refLine = refLine.rstrip()
5339
5340                    if valLine != refLine:
5341                        firstDiff = i + 1
5342                        break
5343                else:
5344                    if i is not None:
5345                        # if one has more lines
5346                        if len(valLines) != len(refLines):
5347                            # In this case, one of the two is longer...
5348                            # If IGNORE_TRAILING_WHITESPACE is on, and
5349                            # the longer one just has a blank extra line
5350                            # (possibly with some whitespace on it), then
5351                            # the difference is just in the presence or
5352                            # absence of a final newline, which we also
5353                            # count as a "trailing whitespace" difference
5354                            # and ignore. Note that multiple final '\n'
5355                            # characters will be counted as a difference,
5356                            # since they result in multiple final
5357                            # lines...
5358                            if (
5359                                IGNORE_TRAILING_WHITESPACE
5360                            and (
5361                                    (
5362                                        len(valLines) == len(refLines) + 1
5363                                    and valLines[i + 1].strip() == ''
5364                                    )
5365                                 or (
5366                                        len(valLines) + 1 == len(refLines)
5367                                    and refLines[i + 1].strip() == ''
5368                                    )
5369                                )
5370                            ):
5371                                return None
5372                            else:
5373                                # If we're attending trailing whitespace,
5374                                # or if there are multiple extra lines or
5375                                # the single extra line is not blank,
5376                                # then that's where our first difference
5377                                # is.
5378                                firstDiff = i + 2
5379                        else:
5380                            # There is no difference once we trim
5381                            # trailing whitespace...
5382                            return None
5383                    else:
5384                        # Note: this is a line number, NOT a line index
5385                        firstDiff = 1
5386
5387                got = "nothing (string had fewer lines than expected)"
5388                expected = "nothing (string had more lines than expected)"
5389                i = firstDiff - 1
5390                if i < len(valLines):
5391                    got = repr(valLines[i])
5392                if i < len(refLines):
5393                    expected = repr(refLines[i])
5394
5395                limit = 60
5396                shortGot = ellipsis(got, limit)
5397                shortExpected = ellipsis(expected, limit)
5398                while (
5399                    shortGot == shortExpected
5400                and limit < len(got) or limit < len(expected)
5401                and limit < 200
5402                ):
5403                    limit += 10
5404                    shortGot = ellipsis(got, limit)
5405                    shortExpected = ellipsis(expected, limit)
5406
5407                return (
5408                    f"strings differ on line {firstDiff} where we got:"
5409                    f"\n  {shortGot}\nbut we expected:"
5410                    f"\n  {shortExpected}"
5411                )
5412            else:
5413                # Single-line strings: find character pos of difference
5414                if IGNORE_TRAILING_WHITESPACE:
5415                    val = val.rstrip()
5416                    ref = ref.rstrip()
5417                    if val == ref:
5418                        return None
5419
5420                # Find character position of first difference
5421                pos = None
5422                i = None
5423                for i in range(min(len(val), len(ref))):
5424                    if val[i] != ref[i]:
5425                        pos = i
5426                        break
5427                else:
5428                    if i is not None:
5429                        pos = i + 1
5430                    else:
5431                        pos = 0  # one string is empty
5432
5433                vchar = None
5434                rchar = None
5435                if pos < len(val):
5436                    vchar = val[pos]
5437                if pos < len(ref):
5438                    rchar = ref[pos]
5439
5440                if vchar is None:
5441                    missing = ellipsis(repr(ref[pos:]), 20)
5442                    return (
5443                        f"expected text missing from end of string:"
5444                        f" {missing}"
5445                    )
5446                    return f"strings {svr} and {srr} differ at position {pos}"
5447                elif rchar is None:
5448                    extra = ellipsis(repr(val[pos:]), 20)
5449                    return (
5450                        f"extra text at end of string:"
5451                        f" {extra}"
5452                    )
5453                else:
5454                    if pos > 6:
5455                        got = ellipsis(repr(val[pos:]), 14)
5456                        expected = ellipsis(repr(ref[pos:]), 14)
5457                        return (
5458                            f"strings differ from position {pos}: got {got}"
5459                            f" but expected {expected}"
5460                        )
5461                    else:
5462                        got = ellipsis(repr(val), 14)
5463                        expected = ellipsis(repr(ref), 14)
5464                        return (
5465                            f"strings are different: got {got}"
5466                            f" but expected {expected}"
5467                        )
5468
5469        elif isinstance(val, (list, tuple)):  # both lists or tuples
5470            svr = ellipsis(repr(val), 10)
5471            srr = ellipsis(repr(ref), 10)
5472            typ = type(val).__name__
5473            if len(val) != len(ref):
5474                return (
5475                    f"{typ}s {svr} and {srr} have different lengths"
5476                    f" ({len(val)} and {len(ref)})"
5477                )
5478            else:
5479                for i in range(len(val)):
5480                    diff = findFirstDifference(val[i], ref[i], comparing)
5481                    if diff is not None:
5482                        return f"in slot {i} of {typ}, " + diff
5483                return None  # no differences in any slot
5484
5485        elif isinstance(val, (set)):  # both sets
5486            svr = ellipsis(repr(val), 10)
5487            srr = ellipsis(repr(ref), 10)
5488            onlyVal = (val - ref)
5489            onlyRef = (ref - val)
5490            # Sort so we can match up different-but-equivalent
5491            # floating-point items...
5492            try:
5493                sonlyVal = sorted(onlyVal)
5494                sonlyRef = sorted(onlyRef)
5495                diff = findFirstDifference(
5496                    sonlyVal,
5497                    sonlyRef,
5498                    comparing
5499                )
5500            except TypeError:
5501                # not sortable, so not just floating-point diffs
5502                diff = "some"
5503
5504            if diff is None:
5505                return None
5506            else:
5507                nMissing = len(onlyRef)
5508                nExtra = len(onlyVal)
5509                if nExtra == 0:
5510                    firstMissing = ellipsis(repr(list(onlyRef)[0]), 12)
5511                    result = f"in a set, missing element {firstMissing}"
5512                    if nMissing > 1:
5513                        result += f" ({nMissing} missing elements in total)"
5514                    return result
5515                elif nMissing == 0:
5516                    firstExtra = ellipsis(repr(list(onlyVal)[0]), 12)
5517                    return f"in a set, extra element {firstExtra}"
5518                    if nExtra > 1:
5519                        result += f" ({nExtra} extra elements in total)"
5520                    return result
5521                else:
5522                    firstMissing = ellipsis(repr(list(onlyRef)[0]), 8)
5523                    firstExtra = ellipsis(repr(list(onlyVal)[0]), 8)
5524                    result = (
5525                        "in a set, elements are different (extra"
5526                        f" element {firstExtra} and missing element"
5527                        f" {firstMissing}"
5528                    )
5529                    if nMissing > 1 and nExtra > 1:
5530                        result += (
5531                            f" ({nExtra} total extra elements and"
5532                            f" {nMissing} total missing elements"
5533                        )
5534                    elif nMissing == 1:
5535                        if nExtra > 1:
5536                            result += (
5537                                f" (1 missing and {nExtra} total extra"
5538                                f" elements)"
5539                            )
5540                    else:  # nExtra must be 1
5541                        if nMissing > 1:
5542                            result += (
5543                                f" (1 extra and {nExtra} total missing"
5544                                f" elements)"
5545                            )
5546                    return result
5547
5548        elif isinstance(val, dict):  # both dicts
5549            svr = ellipsis(repr(val), 14)
5550            srr = ellipsis(repr(ref), 14)
5551
5552            if len(val) != len(ref):
5553                if len(val) < len(ref):
5554                    ldiff = len(ref) - len(val)
5555                    firstMissing = ellipsis(
5556                        repr(list(set(ref.keys()) - set(val.keys()))[0]),
5557                        30
5558                    )
5559                    return (
5560                        f"dictionary is missing key {firstMissing} (has"
5561                        f" {ldiff} fewer key{'s' if ldiff > 1 else ''}"
5562                        f" than expected)"
5563                    )
5564                else:
5565                    ldiff = len(val) - len(ref)
5566                    firstExtra = ellipsis(
5567                        repr(list(set(val.keys()) - set(ref.keys()))[0]),
5568                        30
5569                    )
5570                    return (
5571                        f"dictionary has extra key {firstExtra} (has"
5572                        f" {ldiff} more key{'s' if ldiff > 1 else ''}"
5573                        f" than expected)"
5574                    )
5575                return (
5576                    f"dictionaries {svr} and {srr} have different sizes"
5577                    f" ({len(val)} and {len(ref)})"
5578                )
5579
5580            vkeys = set(val.keys())
5581            rkeys = set(ref.keys())
5582            try:
5583                onlyVal = sorted(vkeys - rkeys)
5584                onlyRef = sorted(rkeys - vkeys)
5585                keyCorrespondence = {}
5586            except TypeError:  # unsortable...
5587                keyCorrespondence = None
5588
5589            # Check for floating-point equivalence of keys if sets are
5590            # sortable...
5591            if keyCorrespondence is not None:
5592                if findFirstDifference(onlyVal, onlyRef, comparing) is None:
5593                    keyCorrespondence = {
5594                        onlyVal[i]: onlyRef[i]
5595                        for i in range(len(onlyVal))
5596                    }
5597                    # Add pass-through mappings for matching keys
5598                    for k in vkeys & rkeys:
5599                        keyCorrespondence[k] = k
5600                else:
5601                    # No actual mapping is available...
5602                    keyCorrespondence = None
5603
5604            # We couldn't find a correspondence between keys, so we
5605            # return a key-based difference
5606            if keyCorrespondence is None:
5607                onlyVal = vkeys - rkeys
5608                onlyRef = rkeys - vkeys
5609                nExtra = len(onlyVal)
5610                nMissing = len(onlyRef)
5611                if nExtra == 0:
5612                    firstMissing = ellipsis(repr(list(onlyRef)[0]), 10)
5613                    result = f"dictionary is missing key {firstMissing}"
5614                    if nMissing > 1:
5615                        result += f" ({nMissing} missing keys in total)"
5616                    return result
5617                elif nMissing == 0:
5618                    firstExtra = ellipsis(repr(list(onlyVal)[0]), 10)
5619                    result = f"dictionary has extra key {firstExtra}"
5620                    if nExtra > 1:
5621                        result += f" ({nExtra} extra keys in total)"
5622                    return result
5623                else:  # neither is 0
5624                    firstMissing = ellipsis(repr(list(onlyRef)[0]), 10)
5625                    firstExtra = ellipsis(repr(list(onlyVal)[0]), 10)
5626                    result = (
5627                        f"dictionary is missing key {firstMissing} and"
5628                        f" has extra key {firstExtra}"
5629                    )
5630                    if nMissing > 1 and nExtra > 1:
5631                        result += (
5632                            f" ({nMissing} missing and {nExtra} extra"
5633                            f" keys in total)"
5634                        )
5635                    elif nMissing == 1:
5636                        if nExtra > 1:
5637                            result += (
5638                                f" (1 missing and {nExtra} extra keys"
5639                                f" in total)"
5640                            )
5641                    else:  # nExtra must be 1
5642                        if nMissing > 1:
5643                            result += (
5644                                f" (1 extra and {nMissing} missing keys"
5645                                f" in total)"
5646                            )
5647                    return result
5648
5649            # if we reach here, keyCorrespondence maps val keys to
5650            # equivalent (but not necessarily identical) ref keys
5651
5652            for vk in keyCorrespondence:
5653                rk = keyCorrespondence[vk]
5654                vdiff = findFirstDifference(val[vk], ref[rk], comparing)
5655                if vdiff is not None:
5656                    krep = ellipsis(repr(vk), 14)
5657                    return f"in dictionary slot {krep}, " + vdiff
5658
5659            return None
5660
5661        else:  # not sure what kind of thing this is...
5662            if val == ref:
5663                return None
5664            else:
5665                limit = 15
5666                vr = repr(val)
5667                rr = repr(ref)
5668                svr = ellipsis(vr, limit)
5669                srr = ellipsis(rr, limit)
5670                while (
5671                    svr == srr
5672                and (limit < len(vr) or limit < len(rr))
5673                and limit < 100
5674                ):
5675                    limit += 10
5676                    svr = ellipsis(vr, limit)
5677                    srr = ellipsis(rr, limit)
5678                return f" objects {svr} and {srr} are different"
5679
5680
5681def checkContainment(val1, val2):
5682    """
5683    Returns True if val1 is 'contained in' to val2, and False otherwise.
5684    If IGNORE_TRAILING_WHITESPACE is True, will ignore trailing
5685    whitespace in two strings when comparing them for containment.
5686    """
5687    if (not isinstance(val1, str)) or (not isinstance(val2, str)):
5688        return val1 in val2  # use regular containment test
5689    # For two strings, pay attention to IGNORE_TRAILING_WHITESPACE
5690    elif IGNORE_TRAILING_WHITESPACE:
5691        # remove trailing whitespace from both strings (on all lines)
5692        return trimWhitespace(val1) in trimWhitespace(val2)
5693    else:
5694        return val1 in val2  # use regular containment test
5695
5696
5697def trimWhitespace(st, requireNewline=False):
5698    """
5699    Assume st a string. Use .rstrip() to remove trailing whitespace from
5700    each line. This has the side effect of replacing complex newlines
5701    with just '\\n'. If requireNewline is set to true, only whitespace
5702    that comes before a newline will be trimmed, and whitespace which
5703    occurs at the end of the string on the last line will be retained if
5704    there is no final newline.
5705    """
5706    if requireNewline:
5707        return re.sub('[ \t\r]*([\r\n])', r'\1', st)
5708    else:
5709        result = '\n'.join(line.rstrip() for line in st.split('\n'))
5710        return result
5711
5712
5713def compare(val, ref):
5714    """
5715    Comparison function returning a boolean which uses
5716    findFirstDifference under the hood.
5717    """
5718    return findFirstDifference(val, ref) is None
5719
5720
5721def test_compare():
5722    "Tests the compare function."
5723    # TODO: test findFirstDifference instead & confirm correct
5724    # messages!!!
5725    # Integers
5726    assert compare(1, 1)
5727    assert compare(1, 2) is False
5728    assert compare(2, 1 + 1)
5729
5730    # Floating point numbers
5731    assert compare(1.1, 1.1)
5732    assert compare(1.1, 1.2) is False
5733    assert compare(1.1, 1.1000000001)
5734    assert compare(1.1, 1.1001) is False
5735
5736    # Complex numbers
5737    assert compare(1.1 + 2.3j, 1.1 + 2.3j)
5738    assert compare(1.1 + 2.3j, 1.1 + 2.4j) is False
5739
5740    # Strings
5741    assert compare('abc', 1.1001) is False
5742    assert compare('abc', 'abc')
5743    assert compare('abc', 'def') is False
5744
5745    # Lists
5746    assert compare([1, 2, 3], [1, 2, 3])
5747    assert compare([1, 2, 3], [1, 2, 4]) is False
5748    assert compare([1, 2, 3], [1, 2, 3.0000000001])
5749
5750    # Tuples
5751    assert compare((1, 2, 3), (1, 2, 3))
5752    assert compare((1, 2, 3), (1, 2, 4)) is False
5753    assert compare((1, 2, 3), (1, 2, 3.0000000001))
5754
5755    # Nested lists + tuples
5756    assert compare(
5757        ['a', 'b', 'cdefg', [1, 2, [3]], (4, 5)],
5758        ['a', 'b', 'cdefg', [1, 2, [3]], (4, 5)]
5759    )
5760    assert compare(
5761        ['a', 'b', 'cdefg', [1, 2, [3]], (4, 5)],
5762        ['a', 'b', 'cdefg', [1, 2, [3]], (4, '5')]
5763    ) is False
5764    assert compare(
5765        ['a', 'b', 'cdefg', [1, 2, [3]], (4, 5)],
5766        ['a', 'b', 'cdefg', [1, 2, [3]], [4, 5]]
5767    ) is False
5768
5769    # Sets
5770    assert compare({1, 2}, {1, 2})
5771    assert compare({1, 2}, {1}) is False
5772    assert compare({1}, {1, 2}) is False
5773    assert compare({1, 2}, {'1', 2}) is False
5774    assert compare({'a', 'b', 'c'}, {'a', 'b', 'c'})
5775    assert compare({'a', 'b', 'c'}, {'a', 'b', 'C'}) is False
5776    # Two tricky cases
5777    assert compare({1, 2}, {1.00000001, 2})
5778    assert compare({(1, 2), 3}, {(1.00000001, 2), 3})
5779
5780    # Dictionaries
5781    assert compare({1: 2, 3: 4}, {1: 2, 3: 4})
5782    assert compare({1: 2, 3: 4}, {1: 2, 3.00000000001: 4})
5783    assert compare({1: 2, 3: 4}, {1: 2, 3: 4.00000000001})
5784    assert compare({1: 2, 3: 4}, {1: 2, 3.1: 4}) is False
5785    assert compare({1: 2, 3: 4}, {1: 2, 3: 4.1}) is False
5786
5787    # Nested dictionaries & lists
5788    assert compare(
5789        {1: {1.1: 2.2}, 2: [2.2, 3.3]},
5790        {1: {1.1: 2.2}, 2: [2.2, 3.3]}
5791    )
5792    assert compare(
5793        {1: {1.1: 2.2}, 2: [2.2, 3.3]},
5794        {1: {1.2: 2.2}, 2: [2.2, 3.3]}
5795    ) is False
5796    assert compare(
5797        {1: {1.1: 2.2}, 2: [2.2, 3.3]},
5798        {1: {1.1: 2.3}, 2: [2.2, 3.3]}
5799    ) is False
5800    assert compare(
5801        {1: {1.1: 2.2}, 2: [2.2, 3.3]},
5802        {1: {1.1: 2.2}, 2: [2.2, 3.4]}
5803    ) is False
5804    assert compare(
5805        {1: {1.1: 2.2}, 2: [2.2, 3.3]},
5806        {1: {1.1: 2.2}, 2: 2.2}
5807    ) is False
5808    assert compare(
5809        {1: {1.1: 2.2}, 2: [2.2, 3.3]},
5810        {1: {1.1: 2.2}, 2: (2.2, 3.3)}
5811    ) is False
5812
5813    # Equivalent infinitely recursive list structures
5814    a = [1, 2, 3]
5815    a.append(a)
5816    a2 = [1, 2, 3]
5817    a2.append(a2)
5818    b = [1, 2, 3, [1, 2, 3]]
5819    b[3].append(b)
5820    c = [1, 2, 3, [1, 2, 3, [1, 2, 3]]]
5821    c[3][3].append(c[3])
5822    d = [1, 2, 3]
5823    d.insert(2, d)
5824
5825    assert compare(a, a)
5826    assert compare(a, a2)
5827    assert compare(a2, a)
5828    assert compare(a, b)
5829    assert compare(a, c)
5830    assert compare(a2, b)
5831    assert compare(b, a)
5832    assert compare(b, a2)
5833    assert compare(c, a)
5834    assert compare(b, c)
5835    assert compare(a, d) is False
5836    assert compare(b, d) is False
5837    assert compare(c, d) is False
5838    assert compare(d, a) is False
5839    assert compare(d, b) is False
5840    assert compare(d, c) is False
5841
5842    # Equivalent infinitely recursive dicitonaries
5843    e = {1: 2}
5844    e[2] = e
5845    e2 = {1: 2}
5846    e2[2] = e2
5847    f = {1: 2}
5848    f2 = {1: 2}
5849    f[2] = f2
5850    f2[2] = f
5851    g = {1: 2, 2: {1: 2.0000000001}}
5852    g[2][2] = g
5853    h = {1: 2, 2: {1: 3}}
5854    h[2][2] = h
5855
5856    assert compare(e, e2)
5857    assert compare(e2, e)
5858    assert compare(f, f2)
5859    assert compare(f2, f)
5860    assert compare(e, f)
5861    assert compare(f, e)
5862    assert compare(e, g)
5863    assert compare(f, g)
5864    assert compare(g, e)
5865    assert compare(g, f)
5866    assert compare(e, h) is False
5867    assert compare(f, h) is False
5868    assert compare(g, h) is False
5869    assert compare(h, e) is False
5870    assert compare(h, f) is False
5871    assert compare(h, g) is False
5872
5873    # Custom types + objects
5874    class T:
5875        pass
5876
5877    assert compare(T, T)
5878    assert compare(T, 1) is False
5879    assert compare(T(), T()) is False
5880
5881    # Custom type w/ a custom __eq__
5882    class E:
5883        def __eq__(self, other):
5884            return isinstance(other, E) or other == 1
5885
5886    assert compare(E, E)
5887    assert compare(E, 1) is False
5888    assert compare(E(), 1)
5889    assert compare(E(), 2) is False
5890    assert compare(E(), E())
5891
5892    # Custom type w/ custom __hash__ and __eq__
5893    class A:
5894        def __init__(self, val):
5895            self.val = val
5896
5897        def __hash__(self):
5898            return 3  # hashes collide
5899
5900        def __eq__(self, other):
5901            return isinstance(other, A) and self.val == other.val
5902
5903    assert compare({A(1), A(2)}, {A(1), A(2)})
5904    assert compare({A(1), A(2)}, {A(1), A(3)}) is False
5905
5906
5907#-----------------------#
5908# Configuration control #
5909#-----------------------#
5910
5911def detailLevel(level):
5912    """
5913    Sets the level of detail for printed messages.
5914    The detail levels are:
5915
5916    * -1: Super-minimal output, with no details beyond success/failure.
5917    * 0: Succinct messages indicating success/failure, with minimal
5918        details when failure occurs.
5919    * 1: More verbose success/failure messages, with details about
5920        successes and more details about failures.
5921    """
5922    global DETAIL_LEVEL
5923    DETAIL_LEVEL = level
5924
5925
5926def attendTrailingWhitespace(on=True):
5927    """
5928    Call this function to force `optimism` to pay attention to
5929    whitespace at the end of lines when checking expectations. By
5930    default, such whitespace is removed both from expected
5931    values/output fragments and from captured outputs/results before
5932    checking expectations. To turn that functionality on again, you
5933    can call this function with False as the argument.
5934    """
5935    global IGNORE_TRAILING_WHITESPACE
5936    IGNORE_TRAILING_WHITESPACE = not on
5937
5938
5939def skipChecksAfterFail(mode="all"):
5940    """
5941    The argument should be either 'case' (the default), 'manager', or
5942    None. In 'manager' mode, when one check fails, any other checks of
5943    cases derived from that manager, including the case where the check
5944    failed, will be skipped. In 'case' mode, once a check fails any
5945    further checks of the same case will be skipped, but checks of other
5946    cases derived from the same manager will not be. In None mode (or if
5947    any other value is provided) no checks will be skipped because of
5948    failed checks (but they might be skipped for other reasons).
5949    """
5950    global SKIP_ON_FAILURE
5951    SKIP_ON_FAILURE = mode
5952
5953
5954def suppressErrorDetailsAfterFail(mode="all"):
5955    """
5956    The argument should be one of the following values:
5957
5958    - `'case'`: Causes error details to be omitted for failed checks
5959      after the first failed check on each particular test case.
5960    - `'manager'`: Causes error details to be omitted for failed checks
5961      after the first failed check on any test case for a particular
5962      manager.
5963    - `'all'`: Causes error details to be omitted for all failed checks
5964      after any check fails. Reset this with `clearFailure`.
5965    - None (or any other value not listed above): Means that full error
5966      details will always be reported.
5967
5968    The default value is `'all`' if you call this function; see
5969    `SUPPRESS_ON_FAILURE` for the default value when `optimism` is
5970    imported.
5971
5972    Note that detail suppression has no effect if the detail level is set
5973    above 0.
5974    """
5975    global SUPPRESS_ON_FAILURE
5976    SUPPRESS_ON_FAILURE = mode
5977
5978
5979def clearFailure():
5980    """
5981    Resets the failure status so that checks will resume when
5982    `SKIP_ON_FAILURE` is set to `'all'`.
5983    """
5984    global CHECK_FAILED
5985    CHECK_FAILED = False
5986
5987
5988#----------------------------------#
5989# Summarization and Trial Tracking #
5990#----------------------------------#
5991
5992def _register_outcome(passed, tag, message):
5993    """
5994    Given a passed/failed boolean, a tag string indicating the file name
5995    + line number where a check was requested, and a message for that
5996    outcome, registers that outcome triple in the `ALL_OUTCOMES`
5997    dictionary under the current test suite name.
5998    """
5999    ALL_OUTCOMES.setdefault(_CURRENT_SUITE_NAME, []).append(
6000        (passed, tag, message)
6001    )
6002
6003
6004def showSummary(suiteName=None):
6005    """
6006    Shows a summary of the number of checks in the current test suite
6007    (see `currentTestSuite`) that have been met or not. You can also
6008    give an argument to specify the name of the test suite to summarize.
6009    Prints output to sys.stderr.
6010
6011    Note that the results of `expect` checks are not included in the
6012    summary, because they aren't trials.
6013    """
6014    # Flush stdout, stderr, and PRINT_TO to improve ordering
6015    sys.stdout.flush()
6016    sys.stderr.flush()
6017    try:
6018        PRINT_TO.flush()
6019    except Exception:
6020        pass
6021
6022    met = []
6023    unmet = []
6024    for passed, tag, msg in listOutcomesInSuite(suiteName):
6025        if passed:
6026            met.append(tag)
6027        else:
6028            unmet.append(tag)
6029
6030    print('---', file=PRINT_TO)
6031
6032    if len(unmet) == 0:
6033        if len(met) == 0:
6034            print("No expectations were established.", file=PRINT_TO)
6035        else:
6036            print(
6037                f"All {len(met)} expectation(s) were met.",
6038                file=PRINT_TO
6039            )
6040    else:
6041        if len(met) == 0:
6042            print(
6043                f"None of the {len(unmet)} expectation(s) were met!",
6044                file=PRINT_TO
6045            )
6046        else:
6047            print(
6048                (
6049                    f"{len(unmet)} of the {len(met) + len(unmet)}"
6050                    f" expectation(s) were NOT met:"
6051                ),
6052                file=PRINT_TO
6053            )
6054        if COLORS:  # bright red
6055            print("\x1b[1;31m", end="", file=PRINT_TO)
6056        for tag in unmet:
6057            print(f"  ✗ {tag}", file=PRINT_TO)
6058        if COLORS:  # reset
6059            print("\x1b[0m", end="", file=PRINT_TO)
6060    print('---', file=PRINT_TO)
6061
6062    # Flush stdout & stderr to improve ordering
6063    sys.stdout.flush()
6064    sys.stderr.flush()
6065    try:
6066        PRINT_TO.flush()
6067    except Exception:
6068        pass
6069
6070
6071def currentTestSuite():
6072    """
6073    Returns the name of the current test suite (a string).
6074    """
6075    return _CURRENT_SUITE_NAME
6076
6077
6078def testSuite(name):
6079    """
6080    Starts a new test suite with the given name, or resumes an old one.
6081    Any cases created subsequently will be registered to that suite.
6082    """
6083    global _CURRENT_SUITE_NAME
6084    if not isinstance(name, str):
6085        raise TypeError(
6086            f"The test suite name must be a string (got: '{repr(name)}'"
6087            f" which is a {type(name)})."
6088        )
6089    _CURRENT_SUITE_NAME = name
6090
6091
6092def resetTestSuite(suiteName=None):
6093    """
6094    Resets the cases and outcomes recorded in the current test suite (or
6095    the named test suite if an argument is provided).
6096    """
6097    if suiteName is None:
6098        suiteName = currentTestSuite()
6099
6100    ALL_TRIALS[suiteName] = []
6101    ALL_OUTCOMES[suiteName] = []
6102
6103
6104def freshTestSuite(name):
6105    """
6106    Works like `testSuite`, but calls `resetTestSuite` for that suite
6107    name first, ensuring no old test suite contents will be included.
6108    """
6109    resetTestSuite(name)
6110    testSuite(name)
6111
6112
6113def deleteAllTestSuites():
6114    """
6115    Deletes all test suites, removing all recorded test cases and
6116    outcomes, and setting the current test suite name back to "default".
6117    """
6118    global ALL_TRIALS, ALL_OUTCOMES, _CURRENT_SUITE_NAME
6119    _CURRENT_SUITE_NAME = "default"
6120    ALL_TRIALS = {}
6121    ALL_OUTCOMES = {}
6122
6123
6124def listTrialsInSuite(suiteName=None):
6125    """
6126    Returns a list of trials (`Trial` objects) in the current test suite
6127    (or the named suite if an argument is provided).
6128    """
6129    if suiteName is None:
6130        suiteName = currentTestSuite()
6131
6132    if suiteName not in ALL_TRIALS:
6133        raise ValueError(f"Test suite '{suiteName}' does not exist.")
6134
6135    return ALL_TRIALS[suiteName][:]
6136
6137
6138def listOutcomesInSuite(suiteName=None):
6139    """
6140    Returns a list of all individual expectation outcomes attached to
6141    trials in the given test suite (default: the current test suite).
6142    Includes `expect` and `expectType` outcomes even though those aren't
6143    attached to trials.
6144    """
6145    if suiteName is None:
6146        suiteName = currentTestSuite()
6147
6148    if suiteName not in ALL_OUTCOMES:
6149        raise ValueError(f"Test suite '{suiteName}' does not exit.")
6150
6151    return ALL_OUTCOMES[suiteName][:]
6152
6153
6154def listAllTrials():
6155    """
6156    Returns a list of all registered trials (`Trial` objects) in any
6157    known test suite. Note that if `deleteAllTestSuites` has been called,
6158    this will not include any `Trial` objects created before that point.
6159    """
6160    result = []
6161    for suiteName in ALL_TRIALS:
6162        result.extend(ALL_TRIALS[suiteName])
6163
6164    return result
6165
6166
6167#---------------#
6168# Color control #
6169#---------------#
6170
6171def colors(enable=False):
6172    """
6173    Enables or disables colors in printed output. If your output does not
6174    support ANSI color codes, the color output will show up as garbage
6175    and you can disable this.
6176    """
6177    global COLORS
6178    COLORS = enable
6179
6180
6181#---------#
6182# Tracing #
6183#---------#
6184
6185def trace(expr):
6186    """
6187    Given an expression (actually, of course, just a value), returns the
6188    value it was given. But also prints a trace message indicating what
6189    the expression was, what value it had, and the line number of that
6190    line of code.
6191
6192    The file name and overlength results are printed only when the
6193    `detailLevel` is set to 1 or higher.
6194    """
6195    # Flush stdout & stderr to improve ordering
6196    sys.stdout.flush()
6197    sys.stderr.flush()
6198    try:
6199        PRINT_TO.flush()
6200    except Exception:
6201        pass
6202
6203    ctx = get_my_context(trace)
6204    rep = repr(expr)
6205    short = ellipsis(repr(expr))
6206    tag = "{line}".format(**ctx)
6207    if DETAIL_LEVEL >= 1:
6208        tag = "{file}:{line}".format(**ctx)
6209    print(
6210        f"{tag} {ctx.get('expr_src', '???')} ⇒ {short}",
6211        file=PRINT_TO
6212    )
6213    if DETAIL_LEVEL >= 1 and short != rep:
6214        print("  Full result is:\n    " + rep, file=PRINT_TO)
6215
6216    # Flush stdout & stderr to improve ordering
6217    sys.stdout.flush()
6218    sys.stderr.flush()
6219    try:
6220        PRINT_TO.flush()
6221    except Exception:
6222        pass
6223
6224    return expr
6225
6226
6227#------------------------------#
6228# Reverse evaluation machinery #
6229#------------------------------#
6230
6231def get_src_index(src, lineno, col_offset):
6232    """
6233    Turns a line number and column offset into an absolute index into
6234    the given source string, assuming length-1 newlines.
6235    """
6236    lines = src.splitlines()
6237    above = lines[:lineno - 1]
6238    return sum(len(line) for line in above) + len(above) + col_offset
6239
6240
6241def test_gsr():
6242    """Tests for get_src_index."""
6243    s = 'a\nb\nc'
6244    assert get_src_index(s, 1, 0) == 0
6245    assert get_src_index(s, 2, 0) == 2
6246    assert get_src_index(s, 3, 0) == 4
6247    assert s[get_src_index(s, 1, 0)] == 'a'
6248    assert s[get_src_index(s, 2, 0)] == 'b'
6249    assert s[get_src_index(s, 3, 0)] == 'c'
6250
6251
6252def find_identifier_end(code, start_index):
6253    """
6254    Given a code string and an index in that string which is the start
6255    of an identifier, returns the index of the end of that identifier.
6256    """
6257    at = start_index + 1
6258    while at < len(code):
6259        ch = code[at]
6260        if not ch.isalpha() and not ch.isdigit() and ch != '_':
6261            break
6262        at += 1
6263    return at - 1
6264
6265
6266def test_find_identifier_end():
6267    """Tests for find_identifier_end."""
6268    assert find_identifier_end("abc.xyz", 0) == 2
6269    assert find_identifier_end("abc.xyz", 1) == 2
6270    assert find_identifier_end("abc.xyz", 2) == 2
6271    assert find_identifier_end("abc.xyz", 4) == 6
6272    assert find_identifier_end("abc.xyz", 5) == 6
6273    assert find_identifier_end("abc.xyz", 6) == 6
6274    assert find_identifier_end("abc_xyz123", 0) == 9
6275    assert find_identifier_end("abc xyz123", 0) == 2
6276    assert find_identifier_end("abc xyz123", 4) == 9
6277    assert find_identifier_end("x", 0) == 0
6278    assert find_identifier_end("  x", 2) == 2
6279    assert find_identifier_end("  xyz1", 2) == 5
6280    s = "def abc(def):\n  print(xyz)\n"
6281    assert find_identifier_end(s, 0) == 2
6282    assert find_identifier_end(s, 4) == 6
6283    assert find_identifier_end(s, 8) == 10
6284    assert find_identifier_end(s, 16) == 20
6285    assert find_identifier_end(s, 22) == 24
6286
6287
6288def unquoted_enumerate(src, start_index):
6289    """
6290    A generator that yields index, character pairs from the given code
6291    string, skipping quotation marks and the strings that they delimit,
6292    including triple-quotes and respecting backslash-escapes within
6293    strings.
6294    """
6295    quote = None
6296    at = start_index
6297
6298    while at < len(src):
6299        char = src[at]
6300
6301        # skip escaped characters in quoted strings
6302        if quote and char == '\\':
6303            # (thank goodness I don't have to worry about r-strings)
6304            at += 2
6305            continue
6306
6307        # handle quoted strings
6308        elif char == '"' or char == "'":
6309            if quote == char:
6310                quote = None  # single end quote
6311                at += 1
6312                continue
6313            elif src[at:at + 3] in ('"""', "'''"):
6314                tq = src[at:at + 3]
6315                at += 3  # going to skip these no matter what
6316                if tq == quote or tq[0] == quote:
6317                    # Ending triple-quote, or matching triple-quote at
6318                    # end of single-quoted string = ending quote +
6319                    # empty string
6320                    quote = None
6321                    continue
6322                else:
6323                    if quote:
6324                        # triple quote of other kind inside single or
6325                        # triple quoted string
6326                        continue
6327                    else:
6328                        quote = tq
6329                        continue
6330            elif quote is None:
6331                # opening single quote
6332                quote = char
6333                at += 1
6334                continue
6335            else:
6336                # single quote inside other quotes
6337                at += 1
6338                continue
6339
6340        # Non-quote characters in quoted strings
6341        elif quote:
6342            at += 1
6343            continue
6344
6345        else:
6346            yield (at, char)
6347            at += 1
6348            continue
6349
6350
6351def test_unquoted_enumerate():
6352    """Tests for unquoted_enumerate."""
6353    uqe = unquoted_enumerate
6354    assert list(uqe("abc'123'", 0)) == list(zip(range(3), "abc"))
6355    assert list(uqe("'abc'123", 0)) == list(zip(range(5, 8), "123"))
6356    assert list(uqe("'abc'123''", 0)) == list(zip(range(5, 8), "123"))
6357    assert list(uqe("'abc'123''", 1)) == [(1, 'a'), (2, 'b'), (3, 'c')]
6358    mls = "'''\na\nb\nc'''\ndef"
6359    assert list(uqe(mls, 0)) == list(zip(range(12, 16), "\ndef"))
6360    tqs = '"""\'\'\'ab\'\'\'\'""" cd'
6361    assert list(uqe(tqs, 0)) == [(15, ' '), (16, 'c'), (17, 'd')]
6362    rqs = "a'b'''c\"\"\"'''\"d\"''''\"\"\"e'''\"\"\"f\"\"\"'''"
6363    assert list(uqe(rqs, 0)) == [(0, 'a'), (6, 'c'), (23, 'e')]
6364    assert list(uqe(rqs, 6)) == [(6, 'c'), (23, 'e')]
6365    bss = "a'\\'b\\''c"
6366    assert list(uqe(bss, 0)) == [(0, 'a'), (8, 'c')]
6367    mqs = "'\"a'b\""
6368    assert list(uqe(mqs, 0)) == [(4, 'b')]
6369
6370
6371def find_nth_attribute_period(code, start_index, n):
6372    """
6373    Given a string of Python code and a start index within that string,
6374    finds the nth period character (counting from first = zero) after
6375    that start point, but only considers periods which are used for
6376    attribute access, i.e., periods outside of quoted strings and which
6377    are not part of ellipses. Returns the index within the string of the
6378    period that it found. A period at the start index (if there is one)
6379    will be counted. Returns None if there are not enough periods in the
6380    code. If the start index is inside a quoted string, things will get
6381    weird, and the results will probably be wrong.
6382    """
6383    for (at, char) in unquoted_enumerate(code, start_index):
6384        if char == '.':
6385            if code[at - 1:at] == '.' or code[at + 1:at + 2] == '.':
6386                # part of an ellipsis, so ignore it
6387                continue
6388            else:
6389                n -= 1
6390                if n < 0:
6391                    break
6392
6393    # Did we hit the end of the string before counting below 0?
6394    if n < 0:
6395        return at
6396    else:
6397        return None
6398
6399
6400def test_find_nth_attribute_period():
6401    """Tests for find_nth_attribute_period."""
6402    assert find_nth_attribute_period("a.b", 0, 0) == 1
6403    assert find_nth_attribute_period("a.b", 0, 1) is None
6404    assert find_nth_attribute_period("a.b", 0, 100) is None
6405    assert find_nth_attribute_period("a.b.c", 0, 1) == 3
6406    assert find_nth_attribute_period("a.b.cde.f", 0, 1) == 3
6407    assert find_nth_attribute_period("a.b.cde.f", 0, 2) == 7
6408    s = "a.b, c.d, 'e.f', g.h"
6409    assert find_nth_attribute_period(s, 0, 0) == 1
6410    assert find_nth_attribute_period(s, 0, 1) == 6
6411    assert find_nth_attribute_period(s, 0, 2) == 18
6412    assert find_nth_attribute_period(s, 0, 3) is None
6413    assert find_nth_attribute_period(s, 0, 3) is None
6414    assert find_nth_attribute_period(s, 1, 0) == 1
6415    assert find_nth_attribute_period(s, 2, 0) == 6
6416    assert find_nth_attribute_period(s, 6, 0) == 6
6417    assert find_nth_attribute_period(s, 7, 0) == 18
6418    assert find_nth_attribute_period(s, 15, 0) == 18
6419
6420
6421def find_closing_item(code, start_index, openclose='()'):
6422    """
6423    Given a string of Python code, a starting index where there's an
6424    open paren, bracket, etc., and a 2-character string containing the
6425    opening and closing delimiters of interest (parentheses by default),
6426    returns the index of the matching closing delimiter, or None if the
6427    opening delimiter is unclosed. Note that the given code must not
6428    contain syntax errors, or the behavior will be undefined.
6429
6430    Does NOT work with quotation marks (single or double).
6431    """
6432    level = 1
6433    open_delim = openclose[0]
6434    close_delim = openclose[1]
6435    for at, char in unquoted_enumerate(code, start_index + 1):
6436        # Non-quoted open delimiters
6437        if char == open_delim:
6438            level += 1
6439
6440        # Non-quoted close delimiters
6441        elif char == close_delim:
6442            level -= 1
6443            if level < 1:
6444                break
6445
6446        # Everything else: ignore it
6447
6448    if level == 0:
6449        return at
6450    else:
6451        return None
6452
6453
6454def test_find_closing_item():
6455    """Tests for find_closing_item."""
6456    assert find_closing_item('()', 0, '()') == 1
6457    assert find_closing_item('()', 0) == 1
6458    assert find_closing_item('(())', 0, '()') == 3
6459    assert find_closing_item('(())', 1, '()') == 2
6460    assert find_closing_item('((word))', 0, '()') == 7
6461    assert find_closing_item('((word))', 1, '()') == 6
6462    assert find_closing_item('(("(("))', 0, '()') == 7
6463    assert find_closing_item('(("(("))', 1, '()') == 6
6464    assert find_closing_item('(("))"))', 0, '()') == 7
6465    assert find_closing_item('(("))"))', 1, '()') == 6
6466    assert find_closing_item('(()())', 0, '()') == 5
6467    assert find_closing_item('(()())', 1, '()') == 2
6468    assert find_closing_item('(()())', 3, '()') == 4
6469    assert find_closing_item('(""")(\n""")', 0, '()') == 10
6470    assert find_closing_item("\"abc(\" + ('''def''')", 9, '()') == 19
6471    assert find_closing_item("\"abc(\" + ('''def''')", 0, '()') is None
6472    assert find_closing_item("\"abc(\" + ('''def''')", 4, '()') is None
6473    assert find_closing_item("(()", 0, '()') is None
6474    assert find_closing_item("(()", 1, '()') == 2
6475    assert find_closing_item("()(", 0, '()') == 1
6476    assert find_closing_item("()(", 2, '()') is None
6477    assert find_closing_item("[]", 0, '[]') == 1
6478    assert find_closing_item("[]", 0) is None
6479    assert find_closing_item("{}", 0, '{}') == 1
6480    assert find_closing_item("aabb", 0, 'ab') == 3
6481
6482
6483def find_unbracketed_comma(code, start_index):
6484    """
6485    Given a string of Python code and a starting index, finds the next
6486    comma at or after that index which isn't surrounded by brackets of
6487    any kind that start at or after that index and which isn't in a
6488    quoted string. Returns the index of the matching comma, or None if
6489    there is none. Stops and returns None if it finds an unmatched
6490    closing bracket. Note that the given code must not contain syntax
6491    errors, or the behavior will be undefined.
6492    """
6493    seeking = []
6494    delims = {
6495        '(': ')',
6496        '[': ']',
6497        '{': '}'
6498    }
6499    closing = delims.values()
6500    for at, char in unquoted_enumerate(code, start_index):
6501        # Non-quoted open delimiter
6502        if char in delims:
6503            seeking.append(delims[char])
6504
6505        # Non-quoted matching close delimiter
6506        elif len(seeking) > 0 and char == seeking[-1]:
6507            seeking.pop()
6508
6509        # Non-quoted non-matching close delimiter
6510        elif char in closing:
6511            return None
6512
6513        # A non-quoted comma
6514        elif char == ',' and len(seeking) == 0:
6515            return at
6516
6517        # Everything else: ignore it
6518
6519    # Got to the end
6520    return None
6521
6522
6523def test_find_unbracketed_comma():
6524    """Tests for find_unbracketed_comma."""
6525    assert find_unbracketed_comma('()', 0) is None
6526    assert find_unbracketed_comma('(),', 0) == 2
6527    assert find_unbracketed_comma('((,),)', 0) is None
6528    assert find_unbracketed_comma('((,),),', 0) == 6
6529    assert find_unbracketed_comma('((,),),', 1) == 4
6530    assert find_unbracketed_comma(',,,', 1) == 1
6531    assert find_unbracketed_comma('",,",","', 0) == 4
6532    assert find_unbracketed_comma('"""\n,,\n""","""\n,,\n"""', 0) == 10
6533    assert find_unbracketed_comma('"""\n,,\n""","""\n,,\n"""', 4) == 4
6534    assert find_unbracketed_comma('"""\n,,\n"""+"""\n,,\n"""', 0) is None
6535    assert find_unbracketed_comma('\n\n,\n', 0) == 2
6536
6537
6538def get_expr_src(src, call_node):
6539    """
6540    Gets the string containing the source code for the expression passed
6541    as the first argument to a function call, given the string source of
6542    the file that defines the function and the AST node for the function
6543    call.
6544    """
6545    # Find the child node for the first (and only) argument
6546    arg_expr = call_node.args[0]
6547
6548    # If get_source_segment is available, use that
6549    if hasattr(ast, "get_source_segment"):
6550        return textwrap.dedent(
6551            ast.get_source_segment(src, arg_expr)
6552        ).strip()
6553    else:
6554        # We're going to have to do this ourself: find the start of the
6555        # expression and state-machine to find a matching paren
6556        start = get_src_index(src, call_node.lineno, call_node.col_offset)
6557        open_paren = src.index('(', start)
6558        end = find_closing_item(src, open_paren, '()')
6559        # Note: can't be None because that would have been a SyntaxError
6560        first_comma = find_unbracketed_comma(src, open_paren + 1)
6561        # Could be None if it's a 1-argument function
6562        if first_comma is not None:
6563            end = min(end, first_comma)
6564        return textwrap.dedent(src[open_paren + 1:end]).strip()
6565
6566
6567def get_ref_src(src, node):
6568    """
6569    Gets the string containing the source code for a variable reference,
6570    attribute, or subscript.
6571    """
6572    # Use get_source_segment if it's available
6573    if hasattr(ast, "get_source_segment"):
6574        return ast.get_source_segment(src, node)
6575    else:
6576        # We're going to have to do this ourself: find the start of the
6577        # expression and state-machine to find its end
6578        start = get_src_index(src, node.lineno, node.col_offset)
6579
6580        # Figure out the end point
6581        if isinstance(node, ast.Attribute):
6582            # Find sub-attributes so we can count syntactic periods to
6583            # figure out where the name part begins to get the span
6584            inner_period_count = 0
6585            for node in ast.walk(node):
6586                if isinstance(node, ast.Attribute):
6587                    inner_period_count += 1
6588            inner_period_count -= 1  # for the node itself
6589            dot = find_nth_attribute_period(src, start, inner_period_count)
6590            end = find_identifier_end(src, dot + 1)
6591
6592        elif isinstance(node, ast.Name):
6593            # It's just an identifier so we can find the end
6594            end = find_identifier_end(src, start)
6595
6596        elif isinstance(node, ast.Subscript):
6597            # Find start of sub-expression so we can find opening brace
6598            # and then match it to find the end
6599            inner = node.slice
6600            if isinstance(inner, ast.Slice):
6601                pass
6602            elif hasattr(ast, "Index") and isinstance(inner, ast.Index):
6603                # 3.7 Index has a "value"
6604                inner = inner.value
6605            elif hasattr(ast, "ExtSlice") and isinstance(inner, ast.ExtSlice):
6606                # 3.7 ExtSlice has "dims"
6607                inner = inner.dims[0]
6608            else:
6609                raise TypeError(
6610                    f"Unexpected subscript slice type {type(inner)} for"
6611                    f" node:\n{ast.dump(node)}"
6612                )
6613            sub_start = get_src_index(src, inner.lineno, inner.col_offset)
6614            end = find_closing_item(src, sub_start - 1, "[]")
6615
6616        return src[start:end + 1]
6617
6618
6619def deepish_copy(obj, memo=None):
6620    """
6621    Returns the deepest possible copy of the given object, using
6622    copy.deepcopy wherever possible and making shallower copies
6623    elsewhere. Basically a middle-ground between copy.deepcopy and
6624    copy.copy.
6625    """
6626    if memo is None:
6627        memo = {}
6628    if id(obj) in memo:
6629        return memo[id(obj)]
6630
6631    try:
6632        result = copy.deepcopy(obj)  # not sure about memo dict compatibility
6633        memo[id(obj)] = result
6634        return result
6635
6636    except Exception:
6637        if isinstance(obj, list):
6638            result = []
6639            memo[id(obj)] = result
6640            result.extend(deepish_copy(item, memo) for item in obj)
6641            return result
6642        elif isinstance(obj, tuple):
6643            # Note: no way to pre-populate the memo, but also no way to
6644            # construct an infinitely-recursive tuple without having
6645            # some mutable structure at some layer...
6646            result = (deepish_copy(item, memo) for item in obj)
6647            memo[id(obj)] = result
6648            return result
6649        elif isinstance(obj, dict):
6650            result = {}
6651            memo[id(obj)] = result
6652            result.update(
6653                {
6654                    deepish_copy(key, memo): deepish_copy(value, memo)
6655                    for key, value in obj.items()
6656                }
6657            )
6658            return result
6659        elif isinstance(obj, set):
6660            result = set()
6661            memo[id(obj)] = result
6662            result |= set(deepish_copy(item, memo) for item in obj)
6663            return result
6664        else:
6665            # Can't go deeper I guess
6666            try:
6667                result = copy.copy(obj)
6668                memo[id(obj)] = result
6669                return result
6670            except Exception:
6671                # Can't even copy (e.g., a module)
6672                result = obj
6673                memo[id(obj)] = result
6674                return result
6675
6676
6677def get_external_calling_frame():
6678    """
6679    Uses the inspect module to get a reference to the stack frame which
6680    called into the `optimism` module. Returns None if it can't find an
6681    appropriate call frame in the current stack.
6682
6683    Remember to del the result after you're done with it, so that
6684    garbage doesn't pile up.
6685    """
6686    myname = __name__
6687    cf = inspect.currentframe()
6688    while (
6689        hasattr(cf, "f_back")
6690    and cf.f_globals.get("__name__") == myname
6691    ):
6692        cf = cf.f_back
6693
6694    return cf
6695
6696
6697def get_module(stack_frame):
6698    """
6699    Given a stack frame, returns a reference to the module where the
6700    code from that frame was defined.
6701
6702    Returns None if it can't figure that out.
6703    """
6704    other_name = stack_frame.f_globals.get("__name__", None)
6705    return sys.modules.get(other_name)
6706
6707
6708def get_filename(stack_frame, speculate_filename=True):
6709    """
6710    Given a stack frame, returns the filename of the file in which the
6711    code which created that stack frame was defined. Returns None if
6712    that information isn't available via a __file__ global, or if
6713    speculate_filename is True (the default), uses the value of the
6714    frame's f_code.co_filename, which may not always be a real file on
6715    disk, or which is weird circumstances could be the name of a file on
6716    disk which is *not* where the code came from.
6717    """
6718    filename = stack_frame.f_globals.get("__file__")
6719    if filename is None and speculate_filename:
6720        filename = stack_frame.f_code.co_filename
6721    return filename
6722
6723
6724def get_code_line(stack_frame):
6725    """
6726    Given a stack frame, returns
6727    """
6728    return stack_frame.f_lineno
6729
6730
6731def evaluate_in_context(node, stack_frame):
6732    """
6733    Given an AST node which is an expression, returns the value of that
6734    expression as evaluated in the context of the given stack frame.
6735
6736    Shallow copies of the stack frame's locals and globals are made in
6737    an attempt to prevent the code being evaluated from having any
6738    impact on the stack frame's values, but of course there's still some
6739    possibility of side effects...
6740    """
6741    expr = ast.Expression(node)
6742    code = compile(
6743        expr,
6744        stack_frame.f_globals.get("__file__", "__unknown__"),
6745        'eval'
6746    )
6747    return eval(
6748        code,
6749        copy.copy(stack_frame.f_globals),
6750        copy.copy(stack_frame.f_locals)
6751    )
6752
6753
6754def walk_ast_in_order(node):
6755    """
6756    Yields all of the descendants of the given node (or list of nodes)
6757    in execution order. Note that this has its limits, for example, if
6758    we run it on the code:
6759
6760    ```py
6761    x = [A for y in C if D]
6762    ```
6763
6764    It will yield the nodes for C, then y, then D, then A, and finally
6765    x, but in actual execution the nodes for D and A may be executed
6766    multiple times before x is assigned.
6767    """
6768    if node is None:
6769        pass  # empty iterator
6770    elif isinstance(node, (list, tuple)):
6771        for child in node:
6772            yield from walk_ast_in_order(child)
6773    else:  # must be an ast.something
6774        # Note: the node itself will be yielded LAST
6775        if isinstance(node, (ast.Module, ast.Interactive, ast.Expression)):
6776            yield from walk_ast_in_order(node.body)
6777        elif (
6778            hasattr(ast, "FunctionType")
6779        and isinstance(node, ast.FunctionType)
6780        ):
6781            yield from walk_ast_in_order(node.argtypes)
6782            yield from walk_ast_in_order(node.returns)
6783        elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
6784            yield from walk_ast_in_order(node.args)
6785            yield from walk_ast_in_order(node.returns)
6786            yield from walk_ast_in_order(reversed(node.decorator_list))
6787            yield from walk_ast_in_order(node.body)
6788        elif isinstance(node, ast.ClassDef):
6789            yield from walk_ast_in_order(node.bases)
6790            yield from walk_ast_in_order(node.keywords)
6791            yield from walk_ast_in_order(reversed(node.decorator_list))
6792            yield from walk_ast_in_order(node.body)
6793        elif isinstance(node, ast.Return):
6794            yield from walk_ast_in_order(node.value)
6795        elif isinstance(node, ast.Delete):
6796            yield from walk_ast_in_order(node.targets)
6797        elif isinstance(node, ast.Assign):
6798            yield from walk_ast_in_order(node.value)
6799            yield from walk_ast_in_order(node.targets)
6800        elif isinstance(node, ast.AugAssign):
6801            yield from walk_ast_in_order(node.value)
6802            yield from walk_ast_in_order(node.target)
6803        elif isinstance(node, ast.AnnAssign):
6804            yield from walk_ast_in_order(node.value)
6805            yield from walk_ast_in_order(node.annotation)
6806            yield from walk_ast_in_order(node.target)
6807        elif isinstance(node, (ast.For, ast.AsyncFor)):
6808            yield from walk_ast_in_order(node.iter)
6809            yield from walk_ast_in_order(node.target)
6810            yield from walk_ast_in_order(node.body)
6811            yield from walk_ast_in_order(node.orelse)
6812        elif isinstance(node, (ast.While, ast.If, ast.IfExp)):
6813            yield from walk_ast_in_order(node.test)
6814            yield from walk_ast_in_order(node.body)
6815            yield from walk_ast_in_order(node.orelse)
6816        elif isinstance(node, (ast.With, ast.AsyncWith)):
6817            yield from walk_ast_in_order(node.items)
6818            yield from walk_ast_in_order(node.body)
6819        elif isinstance(node, ast.Raise):
6820            yield from walk_ast_in_order(node.cause)
6821            yield from walk_ast_in_order(node.exc)
6822        elif isinstance(node, ast.Try):
6823            yield from walk_ast_in_order(node.body)
6824            yield from walk_ast_in_order(node.handlers)
6825            yield from walk_ast_in_order(node.orelse)
6826            yield from walk_ast_in_order(node.finalbody)
6827        elif isinstance(node, ast.Assert):
6828            yield from walk_ast_in_order(node.test)
6829            yield from walk_ast_in_order(node.msg)
6830        elif isinstance(node, ast.Expr):
6831            yield from walk_ast_in_order(node.value)
6832        # Import, ImportFrom, Global, Nonlocal, Pass, Break, and
6833        # Continue each have no executable content, so we'll yield them
6834        # but not any children
6835
6836        elif isinstance(node, ast.BoolOp):
6837            yield from walk_ast_in_order(node.values)
6838        elif HAS_WALRUS and isinstance(node, ast.NamedExpr):
6839            yield from walk_ast_in_order(node.value)
6840            yield from walk_ast_in_order(node.target)
6841        elif isinstance(node, ast.BinOp):
6842            yield from walk_ast_in_order(node.left)
6843            yield from walk_ast_in_order(node.right)
6844        elif isinstance(node, ast.UnaryOp):
6845            yield from walk_ast_in_order(node.operand)
6846        elif isinstance(node, ast.Lambda):
6847            yield from walk_ast_in_order(node.args)
6848            yield from walk_ast_in_order(node.body)
6849        elif isinstance(node, ast.Dict):
6850            for i in range(len(node.keys)):
6851                yield from walk_ast_in_order(node.keys[i])
6852                yield from walk_ast_in_order(node.values[i])
6853        elif isinstance(node, (ast.Tuple, ast.List, ast.Set)):
6854            yield from walk_ast_in_order(node.elts)
6855        elif isinstance(node, (ast.ListComp, ast.SetComp, ast.GeneratorExp)):
6856            yield from walk_ast_in_order(node.generators)
6857            yield from walk_ast_in_order(node.elt)
6858        elif isinstance(node, ast.DictComp):
6859            yield from walk_ast_in_order(node.generators)
6860            yield from walk_ast_in_order(node.key)
6861            yield from walk_ast_in_order(node.value)
6862        elif isinstance(node, (ast.Await, ast.Yield, ast.YieldFrom)):
6863            yield from walk_ast_in_order(node.value)
6864        elif isinstance(node, ast.Compare):
6865            yield from walk_ast_in_order(node.left)
6866            yield from walk_ast_in_order(node.comparators)
6867        elif isinstance(node, ast.Call):
6868            yield from walk_ast_in_order(node.func)
6869            yield from walk_ast_in_order(node.args)
6870            yield from walk_ast_in_order(node.keywords)
6871        elif isinstance(node, ast.FormattedValue):
6872            yield from walk_ast_in_order(node.value)
6873            yield from walk_ast_in_order(node.format_spec)
6874        elif isinstance(node, ast.JoinedStr):
6875            yield from walk_ast_in_order(node.values)
6876        elif isinstance(node, (ast.Attribute, ast.Starred)):
6877            yield from walk_ast_in_order(node.value)
6878        elif isinstance(node, ast.Subscript):
6879            yield from walk_ast_in_order(node.value)
6880            yield from walk_ast_in_order(node.slice)
6881        elif isinstance(node, ast.Slice):
6882            yield from walk_ast_in_order(node.lower)
6883            yield from walk_ast_in_order(node.upper)
6884            yield from walk_ast_in_order(node.step)
6885        # Constant and Name nodes don't have executable contents
6886
6887        elif isinstance(node, ast.comprehension):
6888            yield from walk_ast_in_order(node.iter)
6889            yield from walk_ast_in_order(node.ifs)
6890            yield from walk_ast_in_order(node.target)
6891        elif isinstance(node, ast.ExceptHandler):
6892            yield from walk_ast_in_order(node.type)
6893            yield from walk_ast_in_order(node.body)
6894        elif isinstance(node, ast.arguments):
6895            yield from walk_ast_in_order(node.defaults)
6896            yield from walk_ast_in_order(node.kw_defaults)
6897            if hasattr(node, "posonlyargs"):
6898                yield from walk_ast_in_order(node.posonlyargs)
6899            yield from walk_ast_in_order(node.args)
6900            yield from walk_ast_in_order(node.vararg)
6901            yield from walk_ast_in_order(node.kwonlyargs)
6902            yield from walk_ast_in_order(node.kwarg)
6903        elif isinstance(node, ast.arg):
6904            yield from walk_ast_in_order(node.annotation)
6905        elif isinstance(node, ast.keyword):
6906            yield from walk_ast_in_order(node.value)
6907        elif isinstance(node, ast.withitem):
6908            yield from walk_ast_in_order(node.context_expr)
6909            yield from walk_ast_in_order(node.optional_vars)
6910        # alias and typeignore have no executable members
6911
6912        # Finally, yield this node itself
6913        yield node
6914
6915
6916def find_call_nodes_on_line(node, frame, function, lineno):
6917    """
6918    Given an AST node, a stack frame, a function object, and a line
6919    number, looks for all function calls which occur on the given line
6920    number and which are calls to the given function (as evaluated in
6921    the given stack frame).
6922
6923    Note that calls to functions defined as part of the given AST cannot
6924    be found in this manner, because the objects being called are newly
6925    created and one could not possibly pass a reference to one of them
6926    into this function. For that reason, if the function argument is a
6927    string, any function call whose call part matches the given string
6928    will be matched. Normally only Name nodes can match this way, but if
6929    ast.unparse is available, the string will also attempt to match
6930    (exactly) against the unparsed call expression.
6931
6932    Calls that start on the given line number will match, but if there
6933    are no such calls, then a call on a preceding line whose expression
6934    includes the target line will be looked for and may match.
6935
6936    The return value will be a list of ast.Call nodes, and they will be
6937    ordered in the same order that those nodes would be executed when
6938    the line of code is executed.
6939    """
6940    def call_matches(call_node):
6941        """
6942        Locally-defined matching predicate.
6943        """
6944        nonlocal function
6945        call_expr = call_node.func
6946        return (
6947            (
6948                isinstance(function, str)
6949            and (
6950                    (
6951                        isinstance(call_expr, ast.Name)
6952                    and call_expr.id == function
6953                    )
6954                 or (
6955                        isinstance(call_expr, ast.Attribute)
6956                    and call_expr.attr == function
6957                    )
6958                 or (
6959                        hasattr(ast, "unparse")
6960                    and ast.unparse(call_expr) == function
6961                    )
6962                )
6963            )
6964         or (
6965                not isinstance(function, str)
6966            and evaluate_in_context(call_expr, frame) is function
6967            )
6968        )
6969
6970    result = []
6971    all_on_line = []
6972    for child in walk_ast_in_order(node):
6973        # only consider call nodes on the target line
6974        if (
6975            hasattr(child, "lineno")
6976        and child.lineno == lineno
6977        ):
6978            all_on_line.append(child)
6979            if isinstance(child, ast.Call) and call_matches(child):
6980                result.append(child)
6981
6982    # If we didn't find any candidates, look outwards from ast nodes on
6983    # the target line to find a Call that encompasses them...
6984    if len(result) == 0:
6985        for on_line in all_on_line:
6986            here = getattr(on_line, "parent", None)
6987            while (
6988                here is not None
6989            and not isinstance(
6990                    here,
6991                    # Call (what we're looking for) plus most nodes that
6992                    # indicate there couldn't be a call grandparent:
6993                    (
6994                        ast.Call,
6995                        ast.Module, ast.Interactive, ast.Expression,
6996                        ast.FunctionDef, ast.AsyncFunctionDef,
6997                        ast.ClassDef,
6998                        ast.Return,
6999                        ast.Delete,
7000                        ast.Assign, ast.AugAssign, ast.AnnAssign,
7001                        ast.For, ast.AsyncFor,
7002                        ast.While,
7003                        ast.If,
7004                        ast.With, ast.AsyncWith,
7005                        ast.Raise,
7006                        ast.Try,
7007                        ast.Assert,
7008                        ast.Assert,
7009                        ast.Assert,
7010                        ast.Assert,
7011                        ast.Assert,
7012                        ast.Assert,
7013                        ast.Assert,
7014                        ast.Assert,
7015                    )
7016                )
7017            ):
7018                here = getattr(here, "parent", None)
7019
7020            # If we found a Call that includes the target line as one
7021            # of its children...
7022            if isinstance(here, ast.Call) and call_matches(here):
7023                result.append(here)
7024
7025    return result
7026
7027
7028def assign_parents(root):
7029    """
7030    Given an AST node, assigns "parent" attributes to each sub-node
7031    indicating their parent AST node. Assigns None as the value of the
7032    parent attribute of the root node.
7033    """
7034    for node in ast.walk(root):
7035        for child in ast.iter_child_nodes(node):
7036            child.parent = node
7037
7038    root.parent = None
7039
7040
7041def is_inside_call_func(node):
7042    """
7043    Given an AST node which has a parent attribute, traverses parents to
7044    see if this node is part of the func attribute of a Call node.
7045    """
7046    if not hasattr(node, "parent") or node.parent is None:
7047        return False
7048    if isinstance(node.parent, ast.Call) and node.parent.func is node:
7049        return True
7050    else:
7051        return is_inside_call_func(node.parent)
7052
7053
7054def tag_for(located):
7055    """
7056    Given a dictionary which has 'file' and 'line' slots, returns a
7057    string to be used as the tag for a test with 'filename:line' as the
7058    format. Unless the `DETAIL_LEVEL` is 2 or higher, the filename will
7059    be shown without the full path.
7060    """
7061    filename = located.get('file', '???')
7062    if DETAIL_LEVEL < 2:
7063        filename = os.path.basename(filename)
7064    line = located.get('line', '?')
7065    return f"{filename}:{line}"
7066
7067
7068def get_my_location(speculate_filename=True):
7069    """
7070    Fetches the filename and line number of the external module whose
7071    call into this module ended up invoking this function. Returns a
7072    dictionary with "file" and "line" keys.
7073
7074    If speculate_filename is False, then the filename will be set to
7075    None in cases where a __file__ global cannot be found, instead of
7076    using f_code.co_filename as a backup. In some cases, this is useful
7077    because f_code.co_filename may not be a valid file.
7078    """
7079    frame = get_external_calling_frame()
7080    try:
7081        filename = get_filename(frame, speculate_filename)
7082        lineno = get_code_line(frame)
7083    finally:
7084        del frame
7085
7086    return { "file": filename, "line": lineno }
7087
7088
7089def get_my_context(function_or_name):
7090    """
7091    Returns a dictionary indicating the context of a function call,
7092    assuming that this function is called from within a function with the
7093    given name (or from within the given function), and that that
7094    function is being called from within a different module. The result
7095    has the following keys:
7096
7097    - file: The filename of the calling module
7098    - line: The line number on which the call to the function occurred
7099    - src: The source code string of the calling module
7100    - expr: An AST node storing the expression passed as the first
7101        argument to the function
7102    - expr_src: The source code string of the expression passed as the
7103        first argument to the function
7104    - values: A dictionary mapping source code fragments to their
7105        values, for each variable reference in the test expression. These
7106        are deepish copies of the values encountered.
7107    - relevant: A list of source code fragments which appear in the
7108        values dictionary which are judged to be most-relevant to the
7109        result of the test.
7110
7111    Currently, the relevant list just lists any fragments which aren't
7112    found in the func slot of Call nodes, under the assumption that we
7113    don't care as much about the values of the functions we're calling.
7114
7115    Prints a warning and returns a dictionary with just "file" and
7116    "line" entries if the other context info is unavailable.
7117    """
7118    if isinstance(function_or_name, types.FunctionType):
7119        function_name = function_or_name.__name__
7120    else:
7121        function_name = function_or_name
7122
7123    frame = get_external_calling_frame()
7124    try:
7125        filename = get_filename(frame)
7126        lineno = get_code_line(frame)
7127        if filename is None:
7128            src = None
7129        else:
7130            try:
7131                with open(filename, 'r') as fin:
7132                    src = fin.read()
7133            except Exception:
7134                # Try to get contents from the linecache as a backup...
7135                try:
7136                    src = ''.join(linecache.getlines(filename))
7137                except Exception:
7138                    # We'll assume here that the source is something like
7139                    # an interactive shell so we won't warn unless the
7140                    # detail level is turned up.
7141                    if DETAIL_LEVEL >= 2:
7142                        print(
7143                            "Warning: unable to get calling code's source.",
7144                            file=PRINT_TO
7145                        )
7146                        print(
7147                            (
7148                                "Call is on line {} of module {} from file"
7149                                " '{}'"
7150                            ).format(
7151                                lineno,
7152                                frame.f_globals.get("__name__"),
7153                                filename
7154                            ),
7155                            file=PRINT_TO
7156                        )
7157                    src = None
7158
7159        if src is None:
7160            return {
7161                "file": filename,
7162                "line": lineno
7163            }
7164
7165        src_node = ast.parse(src, filename=filename, mode='exec')
7166        assign_parents(src_node)
7167        candidates = find_call_nodes_on_line(
7168            src_node,
7169            frame,
7170            function_or_name,
7171            lineno
7172        )
7173
7174        # What if there are zero candidates?
7175        if len(candidates) == 0:
7176            print(
7177                f"Warning: unable to find call node for {function_name}"
7178                f" on line {lineno} of file {filename}.",
7179                file=PRINT_TO
7180            )
7181            return {
7182                "file": filename,
7183                "line": lineno
7184            }
7185
7186        # Figure out how many calls to get_my_context have happened
7187        # referencing this line before, so that we know which call on
7188        # this line we might be
7189        prev_this_line = COMPLETED_PER_LINE\
7190            .setdefault(function_name, {})\
7191            .setdefault((filename, lineno), 0)
7192        match = candidates[prev_this_line % len(candidates)]
7193
7194        # Record this call so the next one will grab the subsequent
7195        # candidate
7196        COMPLETED_PER_LINE[function_name][(filename, lineno)] += 1
7197
7198        arg_expr = match.args[0]
7199
7200        # Add .parent attributes
7201        assign_parents(arg_expr)
7202
7203        # Source code for the expression
7204        expr_src = get_expr_src(src, match)
7205
7206        # Prepare our result dictionary
7207        result = {
7208            "file": filename,
7209            "line": lineno,
7210            "src": src,
7211            "expr": arg_expr,
7212            "expr_src": expr_src,
7213            "values": {},
7214            "relevant": set()
7215        }
7216
7217        # Walk expression to find values for each variable
7218        for node in ast.walk(arg_expr):
7219            # If it's potentially a reference to a variable...
7220            if isinstance(
7221                node,
7222                (ast.Attribute, ast.Subscript, ast.Name)
7223            ):
7224                key = get_ref_src(src, node)
7225                if key not in result["values"]:
7226                    # Don't re-evaluate multiply-reference expressions
7227                    # Note: we assume they won't take on multiple
7228                    # values; if they did, even our first evaluation
7229                    # would probably be inaccurate.
7230                    val = deepish_copy(evaluate_in_context(node, frame))
7231                    result["values"][key] = val
7232                    if not is_inside_call_func(node):
7233                        result["relevant"].add(key)
7234
7235        return result
7236
7237    finally:
7238        del frame
7239
7240
7241#----------------#
7242# Output control #
7243#----------------#
7244
7245def messagesAsErrors(activate=True):
7246    """
7247    Sets `PRINT_TO` to `sys.stderr` so that messages from optimism will
7248    appear as error messages, rather than as normal printed output. This
7249    is the default behavior, but you can pass `False` as the argument to
7250    set it to `sys.stdout` instead, causing messages to appear as normal
7251    output.
7252    """
7253    global PRINT_TO
7254    if activate:
7255        PRINT_TO = sys.stderr
7256    else:
7257        PRINT_TO = sys.stdout
ALL_TRIALS = {}

All test cases and code checks that have been created, organized by test-suite names. By default all trials are added to the 'default' test suite, but this can be changed using testSuite. Each entry has a test suite name as the key and a list of Trial (i.e., CodeChecks and/or TestCase) objects as the value.

ALL_OUTCOMES = {}

The outcomes of all checks, including independent expectations (via expect or expectType) and Trial-based expectations (via methods like TestManager.checkCodeContains, TestCase.checkReturnValue, etc.).

These are stored per test suite as lists with the suite name (see _CURRENT_SUITE_NAME) as the key. They are ordered in the order that checks happen in, but may be cleared if resetTestSuite is called.

Each list entry is a 3-tuple with a boolean indicating success/failure, a tag string indicating the file name and line number of the test, and a string indicating the message that was displayed (which might have depended on the current detail level or message suppression, etc.).

COMPLETED_PER_LINE = {}

A dictionary mapping function names to dictionaries mapping (filename, line-number) pairs to counts. Each count represents the number of functions of that name which have finished execution on the given line of the given file already. This allows us to figure out which expression belongs to which invocation if get_my_context is called multiple times from the same line of code.

DETAIL_LEVEL = 0

The current detail level, which controls how verbose our messages are. See detailLevel.

SKIP_ON_FAILURE = None

Controls which checks get skipped when a check fails. If set to 'all', ALL checks will be skipped once one fails, until clearFailure is called. If set to 'case', subsequent checks for the same test case will be skipped when one fails. If set to 'manager', then all checks for any case from a case manager will be skipped when any check for any case derived from that manager fails. Any other value (including the default None) will disable the skipping of checks based on failures.

SUPPRESS_ON_FAILURE = None

Controls how failure messages are suppressed after a check fails. By default, details from failures after the first failure for a given test manager will printed as if the detail level were -1 as long as the default level is 0. You can set this to 'case' to only suppress details on a per-case basis, or 'all' to suppress all detail printing after any failure. clearFailure can be used to reset the failure status, and setting the base detail level above 1 will also undo the suppression.

Set this to None or any other value that's not one of the strings mentioned above to disable this functionality.

CHECK_FAILED = False

Remembers whether we've failed a check yet or not. If True and SKIP_ON_FAILURE is set to 'all', all checks will be skipped, or if SUPPRESS_ON_FAILURE is 'all' and the detail level is 0, failure details will be suppressed. Use clearFailure to reset this and resume checking without changing SKIP_ON_FAILURE if you need to.

COLORS = True

Whether to print ANSI color control sequences to color the printed output or not.

IGNORE_TRAILING_WHITESPACE = True

Controls equality and inclusion tests on strings, including multiline strings and strings within other data structures, causing them to ignore trailing whitespace. True by default, since trailing whitespace is hard to reason about because it's invisible.

Trailing whitespace is any sequence whitespace characters before a newline character (which is the only thing we count as a line break, meaning \r\n breaks are only accepted if IGNORE_TRAILING_WHITESPACE is on). Specifically, we use the rstrip method after splitting on \n.

Additionally, in multi-line scenarios, if there is a single extra line containing just whitespace, that will be ignored, although that's not the same as applying rstrip to the entire string, since if there are multiple extra trailing newline characters, that still counts as a difference.

FLOAT_REL_TOLERANCE = 1e-08

The relative tolerance for floating-point similarity (see cmath.isclose).

FLOAT_ABS_TOLERANCE = 1e-08

The absolute tolerance for floating-point similarity (see cmath.isclose).

class TestError(builtins.Exception): 
485class TestError(Exception):
486    """
487    An error with the testing mechanisms, as opposed to an error with
488    the actual code being tested.
489    """
490    pass

An error with the testing mechanisms, as opposed to an error with the actual code being tested.

Inherited Members
builtins.Exception
Exception
builtins.BaseException
with_traceback
args
class Trial: 
497class Trial:
498    """
499    Base class for both code checks and test cases, delineating common
500    functionality like having outcomes. All trials are derived from a
501    manager.
502    """
503    def __init__(self, manager):
504        """
505        A manager must be specified, but that's it. This does extra
506        things like registering the trial in the current test suite (see
507        `testSuite`) and figuring out the location tag for the trial.
508        """
509        self.manager = manager
510
511        # Location and tag for trial creation
512        self.location = get_my_location()
513        self.tag = tag_for(self.location)
514
515        # List of outcomes of individual checks/tests based on this
516        # trial. Each is a triple with a True/False indicator for
517        # success/failure, a string tag for the expectation, and a full
518        # result message.
519        self.outcomes = []
520
521        # Whether or not a check has failed for this trial yet.
522        self.any_failed = False
523
524        # How to describe this trial; should be overridden
525        self.description = "an unknown trial"
526
527        # Register as a trial
528        ALL_TRIALS.setdefault(_CURRENT_SUITE_NAME, []).append(self)
529
530    def trialDetails(self):
531        """
532        Returns a pair of strings containing base and extra details
533        describing what was tested by this trial. If the base details
534        capture all available information, the extra details value will
535        be None.
536
537        This method is abstract and only sub-class implementations
538        actually do anything.
539        """
540        raise NotImplementedError(
541            "Cannot get trial details for a Trial or TestCase; you must"
542            " create a specific kind of trial like a FunctionCase to be"
543            " able to get trial details."
544        )
545
546    def _create_success_message(
547        self,
548        tag,
549        details,
550        extra_details=None,
551        include_test_details=True
552    ):
553        """
554        Returns an expectation success message (a string) for an
555        expectation with the given tag, using the given details and
556        extra details. Unless `include_test_details` is set to False,
557        details of the test expression/block will also be included (but
558        only when the detail level is at least 1). The tag should be a
559        filename:lineno string indicating where the expectation
560        originated.
561        """
562        # Detail level 1 gives more output for successes
563        if DETAIL_LEVEL < 1:
564            result = f"✓ {tag}"
565        else:  # Detail level is at least 1
566            result = (
567                f"✓ expectation from {tag} met for {self.description}"
568            )
569            detail_msg = indent(details, 2)
570            if not detail_msg.startswith('\n'):
571                detail_msg = '\n' + detail_msg
572
573            if DETAIL_LEVEL >= 2 and extra_details:
574                extra_detail_msg = indent(extra_details, 2)
575                if not extra_detail_msg.startswith('\n'):
576                    extra_detail_msg = '\n' + extra_detail_msg
577
578                detail_msg += extra_detail_msg
579
580            # Test details unless suppressed
581            if include_test_details:
582                test_base, test_extra = self.trialDetails()
583                detail_msg += '\n' + indent(test_base, 2)
584                if DETAIL_LEVEL >= 2 and test_extra is not None:
585                    detail_msg += '\n' + indent(test_extra, 2)
586
587            result += detail_msg
588
589        return result
590
591    def _create_failure_message(
592        self,
593        tag,
594        details,
595        extra_details=None,
596        include_test_details=True
597    ):
598        """
599        Creates a failure message string for an expectation with the
600        given tag that includes the details and/or extra details
601        depending on the current global detail level. Normally,
602        information about the test that was run is included as well, but
603        you can set `include_test_details` to False to prevent this.
604        """
605        # Detail level controls initial message
606        if DETAIL_LEVEL < 1:
607            result = f"✗ {tag}"
608        else:
609            result = (
610                f"✗ expectation from {tag} NOT met for"
611                f" {self.description}"
612            )
613
614        # Assemble our details message
615        detail_msg = ''
616
617        # Figure out if we should suppress details
618        suppress = self._should_suppress()
619
620        # Detail level controls printing of detail messages
621        if (DETAIL_LEVEL == 0 and not suppress) or DETAIL_LEVEL >= 1:
622            detail_msg += '\n' + indent(details, 2)
623        if DETAIL_LEVEL >= 1 and extra_details:
624            detail_msg += '\n' + indent(extra_details, 2)
625
626        # Test details unless suppressed
627        if include_test_details:
628            test_base, test_extra = self.trialDetails()
629            if (DETAIL_LEVEL == 0 and not suppress) or DETAIL_LEVEL >= 1:
630                detail_msg += '\n' + indent(test_base, 2)
631            if DETAIL_LEVEL >= 1 and test_extra is not None:
632                detail_msg += '\n' + indent(test_extra, 2)
633
634        return result + detail_msg
635
636    def _print_skip_message(self, tag, reason):
637        """
638        Prints a standard message about the trial being skipped, using
639        the given tag and a reason (shown only if detail level is 1+).
640        """
641        # Detail level controls initial message
642        if DETAIL_LEVEL < 1:
643            msg = f"~ {tag} (skipped)"
644        else:
645            msg = (
646                f"~ expectation at {tag} for {self.description}"
647                f" skipped ({reason})"
648            )
649        print_message(msg, color=msg_color("skipped"))
650
651    def _should_skip(self):
652        """
653        Returns True if this trial should be skipped based on a previous
654        failure and the `SKIP_ON_FAILURE` mode.
655        """
656        return (
657            (SKIP_ON_FAILURE == "all" and CHECK_FAILED)
658         or (SKIP_ON_FAILURE == "case" and self.any_failed)
659         or (SKIP_ON_FAILURE == "manager" and self.manager.any_failed)
660        )
661
662    def _should_suppress(self):
663        """
664        Returns True if failure details for this trial should be
665        suppressed based on a previous failure and the
666        `SUPPRESS_ON_FAILURE` mode.
667        """
668        return (
669            (SUPPRESS_ON_FAILURE == "all" and CHECK_FAILED)
670         or (SUPPRESS_ON_FAILURE == "case" and self.any_failed)
671         or (SUPPRESS_ON_FAILURE == "manager" and self.manager.any_failed)
672        )
673
674    def _register_outcome(self, passed, tag, message):
675        """
676        Registers an outcome for this trial. `passed` should be either
677        True or False indicating whether the check passed, `tag` is a
678        string to label the outcome with, and `message` is the message
679        displayed by the check. This appends an entry to `self.outcomes`
680        with the passed boolean, the tag, and the message in a tuple, and
681        it sets `self.any_failed` and `self.manager.any_failed` if the
682        outcome is a failure.
683        """
684        global CHECK_FAILED
685        self.outcomes.append((passed, tag, message))
686        _register_outcome(passed, tag, message)
687        if not passed:
688            CHECK_FAILED = True
689            self.any_failed = True
690            self.manager.any_failed = True

Base class for both code checks and test cases, delineating common functionality like having outcomes. All trials are derived from a manager.

Trial(manager) 
503    def __init__(self, manager):
504        """
505        A manager must be specified, but that's it. This does extra
506        things like registering the trial in the current test suite (see
507        `testSuite`) and figuring out the location tag for the trial.
508        """
509        self.manager = manager
510
511        # Location and tag for trial creation
512        self.location = get_my_location()
513        self.tag = tag_for(self.location)
514
515        # List of outcomes of individual checks/tests based on this
516        # trial. Each is a triple with a True/False indicator for
517        # success/failure, a string tag for the expectation, and a full
518        # result message.
519        self.outcomes = []
520
521        # Whether or not a check has failed for this trial yet.
522        self.any_failed = False
523
524        # How to describe this trial; should be overridden
525        self.description = "an unknown trial"
526
527        # Register as a trial
528        ALL_TRIALS.setdefault(_CURRENT_SUITE_NAME, []).append(self)

A manager must be specified, but that's it. This does extra things like registering the trial in the current test suite (see testSuite) and figuring out the location tag for the trial.

def trialDetails(self) 
530    def trialDetails(self):
531        """
532        Returns a pair of strings containing base and extra details
533        describing what was tested by this trial. If the base details
534        capture all available information, the extra details value will
535        be None.
536
537        This method is abstract and only sub-class implementations
538        actually do anything.
539        """
540        raise NotImplementedError(
541            "Cannot get trial details for a Trial or TestCase; you must"
542            " create a specific kind of trial like a FunctionCase to be"
543            " able to get trial details."
544        )

Returns a pair of strings containing base and extra details describing what was tested by this trial. If the base details capture all available information, the extra details value will be None.

This method is abstract and only sub-class implementations actually do anything.

class CodeChecks(Trial): 
697class CodeChecks(Trial):
698    """
699    Represents one or more checks performed against code structure
700    (without running that code) rather than against the behavior of code.
701    Like a `TestCase`, it can have outcomes (one for each check
702    performed) and is tracked globally.
703    """
704    def __init__(self, manager):
705        """
706        A manager must be specified, but that's it.
707        """
708        super().__init__(manager)
709
710        # How to describe this trial
711        self.description = f"code checks for {self.manager.tag}"
712
713    def trialDetails(self):
714        """
715        The base details describe what kind of code was run; the full
716        details include the AST dump.
717        """
718        baseDetails = self.manager.checkDetails()
719
720        # Get representation of the AST we checked:
721        if self.manager.syntaxTree is not None:
722            if sys.version_info < (3, 9):
723                astRepr = ast.dump(self.manager.syntaxTree)
724            else:
725                astRepr = ast.dump(self.manager.syntaxTree, indent=2)
726            return (
727                baseDetails,
728                "The code structure is:" + indent(astRepr, 2)
729            )
730        else:
731            return (
732                baseDetails,
733                "No code was available for checking."
734            )
735
736    def performCheck(self, checkFor):
737        """
738        Performs a check for the given `ASTRequirement` within the AST
739        managed by this code check's manager. Prints a success/failure
740        message, registers an outcome, and returns True on success and
741        False on failure (including when there's a partial match).
742        Returns `None` if the check is skipped (which can happen based on
743        a previous failure depending on settings, or when the AST to
744        check is not available.
745        """
746        tag = tag_for(get_my_location())
747
748        # Skip the check if there's nothing to test
749        if self._should_skip() or self.manager.syntaxTree is None:
750            self._print_skip_message(tag, "source code not available")
751            return None
752        else:
753            # Perform the check
754            matches = checkFor.allMatches(self.manager.syntaxTree)
755            if not matches.isFull:
756                passed = False
757                if checkFor.maxMatches == 0:
758                    contains = "contains a structure that it should not"
759                elif checkFor.minMatches > 1:
760                    contains = (
761                        "does not contain enough of the expected"
762                        " structures"
763                    )
764                else:
765                    contains = "does not contain the expected structure"
766            else:
767                passed = True
768                if checkFor.maxMatches == 0:
769                    contains = (
770                        "does not contain any structures it should not"
771                    )
772                elif checkFor.minMatches > 1:
773                    contains = "contains enough expected structures"
774                else:
775                    contains = "contains the expected structure"
776
777            structureString = checkFor.fullStructure()
778            base_msg = f"""\
779Code {contains}:
780{indent(structureString, 2)}"""
781            if matches.isPartial:
782                base_msg += f"""
783Although it does partially satisfy the requirement:
784{indent(str(matches), 2)}"""
785
786            # TODO: have partial/full structure strings?
787            extra_msg = ""
788
789            if passed:
790                msg = self._create_success_message(tag, base_msg, extra_msg)
791                msg_cat = "succeeded"
792            else:
793                msg = self._create_failure_message(tag, base_msg, extra_msg)
794                msg_cat = "failed"
795
796        # Print our message
797        print_message(msg, color=msg_color(msg_cat))
798
799        # Record outcome
800        self._register_outcome(passed, tag, msg)
801        return passed

Represents one or more checks performed against code structure (without running that code) rather than against the behavior of code. Like a TestCase, it can have outcomes (one for each check performed) and is tracked globally.

CodeChecks(manager) 
704    def __init__(self, manager):
705        """
706        A manager must be specified, but that's it.
707        """
708        super().__init__(manager)
709
710        # How to describe this trial
711        self.description = f"code checks for {self.manager.tag}"

A manager must be specified, but that's it.

def trialDetails(self) 
713    def trialDetails(self):
714        """
715        The base details describe what kind of code was run; the full
716        details include the AST dump.
717        """
718        baseDetails = self.manager.checkDetails()
719
720        # Get representation of the AST we checked:
721        if self.manager.syntaxTree is not None:
722            if sys.version_info < (3, 9):
723                astRepr = ast.dump(self.manager.syntaxTree)
724            else:
725                astRepr = ast.dump(self.manager.syntaxTree, indent=2)
726            return (
727                baseDetails,
728                "The code structure is:" + indent(astRepr, 2)
729            )
730        else:
731            return (
732                baseDetails,
733                "No code was available for checking."
734            )

The base details describe what kind of code was run; the full details include the AST dump.

def performCheck(self, checkFor) 
736    def performCheck(self, checkFor):
737        """
738        Performs a check for the given `ASTRequirement` within the AST
739        managed by this code check's manager. Prints a success/failure
740        message, registers an outcome, and returns True on success and
741        False on failure (including when there's a partial match).
742        Returns `None` if the check is skipped (which can happen based on
743        a previous failure depending on settings, or when the AST to
744        check is not available.
745        """
746        tag = tag_for(get_my_location())
747
748        # Skip the check if there's nothing to test
749        if self._should_skip() or self.manager.syntaxTree is None:
750            self._print_skip_message(tag, "source code not available")
751            return None
752        else:
753            # Perform the check
754            matches = checkFor.allMatches(self.manager.syntaxTree)
755            if not matches.isFull:
756                passed = False
757                if checkFor.maxMatches == 0:
758                    contains = "contains a structure that it should not"
759                elif checkFor.minMatches > 1:
760                    contains = (
761                        "does not contain enough of the expected"
762                        " structures"
763                    )
764                else:
765                    contains = "does not contain the expected structure"
766            else:
767                passed = True
768                if checkFor.maxMatches == 0:
769                    contains = (
770                        "does not contain any structures it should not"
771                    )
772                elif checkFor.minMatches > 1:
773                    contains = "contains enough expected structures"
774                else:
775                    contains = "contains the expected structure"
776
777            structureString = checkFor.fullStructure()
778            base_msg = f"""\
779Code {contains}:
780{indent(structureString, 2)}"""
781            if matches.isPartial:
782                base_msg += f"""
783Although it does partially satisfy the requirement:
784{indent(str(matches), 2)}"""
785
786            # TODO: have partial/full structure strings?
787            extra_msg = ""
788
789            if passed:
790                msg = self._create_success_message(tag, base_msg, extra_msg)
791                msg_cat = "succeeded"
792            else:
793                msg = self._create_failure_message(tag, base_msg, extra_msg)
794                msg_cat = "failed"
795
796        # Print our message
797        print_message(msg, color=msg_color(msg_cat))
798
799        # Record outcome
800        self._register_outcome(passed, tag, msg)
801        return passed

Performs a check for the given ASTRequirement within the AST managed by this code check's manager. Prints a success/failure message, registers an outcome, and returns True on success and False on failure (including when there's a partial match). Returns None if the check is skipped (which can happen based on a previous failure depending on settings, or when the AST to check is not available.

class NoResult: 
808class NoResult:
809    """
810    A special class used to indicate the absence of a result when None
811    is a valid result value.
812    """
813    pass

A special class used to indicate the absence of a result when None is a valid result value.

NoResult()
def mimicInput(prompt) 
816def mimicInput(prompt):
817    """
818    A function which mimics the functionality of the default `input`
819    function: it prints a prompt, reads input from stdin, and then
820    returns that input. Unlike normal input, it prints what it reads
821    from stdin to stdout, which in normal situations would result in
822    that stuff showing up on the console twice, but when stdin is set to
823    an alternate stream (as we do when capturing input/output) that
824    doesn't happen.
825    """
826    print(prompt, end='')
827    incomming = sys.stdin.readline()
828    # Strip newline on incomming value
829    incomming = incomming.rstrip('\n\r')
830    print(incomming, end='\n')
831    return incomming

A function which mimics the functionality of the default input function: it prints a prompt, reads input from stdin, and then returns that input. Unlike normal input, it prints what it reads from stdin to stdout, which in normal situations would result in that stuff showing up on the console twice, but when stdin is set to an alternate stream (as we do when capturing input/output) that doesn't happen.

class TestCase(Trial): 
 834class TestCase(Trial):
 835    """
 836    Represents a specific test to run, managing things like specific
 837    arguments, inputs or available variables that need to be in place.
 838    Derived from a `TestManager` using the `TestManager.case` method.
 839
 840    `TestCase` is abstract; subclasses should override a least the `run`
 841    and `trialDetails` functions.
 842    """
 843    def __init__(self, manager):
 844        """
 845        A manager must be specified, but that's it. This does extra
 846        things like registering the case in the current test suite (see
 847        `testSuite`) and figuring out the location tag for the case.
 848        """
 849        super().__init__(manager)
 850
 851        # How to describe this trial
 852        self.description = f"test case at {self.tag}"
 853
 854        # Inputs to provide on stdin
 855        self.inputs = None
 856
 857        # Results of running this case
 858        self.results = None
 859
 860        # Whether to echo captured printed outputs (overrides global)
 861        self.echo = None
 862
 863    def provideInputs(self, *inputLines):
 864        """
 865        Sets up fake inputs (each argument must be a string and is used
 866        for one line of input) for this test case. When information is
 867        read from stdin during the test, including via the `input`
 868        function, these values are the result. If you don't call
 869        `provideInputs`, then the test will pause and wait for real user
 870        input when `input` is called.
 871
 872        You must call this before the test actually runs (i.e., before
 873        `TestCase.run` or one of the `check` functions is called),
 874        otherwise you'll get an error.
 875        """
 876        if self.results is not None:
 877            raise TestError(
 878                "You cannot provide inputs because this test case has"
 879                " already been run."
 880            )
 881        self.inputs = inputLines
 882
 883    def showPrintedLines(self, show=True):
 884        """
 885        Overrides the global `showPrintedLines` setting for this test.
 886        Use None as the parameter to remove the override.
 887        """
 888        self.echo = show
 889
 890    def _run(self, payload):
 891        """
 892        Given a payload (a zero-argument function that returns a tuple
 893        with a result and a scope dictionary), runs the payload while
 894        managing things like output capturing and input mocking. Sets the
 895        `self.results` field to reflect the results of the run, which
 896        will be a dictionary that has the following slots:
 897
 898        - "result": The result value from a function call. This key
 899            will not be present for tests that don't have a result, like
 900            file or code block tests. To achieve this with a custom
 901            payload, have the payload return `NoResult` as the first part
 902            of the tuple it returns.
 903        - "output": The output printed during the test. Will be an empty
 904            string if nothing gets printed.
 905        - "error": An Exception object representing an error that
 906            occurred during the test, or None if no errors happened.
 907        - "traceback": If an exception occurred, this will be a string
 908            containing the traceback for that exception. Otherwise it
 909            will be None.
 910        - "scope": The second part of the tuple returned by the payload,
 911            which should be a dictionary representing the scope of the
 912            code run by the test. It may also be `None` in cases where no
 913            scope is available (e.g., function alls).
 914
 915        In addition to being added to the results slot, this dictionary
 916        is also returned.
 917        """
 918        # Set up the `input` function to echo what is typed, and to only
 919        # read from stdin (in case we're in a notebook where input would
 920        # do something else).
 921        original_input = builtins.input
 922        builtins.input = mimicInput
 923
 924        # Set up a capturing stream for output
 925        outputCapture = CapturingStream()
 926        outputCapture.install()
 927        if self.echo or (self.echo is None and _SHOW_OUTPUT):
 928            outputCapture.echo()
 929
 930        # Set up fake input contents, AND also monkey-patch the input
 931        # function since in some settings like notebooks input doesn't
 932        # just read from stdin
 933        if self.inputs is not None:
 934            fakeInput = io.StringIO('\n'.join(self.inputs))
 935            original_stdin = sys.stdin
 936            sys.stdin = fakeInput
 937
 938        # Set up default values before we run things
 939        error = None
 940        tb = None
 941        value = NoResult
 942        scope = None
 943
 944        # Actually run the test
 945        try:
 946            value, scope = payload()
 947        except Exception as e:
 948            # Catch any error that occurred
 949            error = e
 950            tb = traceback.format_exc()
 951        finally:
 952            # Release stream captures and reset the input function
 953            outputCapture.uninstall()
 954            builtins.input = original_input
 955            if self.inputs is not None:
 956                sys.stdin = original_stdin
 957
 958        # Grab captured output
 959        output = outputCapture.getvalue()
 960
 961        # Create self.results w/ output, error, and maybe result value
 962        self.results = {
 963            "output": output,
 964            "error": error,
 965            "traceback": tb,
 966            "scope": scope
 967        }
 968        if value is not NoResult:
 969            self.results["result"] = value
 970
 971        # Return new results object
 972        return self.results
 973
 974    def run(self):
 975        """
 976        Runs this test case, capturing printed output and supplying fake
 977        input if `TestCase.provideInputs` has been called. Stores the
 978        results in `self.results`. This will be called once
 979        automatically the first time an expectation method like
 980        `TestCase.checkReturnValue` is used, but the cached value will
 981        be re-used for subsequent expectations, unless you manually call
 982        this method again.
 983
 984        This method is overridden by specific test case types.
 985        """
 986        raise NotImplementedError(
 987            "Cannot run a TestCase; you must create a specific kind of"
 988            " test case like a FunctionCase to be able to run it."
 989        )
 990
 991    def fetchResults(self):
 992        """
 993        Fetches the results of the test, which will run the test if it
 994        hasn't already been run, but otherwise will just return the
 995        latest cached results.
 996
 997        `run` describes the format of the results.
 998        """
 999        if self.results is None:
1000            self.run()
1001        return self.results
1002
1003    def checkReturnValue(self, expectedValue):
1004        """
1005        Checks the result value for this test case, comparing it against
1006        the given expected value and printing a message about success or
1007        failure depending on whether they are considered different by
1008        the `findFirstDifference` function.
1009
1010        If this is the first check performed using this test case, the
1011        test case will run; otherwise a cached result will be used.
1012
1013        This method returns True if the expectation is met and False if
1014        it is not, in addition to printing a message indicating
1015        success/failure and recording that message along with the status
1016        and tag in `self.outcomes`. If the check is skipped, it returns
1017        None and does not add an entry to `self.outcomes`.
1018        """
1019        results = self.fetchResults()
1020
1021        # Figure out the tag for this expectation
1022        tag = tag_for(get_my_location())
1023
1024        # Skip this check if the case has failed already
1025        if self._should_skip():
1026            self._print_skip_message(tag, "prior test failed")
1027            # Note that we don't add an outcome here, and we return None
1028            # instead of True or False
1029            return None
1030
1031        # Figure out whether we've got an error or an actual result
1032        if results["error"] is not None:
1033            # An error during testing
1034            tb = results["traceback"]
1035            tblines = tb.splitlines()
1036            if len(tblines) < 12:
1037                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1038                extra_msg = None
1039            else:
1040                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1041                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1042                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1043
1044            msg = self._create_failure_message(
1045                tag,
1046                base_msg,
1047                extra_msg
1048            )
1049            print_message(msg, color=msg_color("failed"))
1050            self._register_outcome(False, tag, msg)
1051            return False
1052
1053        elif "result" not in results:
1054            # Likely impossible, since we verified the category above
1055            # and we're in a condition where no error was logged...
1056            msg = self._create_failure_message(
1057                tag,
1058                (
1059                    "This test case does not have a result value. (Did"
1060                    " you mean to use checkPrintedLines?)"
1061                )
1062            )
1063            print_message(msg, color=msg_color("failed"))
1064            self._register_outcome(False, tag, msg)
1065            return False
1066
1067        else:
1068            # We produced a result, so check equality
1069
1070            # Check equivalence
1071            passed = False
1072            firstDiff = findFirstDifference(results["result"], expectedValue)
1073            if firstDiff is None:
1074                equivalence = "equivalent to"
1075                passed = True
1076            else:
1077                equivalence = "NOT equivalent to"
1078
1079            # Get short/long versions of result/expected
1080            short_result = ellipsis(repr(results["result"]), 72)
1081            full_result = repr(results["result"])
1082            short_expected = ellipsis(repr(expectedValue), 72)
1083            full_expected = repr(expectedValue)
1084
1085            # Create base/extra messages
1086            if (
1087                short_result == full_result
1088            and short_expected == full_expected
1089            ):
1090                base_msg = (
1091                    f"Result:\n{indent(short_result, 2)}\nwas"
1092                    f" {equivalence} the expected value:\n"
1093                    f"{indent(short_expected, 2)}"
1094                )
1095                extra_msg = None
1096                if (
1097                    firstDiff is not None
1098                and differencesAreSubtle(short_result, short_expected)
1099                ):
1100                    base_msg += (
1101                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1102                    )
1103            else:
1104                base_msg = (
1105                    f"Result:\n{indent(short_result, 2)}\nwas"
1106                    f" {equivalence} the expected value:\n"
1107                    f"{indent(short_expected, 2)}"
1108                )
1109                extra_msg = ""
1110                if (
1111                    firstDiff is not None
1112                and differencesAreSubtle(short_result, short_expected)
1113                ):
1114                    base_msg += (
1115                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1116                    )
1117                if short_result != full_result:
1118                    extra_msg += (
1119                        f"Full result:\n{indent(full_result, 2)}\n"
1120                    )
1121                if short_expected != full_expected:
1122                    extra_msg += (
1123                        f"Full expected value:\n"
1124                        f"{indent(full_expected, 2)}\n"
1125                    )
1126
1127            if passed:
1128                msg = self._create_success_message(
1129                    tag,
1130                    base_msg,
1131                    extra_msg
1132                )
1133                print_message(msg, color=msg_color("succeeded"))
1134                self._register_outcome(True, tag, msg)
1135                return True
1136            else:
1137                msg = self._create_failure_message(
1138                    tag,
1139                    base_msg,
1140                    extra_msg
1141                )
1142                print_message(msg, color=msg_color("failed"))
1143                self._register_outcome(False, tag, msg)
1144                return False
1145
1146    def checkVariableValue(self, varName, expectedValue):
1147        """
1148        Checks the value of a variable established by this test case,
1149        which should be a code block or file test (use `checkReturnValue`
1150        instead for checking the result of a function test). It checks
1151        that a variable with a certain name (given as a string) has a
1152        certain expected value, and prints a message about success or
1153        failure depending on whether the actual value and expected value
1154        are considered different by the `findFirstDifference` function.
1155
1156        If this is the first check performed using this test case, the
1157        test case will run; otherwise a cached result will be used.
1158
1159        This method returns True if the expectation is met and False if
1160        it is not, in addition to printing a message indicating
1161        success/failure and recording that message along with the status
1162        and tag in `self.outcomes`. If the check is skipped, it returns
1163        None and does not add an entry to `self.outcomes`.
1164        """
1165        results = self.fetchResults()
1166
1167        # Figure out the tag for this expectation
1168        tag = tag_for(get_my_location())
1169
1170        # Skip this check if the case has failed already
1171        if self._should_skip():
1172            self._print_skip_message(tag, "prior test failed")
1173            # Note that we don't add an outcome here, and we return None
1174            # instead of True or False
1175            return None
1176
1177        # Figure out whether we've got an error or an actual result
1178        if results["error"] is not None:
1179            # An error during testing
1180            tb = results["traceback"]
1181            tblines = tb.splitlines()
1182            if len(tblines) < 12:
1183                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1184                extra_msg = None
1185            else:
1186                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1187                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1188                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1189
1190            msg = self._create_failure_message(
1191                tag,
1192                base_msg,
1193                extra_msg
1194            )
1195            print_message(msg, color=msg_color("failed"))
1196            self._register_outcome(False, tag, msg)
1197            return False
1198
1199        else:
1200            # No error, so look for our variable
1201            scope = results["scope"]
1202
1203            if varName not in scope:
1204                msg = self._create_failure_message(
1205                    tag,
1206                    f"No variable named '{varName}' was created.",
1207                    None
1208                )
1209                print_message(msg, color=msg_color("failed"))
1210                self._register_outcome(False, tag, msg)
1211                return False
1212
1213            # Check equivalence
1214            passed = False
1215            value = scope[varName]
1216            firstDiff = findFirstDifference(value, expectedValue)
1217            if firstDiff is None:
1218                equivalence = "equivalent to"
1219                passed = True
1220            else:
1221                equivalence = "NOT equivalent to"
1222
1223            # Get short/long versions of result/expected
1224            short_value = ellipsis(repr(value), 72)
1225            full_value = repr(value)
1226            short_expected = ellipsis(repr(expectedValue), 72)
1227            full_expected = repr(expectedValue)
1228
1229            # Create base/extra messages
1230            if (
1231                short_value == full_value
1232            and short_expected == full_expected
1233            ):
1234                base_msg = (
1235                    f"Variable '{varName}' with"
1236                    f" value:\n{indent(short_value, 2)}\nwas"
1237                    f" {equivalence} the expected value:\n"
1238                    f"{indent(short_expected, 2)}"
1239                )
1240                extra_msg = None
1241                if (
1242                    firstDiff is not None
1243                and differencesAreSubtle(short_value, short_expected)
1244                ):
1245                    base_msg += (
1246                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1247                    )
1248            else:
1249                base_msg = (
1250                    f"Variable '{varName}' with"
1251                    f" value:\n{indent(short_value, 2)}\nwas"
1252                    f" {equivalence} the expected value:\n"
1253                    f"{indent(short_expected, 2)}"
1254                )
1255                extra_msg = ""
1256                if (
1257                    firstDiff is not None
1258                and differencesAreSubtle(short_value, short_expected)
1259                ):
1260                    base_msg += (
1261                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1262                    )
1263                if short_value != full_value:
1264                    extra_msg += (
1265                        f"Full value:\n{indent(full_value, 2)}\n"
1266                    )
1267                if short_expected != full_expected:
1268                    extra_msg += (
1269                        f"Full expected value:\n"
1270                        f"{indent(full_expected, 2)}\n"
1271                    )
1272
1273            if passed:
1274                msg = self._create_success_message(
1275                    tag,
1276                    base_msg,
1277                    extra_msg
1278                )
1279                print_message(msg, color=msg_color("succeeded"))
1280                self._register_outcome(True, tag, msg)
1281                return True
1282            else:
1283                msg = self._create_failure_message(
1284                    tag,
1285                    base_msg,
1286                    extra_msg
1287                )
1288                print_message(msg, color=msg_color("failed"))
1289                self._register_outcome(False, tag, msg)
1290                return False
1291
1292    def checkPrintedLines(self, *expectedLines):
1293        """
1294        Checks that the exact printed output captured during the test
1295        matches a sequence of strings each specifying one line of the
1296        output. Note that the global `IGNORE_TRAILING_WHITESPACE`
1297        affects how this function treats line matches.
1298
1299        If this is the first check performed using this test case, the
1300        test case will run; otherwise a cached result will be used.
1301
1302        This method returns True if the check succeeds and False if it
1303        fails, in addition to printing a message indicating
1304        success/failure and recording that message along with the status
1305        and tag in `self.outcomes`. If the check is skipped, it returns
1306        None and does not add an entry to `self.outcomes`.
1307        """
1308        # Fetch captured output
1309        results = self.fetchResults()
1310        output = results["output"]
1311
1312        # Figure out the tag for this expectation
1313        tag = tag_for(get_my_location())
1314
1315        # Skip this check if the case has failed already
1316        if self._should_skip():
1317            self._print_skip_message(tag, "prior test failed")
1318            # Note that we don't add an outcome here, and we return None
1319            # instead of True or False
1320            return None
1321
1322        # Figure out whether we've got an error or an actual result
1323        if results["error"] is not None:
1324            # An error during testing
1325            tb = results["traceback"]
1326            tblines = tb.splitlines()
1327            if len(tblines) < 12:
1328                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1329                extra_msg = None
1330            else:
1331                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1332                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1333                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1334
1335            msg = self._create_failure_message(
1336                tag,
1337                base_msg,
1338                extra_msg
1339            )
1340            print_message(msg, color=msg_color("failed"))
1341            self._register_outcome(False, tag, msg)
1342            return False
1343
1344        else:
1345            # We produced printed output, so check it
1346
1347            # Get lines/single versions
1348            expected = '\n'.join(expectedLines) + '\n'
1349            # If the output doesn't end with a newline, don't add one to
1350            # our expectation either...
1351            if not output.endswith('\n'):
1352                expected = expected[:-1]
1353
1354            # Figure out equivalence category
1355            equivalence = None
1356            passed = False
1357            firstDiff = findFirstDifference(output, expected)
1358            if output == expected:
1359                equivalence = "exactly the same as"
1360                passed = True
1361            elif firstDiff is None:
1362                equivalence = "equivalent to"
1363                passed = True
1364            else:
1365                equivalence = "NOT the same as"
1366                # passed remains False
1367
1368            # Get short/long representations of our strings
1369            short, long = dual_string_repr(output)
1370            short_exp, long_exp = dual_string_repr(expected)
1371
1372            # Construct base and extra messages
1373            if short == long and short_exp == long_exp:
1374                base_msg = (
1375                    f"Printed lines:\n{indent(short, 2)}\nwere"
1376                    f" {equivalence} the expected printed"
1377                    f" lines:\n{indent(short_exp, 2)}"
1378                )
1379                if not passed:
1380                    base_msg += (
1381                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1382                    )
1383                extra_msg = None
1384            else:
1385                base_msg = (
1386                    f"Printed lines:\n{indent(short, 2)}\nwere"
1387                    f" {equivalence} the expected printed"
1388                    f" lines:\n{indent(short_exp, 2)}"
1389                )
1390                if not passed:
1391                    base_msg += (
1392                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1393                    )
1394                extra_msg = ""
1395                if short != long:
1396                    extra_msg += f"Full printed lines:\n{indent(long, 2)}\n"
1397                if short_exp != long_exp:
1398                    extra_msg += (
1399                        f"Full expected printed"
1400                        f" lines:\n{indent(long_exp, 2)}\n"
1401                    )
1402
1403            if passed:
1404                msg = self._create_success_message(
1405                    tag,
1406                    base_msg,
1407                    extra_msg
1408                )
1409                print_message(msg, color=msg_color("succeeded"))
1410                self._register_outcome(True, tag, msg)
1411                return True
1412            else:
1413                msg = self._create_failure_message(
1414                    tag,
1415                    base_msg,
1416                    extra_msg
1417                )
1418                print_message(msg, color="1;31" if COLORS else None)
1419                self._register_outcome(False, tag, msg)
1420                return False
1421
1422    def checkPrintedFragment(self, fragment, copies=1, allowExtra=False):
1423        """
1424        Works like checkPrintedLines, except instead of requiring that
1425        the printed output exactly match a set of lines, it requires that
1426        a certain fragment of text appears somewhere within the printed
1427        output (or perhaps that multiple non-overlapping copies appear,
1428        if the copies argument is set to a number higher than the
1429        default of 1).
1430
1431        If allowExtra is set to True, more than the specified number of
1432        copies will be ignored, but by default, extra copies are not
1433        allowed.
1434
1435        The fragment is matched against the entire output as a single
1436        string, so it may contain newlines and if it does these will
1437        only match newlines in the captured output. If
1438        `IGNORE_TRAILING_WHITESPACE` is active (it's on by default), the
1439        trailing whitespace in the output will be removed before
1440        matching, and trailing whitespace in the fragment will also be
1441        removed IF it has a newline after it (trailing whitespace at the
1442        end of the string with no final newline will be retained).
1443
1444        This function returns True if the check succeeds and False if it
1445        fails, and prints a message either way. If the check is skipped,
1446        it returns None and does not add an entry to `self.outcomes`.
1447        """
1448        # Fetch captured output
1449        results = self.fetchResults()
1450        output = results["output"]
1451
1452        # Figure out the tag for this expectation
1453        tag = tag_for(get_my_location())
1454
1455        # Skip this check if the case has failed already
1456        if self._should_skip():
1457            self._print_skip_message(tag, "prior test failed")
1458            # Note that we don't add an outcome here, and we return None
1459            # instead of True or False
1460            return None
1461
1462        # Figure out whether we've got an error or an actual result
1463        if results["error"] is not None:
1464            # An error during testing
1465            tb = results["traceback"]
1466            tblines = tb.splitlines()
1467            if len(tblines) < 12:
1468                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1469                extra_msg = None
1470            else:
1471                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1472                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1473                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1474
1475            msg = self._create_failure_message(
1476                tag,
1477                base_msg,
1478                extra_msg
1479            )
1480            print_message(msg, color=msg_color("failed"))
1481            self._register_outcome(False, tag, msg)
1482            return False
1483
1484        else:
1485            # We produced printed output, so check it
1486            if IGNORE_TRAILING_WHITESPACE:
1487                matches = re.findall(
1488                    re.escape(trimWhitespace(fragment, True)),
1489                    trimWhitespace(output)
1490                )
1491            else:
1492                matches = re.findall(re.escape(fragment), output)
1493            passed = False
1494            if copies == 1:
1495                copiesPhrase = ""
1496                exactly = ""
1497                atLeast = "at least "
1498            else:
1499                copiesPhrase = f"{copies} copies of "
1500                exactly = "exactly "
1501                atLeast = "at least "
1502
1503            fragShort, fragLong = dual_string_repr(fragment)
1504            outShort, outLong = dual_string_repr(output)
1505
1506            if len(matches) == copies:
1507                passed = True
1508                base_msg = (
1509                    f"Found {exactly}{copiesPhrase}the target"
1510                    f" fragment in the printed output."
1511                    f"\nFragment was:\n{indent(fragShort, 2)}"
1512                    f"\nOutput was:\n{indent(outShort, 2)}"
1513                )
1514            elif allowExtra and len(matches) > copies:
1515                passed = True
1516                base_msg = (
1517                    f"Found {atLeast}{copiesPhrase}the target"
1518                    f" fragment in the printed output (found"
1519                    f" {len(matches)})."
1520                    f"\nFragment was:\n{indent(fragShort, 2)}"
1521                    f"\nOutput was:\n{indent(outShort, 2)}"
1522                )
1523            else:
1524                passed = False
1525                base_msg = (
1526                    f"Did not find {copiesPhrase}the target fragment"
1527                    f" in the printed output (found {len(matches)})."
1528                    f"\nFragment was:\n{indent(fragShort, 2)}"
1529                    f"\nOutput was:\n{indent(outShort, 2)}"
1530                )
1531
1532            extra_msg = ""
1533            if fragLong != fragShort:
1534                extra_msg += f"Full fragment was:\n{indent(fragLong, 2)}"
1535
1536            if outLong != outShort:
1537                if not extra_msg.endswith('\n'):
1538                    extra_msg += '\n'
1539                extra_msg += f"Full output was:\n{indent(outLong, 2)}"
1540
1541            if passed:
1542                msg = self._create_success_message(
1543                    tag,
1544                    base_msg,
1545                    extra_msg
1546                )
1547                print_message(msg, color=msg_color("succeeded"))
1548                self._register_outcome(True, tag, msg)
1549                return True
1550            else:
1551                msg = self._create_failure_message(
1552                    tag,
1553                    base_msg,
1554                    extra_msg
1555                )
1556                print_message(msg, color="1;31" if COLORS else None)
1557                self._register_outcome(False, tag, msg)
1558                return False
1559
1560    def checkFileLines(self, filename, *lines):
1561        """
1562        Works like `checkPrintedLines`, but checks for lines in the
1563        specified file, rather than checking for printed lines.
1564        """
1565        # Figure out the tag for this expectation
1566        tag = tag_for(get_my_location())
1567
1568        # Skip this check if the case has failed already
1569        if self._should_skip():
1570            self._print_skip_message(tag, "prior test failed")
1571            # Note that we don't add an outcome here, and we return None
1572            # instead of True or False
1573            return None
1574
1575        # Fetch the results to actually run the test!
1576        expected = '\n'.join(lines) + '\n'
1577        results = self.fetchResults()
1578
1579        # Figure out whether we've got an error or an actual result
1580        if results["error"] is not None:
1581            # An error during testing
1582            tb = results["traceback"]
1583            tblines = tb.splitlines()
1584            if len(tblines) < 12:
1585                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1586                extra_msg = None
1587            else:
1588                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1589                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1590                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1591
1592            msg = self._create_failure_message(
1593                tag,
1594                base_msg,
1595                extra_msg
1596            )
1597            print_message(msg, color=msg_color("failed"))
1598            self._register_outcome(False, tag, msg)
1599            return False
1600
1601        else:
1602            # The test was able to run, so check the file contents
1603
1604            # Fetch file contents
1605            try:
1606                with open(filename, 'r', newline='') as fileInput:
1607                    fileContents = fileInput.read()
1608            except (OSError, FileNotFoundError, PermissionError):
1609                # We can't even read the file!
1610                msg = self._create_failure_message(
1611                    tag,
1612                    f"Expected file '{filename}' cannot be read.",
1613                    None
1614                )
1615                print_message(msg, color=msg_color("failed"))
1616                self._register_outcome(False, tag, msg)
1617                return False
1618
1619            # If the file doesn't end with a newline, don't add one to
1620            # our expectation either...
1621            if not fileContents.endswith('\n'):
1622                expected = expected[:-1]
1623
1624            # Get lines/single versions
1625            firstDiff = findFirstDifference(fileContents, expected)
1626            equivalence = None
1627            passed = False
1628            if fileContents == expected:
1629                equivalence = "exactly the same as"
1630                passed = True
1631            elif firstDiff is None:
1632                equivalence = "equivalent to"
1633                passed = True
1634            else:
1635                # Some other kind of difference
1636                equivalence = "NOT the same as"
1637                # passed remains False
1638
1639            # Get short/long representations of our strings
1640            short, long = dual_string_repr(fileContents)
1641            short_exp, long_exp = dual_string_repr(expected)
1642
1643            # Construct base and extra messages
1644            if short == long and short_exp == long_exp:
1645                base_msg = (
1646                    f"File contents:\n{indent(short, 2)}\nwere"
1647                    f" {equivalence} the expected file"
1648                    f" contents:\n{indent(short_exp, 2)}"
1649                )
1650                if not passed:
1651                    base_msg += (
1652                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1653                    )
1654                extra_msg = None
1655            else:
1656                base_msg = (
1657                    f"File contents:\n{indent(short, 2)}\nwere"
1658                    f" {equivalence} the expected file"
1659                    f" contents:\n{indent(short_exp, 2)}"
1660                )
1661                if not passed:
1662                    base_msg += (
1663                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1664                    )
1665                extra_msg = ""
1666                if short != long:
1667                    extra_msg += f"Full file contents:\n{indent(long, 2)}\n"
1668                if short_exp != long_exp:
1669                    extra_msg += (
1670                        f"Full expected file"
1671                        f" contents:\n{indent(long_exp, 2)}\n"
1672                    )
1673
1674            if passed:
1675                msg = self._create_success_message(
1676                    tag,
1677                    base_msg,
1678                    extra_msg
1679                )
1680                print_message(msg, color=msg_color("succeeded"))
1681                self._register_outcome(True, tag, msg)
1682                return True
1683            else:
1684                msg = self._create_failure_message(
1685                    tag,
1686                    base_msg,
1687                    extra_msg
1688                )
1689                print_message(msg, color="1;31" if COLORS else None)
1690                self._register_outcome(False, tag, msg)
1691                return False
1692
1693    def checkCustom(self, checker, *args, **kwargs):
1694        """
1695        Sets up a custom check using a testing function. The provided
1696        function will be given one argument, plus any additional
1697        arguments given to this function. The first and/or only argument
1698        to the checker function will be a dictionary with the following
1699        keys:
1700
1701        - "case": The test case object on which `checkCustom` was called.
1702            This could be used to do things like access arguments passed
1703            to the function being tested for a `FunctionCase` for
1704            example.
1705        - "output": Output printed by the test case, as a string.
1706        - "result": the result value (for function tests only, otherwise
1707            this key will not be present).
1708        - "error": the error that occurred (or None if no error
1709            occurred).
1710        - "traceback": the traceback (a string, or None if there was no
1711            error).
1712        - "scope": For file and code block cases, the variable dictionary
1713            created by the file/code block. `None` for function cases.
1714
1715        The testing function must return True to indicate success and
1716        False for failure. If it returns something other than True or
1717        False, it will be counted as a failure, that value will be shown
1718        as part of the test result if the `DETAIL_LEVEL` is 1 or higher,
1719        and this method will return False.
1720
1721        If this check is skipped (e.g., because of a previous failure),
1722        this method returns None and does not add an entry to
1723        `self.outcomes`; the custom checker function won't be called in
1724        that case.
1725        """
1726        results = self.fetchResults()
1727        # Add a 'case' entry
1728        checker_input = copy.copy(results)
1729        checker_input["case"] = self
1730
1731        # Figure out the tag for this expectation
1732        tag = tag_for(get_my_location())
1733
1734        # Skip this check if the case has failed already
1735        if self._should_skip():
1736            self._print_skip_message(tag, "prior test failed")
1737            # Note that we don't add an outcome here, and we return None
1738            # instead of True or False
1739            return None
1740
1741        # Only run the checker if we're not skipping the test
1742        test_result = checker(checker_input, *args, **kwargs)
1743
1744        if test_result is True:
1745            msg = self._create_success_message(tag, "Custom check passed.")
1746            print_message(msg, color=msg_color("succeeded"))
1747            self._register_outcome(True, tag, msg)
1748            return True
1749        elif test_result is False:
1750            msg = self._create_failure_message(tag, "Custom check failed")
1751            print_message(msg, color="1;31" if COLORS else None)
1752            self._register_outcome(False, tag, msg)
1753            return False
1754        else:
1755            msg = self._create_failure_message(
1756                tag,
1757                "Custom check failed:\n" + indent(str(test_result), 2),
1758            )
1759            print_message(msg, color="1;31" if COLORS else None)
1760            self._register_outcome(False, tag, msg)
1761            return False

Represents a specific test to run, managing things like specific arguments, inputs or available variables that need to be in place. Derived from a TestManager using the TestManager.case method.

TestCase is abstract; subclasses should override a least the run and trialDetails functions.

TestCase(manager) 
843    def __init__(self, manager):
844        """
845        A manager must be specified, but that's it. This does extra
846        things like registering the case in the current test suite (see
847        `testSuite`) and figuring out the location tag for the case.
848        """
849        super().__init__(manager)
850
851        # How to describe this trial
852        self.description = f"test case at {self.tag}"
853
854        # Inputs to provide on stdin
855        self.inputs = None
856
857        # Results of running this case
858        self.results = None
859
860        # Whether to echo captured printed outputs (overrides global)
861        self.echo = None

A manager must be specified, but that's it. This does extra things like registering the case in the current test suite (see testSuite) and figuring out the location tag for the case.

def provideInputs(self, *inputLines) 
863    def provideInputs(self, *inputLines):
864        """
865        Sets up fake inputs (each argument must be a string and is used
866        for one line of input) for this test case. When information is
867        read from stdin during the test, including via the `input`
868        function, these values are the result. If you don't call
869        `provideInputs`, then the test will pause and wait for real user
870        input when `input` is called.
871
872        You must call this before the test actually runs (i.e., before
873        `TestCase.run` or one of the `check` functions is called),
874        otherwise you'll get an error.
875        """
876        if self.results is not None:
877            raise TestError(
878                "You cannot provide inputs because this test case has"
879                " already been run."
880            )
881        self.inputs = inputLines

Sets up fake inputs (each argument must be a string and is used for one line of input) for this test case. When information is read from stdin during the test, including via the input function, these values are the result. If you don't call provideInputs, then the test will pause and wait for real user input when input is called.

You must call this before the test actually runs (i.e., before TestCase.run or one of the check functions is called), otherwise you'll get an error.

def showPrintedLines(self, show=True) 
883    def showPrintedLines(self, show=True):
884        """
885        Overrides the global `showPrintedLines` setting for this test.
886        Use None as the parameter to remove the override.
887        """
888        self.echo = show

Overrides the global showPrintedLines setting for this test. Use None as the parameter to remove the override.

def run(self) 
974    def run(self):
975        """
976        Runs this test case, capturing printed output and supplying fake
977        input if `TestCase.provideInputs` has been called. Stores the
978        results in `self.results`. This will be called once
979        automatically the first time an expectation method like
980        `TestCase.checkReturnValue` is used, but the cached value will
981        be re-used for subsequent expectations, unless you manually call
982        this method again.
983
984        This method is overridden by specific test case types.
985        """
986        raise NotImplementedError(
987            "Cannot run a TestCase; you must create a specific kind of"
988            " test case like a FunctionCase to be able to run it."
989        )

Runs this test case, capturing printed output and supplying fake input if TestCase.provideInputs has been called. Stores the results in self.results. This will be called once automatically the first time an expectation method like TestCase.checkReturnValue is used, but the cached value will be re-used for subsequent expectations, unless you manually call this method again.

This method is overridden by specific test case types.

def fetchResults(self) 
 991    def fetchResults(self):
 992        """
 993        Fetches the results of the test, which will run the test if it
 994        hasn't already been run, but otherwise will just return the
 995        latest cached results.
 996
 997        `run` describes the format of the results.
 998        """
 999        if self.results is None:
1000            self.run()
1001        return self.results

Fetches the results of the test, which will run the test if it hasn't already been run, but otherwise will just return the latest cached results.

run describes the format of the results.

def checkReturnValue(self, expectedValue) 
1003    def checkReturnValue(self, expectedValue):
1004        """
1005        Checks the result value for this test case, comparing it against
1006        the given expected value and printing a message about success or
1007        failure depending on whether they are considered different by
1008        the `findFirstDifference` function.
1009
1010        If this is the first check performed using this test case, the
1011        test case will run; otherwise a cached result will be used.
1012
1013        This method returns True if the expectation is met and False if
1014        it is not, in addition to printing a message indicating
1015        success/failure and recording that message along with the status
1016        and tag in `self.outcomes`. If the check is skipped, it returns
1017        None and does not add an entry to `self.outcomes`.
1018        """
1019        results = self.fetchResults()
1020
1021        # Figure out the tag for this expectation
1022        tag = tag_for(get_my_location())
1023
1024        # Skip this check if the case has failed already
1025        if self._should_skip():
1026            self._print_skip_message(tag, "prior test failed")
1027            # Note that we don't add an outcome here, and we return None
1028            # instead of True or False
1029            return None
1030
1031        # Figure out whether we've got an error or an actual result
1032        if results["error"] is not None:
1033            # An error during testing
1034            tb = results["traceback"]
1035            tblines = tb.splitlines()
1036            if len(tblines) < 12:
1037                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1038                extra_msg = None
1039            else:
1040                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1041                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1042                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1043
1044            msg = self._create_failure_message(
1045                tag,
1046                base_msg,
1047                extra_msg
1048            )
1049            print_message(msg, color=msg_color("failed"))
1050            self._register_outcome(False, tag, msg)
1051            return False
1052
1053        elif "result" not in results:
1054            # Likely impossible, since we verified the category above
1055            # and we're in a condition where no error was logged...
1056            msg = self._create_failure_message(
1057                tag,
1058                (
1059                    "This test case does not have a result value. (Did"
1060                    " you mean to use checkPrintedLines?)"
1061                )
1062            )
1063            print_message(msg, color=msg_color("failed"))
1064            self._register_outcome(False, tag, msg)
1065            return False
1066
1067        else:
1068            # We produced a result, so check equality
1069
1070            # Check equivalence
1071            passed = False
1072            firstDiff = findFirstDifference(results["result"], expectedValue)
1073            if firstDiff is None:
1074                equivalence = "equivalent to"
1075                passed = True
1076            else:
1077                equivalence = "NOT equivalent to"
1078
1079            # Get short/long versions of result/expected
1080            short_result = ellipsis(repr(results["result"]), 72)
1081            full_result = repr(results["result"])
1082            short_expected = ellipsis(repr(expectedValue), 72)
1083            full_expected = repr(expectedValue)
1084
1085            # Create base/extra messages
1086            if (
1087                short_result == full_result
1088            and short_expected == full_expected
1089            ):
1090                base_msg = (
1091                    f"Result:\n{indent(short_result, 2)}\nwas"
1092                    f" {equivalence} the expected value:\n"
1093                    f"{indent(short_expected, 2)}"
1094                )
1095                extra_msg = None
1096                if (
1097                    firstDiff is not None
1098                and differencesAreSubtle(short_result, short_expected)
1099                ):
1100                    base_msg += (
1101                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1102                    )
1103            else:
1104                base_msg = (
1105                    f"Result:\n{indent(short_result, 2)}\nwas"
1106                    f" {equivalence} the expected value:\n"
1107                    f"{indent(short_expected, 2)}"
1108                )
1109                extra_msg = ""
1110                if (
1111                    firstDiff is not None
1112                and differencesAreSubtle(short_result, short_expected)
1113                ):
1114                    base_msg += (
1115                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1116                    )
1117                if short_result != full_result:
1118                    extra_msg += (
1119                        f"Full result:\n{indent(full_result, 2)}\n"
1120                    )
1121                if short_expected != full_expected:
1122                    extra_msg += (
1123                        f"Full expected value:\n"
1124                        f"{indent(full_expected, 2)}\n"
1125                    )
1126
1127            if passed:
1128                msg = self._create_success_message(
1129                    tag,
1130                    base_msg,
1131                    extra_msg
1132                )
1133                print_message(msg, color=msg_color("succeeded"))
1134                self._register_outcome(True, tag, msg)
1135                return True
1136            else:
1137                msg = self._create_failure_message(
1138                    tag,
1139                    base_msg,
1140                    extra_msg
1141                )
1142                print_message(msg, color=msg_color("failed"))
1143                self._register_outcome(False, tag, msg)
1144                return False

Checks the result value for this test case, comparing it against the given expected value and printing a message about success or failure depending on whether they are considered different by the findFirstDifference function.

If this is the first check performed using this test case, the test case will run; otherwise a cached result will be used.

This method returns True if the expectation is met and False if it is not, in addition to printing a message indicating success/failure and recording that message along with the status and tag in self.outcomes. If the check is skipped, it returns None and does not add an entry to self.outcomes.

def checkVariableValue(self, varName, expectedValue) 
1146    def checkVariableValue(self, varName, expectedValue):
1147        """
1148        Checks the value of a variable established by this test case,
1149        which should be a code block or file test (use `checkReturnValue`
1150        instead for checking the result of a function test). It checks
1151        that a variable with a certain name (given as a string) has a
1152        certain expected value, and prints a message about success or
1153        failure depending on whether the actual value and expected value
1154        are considered different by the `findFirstDifference` function.
1155
1156        If this is the first check performed using this test case, the
1157        test case will run; otherwise a cached result will be used.
1158
1159        This method returns True if the expectation is met and False if
1160        it is not, in addition to printing a message indicating
1161        success/failure and recording that message along with the status
1162        and tag in `self.outcomes`. If the check is skipped, it returns
1163        None and does not add an entry to `self.outcomes`.
1164        """
1165        results = self.fetchResults()
1166
1167        # Figure out the tag for this expectation
1168        tag = tag_for(get_my_location())
1169
1170        # Skip this check if the case has failed already
1171        if self._should_skip():
1172            self._print_skip_message(tag, "prior test failed")
1173            # Note that we don't add an outcome here, and we return None
1174            # instead of True or False
1175            return None
1176
1177        # Figure out whether we've got an error or an actual result
1178        if results["error"] is not None:
1179            # An error during testing
1180            tb = results["traceback"]
1181            tblines = tb.splitlines()
1182            if len(tblines) < 12:
1183                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1184                extra_msg = None
1185            else:
1186                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1187                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1188                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1189
1190            msg = self._create_failure_message(
1191                tag,
1192                base_msg,
1193                extra_msg
1194            )
1195            print_message(msg, color=msg_color("failed"))
1196            self._register_outcome(False, tag, msg)
1197            return False
1198
1199        else:
1200            # No error, so look for our variable
1201            scope = results["scope"]
1202
1203            if varName not in scope:
1204                msg = self._create_failure_message(
1205                    tag,
1206                    f"No variable named '{varName}' was created.",
1207                    None
1208                )
1209                print_message(msg, color=msg_color("failed"))
1210                self._register_outcome(False, tag, msg)
1211                return False
1212
1213            # Check equivalence
1214            passed = False
1215            value = scope[varName]
1216            firstDiff = findFirstDifference(value, expectedValue)
1217            if firstDiff is None:
1218                equivalence = "equivalent to"
1219                passed = True
1220            else:
1221                equivalence = "NOT equivalent to"
1222
1223            # Get short/long versions of result/expected
1224            short_value = ellipsis(repr(value), 72)
1225            full_value = repr(value)
1226            short_expected = ellipsis(repr(expectedValue), 72)
1227            full_expected = repr(expectedValue)
1228
1229            # Create base/extra messages
1230            if (
1231                short_value == full_value
1232            and short_expected == full_expected
1233            ):
1234                base_msg = (
1235                    f"Variable '{varName}' with"
1236                    f" value:\n{indent(short_value, 2)}\nwas"
1237                    f" {equivalence} the expected value:\n"
1238                    f"{indent(short_expected, 2)}"
1239                )
1240                extra_msg = None
1241                if (
1242                    firstDiff is not None
1243                and differencesAreSubtle(short_value, short_expected)
1244                ):
1245                    base_msg += (
1246                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1247                    )
1248            else:
1249                base_msg = (
1250                    f"Variable '{varName}' with"
1251                    f" value:\n{indent(short_value, 2)}\nwas"
1252                    f" {equivalence} the expected value:\n"
1253                    f"{indent(short_expected, 2)}"
1254                )
1255                extra_msg = ""
1256                if (
1257                    firstDiff is not None
1258                and differencesAreSubtle(short_value, short_expected)
1259                ):
1260                    base_msg += (
1261                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1262                    )
1263                if short_value != full_value:
1264                    extra_msg += (
1265                        f"Full value:\n{indent(full_value, 2)}\n"
1266                    )
1267                if short_expected != full_expected:
1268                    extra_msg += (
1269                        f"Full expected value:\n"
1270                        f"{indent(full_expected, 2)}\n"
1271                    )
1272
1273            if passed:
1274                msg = self._create_success_message(
1275                    tag,
1276                    base_msg,
1277                    extra_msg
1278                )
1279                print_message(msg, color=msg_color("succeeded"))
1280                self._register_outcome(True, tag, msg)
1281                return True
1282            else:
1283                msg = self._create_failure_message(
1284                    tag,
1285                    base_msg,
1286                    extra_msg
1287                )
1288                print_message(msg, color=msg_color("failed"))
1289                self._register_outcome(False, tag, msg)
1290                return False

Checks the value of a variable established by this test case, which should be a code block or file test (use checkReturnValue instead for checking the result of a function test). It checks that a variable with a certain name (given as a string) has a certain expected value, and prints a message about success or failure depending on whether the actual value and expected value are considered different by the findFirstDifference function.

If this is the first check performed using this test case, the test case will run; otherwise a cached result will be used.

This method returns True if the expectation is met and False if it is not, in addition to printing a message indicating success/failure and recording that message along with the status and tag in self.outcomes. If the check is skipped, it returns None and does not add an entry to self.outcomes.

def checkPrintedLines(self, *expectedLines) 
1292    def checkPrintedLines(self, *expectedLines):
1293        """
1294        Checks that the exact printed output captured during the test
1295        matches a sequence of strings each specifying one line of the
1296        output. Note that the global `IGNORE_TRAILING_WHITESPACE`
1297        affects how this function treats line matches.
1298
1299        If this is the first check performed using this test case, the
1300        test case will run; otherwise a cached result will be used.
1301
1302        This method returns True if the check succeeds and False if it
1303        fails, in addition to printing a message indicating
1304        success/failure and recording that message along with the status
1305        and tag in `self.outcomes`. If the check is skipped, it returns
1306        None and does not add an entry to `self.outcomes`.
1307        """
1308        # Fetch captured output
1309        results = self.fetchResults()
1310        output = results["output"]
1311
1312        # Figure out the tag for this expectation
1313        tag = tag_for(get_my_location())
1314
1315        # Skip this check if the case has failed already
1316        if self._should_skip():
1317            self._print_skip_message(tag, "prior test failed")
1318            # Note that we don't add an outcome here, and we return None
1319            # instead of True or False
1320            return None
1321
1322        # Figure out whether we've got an error or an actual result
1323        if results["error"] is not None:
1324            # An error during testing
1325            tb = results["traceback"]
1326            tblines = tb.splitlines()
1327            if len(tblines) < 12:
1328                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1329                extra_msg = None
1330            else:
1331                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1332                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1333                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1334
1335            msg = self._create_failure_message(
1336                tag,
1337                base_msg,
1338                extra_msg
1339            )
1340            print_message(msg, color=msg_color("failed"))
1341            self._register_outcome(False, tag, msg)
1342            return False
1343
1344        else:
1345            # We produced printed output, so check it
1346
1347            # Get lines/single versions
1348            expected = '\n'.join(expectedLines) + '\n'
1349            # If the output doesn't end with a newline, don't add one to
1350            # our expectation either...
1351            if not output.endswith('\n'):
1352                expected = expected[:-1]
1353
1354            # Figure out equivalence category
1355            equivalence = None
1356            passed = False
1357            firstDiff = findFirstDifference(output, expected)
1358            if output == expected:
1359                equivalence = "exactly the same as"
1360                passed = True
1361            elif firstDiff is None:
1362                equivalence = "equivalent to"
1363                passed = True
1364            else:
1365                equivalence = "NOT the same as"
1366                # passed remains False
1367
1368            # Get short/long representations of our strings
1369            short, long = dual_string_repr(output)
1370            short_exp, long_exp = dual_string_repr(expected)
1371
1372            # Construct base and extra messages
1373            if short == long and short_exp == long_exp:
1374                base_msg = (
1375                    f"Printed lines:\n{indent(short, 2)}\nwere"
1376                    f" {equivalence} the expected printed"
1377                    f" lines:\n{indent(short_exp, 2)}"
1378                )
1379                if not passed:
1380                    base_msg += (
1381                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1382                    )
1383                extra_msg = None
1384            else:
1385                base_msg = (
1386                    f"Printed lines:\n{indent(short, 2)}\nwere"
1387                    f" {equivalence} the expected printed"
1388                    f" lines:\n{indent(short_exp, 2)}"
1389                )
1390                if not passed:
1391                    base_msg += (
1392                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1393                    )
1394                extra_msg = ""
1395                if short != long:
1396                    extra_msg += f"Full printed lines:\n{indent(long, 2)}\n"
1397                if short_exp != long_exp:
1398                    extra_msg += (
1399                        f"Full expected printed"
1400                        f" lines:\n{indent(long_exp, 2)}\n"
1401                    )
1402
1403            if passed:
1404                msg = self._create_success_message(
1405                    tag,
1406                    base_msg,
1407                    extra_msg
1408                )
1409                print_message(msg, color=msg_color("succeeded"))
1410                self._register_outcome(True, tag, msg)
1411                return True
1412            else:
1413                msg = self._create_failure_message(
1414                    tag,
1415                    base_msg,
1416                    extra_msg
1417                )
1418                print_message(msg, color="1;31" if COLORS else None)
1419                self._register_outcome(False, tag, msg)
1420                return False

Checks that the exact printed output captured during the test matches a sequence of strings each specifying one line of the output. Note that the global IGNORE_TRAILING_WHITESPACE affects how this function treats line matches.

If this is the first check performed using this test case, the test case will run; otherwise a cached result will be used.

This method returns True if the check succeeds and False if it fails, in addition to printing a message indicating success/failure and recording that message along with the status and tag in self.outcomes. If the check is skipped, it returns None and does not add an entry to self.outcomes.

def checkPrintedFragment(self, fragment, copies=1, allowExtra=False) 
1422    def checkPrintedFragment(self, fragment, copies=1, allowExtra=False):
1423        """
1424        Works like checkPrintedLines, except instead of requiring that
1425        the printed output exactly match a set of lines, it requires that
1426        a certain fragment of text appears somewhere within the printed
1427        output (or perhaps that multiple non-overlapping copies appear,
1428        if the copies argument is set to a number higher than the
1429        default of 1).
1430
1431        If allowExtra is set to True, more than the specified number of
1432        copies will be ignored, but by default, extra copies are not
1433        allowed.
1434
1435        The fragment is matched against the entire output as a single
1436        string, so it may contain newlines and if it does these will
1437        only match newlines in the captured output. If
1438        `IGNORE_TRAILING_WHITESPACE` is active (it's on by default), the
1439        trailing whitespace in the output will be removed before
1440        matching, and trailing whitespace in the fragment will also be
1441        removed IF it has a newline after it (trailing whitespace at the
1442        end of the string with no final newline will be retained).
1443
1444        This function returns True if the check succeeds and False if it
1445        fails, and prints a message either way. If the check is skipped,
1446        it returns None and does not add an entry to `self.outcomes`.
1447        """
1448        # Fetch captured output
1449        results = self.fetchResults()
1450        output = results["output"]
1451
1452        # Figure out the tag for this expectation
1453        tag = tag_for(get_my_location())
1454
1455        # Skip this check if the case has failed already
1456        if self._should_skip():
1457            self._print_skip_message(tag, "prior test failed")
1458            # Note that we don't add an outcome here, and we return None
1459            # instead of True or False
1460            return None
1461
1462        # Figure out whether we've got an error or an actual result
1463        if results["error"] is not None:
1464            # An error during testing
1465            tb = results["traceback"]
1466            tblines = tb.splitlines()
1467            if len(tblines) < 12:
1468                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1469                extra_msg = None
1470            else:
1471                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1472                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1473                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1474
1475            msg = self._create_failure_message(
1476                tag,
1477                base_msg,
1478                extra_msg
1479            )
1480            print_message(msg, color=msg_color("failed"))
1481            self._register_outcome(False, tag, msg)
1482            return False
1483
1484        else:
1485            # We produced printed output, so check it
1486            if IGNORE_TRAILING_WHITESPACE:
1487                matches = re.findall(
1488                    re.escape(trimWhitespace(fragment, True)),
1489                    trimWhitespace(output)
1490                )
1491            else:
1492                matches = re.findall(re.escape(fragment), output)
1493            passed = False
1494            if copies == 1:
1495                copiesPhrase = ""
1496                exactly = ""
1497                atLeast = "at least "
1498            else:
1499                copiesPhrase = f"{copies} copies of "
1500                exactly = "exactly "
1501                atLeast = "at least "
1502
1503            fragShort, fragLong = dual_string_repr(fragment)
1504            outShort, outLong = dual_string_repr(output)
1505
1506            if len(matches) == copies:
1507                passed = True
1508                base_msg = (
1509                    f"Found {exactly}{copiesPhrase}the target"
1510                    f" fragment in the printed output."
1511                    f"\nFragment was:\n{indent(fragShort, 2)}"
1512                    f"\nOutput was:\n{indent(outShort, 2)}"
1513                )
1514            elif allowExtra and len(matches) > copies:
1515                passed = True
1516                base_msg = (
1517                    f"Found {atLeast}{copiesPhrase}the target"
1518                    f" fragment in the printed output (found"
1519                    f" {len(matches)})."
1520                    f"\nFragment was:\n{indent(fragShort, 2)}"
1521                    f"\nOutput was:\n{indent(outShort, 2)}"
1522                )
1523            else:
1524                passed = False
1525                base_msg = (
1526                    f"Did not find {copiesPhrase}the target fragment"
1527                    f" in the printed output (found {len(matches)})."
1528                    f"\nFragment was:\n{indent(fragShort, 2)}"
1529                    f"\nOutput was:\n{indent(outShort, 2)}"
1530                )
1531
1532            extra_msg = ""
1533            if fragLong != fragShort:
1534                extra_msg += f"Full fragment was:\n{indent(fragLong, 2)}"
1535
1536            if outLong != outShort:
1537                if not extra_msg.endswith('\n'):
1538                    extra_msg += '\n'
1539                extra_msg += f"Full output was:\n{indent(outLong, 2)}"
1540
1541            if passed:
1542                msg = self._create_success_message(
1543                    tag,
1544                    base_msg,
1545                    extra_msg
1546                )
1547                print_message(msg, color=msg_color("succeeded"))
1548                self._register_outcome(True, tag, msg)
1549                return True
1550            else:
1551                msg = self._create_failure_message(
1552                    tag,
1553                    base_msg,
1554                    extra_msg
1555                )
1556                print_message(msg, color="1;31" if COLORS else None)
1557                self._register_outcome(False, tag, msg)
1558                return False

Works like checkPrintedLines, except instead of requiring that the printed output exactly match a set of lines, it requires that a certain fragment of text appears somewhere within the printed output (or perhaps that multiple non-overlapping copies appear, if the copies argument is set to a number higher than the default of 1).

If allowExtra is set to True, more than the specified number of copies will be ignored, but by default, extra copies are not allowed.

The fragment is matched against the entire output as a single string, so it may contain newlines and if it does these will only match newlines in the captured output. If IGNORE_TRAILING_WHITESPACE is active (it's on by default), the trailing whitespace in the output will be removed before matching, and trailing whitespace in the fragment will also be removed IF it has a newline after it (trailing whitespace at the end of the string with no final newline will be retained).

This function returns True if the check succeeds and False if it fails, and prints a message either way. If the check is skipped, it returns None and does not add an entry to self.outcomes.

def checkFileLines(self, filename, *lines) 
1560    def checkFileLines(self, filename, *lines):
1561        """
1562        Works like `checkPrintedLines`, but checks for lines in the
1563        specified file, rather than checking for printed lines.
1564        """
1565        # Figure out the tag for this expectation
1566        tag = tag_for(get_my_location())
1567
1568        # Skip this check if the case has failed already
1569        if self._should_skip():
1570            self._print_skip_message(tag, "prior test failed")
1571            # Note that we don't add an outcome here, and we return None
1572            # instead of True or False
1573            return None
1574
1575        # Fetch the results to actually run the test!
1576        expected = '\n'.join(lines) + '\n'
1577        results = self.fetchResults()
1578
1579        # Figure out whether we've got an error or an actual result
1580        if results["error"] is not None:
1581            # An error during testing
1582            tb = results["traceback"]
1583            tblines = tb.splitlines()
1584            if len(tblines) < 12:
1585                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1586                extra_msg = None
1587            else:
1588                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1589                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1590                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1591
1592            msg = self._create_failure_message(
1593                tag,
1594                base_msg,
1595                extra_msg
1596            )
1597            print_message(msg, color=msg_color("failed"))
1598            self._register_outcome(False, tag, msg)
1599            return False
1600
1601        else:
1602            # The test was able to run, so check the file contents
1603
1604            # Fetch file contents
1605            try:
1606                with open(filename, 'r', newline='') as fileInput:
1607                    fileContents = fileInput.read()
1608            except (OSError, FileNotFoundError, PermissionError):
1609                # We can't even read the file!
1610                msg = self._create_failure_message(
1611                    tag,
1612                    f"Expected file '{filename}' cannot be read.",
1613                    None
1614                )
1615                print_message(msg, color=msg_color("failed"))
1616                self._register_outcome(False, tag, msg)
1617                return False
1618
1619            # If the file doesn't end with a newline, don't add one to
1620            # our expectation either...
1621            if not fileContents.endswith('\n'):
1622                expected = expected[:-1]
1623
1624            # Get lines/single versions
1625            firstDiff = findFirstDifference(fileContents, expected)
1626            equivalence = None
1627            passed = False
1628            if fileContents == expected:
1629                equivalence = "exactly the same as"
1630                passed = True
1631            elif firstDiff is None:
1632                equivalence = "equivalent to"
1633                passed = True
1634            else:
1635                # Some other kind of difference
1636                equivalence = "NOT the same as"
1637                # passed remains False
1638
1639            # Get short/long representations of our strings
1640            short, long = dual_string_repr(fileContents)
1641            short_exp, long_exp = dual_string_repr(expected)
1642
1643            # Construct base and extra messages
1644            if short == long and short_exp == long_exp:
1645                base_msg = (
1646                    f"File contents:\n{indent(short, 2)}\nwere"
1647                    f" {equivalence} the expected file"
1648                    f" contents:\n{indent(short_exp, 2)}"
1649                )
1650                if not passed:
1651                    base_msg += (
1652                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1653                    )
1654                extra_msg = None
1655            else:
1656                base_msg = (
1657                    f"File contents:\n{indent(short, 2)}\nwere"
1658                    f" {equivalence} the expected file"
1659                    f" contents:\n{indent(short_exp, 2)}"
1660                )
1661                if not passed:
1662                    base_msg += (
1663                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1664                    )
1665                extra_msg = ""
1666                if short != long:
1667                    extra_msg += f"Full file contents:\n{indent(long, 2)}\n"
1668                if short_exp != long_exp:
1669                    extra_msg += (
1670                        f"Full expected file"
1671                        f" contents:\n{indent(long_exp, 2)}\n"
1672                    )
1673
1674            if passed:
1675                msg = self._create_success_message(
1676                    tag,
1677                    base_msg,
1678                    extra_msg
1679                )
1680                print_message(msg, color=msg_color("succeeded"))
1681                self._register_outcome(True, tag, msg)
1682                return True
1683            else:
1684                msg = self._create_failure_message(
1685                    tag,
1686                    base_msg,
1687                    extra_msg
1688                )
1689                print_message(msg, color="1;31" if COLORS else None)
1690                self._register_outcome(False, tag, msg)
1691                return False

Works like checkPrintedLines, but checks for lines in the specified file, rather than checking for printed lines.

def checkCustom(self, checker, *args, **kwargs) 
1693    def checkCustom(self, checker, *args, **kwargs):
1694        """
1695        Sets up a custom check using a testing function. The provided
1696        function will be given one argument, plus any additional
1697        arguments given to this function. The first and/or only argument
1698        to the checker function will be a dictionary with the following
1699        keys:
1700
1701        - "case": The test case object on which `checkCustom` was called.
1702            This could be used to do things like access arguments passed
1703            to the function being tested for a `FunctionCase` for
1704            example.
1705        - "output": Output printed by the test case, as a string.
1706        - "result": the result value (for function tests only, otherwise
1707            this key will not be present).
1708        - "error": the error that occurred (or None if no error
1709            occurred).
1710        - "traceback": the traceback (a string, or None if there was no
1711            error).
1712        - "scope": For file and code block cases, the variable dictionary
1713            created by the file/code block. `None` for function cases.
1714
1715        The testing function must return True to indicate success and
1716        False for failure. If it returns something other than True or
1717        False, it will be counted as a failure, that value will be shown
1718        as part of the test result if the `DETAIL_LEVEL` is 1 or higher,
1719        and this method will return False.
1720
1721        If this check is skipped (e.g., because of a previous failure),
1722        this method returns None and does not add an entry to
1723        `self.outcomes`; the custom checker function won't be called in
1724        that case.
1725        """
1726        results = self.fetchResults()
1727        # Add a 'case' entry
1728        checker_input = copy.copy(results)
1729        checker_input["case"] = self
1730
1731        # Figure out the tag for this expectation
1732        tag = tag_for(get_my_location())
1733
1734        # Skip this check if the case has failed already
1735        if self._should_skip():
1736            self._print_skip_message(tag, "prior test failed")
1737            # Note that we don't add an outcome here, and we return None
1738            # instead of True or False
1739            return None
1740
1741        # Only run the checker if we're not skipping the test
1742        test_result = checker(checker_input, *args, **kwargs)
1743
1744        if test_result is True:
1745            msg = self._create_success_message(tag, "Custom check passed.")
1746            print_message(msg, color=msg_color("succeeded"))
1747            self._register_outcome(True, tag, msg)
1748            return True
1749        elif test_result is False:
1750            msg = self._create_failure_message(tag, "Custom check failed")
1751            print_message(msg, color="1;31" if COLORS else None)
1752            self._register_outcome(False, tag, msg)
1753            return False
1754        else:
1755            msg = self._create_failure_message(
1756                tag,
1757                "Custom check failed:\n" + indent(str(test_result), 2),
1758            )
1759            print_message(msg, color="1;31" if COLORS else None)
1760            self._register_outcome(False, tag, msg)
1761            return False

Sets up a custom check using a testing function. The provided function will be given one argument, plus any additional arguments given to this function. The first and/or only argument to the checker function will be a dictionary with the following keys:

  • "case": The test case object on which checkCustom was called. This could be used to do things like access arguments passed to the function being tested for a FunctionCase for example.
  • "output": Output printed by the test case, as a string.
  • "result": the result value (for function tests only, otherwise this key will not be present).
  • "error": the error that occurred (or None if no error occurred).
  • "traceback": the traceback (a string, or None if there was no error).
  • "scope": For file and code block cases, the variable dictionary created by the file/code block. None for function cases.

The testing function must return True to indicate success and False for failure. If it returns something other than True or False, it will be counted as a failure, that value will be shown as part of the test result if the DETAIL_LEVEL is 1 or higher, and this method will return False.

If this check is skipped (e.g., because of a previous failure), this method returns None and does not add an entry to self.outcomes; the custom checker function won't be called in that case.

Inherited Members
Trial
trialDetails
class FileCase(TestCase): 
1764class FileCase(TestCase):
1765    """
1766    Runs a particular file when executed. Its manager should be a
1767    `FileManager`.
1768    """
1769    # __init__ is inherited
1770
1771    def run(self):
1772        """
1773        Runs the code in the target file in an empty environment (except
1774        that `__name__` is set to `'__main__'`, to make the file behave
1775        as if it were run as the main file).
1776
1777        Note that the code is read and parsed when the `FileManager` is
1778        created, not when the test case is run.
1779        """
1780        def payload():
1781            "Payload function to run a file."
1782            global _RUNNING_TEST_CODE
1783
1784            # Fetch syntax tree from our manager
1785            node = self.manager.syntaxTree
1786
1787            if node is None:
1788                raise RuntimeError(
1789                    "Manager of a FileCase was missing a syntax tree!"
1790                )
1791
1792            # Compile the syntax tree
1793            code = compile(node, self.manager.target, 'exec')
1794
1795            # Run the code, setting __name__ to __main__ (this is
1796            # why we don't just import the file)
1797            env = {"__name__": "__main__"}
1798            try:
1799                _RUNNING_TEST_CODE = True
1800                exec(code, env)
1801            finally:
1802                _RUNNING_TEST_CODE = False
1803
1804            # Running a file doesn't have a result value, but it does
1805            # provide a module scope.
1806            return (NoResult, deepish_copy(env))
1807
1808        return self._run(payload)
1809
1810    def trialDetails(self):
1811        """
1812        Returns a pair of strings containing base and extra details
1813        describing what was tested by this test case. If the base
1814        details capture all available information, the extra details
1815        value will be None.
1816        """
1817        return (
1818            f"Ran file '{self.manager.target}'",
1819            None  # no further details to report
1820        )

Runs a particular file when executed. Its manager should be a FileManager.

def run(self) 
1771    def run(self):
1772        """
1773        Runs the code in the target file in an empty environment (except
1774        that `__name__` is set to `'__main__'`, to make the file behave
1775        as if it were run as the main file).
1776
1777        Note that the code is read and parsed when the `FileManager` is
1778        created, not when the test case is run.
1779        """
1780        def payload():
1781            "Payload function to run a file."
1782            global _RUNNING_TEST_CODE
1783
1784            # Fetch syntax tree from our manager
1785            node = self.manager.syntaxTree
1786
1787            if node is None:
1788                raise RuntimeError(
1789                    "Manager of a FileCase was missing a syntax tree!"
1790                )
1791
1792            # Compile the syntax tree
1793            code = compile(node, self.manager.target, 'exec')
1794
1795            # Run the code, setting __name__ to __main__ (this is
1796            # why we don't just import the file)
1797            env = {"__name__": "__main__"}
1798            try:
1799                _RUNNING_TEST_CODE = True
1800                exec(code, env)
1801            finally:
1802                _RUNNING_TEST_CODE = False
1803
1804            # Running a file doesn't have a result value, but it does
1805            # provide a module scope.
1806            return (NoResult, deepish_copy(env))
1807
1808        return self._run(payload)

Runs the code in the target file in an empty environment (except that __name__ is set to '__main__', to make the file behave as if it were run as the main file).

Note that the code is read and parsed when the FileManager is created, not when the test case is run.

def trialDetails(self) 
1810    def trialDetails(self):
1811        """
1812        Returns a pair of strings containing base and extra details
1813        describing what was tested by this test case. If the base
1814        details capture all available information, the extra details
1815        value will be None.
1816        """
1817        return (
1818            f"Ran file '{self.manager.target}'",
1819            None  # no further details to report
1820        )

Returns a pair of strings containing base and extra details describing what was tested by this test case. If the base details capture all available information, the extra details value will be None.

Inherited Members
TestCase
TestCase
provideInputs
showPrintedLines
fetchResults
checkReturnValue
checkVariableValue
checkPrintedLines
checkPrintedFragment
checkFileLines
checkCustom
class FunctionCase(TestCase): 
1823class FunctionCase(TestCase):
1824    """
1825    Calls a particular function with specific arguments when run.
1826    """
1827    def __init__(self, manager, args=None, kwargs=None):
1828        """
1829        The arguments and/or keyword arguments to be used for the case
1830        are provided after the manager (as a list and a dictionary, NOT
1831        as actual arguments). If omitted, the function will be called
1832        with no arguments.
1833        """
1834        super().__init__(manager)
1835        self.args = args or ()
1836        self.kwargs = kwargs or {}
1837
1838    def run(self):
1839        """
1840        Runs the target function with the arguments specified for this
1841        case. The 'result' slot of the `self.results` dictionary that it
1842        creates holds the return value of the function.
1843        """
1844        def payload():
1845            "Payload for running a function with specific arguments."
1846            global _RUNNING_TEST_CODE
1847            try:
1848                _RUNNING_TEST_CODE = True
1849                result = (
1850                    self.manager.target(*self.args, **self.kwargs),
1851                    None  # no scope for a function TODO: Get locals?
1852                )
1853            finally:
1854                _RUNNING_TEST_CODE = False
1855            return result
1856
1857        return self._run(payload)
1858
1859    def trialDetails(self):
1860        """
1861        Returns a pair of strings containing base and extra details
1862        describing what was tested by this test case. If the base
1863        details capture all available information, the extra details
1864        value will be None.
1865        """
1866        # Show function name + args, possibly with some abbreviation
1867        fn = self.manager.target
1868        msg = f"Called function '{fn.__name__}'"
1869
1870        args = self.args if self.args is not None else []
1871        kwargs = self.kwargs if self.args is not None else {}
1872        all_args = len(args) + len(kwargs)
1873
1874        argnames = fn.__code__.co_varnames[:all_args]
1875        if len(args) > len(argnames):
1876            msg += " with too many arguments (!):"
1877        elif all_args > 0:
1878            msg += " with arguments:"
1879
1880        # TODO: Proper handling of *args and **kwargs entries!
1881
1882        # Create lists of full and maybe-abbreviated argument
1883        # strings
1884        argstrings = []
1885        short_argstrings = []
1886        for i, arg in enumerate(args):
1887            if i < len(argnames):
1888                name = argnames[i]
1889            else:
1890                name = f"extra argument #{i - len(argnames) + 1}"
1891            short_name = ellipsis(name, 20)
1892
1893            argstrings.append(f"{name} = {repr(arg)}")
1894            short_argstrings.append(
1895                f"{short_name} = {ellipsis(repr(arg), 60)}"
1896            )
1897
1898        # Order kwargs by original kwargs order and then by natural
1899        # order of kwargs dictionary
1900        keyset = set(kwargs)
1901        ordered = list(filter(lambda x: x in keyset, argnames))
1902        rest = [k for k in kwargs if k not in ordered]
1903        for k in ordered + rest:
1904            argstrings.append(f"{k} = {repr(kwargs[k])}")
1905            short_name = ellipsis(k, 20)
1906            short_argstrings.append(
1907                f"{short_name} = {ellipsis(repr(kwargs[k]), 60)}"
1908            )
1909
1910        full_args = '  ' + '\n  '.join(argstrings)
1911        # In case there are too many arguments
1912        if len(short_argstrings) < 20:
1913            short_args = '  ' + '\n  '.join(short_argstrings)
1914        else:
1915            short_args = (
1916                '  '
1917              + '\n  '.join(short_argstrings[:19])
1918              + f"...plus {len(argstrings) - 19} more arguments..."
1919            )
1920
1921        if short_args == full_args:
1922            return (
1923                msg + '\n' + short_args,
1924                None
1925            )
1926        else:
1927            return (
1928                msg + '\n' + short_args,
1929                "Full arguments were:\n" + full_args
1930            )

Calls a particular function with specific arguments when run.

FunctionCase(manager, args=None, kwargs=None) 
1827    def __init__(self, manager, args=None, kwargs=None):
1828        """
1829        The arguments and/or keyword arguments to be used for the case
1830        are provided after the manager (as a list and a dictionary, NOT
1831        as actual arguments). If omitted, the function will be called
1832        with no arguments.
1833        """
1834        super().__init__(manager)
1835        self.args = args or ()
1836        self.kwargs = kwargs or {}

The arguments and/or keyword arguments to be used for the case are provided after the manager (as a list and a dictionary, NOT as actual arguments). If omitted, the function will be called with no arguments.

def run(self) 
1838    def run(self):
1839        """
1840        Runs the target function with the arguments specified for this
1841        case. The 'result' slot of the `self.results` dictionary that it
1842        creates holds the return value of the function.
1843        """
1844        def payload():
1845            "Payload for running a function with specific arguments."
1846            global _RUNNING_TEST_CODE
1847            try:
1848                _RUNNING_TEST_CODE = True
1849                result = (
1850                    self.manager.target(*self.args, **self.kwargs),
1851                    None  # no scope for a function TODO: Get locals?
1852                )
1853            finally:
1854                _RUNNING_TEST_CODE = False
1855            return result
1856
1857        return self._run(payload)

Runs the target function with the arguments specified for this case. The 'result' slot of the self.results dictionary that it creates holds the return value of the function.

def trialDetails(self) 
1859    def trialDetails(self):
1860        """
1861        Returns a pair of strings containing base and extra details
1862        describing what was tested by this test case. If the base
1863        details capture all available information, the extra details
1864        value will be None.
1865        """
1866        # Show function name + args, possibly with some abbreviation
1867        fn = self.manager.target
1868        msg = f"Called function '{fn.__name__}'"
1869
1870        args = self.args if self.args is not None else []
1871        kwargs = self.kwargs if self.args is not None else {}
1872        all_args = len(args) + len(kwargs)
1873
1874        argnames = fn.__code__.co_varnames[:all_args]
1875        if len(args) > len(argnames):
1876            msg += " with too many arguments (!):"
1877        elif all_args > 0:
1878            msg += " with arguments:"
1879
1880        # TODO: Proper handling of *args and **kwargs entries!
1881
1882        # Create lists of full and maybe-abbreviated argument
1883        # strings
1884        argstrings = []
1885        short_argstrings = []
1886        for i, arg in enumerate(args):
1887            if i < len(argnames):
1888                name = argnames[i]
1889            else:
1890                name = f"extra argument #{i - len(argnames) + 1}"
1891            short_name = ellipsis(name, 20)
1892
1893            argstrings.append(f"{name} = {repr(arg)}")
1894            short_argstrings.append(
1895                f"{short_name} = {ellipsis(repr(arg), 60)}"
1896            )
1897
1898        # Order kwargs by original kwargs order and then by natural
1899        # order of kwargs dictionary
1900        keyset = set(kwargs)
1901        ordered = list(filter(lambda x: x in keyset, argnames))
1902        rest = [k for k in kwargs if k not in ordered]
1903        for k in ordered + rest:
1904            argstrings.append(f"{k} = {repr(kwargs[k])}")
1905            short_name = ellipsis(k, 20)
1906            short_argstrings.append(
1907                f"{short_name} = {ellipsis(repr(kwargs[k]), 60)}"
1908            )
1909
1910        full_args = '  ' + '\n  '.join(argstrings)
1911        # In case there are too many arguments
1912        if len(short_argstrings) < 20:
1913            short_args = '  ' + '\n  '.join(short_argstrings)
1914        else:
1915            short_args = (
1916                '  '
1917              + '\n  '.join(short_argstrings[:19])
1918              + f"...plus {len(argstrings) - 19} more arguments..."
1919            )
1920
1921        if short_args == full_args:
1922            return (
1923                msg + '\n' + short_args,
1924                None
1925            )
1926        else:
1927            return (
1928                msg + '\n' + short_args,
1929                "Full arguments were:\n" + full_args
1930            )

Returns a pair of strings containing base and extra details describing what was tested by this test case. If the base details capture all available information, the extra details value will be None.

Inherited Members
TestCase
provideInputs
showPrintedLines
fetchResults
checkReturnValue
checkVariableValue
checkPrintedLines
checkPrintedFragment
checkFileLines
checkCustom
class BlockCase(TestCase): 
1933class BlockCase(TestCase):
1934    """
1935    Executes a block of code (provided as text) when run. Per-case
1936    variables may be defined for the execution environment, which
1937    otherwise just has builtins.
1938    """
1939    def __init__(self, manager, assignments=None):
1940        """
1941        A dictionary of variable name : value assignments may be
1942        provided and these will be inserted into the execution
1943        environment for the code block. If omitted, no extra variables
1944        will be defined (this means that global variables available when
1945        the test manager and/or code block is set up are NOT available to
1946        the code in the code block by default).
1947        """
1948        super().__init__(manager)
1949        self.assignments = assignments or {}
1950
1951    def run(self):
1952        """
1953        Compiles and runs the target code block in an environment which
1954        is empty except for the assignments specified in this case (and
1955        builtins).
1956        """
1957        def payload():
1958            "Payload for running a code block specific variables active."
1959            global _RUNNING_TEST_CODE
1960            env = dict(self.assignments)
1961            try:
1962                _RUNNING_TEST_CODE = True
1963                exec(self.manager.code, env)
1964            finally:
1965                _RUNNING_TEST_CODE = False
1966            return (NoResult, deepish_copy(env))
1967
1968        return self._run(payload)
1969
1970    def trialDetails(self):
1971        """
1972        Returns a pair of strings containing base and extra details
1973        describing what was tested by this test case. If the base
1974        details capture all available information, the extra details
1975        value will be None.
1976        """
1977        block = self.manager.code
1978        short = limited_repr(block)
1979        if block == short:
1980            # Short enough to show whole block
1981            return (
1982                "Ran code:\n" + indent(block, 2),
1983                None
1984            )
1985
1986        else:
1987            # Too long to show whole block in short view...
1988            return (
1989                "Ran code:\n" + indent(short, 2),
1990                "Full code was:\n" + indent(block, 2)
1991            )

Executes a block of code (provided as text) when run. Per-case variables may be defined for the execution environment, which otherwise just has builtins.

BlockCase(manager, assignments=None) 
1939    def __init__(self, manager, assignments=None):
1940        """
1941        A dictionary of variable name : value assignments may be
1942        provided and these will be inserted into the execution
1943        environment for the code block. If omitted, no extra variables
1944        will be defined (this means that global variables available when
1945        the test manager and/or code block is set up are NOT available to
1946        the code in the code block by default).
1947        """
1948        super().__init__(manager)
1949        self.assignments = assignments or {}

A dictionary of variable name : value assignments may be provided and these will be inserted into the execution environment for the code block. If omitted, no extra variables will be defined (this means that global variables available when the test manager and/or code block is set up are NOT available to the code in the code block by default).

def run(self) 
1951    def run(self):
1952        """
1953        Compiles and runs the target code block in an environment which
1954        is empty except for the assignments specified in this case (and
1955        builtins).
1956        """
1957        def payload():
1958            "Payload for running a code block specific variables active."
1959            global _RUNNING_TEST_CODE
1960            env = dict(self.assignments)
1961            try:
1962                _RUNNING_TEST_CODE = True
1963                exec(self.manager.code, env)
1964            finally:
1965                _RUNNING_TEST_CODE = False
1966            return (NoResult, deepish_copy(env))
1967
1968        return self._run(payload)

Compiles and runs the target code block in an environment which is empty except for the assignments specified in this case (and builtins).

def trialDetails(self) 
1970    def trialDetails(self):
1971        """
1972        Returns a pair of strings containing base and extra details
1973        describing what was tested by this test case. If the base
1974        details capture all available information, the extra details
1975        value will be None.
1976        """
1977        block = self.manager.code
1978        short = limited_repr(block)
1979        if block == short:
1980            # Short enough to show whole block
1981            return (
1982                "Ran code:\n" + indent(block, 2),
1983                None
1984            )
1985
1986        else:
1987            # Too long to show whole block in short view...
1988            return (
1989                "Ran code:\n" + indent(short, 2),
1990                "Full code was:\n" + indent(block, 2)
1991            )

Returns a pair of strings containing base and extra details describing what was tested by this test case. If the base details capture all available information, the extra details value will be None.

Inherited Members
TestCase
provideInputs
showPrintedLines
fetchResults
checkReturnValue
checkVariableValue
checkPrintedLines
checkPrintedFragment
checkFileLines
checkCustom
class SkipCase(TestCase): 
1994class SkipCase(TestCase):
1995    """
1996    A type of test case which actually doesn't run checks, but instead
1997    prints a message that the check was skipped.
1998    """
1999    # __init__ is inherited
2000
2001    def run(self):
2002        """
2003        Since there is no real test, our results are fake. The keys
2004        "error" and "traceback" have None as their value, and "output"
2005        also has None. We add a key "skipped" with value True.
2006        """
2007        self.results = {
2008            "output": None,
2009            "error": None,
2010            "traceback": None,
2011            "skipped": True
2012        }
2013        return self.results
2014
2015    def trialDetails(self):
2016        """
2017        Provides a pair of topic/details strings about this test.
2018        """
2019        return (f"Skipped check of '{self.manager.target}'", None)
2020
2021    def checkReturnValue(self, _, **__):
2022        """
2023        Skips the check.
2024        """
2025        self._print_skip_message(
2026            tag_for(get_my_location()),
2027            "testing target not available"
2028        )
2029
2030    def checkVariableValue(self, *_, **__):
2031        """
2032        Skips the check.
2033        """
2034        self._print_skip_message(
2035            tag_for(get_my_location()),
2036            "testing target not available"
2037        )
2038
2039    def checkPrintedLines(self, *_, **__):
2040        """
2041        Skips the check.
2042        """
2043        self._print_skip_message(
2044            tag_for(get_my_location()),
2045            "testing target not available"
2046        )
2047
2048    def checkPrintedFragment(self, *_, **__):
2049        """
2050        Skips the check.
2051        """
2052        self._print_skip_message(
2053            tag_for(get_my_location()),
2054            "testing target not available"
2055        )
2056
2057    def checkFileLines(self, *_, **__):
2058        """
2059        Skips the check.
2060        """
2061        self._print_skip_message(
2062            tag_for(get_my_location()),
2063            "testing target not available"
2064        )
2065
2066    def checkCustom(self, _, **__):
2067        """
2068        Skips the check.
2069        """
2070        self._print_skip_message(
2071            tag_for(get_my_location()),
2072            "testing target not available"
2073        )

A type of test case which actually doesn't run checks, but instead prints a message that the check was skipped.

def run(self) 
2001    def run(self):
2002        """
2003        Since there is no real test, our results are fake. The keys
2004        "error" and "traceback" have None as their value, and "output"
2005        also has None. We add a key "skipped" with value True.
2006        """
2007        self.results = {
2008            "output": None,
2009            "error": None,
2010            "traceback": None,
2011            "skipped": True
2012        }
2013        return self.results

Since there is no real test, our results are fake. The keys "error" and "traceback" have None as their value, and "output" also has None. We add a key "skipped" with value True.

def trialDetails(self) 
2015    def trialDetails(self):
2016        """
2017        Provides a pair of topic/details strings about this test.
2018        """
2019        return (f"Skipped check of '{self.manager.target}'", None)

Provides a pair of topic/details strings about this test.

def checkReturnValue(self, _, **__) 
2021    def checkReturnValue(self, _, **__):
2022        """
2023        Skips the check.
2024        """
2025        self._print_skip_message(
2026            tag_for(get_my_location()),
2027            "testing target not available"
2028        )

Skips the check.

def checkVariableValue(self, *_, **__) 
2030    def checkVariableValue(self, *_, **__):
2031        """
2032        Skips the check.
2033        """
2034        self._print_skip_message(
2035            tag_for(get_my_location()),
2036            "testing target not available"
2037        )

Skips the check.

def checkPrintedLines(self, *_, **__) 
2039    def checkPrintedLines(self, *_, **__):
2040        """
2041        Skips the check.
2042        """
2043        self._print_skip_message(
2044            tag_for(get_my_location()),
2045            "testing target not available"
2046        )

Skips the check.

def checkPrintedFragment(self, *_, **__) 
2048    def checkPrintedFragment(self, *_, **__):
2049        """
2050        Skips the check.
2051        """
2052        self._print_skip_message(
2053            tag_for(get_my_location()),
2054            "testing target not available"
2055        )

Skips the check.

def checkFileLines(self, *_, **__) 
2057    def checkFileLines(self, *_, **__):
2058        """
2059        Skips the check.
2060        """
2061        self._print_skip_message(
2062            tag_for(get_my_location()),
2063            "testing target not available"
2064        )

Skips the check.

def checkCustom(self, _, **__) 
2066    def checkCustom(self, _, **__):
2067        """
2068        Skips the check.
2069        """
2070        self._print_skip_message(
2071            tag_for(get_my_location()),
2072            "testing target not available"
2073        )

Skips the check.

Inherited Members
TestCase
TestCase
provideInputs
showPrintedLines
fetchResults
class SilentCase(TestCase): 
2076class SilentCase(TestCase):
2077    """
2078    A type of test case which actually doesn't run checks, and also
2079    prints nothing. Just exists so that errors won't be thrown when
2080    checks are attempted. Testing methods return `None` instead of `True`
2081    or `False`, although this is not counted as a test failure.
2082    """
2083    # __init__ is inherited
2084
2085    def run(self):
2086        "Returns fake empty results."
2087        self.results = {
2088            "output": None,
2089            "error": None,
2090            "traceback": None,
2091            "skipped": True
2092        }
2093        return self.results
2094
2095    def trialDetails(self):
2096        """
2097        Provides a pair of topic/details strings about this test.
2098        """
2099        return ("Silently skipped check", None)
2100
2101    def checkReturnValue(self, _, **__):
2102        "Returns `None`."
2103        return None
2104
2105    def checkVariableValue(self, *_, **__):
2106        "Returns `None`."
2107        return None
2108
2109    def checkPrintedLines(self, *_, **__):
2110        "Returns `None`."
2111        return None
2112
2113    def checkPrintedFragment(self, *_, **__):
2114        "Returns `None`."
2115        return None
2116
2117    def checkFileLines(self, *_, **__):
2118        "Returns `None`."
2119        return None
2120
2121    def checkCustom(self, _, **__):
2122        "Returns `None`."
2123        return None

A type of test case which actually doesn't run checks, and also prints nothing. Just exists so that errors won't be thrown when checks are attempted. Testing methods return None instead of True or False, although this is not counted as a test failure.

def run(self) 
2085    def run(self):
2086        "Returns fake empty results."
2087        self.results = {
2088            "output": None,
2089            "error": None,
2090            "traceback": None,
2091            "skipped": True
2092        }
2093        return self.results

Returns fake empty results.

def trialDetails(self) 
2095    def trialDetails(self):
2096        """
2097        Provides a pair of topic/details strings about this test.
2098        """
2099        return ("Silently skipped check", None)

Provides a pair of topic/details strings about this test.

def checkReturnValue(self, _, **__) 
2101    def checkReturnValue(self, _, **__):
2102        "Returns `None`."
2103        return None

Returns None.

def checkVariableValue(self, *_, **__) 
2105    def checkVariableValue(self, *_, **__):
2106        "Returns `None`."
2107        return None

Returns None.

def checkPrintedLines(self, *_, **__) 
2109    def checkPrintedLines(self, *_, **__):
2110        "Returns `None`."
2111        return None

Returns None.

def checkPrintedFragment(self, *_, **__) 
2113    def checkPrintedFragment(self, *_, **__):
2114        "Returns `None`."
2115        return None

Returns None.

def checkFileLines(self, *_, **__) 
2117    def checkFileLines(self, *_, **__):
2118        "Returns `None`."
2119        return None

Returns None.

def checkCustom(self, _, **__) 
2121    def checkCustom(self, _, **__):
2122        "Returns `None`."
2123        return None

Returns None.

Inherited Members
TestCase
TestCase
provideInputs
showPrintedLines
fetchResults
class TestManager: 
2130class TestManager:
2131    """
2132    Abstract base class for managing tests for a certain function, file,
2133    or block of code. Create these using the `testFunction`, `testFile`,
2134    and/or `testBlock` factory functions. The `TestManager.case`
2135    function can be used to derive `TestCase` objects which can then be
2136    used to set up checks.
2137
2138    It can also be used to directly check structural properties of the
2139    function, file, or block it manages tests for TODO
2140    """
2141    case_type = TestCase
2142    """
2143    The case type determines what kind of test case will be constructed
2144    when calling the `TestManager.case` method. Subclasses override
2145    this.
2146    """
2147
2148    def __init__(self, target, code):
2149        """
2150        A testing target (a filename string, function object, code
2151        string, or test label string) must be provided. The relevant
2152        code text must also be provided, although this can be set to
2153        None in cases where it isn't available.
2154        """
2155        self.target = target
2156
2157        self.code = code
2158
2159        if code is not None:
2160            self.syntaxTree = ast.parse(code, filename=self.codeFilename())
2161        else:
2162            self.syntaxTree = None
2163
2164        # Keeps track of whether any cases derived from this manager have
2165        # failed so far
2166        self.any_failed = False
2167
2168        self.code_checks = None
2169
2170        self.tag = tag_for(get_my_location())
2171
2172    def codeFilename(self):
2173        """
2174        Returns the filename to be used when parsing the code for this
2175        test case.
2176        """
2177        return f"code specified at {self.tag}"
2178
2179    def checkDetails(self):
2180        """
2181        Returns base details string describing what code was checked for
2182        a `checkCodeContains` check.
2183        """
2184        return "checked unknown code"
2185
2186    def case(self):
2187        """
2188        Returns a `TestCase` object that will test the target
2189        file/function/block. Some manager types allow arguments to this
2190        function.
2191        """
2192        return self.case_type(self)
2193
2194    def checkCodeContains(self, checkFor):
2195        """
2196        Given an `ASTRequirement` object, ensures that some part of the
2197        code that this manager would run during a test case contains the
2198        structure specified by that check object. Immediately performs
2199        the check and prints a pass/fail message. The check's result will
2200        be added to the `CodeChecks` outcomes for this manager; a new
2201        `CodeChecks` trial will be created and registered if one hasn't
2202        been already.
2203
2204        Returns `True` if the check succeeds and `False` if it fails
2205        (including cases where there's a partial match). Returns `None`
2206        if the check was skipped.
2207        """
2208        # Create a code checks trial if we haven't already
2209        if self.code_checks is None:
2210            self.code_checks = CodeChecks(self)
2211        trial = self.code_checks
2212
2213        return trial.performCheck(checkFor)
2214
2215    def validateTrace(self):
2216        """
2217        Not implemented yet.
2218        """
2219        raise NotImplementedError(
2220            "validateTrace is a planned feature, but has not been"
2221            " implemented yet."
2222        )

Abstract base class for managing tests for a certain function, file, or block of code. Create these using the testFunction, testFile, and/or testBlock factory functions. The TestManager.case function can be used to derive TestCase objects which can then be used to set up checks.

It can also be used to directly check structural properties of the function, file, or block it manages tests for TODO

TestManager(target, code) 
2148    def __init__(self, target, code):
2149        """
2150        A testing target (a filename string, function object, code
2151        string, or test label string) must be provided. The relevant
2152        code text must also be provided, although this can be set to
2153        None in cases where it isn't available.
2154        """
2155        self.target = target
2156
2157        self.code = code
2158
2159        if code is not None:
2160            self.syntaxTree = ast.parse(code, filename=self.codeFilename())
2161        else:
2162            self.syntaxTree = None
2163
2164        # Keeps track of whether any cases derived from this manager have
2165        # failed so far
2166        self.any_failed = False
2167
2168        self.code_checks = None
2169
2170        self.tag = tag_for(get_my_location())

A testing target (a filename string, function object, code string, or test label string) must be provided. The relevant code text must also be provided, although this can be set to None in cases where it isn't available.

def codeFilename(self) 
2172    def codeFilename(self):
2173        """
2174        Returns the filename to be used when parsing the code for this
2175        test case.
2176        """
2177        return f"code specified at {self.tag}"

Returns the filename to be used when parsing the code for this test case.

def checkDetails(self) 
2179    def checkDetails(self):
2180        """
2181        Returns base details string describing what code was checked for
2182        a `checkCodeContains` check.
2183        """
2184        return "checked unknown code"

Returns base details string describing what code was checked for a checkCodeContains check.

def case(self) 
2186    def case(self):
2187        """
2188        Returns a `TestCase` object that will test the target
2189        file/function/block. Some manager types allow arguments to this
2190        function.
2191        """
2192        return self.case_type(self)

Returns a TestCase object that will test the target file/function/block. Some manager types allow arguments to this function.

def checkCodeContains(self, checkFor) 
2194    def checkCodeContains(self, checkFor):
2195        """
2196        Given an `ASTRequirement` object, ensures that some part of the
2197        code that this manager would run during a test case contains the
2198        structure specified by that check object. Immediately performs
2199        the check and prints a pass/fail message. The check's result will
2200        be added to the `CodeChecks` outcomes for this manager; a new
2201        `CodeChecks` trial will be created and registered if one hasn't
2202        been already.
2203
2204        Returns `True` if the check succeeds and `False` if it fails
2205        (including cases where there's a partial match). Returns `None`
2206        if the check was skipped.
2207        """
2208        # Create a code checks trial if we haven't already
2209        if self.code_checks is None:
2210            self.code_checks = CodeChecks(self)
2211        trial = self.code_checks
2212
2213        return trial.performCheck(checkFor)

Given an ASTRequirement object, ensures that some part of the code that this manager would run during a test case contains the structure specified by that check object. Immediately performs the check and prints a pass/fail message. The check's result will be added to the CodeChecks outcomes for this manager; a new CodeChecks trial will be created and registered if one hasn't been already.

Returns True if the check succeeds and False if it fails (including cases where there's a partial match). Returns None if the check was skipped.

def validateTrace(self) 
2215    def validateTrace(self):
2216        """
2217        Not implemented yet.
2218        """
2219        raise NotImplementedError(
2220            "validateTrace is a planned feature, but has not been"
2221            " implemented yet."
2222        )

Not implemented yet.

class TestManager.case_type(Trial): 
 834class TestCase(Trial):
 835    """
 836    Represents a specific test to run, managing things like specific
 837    arguments, inputs or available variables that need to be in place.
 838    Derived from a `TestManager` using the `TestManager.case` method.
 839
 840    `TestCase` is abstract; subclasses should override a least the `run`
 841    and `trialDetails` functions.
 842    """
 843    def __init__(self, manager):
 844        """
 845        A manager must be specified, but that's it. This does extra
 846        things like registering the case in the current test suite (see
 847        `testSuite`) and figuring out the location tag for the case.
 848        """
 849        super().__init__(manager)
 850
 851        # How to describe this trial
 852        self.description = f"test case at {self.tag}"
 853
 854        # Inputs to provide on stdin
 855        self.inputs = None
 856
 857        # Results of running this case
 858        self.results = None
 859
 860        # Whether to echo captured printed outputs (overrides global)
 861        self.echo = None
 862
 863    def provideInputs(self, *inputLines):
 864        """
 865        Sets up fake inputs (each argument must be a string and is used
 866        for one line of input) for this test case. When information is
 867        read from stdin during the test, including via the `input`
 868        function, these values are the result. If you don't call
 869        `provideInputs`, then the test will pause and wait for real user
 870        input when `input` is called.
 871
 872        You must call this before the test actually runs (i.e., before
 873        `TestCase.run` or one of the `check` functions is called),
 874        otherwise you'll get an error.
 875        """
 876        if self.results is not None:
 877            raise TestError(
 878                "You cannot provide inputs because this test case has"
 879                " already been run."
 880            )
 881        self.inputs = inputLines
 882
 883    def showPrintedLines(self, show=True):
 884        """
 885        Overrides the global `showPrintedLines` setting for this test.
 886        Use None as the parameter to remove the override.
 887        """
 888        self.echo = show
 889
 890    def _run(self, payload):
 891        """
 892        Given a payload (a zero-argument function that returns a tuple
 893        with a result and a scope dictionary), runs the payload while
 894        managing things like output capturing and input mocking. Sets the
 895        `self.results` field to reflect the results of the run, which
 896        will be a dictionary that has the following slots:
 897
 898        - "result": The result value from a function call. This key
 899            will not be present for tests that don't have a result, like
 900            file or code block tests. To achieve this with a custom
 901            payload, have the payload return `NoResult` as the first part
 902            of the tuple it returns.
 903        - "output": The output printed during the test. Will be an empty
 904            string if nothing gets printed.
 905        - "error": An Exception object representing an error that
 906            occurred during the test, or None if no errors happened.
 907        - "traceback": If an exception occurred, this will be a string
 908            containing the traceback for that exception. Otherwise it
 909            will be None.
 910        - "scope": The second part of the tuple returned by the payload,
 911            which should be a dictionary representing the scope of the
 912            code run by the test. It may also be `None` in cases where no
 913            scope is available (e.g., function alls).
 914
 915        In addition to being added to the results slot, this dictionary
 916        is also returned.
 917        """
 918        # Set up the `input` function to echo what is typed, and to only
 919        # read from stdin (in case we're in a notebook where input would
 920        # do something else).
 921        original_input = builtins.input
 922        builtins.input = mimicInput
 923
 924        # Set up a capturing stream for output
 925        outputCapture = CapturingStream()
 926        outputCapture.install()
 927        if self.echo or (self.echo is None and _SHOW_OUTPUT):
 928            outputCapture.echo()
 929
 930        # Set up fake input contents, AND also monkey-patch the input
 931        # function since in some settings like notebooks input doesn't
 932        # just read from stdin
 933        if self.inputs is not None:
 934            fakeInput = io.StringIO('\n'.join(self.inputs))
 935            original_stdin = sys.stdin
 936            sys.stdin = fakeInput
 937
 938        # Set up default values before we run things
 939        error = None
 940        tb = None
 941        value = NoResult
 942        scope = None
 943
 944        # Actually run the test
 945        try:
 946            value, scope = payload()
 947        except Exception as e:
 948            # Catch any error that occurred
 949            error = e
 950            tb = traceback.format_exc()
 951        finally:
 952            # Release stream captures and reset the input function
 953            outputCapture.uninstall()
 954            builtins.input = original_input
 955            if self.inputs is not None:
 956                sys.stdin = original_stdin
 957
 958        # Grab captured output
 959        output = outputCapture.getvalue()
 960
 961        # Create self.results w/ output, error, and maybe result value
 962        self.results = {
 963            "output": output,
 964            "error": error,
 965            "traceback": tb,
 966            "scope": scope
 967        }
 968        if value is not NoResult:
 969            self.results["result"] = value
 970
 971        # Return new results object
 972        return self.results
 973
 974    def run(self):
 975        """
 976        Runs this test case, capturing printed output and supplying fake
 977        input if `TestCase.provideInputs` has been called. Stores the
 978        results in `self.results`. This will be called once
 979        automatically the first time an expectation method like
 980        `TestCase.checkReturnValue` is used, but the cached value will
 981        be re-used for subsequent expectations, unless you manually call
 982        this method again.
 983
 984        This method is overridden by specific test case types.
 985        """
 986        raise NotImplementedError(
 987            "Cannot run a TestCase; you must create a specific kind of"
 988            " test case like a FunctionCase to be able to run it."
 989        )
 990
 991    def fetchResults(self):
 992        """
 993        Fetches the results of the test, which will run the test if it
 994        hasn't already been run, but otherwise will just return the
 995        latest cached results.
 996
 997        `run` describes the format of the results.
 998        """
 999        if self.results is None:
1000            self.run()
1001        return self.results
1002
1003    def checkReturnValue(self, expectedValue):
1004        """
1005        Checks the result value for this test case, comparing it against
1006        the given expected value and printing a message about success or
1007        failure depending on whether they are considered different by
1008        the `findFirstDifference` function.
1009
1010        If this is the first check performed using this test case, the
1011        test case will run; otherwise a cached result will be used.
1012
1013        This method returns True if the expectation is met and False if
1014        it is not, in addition to printing a message indicating
1015        success/failure and recording that message along with the status
1016        and tag in `self.outcomes`. If the check is skipped, it returns
1017        None and does not add an entry to `self.outcomes`.
1018        """
1019        results = self.fetchResults()
1020
1021        # Figure out the tag for this expectation
1022        tag = tag_for(get_my_location())
1023
1024        # Skip this check if the case has failed already
1025        if self._should_skip():
1026            self._print_skip_message(tag, "prior test failed")
1027            # Note that we don't add an outcome here, and we return None
1028            # instead of True or False
1029            return None
1030
1031        # Figure out whether we've got an error or an actual result
1032        if results["error"] is not None:
1033            # An error during testing
1034            tb = results["traceback"]
1035            tblines = tb.splitlines()
1036            if len(tblines) < 12:
1037                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1038                extra_msg = None
1039            else:
1040                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1041                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1042                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1043
1044            msg = self._create_failure_message(
1045                tag,
1046                base_msg,
1047                extra_msg
1048            )
1049            print_message(msg, color=msg_color("failed"))
1050            self._register_outcome(False, tag, msg)
1051            return False
1052
1053        elif "result" not in results:
1054            # Likely impossible, since we verified the category above
1055            # and we're in a condition where no error was logged...
1056            msg = self._create_failure_message(
1057                tag,
1058                (
1059                    "This test case does not have a result value. (Did"
1060                    " you mean to use checkPrintedLines?)"
1061                )
1062            )
1063            print_message(msg, color=msg_color("failed"))
1064            self._register_outcome(False, tag, msg)
1065            return False
1066
1067        else:
1068            # We produced a result, so check equality
1069
1070            # Check equivalence
1071            passed = False
1072            firstDiff = findFirstDifference(results["result"], expectedValue)
1073            if firstDiff is None:
1074                equivalence = "equivalent to"
1075                passed = True
1076            else:
1077                equivalence = "NOT equivalent to"
1078
1079            # Get short/long versions of result/expected
1080            short_result = ellipsis(repr(results["result"]), 72)
1081            full_result = repr(results["result"])
1082            short_expected = ellipsis(repr(expectedValue), 72)
1083            full_expected = repr(expectedValue)
1084
1085            # Create base/extra messages
1086            if (
1087                short_result == full_result
1088            and short_expected == full_expected
1089            ):
1090                base_msg = (
1091                    f"Result:\n{indent(short_result, 2)}\nwas"
1092                    f" {equivalence} the expected value:\n"
1093                    f"{indent(short_expected, 2)}"
1094                )
1095                extra_msg = None
1096                if (
1097                    firstDiff is not None
1098                and differencesAreSubtle(short_result, short_expected)
1099                ):
1100                    base_msg += (
1101                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1102                    )
1103            else:
1104                base_msg = (
1105                    f"Result:\n{indent(short_result, 2)}\nwas"
1106                    f" {equivalence} the expected value:\n"
1107                    f"{indent(short_expected, 2)}"
1108                )
1109                extra_msg = ""
1110                if (
1111                    firstDiff is not None
1112                and differencesAreSubtle(short_result, short_expected)
1113                ):
1114                    base_msg += (
1115                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1116                    )
1117                if short_result != full_result:
1118                    extra_msg += (
1119                        f"Full result:\n{indent(full_result, 2)}\n"
1120                    )
1121                if short_expected != full_expected:
1122                    extra_msg += (
1123                        f"Full expected value:\n"
1124                        f"{indent(full_expected, 2)}\n"
1125                    )
1126
1127            if passed:
1128                msg = self._create_success_message(
1129                    tag,
1130                    base_msg,
1131                    extra_msg
1132                )
1133                print_message(msg, color=msg_color("succeeded"))
1134                self._register_outcome(True, tag, msg)
1135                return True
1136            else:
1137                msg = self._create_failure_message(
1138                    tag,
1139                    base_msg,
1140                    extra_msg
1141                )
1142                print_message(msg, color=msg_color("failed"))
1143                self._register_outcome(False, tag, msg)
1144                return False
1145
1146    def checkVariableValue(self, varName, expectedValue):
1147        """
1148        Checks the value of a variable established by this test case,
1149        which should be a code block or file test (use `checkReturnValue`
1150        instead for checking the result of a function test). It checks
1151        that a variable with a certain name (given as a string) has a
1152        certain expected value, and prints a message about success or
1153        failure depending on whether the actual value and expected value
1154        are considered different by the `findFirstDifference` function.
1155
1156        If this is the first check performed using this test case, the
1157        test case will run; otherwise a cached result will be used.
1158
1159        This method returns True if the expectation is met and False if
1160        it is not, in addition to printing a message indicating
1161        success/failure and recording that message along with the status
1162        and tag in `self.outcomes`. If the check is skipped, it returns
1163        None and does not add an entry to `self.outcomes`.
1164        """
1165        results = self.fetchResults()
1166
1167        # Figure out the tag for this expectation
1168        tag = tag_for(get_my_location())
1169
1170        # Skip this check if the case has failed already
1171        if self._should_skip():
1172            self._print_skip_message(tag, "prior test failed")
1173            # Note that we don't add an outcome here, and we return None
1174            # instead of True or False
1175            return None
1176
1177        # Figure out whether we've got an error or an actual result
1178        if results["error"] is not None:
1179            # An error during testing
1180            tb = results["traceback"]
1181            tblines = tb.splitlines()
1182            if len(tblines) < 12:
1183                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1184                extra_msg = None
1185            else:
1186                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1187                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1188                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1189
1190            msg = self._create_failure_message(
1191                tag,
1192                base_msg,
1193                extra_msg
1194            )
1195            print_message(msg, color=msg_color("failed"))
1196            self._register_outcome(False, tag, msg)
1197            return False
1198
1199        else:
1200            # No error, so look for our variable
1201            scope = results["scope"]
1202
1203            if varName not in scope:
1204                msg = self._create_failure_message(
1205                    tag,
1206                    f"No variable named '{varName}' was created.",
1207                    None
1208                )
1209                print_message(msg, color=msg_color("failed"))
1210                self._register_outcome(False, tag, msg)
1211                return False
1212
1213            # Check equivalence
1214            passed = False
1215            value = scope[varName]
1216            firstDiff = findFirstDifference(value, expectedValue)
1217            if firstDiff is None:
1218                equivalence = "equivalent to"
1219                passed = True
1220            else:
1221                equivalence = "NOT equivalent to"
1222
1223            # Get short/long versions of result/expected
1224            short_value = ellipsis(repr(value), 72)
1225            full_value = repr(value)
1226            short_expected = ellipsis(repr(expectedValue), 72)
1227            full_expected = repr(expectedValue)
1228
1229            # Create base/extra messages
1230            if (
1231                short_value == full_value
1232            and short_expected == full_expected
1233            ):
1234                base_msg = (
1235                    f"Variable '{varName}' with"
1236                    f" value:\n{indent(short_value, 2)}\nwas"
1237                    f" {equivalence} the expected value:\n"
1238                    f"{indent(short_expected, 2)}"
1239                )
1240                extra_msg = None
1241                if (
1242                    firstDiff is not None
1243                and differencesAreSubtle(short_value, short_expected)
1244                ):
1245                    base_msg += (
1246                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1247                    )
1248            else:
1249                base_msg = (
1250                    f"Variable '{varName}' with"
1251                    f" value:\n{indent(short_value, 2)}\nwas"
1252                    f" {equivalence} the expected value:\n"
1253                    f"{indent(short_expected, 2)}"
1254                )
1255                extra_msg = ""
1256                if (
1257                    firstDiff is not None
1258                and differencesAreSubtle(short_value, short_expected)
1259                ):
1260                    base_msg += (
1261                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1262                    )
1263                if short_value != full_value:
1264                    extra_msg += (
1265                        f"Full value:\n{indent(full_value, 2)}\n"
1266                    )
1267                if short_expected != full_expected:
1268                    extra_msg += (
1269                        f"Full expected value:\n"
1270                        f"{indent(full_expected, 2)}\n"
1271                    )
1272
1273            if passed:
1274                msg = self._create_success_message(
1275                    tag,
1276                    base_msg,
1277                    extra_msg
1278                )
1279                print_message(msg, color=msg_color("succeeded"))
1280                self._register_outcome(True, tag, msg)
1281                return True
1282            else:
1283                msg = self._create_failure_message(
1284                    tag,
1285                    base_msg,
1286                    extra_msg
1287                )
1288                print_message(msg, color=msg_color("failed"))
1289                self._register_outcome(False, tag, msg)
1290                return False
1291
1292    def checkPrintedLines(self, *expectedLines):
1293        """
1294        Checks that the exact printed output captured during the test
1295        matches a sequence of strings each specifying one line of the
1296        output. Note that the global `IGNORE_TRAILING_WHITESPACE`
1297        affects how this function treats line matches.
1298
1299        If this is the first check performed using this test case, the
1300        test case will run; otherwise a cached result will be used.
1301
1302        This method returns True if the check succeeds and False if it
1303        fails, in addition to printing a message indicating
1304        success/failure and recording that message along with the status
1305        and tag in `self.outcomes`. If the check is skipped, it returns
1306        None and does not add an entry to `self.outcomes`.
1307        """
1308        # Fetch captured output
1309        results = self.fetchResults()
1310        output = results["output"]
1311
1312        # Figure out the tag for this expectation
1313        tag = tag_for(get_my_location())
1314
1315        # Skip this check if the case has failed already
1316        if self._should_skip():
1317            self._print_skip_message(tag, "prior test failed")
1318            # Note that we don't add an outcome here, and we return None
1319            # instead of True or False
1320            return None
1321
1322        # Figure out whether we've got an error or an actual result
1323        if results["error"] is not None:
1324            # An error during testing
1325            tb = results["traceback"]
1326            tblines = tb.splitlines()
1327            if len(tblines) < 12:
1328                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1329                extra_msg = None
1330            else:
1331                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1332                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1333                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1334
1335            msg = self._create_failure_message(
1336                tag,
1337                base_msg,
1338                extra_msg
1339            )
1340            print_message(msg, color=msg_color("failed"))
1341            self._register_outcome(False, tag, msg)
1342            return False
1343
1344        else:
1345            # We produced printed output, so check it
1346
1347            # Get lines/single versions
1348            expected = '\n'.join(expectedLines) + '\n'
1349            # If the output doesn't end with a newline, don't add one to
1350            # our expectation either...
1351            if not output.endswith('\n'):
1352                expected = expected[:-1]
1353
1354            # Figure out equivalence category
1355            equivalence = None
1356            passed = False
1357            firstDiff = findFirstDifference(output, expected)
1358            if output == expected:
1359                equivalence = "exactly the same as"
1360                passed = True
1361            elif firstDiff is None:
1362                equivalence = "equivalent to"
1363                passed = True
1364            else:
1365                equivalence = "NOT the same as"
1366                # passed remains False
1367
1368            # Get short/long representations of our strings
1369            short, long = dual_string_repr(output)
1370            short_exp, long_exp = dual_string_repr(expected)
1371
1372            # Construct base and extra messages
1373            if short == long and short_exp == long_exp:
1374                base_msg = (
1375                    f"Printed lines:\n{indent(short, 2)}\nwere"
1376                    f" {equivalence} the expected printed"
1377                    f" lines:\n{indent(short_exp, 2)}"
1378                )
1379                if not passed:
1380                    base_msg += (
1381                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1382                    )
1383                extra_msg = None
1384            else:
1385                base_msg = (
1386                    f"Printed lines:\n{indent(short, 2)}\nwere"
1387                    f" {equivalence} the expected printed"
1388                    f" lines:\n{indent(short_exp, 2)}"
1389                )
1390                if not passed:
1391                    base_msg += (
1392                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1393                    )
1394                extra_msg = ""
1395                if short != long:
1396                    extra_msg += f"Full printed lines:\n{indent(long, 2)}\n"
1397                if short_exp != long_exp:
1398                    extra_msg += (
1399                        f"Full expected printed"
1400                        f" lines:\n{indent(long_exp, 2)}\n"
1401                    )
1402
1403            if passed:
1404                msg = self._create_success_message(
1405                    tag,
1406                    base_msg,
1407                    extra_msg
1408                )
1409                print_message(msg, color=msg_color("succeeded"))
1410                self._register_outcome(True, tag, msg)
1411                return True
1412            else:
1413                msg = self._create_failure_message(
1414                    tag,
1415                    base_msg,
1416                    extra_msg
1417                )
1418                print_message(msg, color="1;31" if COLORS else None)
1419                self._register_outcome(False, tag, msg)
1420                return False
1421
1422    def checkPrintedFragment(self, fragment, copies=1, allowExtra=False):
1423        """
1424        Works like checkPrintedLines, except instead of requiring that
1425        the printed output exactly match a set of lines, it requires that
1426        a certain fragment of text appears somewhere within the printed
1427        output (or perhaps that multiple non-overlapping copies appear,
1428        if the copies argument is set to a number higher than the
1429        default of 1).
1430
1431        If allowExtra is set to True, more than the specified number of
1432        copies will be ignored, but by default, extra copies are not
1433        allowed.
1434
1435        The fragment is matched against the entire output as a single
1436        string, so it may contain newlines and if it does these will
1437        only match newlines in the captured output. If
1438        `IGNORE_TRAILING_WHITESPACE` is active (it's on by default), the
1439        trailing whitespace in the output will be removed before
1440        matching, and trailing whitespace in the fragment will also be
1441        removed IF it has a newline after it (trailing whitespace at the
1442        end of the string with no final newline will be retained).
1443
1444        This function returns True if the check succeeds and False if it
1445        fails, and prints a message either way. If the check is skipped,
1446        it returns None and does not add an entry to `self.outcomes`.
1447        """
1448        # Fetch captured output
1449        results = self.fetchResults()
1450        output = results["output"]
1451
1452        # Figure out the tag for this expectation
1453        tag = tag_for(get_my_location())
1454
1455        # Skip this check if the case has failed already
1456        if self._should_skip():
1457            self._print_skip_message(tag, "prior test failed")
1458            # Note that we don't add an outcome here, and we return None
1459            # instead of True or False
1460            return None
1461
1462        # Figure out whether we've got an error or an actual result
1463        if results["error"] is not None:
1464            # An error during testing
1465            tb = results["traceback"]
1466            tblines = tb.splitlines()
1467            if len(tblines) < 12:
1468                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1469                extra_msg = None
1470            else:
1471                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1472                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1473                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1474
1475            msg = self._create_failure_message(
1476                tag,
1477                base_msg,
1478                extra_msg
1479            )
1480            print_message(msg, color=msg_color("failed"))
1481            self._register_outcome(False, tag, msg)
1482            return False
1483
1484        else:
1485            # We produced printed output, so check it
1486            if IGNORE_TRAILING_WHITESPACE:
1487                matches = re.findall(
1488                    re.escape(trimWhitespace(fragment, True)),
1489                    trimWhitespace(output)
1490                )
1491            else:
1492                matches = re.findall(re.escape(fragment), output)
1493            passed = False
1494            if copies == 1:
1495                copiesPhrase = ""
1496                exactly = ""
1497                atLeast = "at least "
1498            else:
1499                copiesPhrase = f"{copies} copies of "
1500                exactly = "exactly "
1501                atLeast = "at least "
1502
1503            fragShort, fragLong = dual_string_repr(fragment)
1504            outShort, outLong = dual_string_repr(output)
1505
1506            if len(matches) == copies:
1507                passed = True
1508                base_msg = (
1509                    f"Found {exactly}{copiesPhrase}the target"
1510                    f" fragment in the printed output."
1511                    f"\nFragment was:\n{indent(fragShort, 2)}"
1512                    f"\nOutput was:\n{indent(outShort, 2)}"
1513                )
1514            elif allowExtra and len(matches) > copies:
1515                passed = True
1516                base_msg = (
1517                    f"Found {atLeast}{copiesPhrase}the target"
1518                    f" fragment in the printed output (found"
1519                    f" {len(matches)})."
1520                    f"\nFragment was:\n{indent(fragShort, 2)}"
1521                    f"\nOutput was:\n{indent(outShort, 2)}"
1522                )
1523            else:
1524                passed = False
1525                base_msg = (
1526                    f"Did not find {copiesPhrase}the target fragment"
1527                    f" in the printed output (found {len(matches)})."
1528                    f"\nFragment was:\n{indent(fragShort, 2)}"
1529                    f"\nOutput was:\n{indent(outShort, 2)}"
1530                )
1531
1532            extra_msg = ""
1533            if fragLong != fragShort:
1534                extra_msg += f"Full fragment was:\n{indent(fragLong, 2)}"
1535
1536            if outLong != outShort:
1537                if not extra_msg.endswith('\n'):
1538                    extra_msg += '\n'
1539                extra_msg += f"Full output was:\n{indent(outLong, 2)}"
1540
1541            if passed:
1542                msg = self._create_success_message(
1543                    tag,
1544                    base_msg,
1545                    extra_msg
1546                )
1547                print_message(msg, color=msg_color("succeeded"))
1548                self._register_outcome(True, tag, msg)
1549                return True
1550            else:
1551                msg = self._create_failure_message(
1552                    tag,
1553                    base_msg,
1554                    extra_msg
1555                )
1556                print_message(msg, color="1;31" if COLORS else None)
1557                self._register_outcome(False, tag, msg)
1558                return False
1559
1560    def checkFileLines(self, filename, *lines):
1561        """
1562        Works like `checkPrintedLines`, but checks for lines in the
1563        specified file, rather than checking for printed lines.
1564        """
1565        # Figure out the tag for this expectation
1566        tag = tag_for(get_my_location())
1567
1568        # Skip this check if the case has failed already
1569        if self._should_skip():
1570            self._print_skip_message(tag, "prior test failed")
1571            # Note that we don't add an outcome here, and we return None
1572            # instead of True or False
1573            return None
1574
1575        # Fetch the results to actually run the test!
1576        expected = '\n'.join(lines) + '\n'
1577        results = self.fetchResults()
1578
1579        # Figure out whether we've got an error or an actual result
1580        if results["error"] is not None:
1581            # An error during testing
1582            tb = results["traceback"]
1583            tblines = tb.splitlines()
1584            if len(tblines) < 12:
1585                base_msg = "Failed due to an error:\n" + indent(tb, 2)
1586                extra_msg = None
1587            else:
1588                short_tb = '\n'.join(tblines[:4] + ['...'] + tblines[-4:])
1589                base_msg = "Failed due to an error:\n" + indent(short_tb, 2)
1590                extra_msg = "Full traceback is:\n" + indent(tb, 2)
1591
1592            msg = self._create_failure_message(
1593                tag,
1594                base_msg,
1595                extra_msg
1596            )
1597            print_message(msg, color=msg_color("failed"))
1598            self._register_outcome(False, tag, msg)
1599            return False
1600
1601        else:
1602            # The test was able to run, so check the file contents
1603
1604            # Fetch file contents
1605            try:
1606                with open(filename, 'r', newline='') as fileInput:
1607                    fileContents = fileInput.read()
1608            except (OSError, FileNotFoundError, PermissionError):
1609                # We can't even read the file!
1610                msg = self._create_failure_message(
1611                    tag,
1612                    f"Expected file '{filename}' cannot be read.",
1613                    None
1614                )
1615                print_message(msg, color=msg_color("failed"))
1616                self._register_outcome(False, tag, msg)
1617                return False
1618
1619            # If the file doesn't end with a newline, don't add one to
1620            # our expectation either...
1621            if not fileContents.endswith('\n'):
1622                expected = expected[:-1]
1623
1624            # Get lines/single versions
1625            firstDiff = findFirstDifference(fileContents, expected)
1626            equivalence = None
1627            passed = False
1628            if fileContents == expected:
1629                equivalence = "exactly the same as"
1630                passed = True
1631            elif firstDiff is None:
1632                equivalence = "equivalent to"
1633                passed = True
1634            else:
1635                # Some other kind of difference
1636                equivalence = "NOT the same as"
1637                # passed remains False
1638
1639            # Get short/long representations of our strings
1640            short, long = dual_string_repr(fileContents)
1641            short_exp, long_exp = dual_string_repr(expected)
1642
1643            # Construct base and extra messages
1644            if short == long and short_exp == long_exp:
1645                base_msg = (
1646                    f"File contents:\n{indent(short, 2)}\nwere"
1647                    f" {equivalence} the expected file"
1648                    f" contents:\n{indent(short_exp, 2)}"
1649                )
1650                if not passed:
1651                    base_msg += (
1652                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1653                    )
1654                extra_msg = None
1655            else:
1656                base_msg = (
1657                    f"File contents:\n{indent(short, 2)}\nwere"
1658                    f" {equivalence} the expected file"
1659                    f" contents:\n{indent(short_exp, 2)}"
1660                )
1661                if not passed:
1662                    base_msg += (
1663                        f"\nFirst difference was:\n{indent(firstDiff, 2)}"
1664                    )
1665                extra_msg = ""
1666                if short != long:
1667                    extra_msg += f"Full file contents:\n{indent(long, 2)}\n"
1668                if short_exp != long_exp:
1669                    extra_msg += (
1670                        f"Full expected file"
1671                        f" contents:\n{indent(long_exp, 2)}\n"
1672                    )
1673
1674            if passed:
1675                msg = self._create_success_message(
1676                    tag,
1677                    base_msg,
1678                    extra_msg
1679                )
1680                print_message(msg, color=msg_color("succeeded"))
1681                self._register_outcome(True, tag, msg)
1682                return True
1683            else:
1684                msg = self._create_failure_message(
1685                    tag,
1686                    base_msg,
1687                    extra_msg
1688                )
1689                print_message(msg, color="1;31" if COLORS else None)
1690                self._register_outcome(False, tag, msg)
1691                return False
1692
1693    def checkCustom(self, checker, *args, **kwargs):
1694        """
1695        Sets up a custom check using a testing function. The provided
1696        function will be given one argument, plus any additional
1697        arguments given to this function. The first and/or only argument
1698        to the checker function will be a dictionary with the following
1699        keys:
1700
1701        - "case": The test case object on which `checkCustom` was called.
1702            This could be used to do things like access arguments passed
1703            to the function being tested for a `FunctionCase` for
1704            example.
1705        - "output": Output printed by the test case, as a string.
1706        - "result": the result value (for function tests only, otherwise
1707            this key will not be present).
1708        - "error": the error that occurred (or None if no error
1709            occurred).
1710        - "traceback": the traceback (a string, or None if there was no
1711            error).
1712        - "scope": For file and code block cases, the variable dictionary
1713            created by the file/code block. `None` for function cases.
1714
1715        The testing function must return True to indicate success and
1716        False for failure. If it returns something other than True or
1717        False, it will be counted as a failure, that value will be shown
1718        as part of the test result if the `DETAIL_LEVEL` is 1 or higher,
1719        and this method will return False.
1720
1721        If this check is skipped (e.g., because of a previous failure),
1722        this method returns None and does not add an entry to
1723        `self.outcomes`; the custom checker function won't be called in
1724        that case.
1725        """
1726        results = self.fetchResults()
1727        # Add a 'case' entry
1728        checker_input = copy.copy(results)
1729        checker_input["case"] = self
1730
1731        # Figure out the tag for this expectation
1732        tag = tag_for(get_my_location())
1733
1734        # Skip this check if the case has failed already
1735        if self._should_skip():
1736            self._print_skip_message(tag, "prior test failed")
1737            # Note that we don't add an outcome here, and we return None
1738            # instead of True or False
1739            return None
1740
1741        # Only run the checker if we're not skipping the test
1742        test_result = checker(checker_input, *args, **kwargs)
1743
1744        if test_result is True:
1745            msg = self._create_success_message(tag, "Custom check passed.")
1746            print_message(msg, color=msg_color("succeeded"))
1747            self._register_outcome(True, tag, msg)
1748            return True
1749        elif test_result is False:
1750            msg = self._create_failure_message(tag, "Custom check failed")
1751            print_message(msg, color="1;31" if COLORS else None)
1752            self._register_outcome(False, tag, msg)
1753            return False
1754        else:
1755            msg = self._create_failure_message(
1756                tag,
1757                "Custom check failed:\n" + indent(str(test_result), 2),
1758            )
1759            print_message(msg, color="1;31" if COLORS else None)
1760            self._register_outcome(False, tag, msg)
1761            return False

The case type determines what kind of test case will be constructed when calling the TestManager.case method. Subclasses override this.

Inherited Members
TestCase
TestCase
provideInputs
showPrintedLines
run
fetchResults
checkReturnValue
checkVariableValue
checkPrintedLines
checkPrintedFragment
checkFileLines
checkCustom
Trial
trialDetails
class FileManager(TestManager): 
2225class FileManager(TestManager):
2226    """
2227    Manages test cases for running an entire file. Unlike other
2228    managers, cases for a file cannot have parameters. Calling
2229    `TestCase.provideInputs` on a case to provide inputs still means
2230    that having multiple cases can be useful, however.
2231    """
2232    case_type = FileCase
2233
2234    def __init__(self, filename):
2235        """
2236        A FileManager needs a filename string that specifies which file
2237        we'll run when we run a test case.
2238        """
2239        if not isinstance(filename, str):
2240            raise TypeError(
2241                f"For a file test manager, the target must be a file"
2242                f" name string. (You provided a/an {type(filename)}.)"
2243            )
2244
2245        with open(filename, 'r') as inputFile:
2246            code = inputFile.read()
2247
2248        super().__init__(filename, code)
2249
2250    def codeFilename(self):
2251        return self.target
2252
2253    def checkDetails(self):
2254        return f"checked code in file '{self.target}'"
2255
2256    # case is inherited as-is

Manages test cases for running an entire file. Unlike other managers, cases for a file cannot have parameters. Calling TestCase.provideInputs on a case to provide inputs still means that having multiple cases can be useful, however.

FileManager(filename) 
2234    def __init__(self, filename):
2235        """
2236        A FileManager needs a filename string that specifies which file
2237        we'll run when we run a test case.
2238        """
2239        if not isinstance(filename, str):
2240            raise TypeError(
2241                f"For a file test manager, the target must be a file"
2242                f" name string. (You provided a/an {type(filename)}.)"
2243            )
2244
2245        with open(filename, 'r') as inputFile:
2246            code = inputFile.read()
2247
2248        super().__init__(filename, code)

A FileManager needs a filename string that specifies which file we'll run when we run a test case.

def codeFilename(self) 
2250    def codeFilename(self):
2251        return self.target

Returns the filename to be used when parsing the code for this test case.

def checkDetails(self) 
2253    def checkDetails(self):
2254        return f"checked code in file '{self.target}'"

Returns base details string describing what code was checked for a checkCodeContains check.

Inherited Members
TestManager
case
checkCodeContains
validateTrace
class FileManager.case_type(TestCase): 
1764class FileCase(TestCase):
1765    """
1766    Runs a particular file when executed. Its manager should be a
1767    `FileManager`.
1768    """
1769    # __init__ is inherited
1770
1771    def run(self):
1772        """
1773        Runs the code in the target file in an empty environment (except
1774        that `__name__` is set to `'__main__'`, to make the file behave
1775        as if it were run as the main file).
1776
1777        Note that the code is read and parsed when the `FileManager` is
1778        created, not when the test case is run.
1779        """
1780        def payload():
1781            "Payload function to run a file."
1782            global _RUNNING_TEST_CODE
1783
1784            # Fetch syntax tree from our manager
1785            node = self.manager.syntaxTree
1786
1787            if node is None:
1788                raise RuntimeError(
1789                    "Manager of a FileCase was missing a syntax tree!"
1790                )
1791
1792            # Compile the syntax tree
1793            code = compile(node, self.manager.target, 'exec')
1794
1795            # Run the code, setting __name__ to __main__ (this is
1796            # why we don't just import the file)
1797            env = {"__name__": "__main__"}
1798            try:
1799                _RUNNING_TEST_CODE = True
1800                exec(code, env)
1801            finally:
1802                _RUNNING_TEST_CODE = False
1803
1804            # Running a file doesn't have a result value, but it does
1805            # provide a module scope.
1806            return (NoResult, deepish_copy(env))
1807
1808        return self._run(payload)
1809
1810    def trialDetails(self):
1811        """
1812        Returns a pair of strings containing base and extra details
1813        describing what was tested by this test case. If the base
1814        details capture all available information, the extra details
1815        value will be None.
1816        """
1817        return (
1818            f"Ran file '{self.manager.target}'",
1819            None  # no further details to report
1820        )

The case type determines what kind of test case will be constructed when calling the TestManager.case method. Subclasses override this.

Inherited Members
TestCase
TestCase
provideInputs
showPrintedLines
fetchResults
checkReturnValue
checkVariableValue
checkPrintedLines
checkPrintedFragment
checkFileLines
checkCustom
FileCase
run
trialDetails
class FunctionManager(TestManager): 
2259class FunctionManager(TestManager):
2260    """
2261    Manages test cases for running a specific function. Arguments to the
2262    `TestManager.case` function are passed to the function being tested
2263    for that case.
2264    """
2265    case_type = FunctionCase
2266
2267    def __init__(self, function):
2268        """
2269        A FunctionManager needs a function object as the target. Each
2270        case will call that function with arguments provided when the
2271        case is created.
2272        """
2273        if not isinstance(function, types.FunctionType):
2274            raise TypeError(
2275                f"For a function test manager, the target must be a"
2276                f" function. (You provided a/an {type(function)}.)"
2277            )
2278
2279        # We need to track down the source code for this function;
2280        # luckily the inspect module makes that easy :)
2281        try:
2282            sourceCode = inspect.getsource(function)
2283        except OSError:
2284            # In some cases code might not be available, for example
2285            # when testing a function that was defined using exec.
2286            sourceCode = None
2287
2288        super().__init__(function, sourceCode)
2289
2290    def codeFilename(self):
2291        return f"function {self.target.__name__}"
2292
2293    def checkDetails(self):
2294        return f"checked code of function '{self.target.__name__}'"
2295
2296    def case(self, *args, **kwargs):
2297        """
2298        Arguments supplied here are used when calling the function which
2299        is what happens when the case is run. Returns a `FunctionCase`
2300        object.
2301        """
2302        return self.case_type(self, args, kwargs)

Manages test cases for running a specific function. Arguments to the TestManager.case function are passed to the function being tested for that case.

FunctionManager(function) 
2267    def __init__(self, function):
2268        """
2269        A FunctionManager needs a function object as the target. Each
2270        case will call that function with arguments provided when the
2271        case is created.
2272        """
2273        if not isinstance(function, types.FunctionType):
2274            raise TypeError(
2275                f"For a function test manager, the target must be a"
2276                f" function. (You provided a/an {type(function)}.)"
2277            )
2278
2279        # We need to track down the source code for this function;
2280        # luckily the inspect module makes that easy :)
2281        try:
2282            sourceCode = inspect.getsource(function)
2283        except OSError:
2284            # In some cases code might not be available, for example
2285            # when testing a function that was defined using exec.
2286            sourceCode = None
2287
2288        super().__init__(function, sourceCode)

A FunctionManager needs a function object as the target. Each case will call that function with arguments provided when the case is created.

def codeFilename(self) 
2290    def codeFilename(self):
2291        return f"function {self.target.__name__}"

Returns the filename to be used when parsing the code for this test case.

def checkDetails(self) 
2293    def checkDetails(self):
2294        return f"checked code of function '{self.target.__name__}'"

Returns base details string describing what code was checked for a checkCodeContains check.

def case(self, *args, **kwargs) 
2296    def case(self, *args, **kwargs):
2297        """
2298        Arguments supplied here are used when calling the function which
2299        is what happens when the case is run. Returns a `FunctionCase`
2300        object.
2301        """
2302        return self.case_type(self, args, kwargs)

Arguments supplied here are used when calling the function which is what happens when the case is run. Returns a FunctionCase object.

Inherited Members
TestManager
checkCodeContains
validateTrace
class FunctionManager.case_type(TestCase): 
1823class FunctionCase(TestCase):
1824    """
1825    Calls a particular function with specific arguments when run.
1826    """
1827    def __init__(self, manager, args=None, kwargs=None):
1828        """
1829        The arguments and/or keyword arguments to be used for the case
1830        are provided after the manager (as a list and a dictionary, NOT
1831        as actual arguments). If omitted, the function will be called
1832        with no arguments.
1833        """
1834        super().__init__(manager)
1835        self.args = args or ()
1836        self.kwargs = kwargs or {}
1837
1838    def run(self):
1839        """
1840        Runs the target function with the arguments specified for this
1841        case. The 'result' slot of the `self.results` dictionary that it
1842        creates holds the return value of the function.
1843        """
1844        def payload():
1845            "Payload for running a function with specific arguments."
1846            global _RUNNING_TEST_CODE
1847            try:
1848                _RUNNING_TEST_CODE = True
1849                result = (
1850                    self.manager.target(*self.args, **self.kwargs),
1851                    None  # no scope for a function TODO: Get locals?
1852                )
1853            finally:
1854                _RUNNING_TEST_CODE = False
1855            return result
1856
1857        return self._run(payload)
1858
1859    def trialDetails(self):
1860        """
1861        Returns a pair of strings containing base and extra details
1862        describing what was tested by this test case. If the base
1863        details capture all available information, the extra details
1864        value will be None.
1865        """
1866        # Show function name + args, possibly with some abbreviation
1867        fn = self.manager.target
1868        msg = f"Called function '{fn.__name__}'"
1869
1870        args = self.args if self.args is not None else []
1871        kwargs = self.kwargs if self.args is not None else {}
1872        all_args = len(args) + len(kwargs)
1873
1874        argnames = fn.__code__.co_varnames[:all_args]
1875        if len(args) > len(argnames):
1876            msg += " with too many arguments (!):"
1877        elif all_args > 0:
1878            msg += " with arguments:"
1879
1880        # TODO: Proper handling of *args and **kwargs entries!
1881
1882        # Create lists of full and maybe-abbreviated argument
1883        # strings
1884        argstrings = []
1885        short_argstrings = []
1886        for i, arg in enumerate(args):
1887            if i < len(argnames):
1888                name = argnames[i]
1889            else:
1890                name = f"extra argument #{i - len(argnames) + 1}"
1891            short_name = ellipsis(name, 20)
1892
1893            argstrings.append(f"{name} = {repr(arg)}")
1894            short_argstrings.append(
1895                f"{short_name} = {ellipsis(repr(arg), 60)}"
1896            )
1897
1898        # Order kwargs by original kwargs order and then by natural
1899        # order of kwargs dictionary
1900        keyset = set(kwargs)
1901        ordered = list(filter(lambda x: x in keyset, argnames))
1902        rest = [k for k in kwargs if k not in ordered]
1903        for k in ordered + rest:
1904            argstrings.append(f"{k} = {repr(kwargs[k])}")
1905            short_name = ellipsis(k, 20)
1906            short_argstrings.append(
1907                f"{short_name} = {ellipsis(repr(kwargs[k]), 60)}"
1908            )
1909
1910        full_args = '  ' + '\n  '.join(argstrings)
1911        # In case there are too many arguments
1912        if len(short_argstrings) < 20:
1913            short_args = '  ' + '\n  '.join(short_argstrings)
1914        else:
1915            short_args = (
1916                '  '
1917              + '\n  '.join(short_argstrings[:19])
1918              + f"...plus {len(argstrings) - 19} more arguments..."
1919            )
1920
1921        if short_args == full_args:
1922            return (
1923                msg + '\n' + short_args,
1924                None
1925            )
1926        else:
1927            return (
1928                msg + '\n' + short_args,
1929                "Full arguments were:\n" + full_args
1930            )

The case type determines what kind of test case will be constructed when calling the TestManager.case method. Subclasses override this.

Inherited Members
FunctionCase
FunctionCase
run
trialDetails
TestCase
provideInputs
showPrintedLines
fetchResults
checkReturnValue
checkVariableValue
checkPrintedLines
checkPrintedFragment
checkFileLines
checkCustom
class BlockManager(TestManager): 
2305class BlockManager(TestManager):
2306    """
2307    Manages test cases for running a block of code (from a string).
2308    Keyword arguments to the `TestManager.case` function are defined as
2309    variables before the block is executed in that case.
2310    """
2311    case_type = BlockCase
2312
2313    def __init__(self, code, includeGlobals=False):
2314        """
2315        A BlockManager needs a code string as the target (the actual
2316        target value will be set to `None`). Optionally, the
2317        `use_globals` argument (default `False`) can be set to `True` to
2318        make globals defined at case-creation time accessible to the
2319        code in the case.
2320        """
2321        if not isinstance(code, str):
2322            raise TypeError(
2323                f"For a 'block' test manager, the target must be a"
2324                f" string. (You provided a/an {type(code)}.)"
2325            )
2326
2327        # TODO: This check is good, but avoiding multiple parsing passes
2328        # might be nice for larger code blocks...
2329        try:
2330            ast.parse(code)
2331        except Exception:
2332            raise ValueError(
2333                "The code block you provided could not be parsed as Python"
2334                " code."
2335            )
2336
2337        self.includeGlobals = bool(includeGlobals)
2338
2339        super().__init__("a code block", code)
2340        # Now that we have a tag, update our target
2341        self.target = f"code block from {self.tag}"
2342
2343    def codeFilename(self):
2344        return self.target
2345
2346    def checkDetails(self):
2347        return f"checked code from block at {self.tag}"
2348
2349    def case(self, **assignments):
2350        """
2351        Keyword argument supplied here will be defined as variables
2352        in the environment used to run the code block, and will override
2353        any global variable values (which are only included if
2354        `includeGlobals` was set to true when the manager was created).
2355        Returns a `BlockCase` object.
2356        """
2357        if self.includeGlobals:
2358            provide = copy.copy(get_external_calling_frame().f_globals)
2359            provide.update(assignments)
2360        else:
2361            provide = assignments
2362
2363        return self.case_type(self, provide)

Manages test cases for running a block of code (from a string). Keyword arguments to the TestManager.case function are defined as variables before the block is executed in that case.

BlockManager(code, includeGlobals=False) 
2313    def __init__(self, code, includeGlobals=False):
2314        """
2315        A BlockManager needs a code string as the target (the actual
2316        target value will be set to `None`). Optionally, the
2317        `use_globals` argument (default `False`) can be set to `True` to
2318        make globals defined at case-creation time accessible to the
2319        code in the case.
2320        """
2321        if not isinstance(code, str):
2322            raise TypeError(
2323                f"For a 'block' test manager, the target must be a"
2324                f" string. (You provided a/an {type(code)}.)"
2325            )
2326
2327        # TODO: This check is good, but avoiding multiple parsing passes
2328        # might be nice for larger code blocks...
2329        try:
2330            ast.parse(code)
2331        except Exception:
2332            raise ValueError(
2333                "The code block you provided could not be parsed as Python"
2334                " code."
2335            )
2336
2337        self.includeGlobals = bool(includeGlobals)
2338
2339        super().__init__("a code block", code)
2340        # Now that we have a tag, update our target
2341        self.target = f"code block from {self.tag}"

A BlockManager needs a code string as the target (the actual target value will be set to None). Optionally, the use_globals argument (default False) can be set to True to make globals defined at case-creation time accessible to the code in the case.

def codeFilename(self) 
2343    def codeFilename(self):
2344        return self.target

Returns the filename to be used when parsing the code for this test case.

def checkDetails(self) 
2346    def checkDetails(self):
2347        return f"checked code from block at {self.tag}"

Returns base details string describing what code was checked for a checkCodeContains check.

def case(self, **assignments) 
2349    def case(self, **assignments):
2350        """
2351        Keyword argument supplied here will be defined as variables
2352        in the environment used to run the code block, and will override
2353        any global variable values (which are only included if
2354        `includeGlobals` was set to true when the manager was created).
2355        Returns a `BlockCase` object.
2356        """
2357        if self.includeGlobals:
2358            provide = copy.copy(get_external_calling_frame().f_globals)
2359            provide.update(assignments)
2360        else:
2361            provide = assignments
2362
2363        return self.case_type(self, provide)

Keyword argument supplied here will be defined as variables in the environment used to run the code block, and will override any global variable values (which are only included if includeGlobals was set to true when the manager was created). Returns a BlockCase object.

Inherited Members
TestManager
checkCodeContains
validateTrace
class BlockManager.case_type(TestCase): 
1933class BlockCase(TestCase):
1934    """
1935    Executes a block of code (provided as text) when run. Per-case
1936    variables may be defined for the execution environment, which
1937    otherwise just has builtins.
1938    """
1939    def __init__(self, manager, assignments=None):
1940        """
1941        A dictionary of variable name : value assignments may be
1942        provided and these will be inserted into the execution
1943        environment for the code block. If omitted, no extra variables
1944        will be defined (this means that global variables available when
1945        the test manager and/or code block is set up are NOT available to
1946        the code in the code block by default).
1947        """
1948        super().__init__(manager)
1949        self.assignments = assignments or {}
1950
1951    def run(self):
1952        """
1953        Compiles and runs the target code block in an environment which
1954        is empty except for the assignments specified in this case (and
1955        builtins).
1956        """
1957        def payload():
1958            "Payload for running a code block specific variables active."
1959            global _RUNNING_TEST_CODE
1960            env = dict(self.assignments)
1961            try:
1962                _RUNNING_TEST_CODE = True
1963                exec(self.manager.code, env)
1964            finally:
1965                _RUNNING_TEST_CODE = False
1966            return (NoResult, deepish_copy(env))
1967
1968        return self._run(payload)
1969
1970    def trialDetails(self):
1971        """
1972        Returns a pair of strings containing base and extra details
1973        describing what was tested by this test case. If the base
1974        details capture all available information, the extra details
1975        value will be None.
1976        """
1977        block = self.manager.code
1978        short = limited_repr(block)
1979        if block == short:
1980            # Short enough to show whole block
1981            return (
1982                "Ran code:\n" + indent(block, 2),
1983                None
1984            )
1985
1986        else:
1987            # Too long to show whole block in short view...
1988            return (
1989                "Ran code:\n" + indent(short, 2),
1990                "Full code was:\n" + indent(block, 2)
1991            )

The case type determines what kind of test case will be constructed when calling the TestManager.case method. Subclasses override this.

Inherited Members
BlockCase
BlockCase
run
trialDetails
TestCase
provideInputs
showPrintedLines
fetchResults
checkReturnValue
checkVariableValue
checkPrintedLines
checkPrintedFragment
checkFileLines
checkCustom
class SkipManager(TestManager): 
2366class SkipManager(TestManager):
2367    """
2368    Manages fake test cases for a file, function, or code block that
2369    needs to be skipped (perhaps for a function that doesn't yet exist,
2370    for example). Cases derived are `SkipCase` objects which just print
2371    skip messages for any checks requested.
2372    """
2373    case_type = SkipCase
2374
2375    def __init__(self, label):
2376        """
2377        Needs a label string to identify which tests are being skipped.
2378        """
2379        if not isinstance(label, str):
2380            raise TypeError(
2381                f"For a skip test manager, the target must be a string."
2382                f" (You provided a/an {type(label)}.)"
2383            )
2384        super().__init__(label, None)
2385
2386    def codeFilename(self):
2387        return "no code (cases skipped)"
2388
2389    def checkDetails(self):
2390        return "skipped check (no code available)"
2391
2392    def case(self, *_, **__):
2393        """
2394        Accepts (and ignores) any extra arguments.
2395        """
2396        return super().case()
2397
2398    def checkCodeContains(self, checkFor):
2399        """
2400        Skips checking the AST of the target; see
2401        `TestManager.checkCodeContains`.
2402        """
2403        tag = tag_for(get_my_location())
2404        # Detail level controls initial message
2405        if DETAIL_LEVEL < 1:
2406            msg = f"~ {tag} (skipped)"
2407        else:
2408            msg = (
2409                f"~ code check at {tag} skipped"
2410            )
2411        print_message(msg, color=msg_color("skipped"))

Manages fake test cases for a file, function, or code block that needs to be skipped (perhaps for a function that doesn't yet exist, for example). Cases derived are SkipCase objects which just print skip messages for any checks requested.

SkipManager(label) 
2375    def __init__(self, label):
2376        """
2377        Needs a label string to identify which tests are being skipped.
2378        """
2379        if not isinstance(label, str):
2380            raise TypeError(
2381                f"For a skip test manager, the target must be a string."
2382                f" (You provided a/an {type(label)}.)"
2383            )
2384        super().__init__(label, None)

Needs a label string to identify which tests are being skipped.

def codeFilename(self) 
2386    def codeFilename(self):
2387        return "no code (cases skipped)"

Returns the filename to be used when parsing the code for this test case.

def checkDetails(self) 
2389    def checkDetails(self):
2390        return "skipped check (no code available)"

Returns base details string describing what code was checked for a checkCodeContains check.

def case(self, *_, **__) 
2392    def case(self, *_, **__):
2393        """
2394        Accepts (and ignores) any extra arguments.
2395        """
2396        return super().case()

Accepts (and ignores) any extra arguments.

def checkCodeContains(self, checkFor) 
2398    def checkCodeContains(self, checkFor):
2399        """
2400        Skips checking the AST of the target; see
2401        `TestManager.checkCodeContains`.
2402        """
2403        tag = tag_for(get_my_location())
2404        # Detail level controls initial message
2405        if DETAIL_LEVEL < 1:
2406            msg = f"~ {tag} (skipped)"
2407        else:
2408            msg = (
2409                f"~ code check at {tag} skipped"
2410            )
2411        print_message(msg, color=msg_color("skipped"))

Skips checking the AST of the target; see TestManager.checkCodeContains.

Inherited Members
TestManager
validateTrace
class SkipManager.case_type(TestCase): 
1994class SkipCase(TestCase):
1995    """
1996    A type of test case which actually doesn't run checks, but instead
1997    prints a message that the check was skipped.
1998    """
1999    # __init__ is inherited
2000
2001    def run(self):
2002        """
2003        Since there is no real test, our results are fake. The keys
2004        "error" and "traceback" have None as their value, and "output"
2005        also has None. We add a key "skipped" with value True.
2006        """
2007        self.results = {
2008            "output": None,
2009            "error": None,
2010            "traceback": None,
2011            "skipped": True
2012        }
2013        return self.results
2014
2015    def trialDetails(self):
2016        """
2017        Provides a pair of topic/details strings about this test.
2018        """
2019        return (f"Skipped check of '{self.manager.target}'", None)
2020
2021    def checkReturnValue(self, _, **__):
2022        """
2023        Skips the check.
2024        """
2025        self._print_skip_message(
2026            tag_for(get_my_location()),
2027            "testing target not available"
2028        )
2029
2030    def checkVariableValue(self, *_, **__):
2031        """
2032        Skips the check.
2033        """
2034        self._print_skip_message(
2035            tag_for(get_my_location()),
2036            "testing target not available"
2037        )
2038
2039    def checkPrintedLines(self, *_, **__):
2040        """
2041        Skips the check.
2042        """
2043        self._print_skip_message(
2044            tag_for(get_my_location()),
2045            "testing target not available"
2046        )
2047
2048    def checkPrintedFragment(self, *_, **__):
2049        """
2050        Skips the check.
2051        """
2052        self._print_skip_message(
2053            tag_for(get_my_location()),
2054            "testing target not available"
2055        )
2056
2057    def checkFileLines(self, *_, **__):
2058        """
2059        Skips the check.
2060        """
2061        self._print_skip_message(
2062            tag_for(get_my_location()),
2063            "testing target not available"
2064        )
2065
2066    def checkCustom(self, _, **__):
2067        """
2068        Skips the check.
2069        """
2070        self._print_skip_message(
2071            tag_for(get_my_location()),
2072            "testing target not available"
2073        )

The case type determines what kind of test case will be constructed when calling the TestManager.case method. Subclasses override this.

Inherited Members
TestCase
TestCase
provideInputs
showPrintedLines
fetchResults
SkipCase
run
trialDetails
checkReturnValue
checkVariableValue
checkPrintedLines
checkPrintedFragment
checkFileLines
checkCustom
class QuietSkipManager(TestManager): 
2414class QuietSkipManager(TestManager):
2415    """
2416    Manages fake test cases that should be skipped silently, without any
2417    notifications. Cases derived are `SilentCase` objects which don't
2418    print anything.
2419    """
2420    case_type = SilentCase
2421
2422    def __init__(self):
2423        """
2424        No arguments needed.
2425        """
2426        super().__init__("ignored", None)
2427
2428    def codeFilename(self):
2429        return "no code (cases skipped)"
2430
2431    def checkDetails(self):
2432        return "skipped check (no code available)"
2433
2434    def case(self, *_, **__):
2435        """
2436        Accepts (and ignores) any extra arguments.
2437        """
2438        return super().case()
2439
2440    def checkCodeContains(self, checkFor):
2441        """
2442        Skips checking the AST; returns `None`.
2443        """
2444        return None

Manages fake test cases that should be skipped silently, without any notifications. Cases derived are SilentCase objects which don't print anything.

QuietSkipManager() 
2422    def __init__(self):
2423        """
2424        No arguments needed.
2425        """
2426        super().__init__("ignored", None)

No arguments needed.

def codeFilename(self) 
2428    def codeFilename(self):
2429        return "no code (cases skipped)"

Returns the filename to be used when parsing the code for this test case.

def checkDetails(self) 
2431    def checkDetails(self):
2432        return "skipped check (no code available)"

Returns base details string describing what code was checked for a checkCodeContains check.

def case(self, *_, **__) 
2434    def case(self, *_, **__):
2435        """
2436        Accepts (and ignores) any extra arguments.
2437        """
2438        return super().case()

Accepts (and ignores) any extra arguments.

def checkCodeContains(self, checkFor) 
2440    def checkCodeContains(self, checkFor):
2441        """
2442        Skips checking the AST; returns `None`.
2443        """
2444        return None

Skips checking the AST; returns None.

Inherited Members
TestManager
validateTrace
class QuietSkipManager.case_type(TestCase): 
2076class SilentCase(TestCase):
2077    """
2078    A type of test case which actually doesn't run checks, and also
2079    prints nothing. Just exists so that errors won't be thrown when
2080    checks are attempted. Testing methods return `None` instead of `True`
2081    or `False`, although this is not counted as a test failure.
2082    """
2083    # __init__ is inherited
2084
2085    def run(self):
2086        "Returns fake empty results."
2087        self.results = {
2088            "output": None,
2089            "error": None,
2090            "traceback": None,
2091            "skipped": True
2092        }
2093        return self.results
2094
2095    def trialDetails(self):
2096        """
2097        Provides a pair of topic/details strings about this test.
2098        """
2099        return ("Silently skipped check", None)
2100
2101    def checkReturnValue(self, _, **__):
2102        "Returns `None`."
2103        return None
2104
2105    def checkVariableValue(self, *_, **__):
2106        "Returns `None`."
2107        return None
2108
2109    def checkPrintedLines(self, *_, **__):
2110        "Returns `None`."
2111        return None
2112
2113    def checkPrintedFragment(self, *_, **__):
2114        "Returns `None`."
2115        return None
2116
2117    def checkFileLines(self, *_, **__):
2118        "Returns `None`."
2119        return None
2120
2121    def checkCustom(self, _, **__):
2122        "Returns `None`."
2123        return None

The case type determines what kind of test case will be constructed when calling the TestManager.case method. Subclasses override this.

Inherited Members
TestCase
TestCase
provideInputs
showPrintedLines
fetchResults
SilentCase
run
trialDetails
checkReturnValue
checkVariableValue
checkPrintedLines
checkPrintedFragment
checkFileLines
checkCustom
def testFunction(fn) 
2451def testFunction(fn):
2452    """
2453    Creates a test-manager for the given function.
2454    """
2455    if not isinstance(fn, types.FunctionType):
2456        raise TypeError(
2457            "Test target must be a function (use testFile or testBlock"
2458            " instead to test a file or block of code)."
2459        )
2460
2461    return FunctionManager(fn)

Creates a test-manager for the given function.

def testFunctionMaybe(module, fname) 
2464def testFunctionMaybe(module, fname):
2465    """
2466    This function creates a test-manager for a named function from a
2467    specific module, but displays an alternate message and returns a
2468    dummy manager if that module doesn't define any variable with the
2469    target name. Useful for defining tests for functions that will be
2470    skipped if the functions aren't done yet.
2471    """
2472    # Message if we can't find the function
2473    if not hasattr(module, fname):
2474        print_message(
2475            f"Did not find '{fname}' in module '{module.__name__}'...",
2476            color=msg_color("skipped")
2477        )
2478        return SkipManager(f"{module.__name__}.{fname}")
2479    else:
2480        target = getattr(module, fname)
2481        if not isinstance(target, types.FunctionType):
2482            print_message(
2483                (
2484                    f"'{fname}' in module '{module.__name__}' is not a"
2485                    f" function..."
2486                ),
2487                color=msg_color("skipped")
2488            )
2489            return SkipManager(f"{module.__name__}.{fname}")
2490        else:
2491            return FunctionManager(target)

This function creates a test-manager for a named function from a specific module, but displays an alternate message and returns a dummy manager if that module doesn't define any variable with the target name. Useful for defining tests for functions that will be skipped if the functions aren't done yet.

def testFile(filename) 
2494def testFile(filename):
2495    """
2496    Creates a test-manager for running the named file.
2497    """
2498    if not isinstance(filename, str):
2499        raise TypeError(
2500            "Test target must be a file name (use testFunction instead"
2501            " to test a function)."
2502        )
2503
2504    if not os.path.exists(filename):
2505        raise FileNotFoundError(
2506            f"We cannot create a test for running '{filename}' because"
2507            f" that file does not exist."
2508        )
2509
2510    return FileManager(filename)

Creates a test-manager for running the named file.

def testBlock(code, includeGlobals=False) 
2513def testBlock(code, includeGlobals=False):
2514    """
2515    Creates a test-manager for running a block of code (provided as a
2516    string). If `includeGlobals` is set to true, global variables which
2517    are defined at the time a case is created from the manager will be
2518    available to the code in that test case; if not (the default) no
2519    variables defined outside of the test block are available to the code
2520    in the block, except for explicit definitions supplied when creating
2521    a test case (see `BlockManager.case`).
2522    """
2523    if not isinstance(code, str):
2524        raise TypeError(
2525            "Test target must be a code string (use testFunction instead"
2526            " to test a function)."
2527        )
2528
2529    return BlockManager(code, includeGlobals)

Creates a test-manager for running a block of code (provided as a string). If includeGlobals is set to true, global variables which are defined at the time a case is created from the manager will be available to the code in that test case; if not (the default) no variables defined outside of the test block are available to the code in the block, except for explicit definitions supplied when creating a test case (see BlockManager.case).

SKIP_NOTEBOOK_CELL_CHECKS = False

If set to true, notebook cell checks will be skipped silently. This is used to avoid recursive checking problems.

def beginSkippingNotebookCellChecks() 
2545def beginSkippingNotebookCellChecks():
2546    """
2547    Sets `SKIP_NOTEBOOK_CELL_CHECKS` to True, and saves the old value in
2548    `_SKIP_NOTEBOOK_CELL_CHECKS`.
2549    """
2550    global SKIP_NOTEBOOK_CELL_CHECKS, _SKIP_NOTEBOOK_CELL_CHECKS
2551    _SKIP_NOTEBOOK_CELL_CHECKS = SKIP_NOTEBOOK_CELL_CHECKS
2552    SKIP_NOTEBOOK_CELL_CHECKS = True

Sets SKIP_NOTEBOOK_CELL_CHECKS to True, and saves the old value in _SKIP_NOTEBOOK_CELL_CHECKS.

def endSkippingNotebookCellChecks() 
2555def endSkippingNotebookCellChecks():
2556    """
2557    Sets `SKIP_NOTEBOOK_CELL_CHECKS` back to whatever value was stored
2558    when `beginSkippingNotebookCellChecks` was called (might not actually
2559    end skipping, because of that).
2560    """
2561    global SKIP_NOTEBOOK_CELL_CHECKS, _SKIP_NOTEBOOK_CELL_CHECKS
2562    SKIP_NOTEBOOK_CELL_CHECKS = _SKIP_NOTEBOOK_CELL_CHECKS

Sets SKIP_NOTEBOOK_CELL_CHECKS back to whatever value was stored when beginSkippingNotebookCellChecks was called (might not actually end skipping, because of that).

def testThisNotebookCell(includeGlobals=True) 
2565def testThisNotebookCell(includeGlobals=True):
2566    """
2567    Creates a test manager for running code in an IPython (and by
2568    implication also Jupyter) notebook cell (without any
2569    other cells being run). The current cell that is executing when the
2570    function is called is captured as a string and a `BlockManager` is
2571    created for that string, with `includeGlobals` set to `True` (you
2572    can override that by providing `False` as an argument to this
2573    function).
2574
2575    This function will raise an error if it is called outside of an
2576    IPython context, although this will not happen if
2577    `SKIP_NOTEBOOK_CELL_CHECKS` is set (see below).
2578
2579    If the `SKIP_NOTEBOOK_CELL_CHECKS` global variable is `True`, the
2580    result will be a special silent `QuietSkipManager` instead of a
2581    `BlockManager`. The code block captured from the notebook cell is
2582    augmented to set that variable to True at the beginning and back to
2583    its original value at the end, to avoid infinite recursion.
2584    """
2585    if SKIP_NOTEBOOK_CELL_CHECKS:
2586        return QuietSkipManager()
2587
2588    try:
2589        hist = get_ipython().history_manager  # noqa F821
2590    except Exception:
2591        raise RuntimeError(
2592            "Failed to get IPython context; testThisNotebookCell will"
2593            " only work when run from within a notebook."
2594        )
2595
2596    sessionID = hist.get_last_session_id()
2597    thisCellCode = next(hist.get_range(sessionID, start=-1, stop=None))[2]
2598    return BlockManager(
2599        (
2600            "import optimism\n"
2601          + "optimism.beginSkippingNotebookCellChecks()\n"
2602          + "try:\n"
2603          + indent(thisCellCode, 4)
2604          + "\nfinally:\n"
2605          + "    optimism.endSkippingNotebookCellChecks()\n"
2606        ),
2607        includeGlobals
2608    )

Creates a test manager for running code in an IPython (and by implication also Jupyter) notebook cell (without any other cells being run). The current cell that is executing when the function is called is captured as a string and a BlockManager is created for that string, with includeGlobals set to True (you can override that by providing False as an argument to this function).

This function will raise an error if it is called outside of an IPython context, although this will not happen if SKIP_NOTEBOOK_CELL_CHECKS is set (see below).

If the SKIP_NOTEBOOK_CELL_CHECKS global variable is True, the result will be a special silent QuietSkipManager instead of a BlockManager. The code block captured from the notebook cell is augmented to set that variable to True at the beginning and back to its original value at the end, to avoid infinite recursion.

def mark(name) 
2611def mark(name):
2612    """
2613    Collects the code of the file or notebook cell within which the
2614    function call occurs, and caches it for later testing using
2615    `testMarkedCode`. Note that all code in a file or notebook cell is
2616    collected: if you use `mark` multiple times in the same file each
2617    `testMarkedCode` call will still test the entire file.
2618
2619    Also, if this function is called during the run of another test and a
2620    code block is not available but an old code block was under the same
2621    name, that old code block will not be modified.
2622    """
2623    global _MARKED_CODE_BLOCKS
2624    block_filename = get_filename(get_external_calling_frame())
2625    contents = ''.join(linecache.getlines(block_filename))
2626    if contents or not _RUNNING_TEST_CODE:
2627        _MARKED_CODE_BLOCKS[name] = contents or None

Collects the code of the file or notebook cell within which the function call occurs, and caches it for later testing using testMarkedCode. Note that all code in a file or notebook cell is collected: if you use mark multiple times in the same file each testMarkedCode call will still test the entire file.

Also, if this function is called during the run of another test and a code block is not available but an old code block was under the same name, that old code block will not be modified.

def getMarkedCode(markName) 
2630def getMarkedCode(markName):
2631    """
2632    Gets the block of code (e.g., Python file; notebook cell; etc.)
2633    within which `mark` was called with the specified name. Returns
2634    `None` if that information isn't available. Reasons it isn't
2635    available include that `mark` was never called with that name, and
2636    that `mark` was called, but we weren't able to extract the source
2637    code of the block it was called in (e.g., because it was called in
2638    an interactive interpreter session).
2639    """
2640    return _MARKED_CODE_BLOCKS.get(markName)

Gets the block of code (e.g., Python file; notebook cell; etc.) within which mark was called with the specified name. Returns None if that information isn't available. Reasons it isn't available include that mark was never called with that name, and that mark was called, but we weren't able to extract the source code of the block it was called in (e.g., because it was called in an interactive interpreter session).

def testMarkedCode(markName, includeGlobals=True) 
2643def testMarkedCode(markName, includeGlobals=True):
2644    """
2645    Creates a test manager for running the code block (e.g., Python file;
2646    notebook cell; etc.) within which `mark` was called using the given
2647    mark name. `mark` must have already been called with the specified
2648    name, and changes to the code around it may or may not be picked up
2649    if they were made since the call happened. A `BlockManager` is
2650    created for that code, with `includeGlobals` set based on the value
2651    provided here (default `True`).
2652
2653    If no code is available, a `SkipManager` will be returned.
2654    """
2655    code = getMarkedCode(markName)
2656    if code is None:
2657        print(
2658            (
2659                f"Warning: unable to find code for test suite"
2660                f" '{markName}'. Have you called 'mark' already with"
2661                f" that name?"
2662            ),
2663            file=PRINT_TO
2664        )
2665        return SkipManager("Code around mark '{markName}'")
2666    else:
2667        return BlockManager(code, includeGlobals)

Creates a test manager for running the code block (e.g., Python file; notebook cell; etc.) within which mark was called using the given mark name. mark must have already been called with the specified name, and changes to the code around it may or may not be picked up if they were made since the call happened. A BlockManager is created for that code, with includeGlobals set based on the value provided here (default True).

If no code is available, a SkipManager will be returned.

class CapturingStream(_io.StringIO): 
2674class CapturingStream(io.StringIO):
2675    """
2676    An output capture object which is an `io.StringIO` underneath, but
2677    which has an option to also write incoming text to normal
2678    `sys.stdout`. Call the install function to begin capture.
2679    """
2680    def __init__(self, *args, **kwargs):
2681        """
2682        Passes arguments through to `io.StringIO`'s constructor.
2683        """
2684        self.original_stdout = None
2685        self.tee = False
2686        super().__init__(*args, **kwargs)
2687
2688    def echo(self, doit=True):
2689        """
2690        Turn on echoing to stdout along with capture, or turn it off if
2691        False is given.
2692        """
2693        self.tee = doit
2694
2695    def install(self):
2696        """
2697        Replaces `sys.stdout` to begin capturing printed output.
2698        Remembers the old `sys.stdout` value so that `uninstall` can
2699        work. Note that if someone else changes `sys.stdout` after this
2700        is installed, uninstall will set `sys.stdout` back to what it was
2701        when `install` was called, which could cause issues. For example,
2702        if we have two capturing streams A and B, and we call:
2703
2704        ```py
2705        A.install()
2706        B.install()
2707        A.uninstall()
2708        B.uninstall()
2709        ```
2710
2711        The original `sys.stdout` will not be restored. In general, you
2712        must uninstall capturing streams in the reverse order that you
2713        installed them.
2714        """
2715        self.original_stdout = sys.stdout
2716        sys.stdout = self
2717
2718    def uninstall(self):
2719        """
2720        Returns `sys.stdout` to what it was before `install` was called,
2721        or does nothing if `install` was never called.
2722        """
2723        if self.original_stdout is not None:
2724            sys.stdout = self.original_stdout
2725
2726    def reset(self):
2727        """
2728        Resets the captured output.
2729        """
2730        self.seek(0)
2731        self.truncate(0)
2732
2733    def writelines(self, lines):
2734        """
2735        Override writelines to work through write.
2736        """
2737        for line in lines:
2738            self.write(line)
2739
2740    def write(self, stuff):
2741        """
2742        Accepts a string and writes to our capture buffer (and to
2743        original stdout if `echo` has been called). Returns the number
2744        of characters written.
2745        """
2746        if self.tee and self.original_stdout is not None:
2747            self.original_stdout.write(stuff)
2748        super().write(stuff)

An output capture object which is an io.StringIO underneath, but which has an option to also write incoming text to normal sys.stdout. Call the install function to begin capture.

CapturingStream(*args, **kwargs) 
2680    def __init__(self, *args, **kwargs):
2681        """
2682        Passes arguments through to `io.StringIO`'s constructor.
2683        """
2684        self.original_stdout = None
2685        self.tee = False
2686        super().__init__(*args, **kwargs)

Passes arguments through to io.StringIO's constructor.

def echo(self, doit=True) 
2688    def echo(self, doit=True):
2689        """
2690        Turn on echoing to stdout along with capture, or turn it off if
2691        False is given.
2692        """
2693        self.tee = doit

Turn on echoing to stdout along with capture, or turn it off if False is given.

def install(self) 
2695    def install(self):
2696        """
2697        Replaces `sys.stdout` to begin capturing printed output.
2698        Remembers the old `sys.stdout` value so that `uninstall` can
2699        work. Note that if someone else changes `sys.stdout` after this
2700        is installed, uninstall will set `sys.stdout` back to what it was
2701        when `install` was called, which could cause issues. For example,
2702        if we have two capturing streams A and B, and we call:
2703
2704        ```py
2705        A.install()
2706        B.install()
2707        A.uninstall()
2708        B.uninstall()
2709        ```
2710
2711        The original `sys.stdout` will not be restored. In general, you
2712        must uninstall capturing streams in the reverse order that you
2713        installed them.
2714        """
2715        self.original_stdout = sys.stdout
2716        sys.stdout = self

Replaces sys.stdout to begin capturing printed output. Remembers the old sys.stdout value so that uninstall can work. Note that if someone else changes sys.stdout after this is installed, uninstall will set sys.stdout back to what it was when install was called, which could cause issues. For example, if we have two capturing streams A and B, and we call:

A.install()
B.install()
A.uninstall()
B.uninstall()

The original sys.stdout will not be restored. In general, you must uninstall capturing streams in the reverse order that you installed them.

def uninstall(self) 
2718    def uninstall(self):
2719        """
2720        Returns `sys.stdout` to what it was before `install` was called,
2721        or does nothing if `install` was never called.
2722        """
2723        if self.original_stdout is not None:
2724            sys.stdout = self.original_stdout

Returns sys.stdout to what it was before install was called, or does nothing if install was never called.

def reset(self) 
2726    def reset(self):
2727        """
2728        Resets the captured output.
2729        """
2730        self.seek(0)
2731        self.truncate(0)

Resets the captured output.

def writelines(self, lines) 
2733    def writelines(self, lines):
2734        """
2735        Override writelines to work through write.
2736        """
2737        for line in lines:
2738            self.write(line)

Override writelines to work through write.

def write(self, stuff) 
2740    def write(self, stuff):
2741        """
2742        Accepts a string and writes to our capture buffer (and to
2743        original stdout if `echo` has been called). Returns the number
2744        of characters written.
2745        """
2746        if self.tee and self.original_stdout is not None:
2747            self.original_stdout.write(stuff)
2748        super().write(stuff)

Accepts a string and writes to our capture buffer (and to original stdout if echo has been called). Returns the number of characters written.

Inherited Members
_io.StringIO
close
getvalue
read
readline
tell
truncate
seek
seekable
readable
writable
closed
newlines
line_buffering
_io._TextIOBase
detach
encoding
errors
_io._IOBase
flush
fileno
isatty
readlines
def showPrintedLines(show=True) 
2751def showPrintedLines(show=True):
2752    """
2753    Changes the testing mechanisms so that printed output produced during
2754    tests is shown as normal in addition to being captured. Call it with
2755    False as an argument to disable this.
2756    """
2757    global _SHOW_OUTPUT
2758    _SHOW_OUTPUT = show

Changes the testing mechanisms so that printed output produced during tests is shown as normal in addition to being captured. Call it with False as an argument to disable this.

def differencesAreSubtle(val, ref) 
2765def differencesAreSubtle(val, ref):
2766    """
2767    Judges whether differences between two strings are 'subtle' in which
2768    case the first difference details will be displayed. Returns true if
2769    either value is longer than a typical floating-point number, or if
2770    the representations are the same once all whitespace is stripped
2771    out.
2772    """
2773    # If either has non-trivial length, we'll include the first
2774    # difference report. 18 is the length of a floating-point number
2775    # with two digits before the decimal point and max digits afterwards
2776    if len(val) > 18 or len(ref) > 18:
2777        return True
2778
2779    valNoWS = re.sub(r'\s', '', val)
2780    refNoWS = re.sub(r'\s', '', ref)
2781    # If they're the same modulo whitespace, then it's probably useful
2782    # to report first difference, otherwise we won't
2783    return valNoWS == refNoWS

Judges whether differences between two strings are 'subtle' in which case the first difference details will be displayed. Returns true if either value is longer than a typical floating-point number, or if the representations are the same once all whitespace is stripped out.

def expect(expr, value) 
2786def expect(expr, value):
2787    """
2788    Establishes an immediate expectation that the values of the two
2789    arguments should be equivalent. The expression provided will be
2790    picked out of the source code of the module calling `expect` (see
2791    `get_my_context`). The expression and sub-values will be displayed
2792    if the expectation is not met, and either way a message indicating
2793    success or failure will be printed. Use `detailLevel` to control how
2794    detailed the messages are.
2795
2796    For `expect` to work properly, the following rules must be followed:
2797
2798    1. When multiple calls to `expect` appear on a single line of the
2799        source code (something you should probably avoid anyway), none of
2800        the calls should execute more times than another when that line
2801        is executed (it's difficult to violate this, but examples
2802        include the use of `expect` multiple times on one line within
2803        generator or if/else expressions)
2804    2. None of the following components of the expression passed to
2805        `expect` should have side effects when evaluated:
2806        - Attribute accesses
2807        - Subscripts (including expressions inside brackets)
2808        - Variable lookups
2809        (Note that those things don't normally have side effects!)
2810
2811    This function returns True if the expectation is met and False
2812    otherwise. It returns None if the check is skipped, which will
2813    happen when `SKIP_ON_FAILURE` is `'all'` and a previous check failed.
2814    If the values are not equivalent, this will count as a failed check
2815    and other checks may be skipped.
2816
2817    If not skipped, this function registers an outcome in `ALL_OUTCOMES`.
2818    """
2819    global CHECK_FAILED
2820    context = get_my_context(expect)
2821    tag = tag_for(context)
2822
2823    # Skip this expectation if necessary
2824    if SKIP_ON_FAILURE == 'all' and CHECK_FAILED:
2825        if DETAIL_LEVEL < 1:
2826            msg = f"~ {tag} (skipped)"
2827        else:
2828            msg = (
2829                f"~ direct expectation at {tag} for skipped because a"
2830                f" prior check failed"
2831            )
2832            print_message(msg, color=msg_color("skipped"))
2833            return None
2834
2835    # Figure out if we want to suppress any failure message
2836    suppress = SUPPRESS_ON_FAILURE == 'all' and CHECK_FAILED
2837
2838    short_result = ellipsis(repr(expr), 78)
2839    short_expected = ellipsis(repr(value), 78)
2840    full_result = repr(expr)
2841    full_expected = repr(value)
2842
2843    firstDiff = findFirstDifference(expr, value)
2844    if firstDiff is None:
2845        message = f"✓ {tag}"
2846        equivalent = "equivalent to"
2847        msg_cat = "succeeded"
2848        same = True
2849    else:
2850        message = f"✗ {tag}"
2851        equivalent = "NOT equivalent to"
2852        msg_cat = "failed"
2853        same = False
2854
2855    # At higher detail for success or any detail for unsuppressed
2856    # failure:
2857    if DETAIL_LEVEL >= 1 or (not same and not suppress):
2858        message += f"""
2859  Result:
2860{indent(short_result, 4)}
2861  was {equivalent} the expected value:
2862{indent(short_expected, 4)}"""
2863
2864    if (
2865        not same
2866    and not suppress
2867    and differencesAreSubtle(short_result, short_expected)
2868    ):
2869        message += f"\n  First difference was:\n{indent(firstDiff, 4)}"
2870
2871    # Report full values if detail level is turned up and the short
2872    # values were abbreviations
2873    if DETAIL_LEVEL >= 1:
2874        if short_result != full_result:
2875            message += f"\n  Full result:\n{indent(full_result, 4)}"
2876        if short_expected != full_expected:
2877            message += (
2878                f"\n  Full expected value:\n{indent(full_expected, 4)}"
2879            )
2880
2881    # Report info about the test expression
2882    base, extra = expr_details(context)
2883    if (
2884           (same and DETAIL_LEVEL >= 1)
2885        or (not same and not suppress and DETAIL_LEVEL >= 0)
2886    ):
2887        message += '\n' + indent(base, 2)
2888
2889    if DETAIL_LEVEL >= 1 and extra:
2890        message += '\n' + indent(extra, 2)
2891
2892    # Register a check failure if the expectation was not met
2893    if not same:
2894        CHECK_FAILED = True
2895
2896    # Print our message
2897    print_message(message, color=msg_color(msg_cat))
2898
2899    # Register our outcome
2900    _register_outcome(same, tag, message)
2901
2902    # Return our result
2903    return same

Establishes an immediate expectation that the values of the two arguments should be equivalent. The expression provided will be picked out of the source code of the module calling expect (see get_my_context). The expression and sub-values will be displayed if the expectation is not met, and either way a message indicating success or failure will be printed. Use detailLevel to control how detailed the messages are.

For expect to work properly, the following rules must be followed:

  1. When multiple calls to expect appear on a single line of the source code (something you should probably avoid anyway), none of the calls should execute more times than another when that line is executed (it's difficult to violate this, but examples include the use of expect multiple times on one line within generator or if/else expressions)
  2. None of the following components of the expression passed to expect should have side effects when evaluated:
    • Attribute accesses
    • Subscripts (including expressions inside brackets)
    • Variable lookups (Note that those things don't normally have side effects!)

This function returns True if the expectation is met and False otherwise. It returns None if the check is skipped, which will happen when SKIP_ON_FAILURE is 'all' and a previous check failed. If the values are not equivalent, this will count as a failed check and other checks may be skipped.

If not skipped, this function registers an outcome in ALL_OUTCOMES.

def expectType(expr, typ) 
2906def expectType(expr, typ):
2907    """
2908    Works like `expect`, but establishes an expectation for the type of
2909    the result of the expression, not for the exact value. The same
2910    rules must be followed as for `expect` for this to work properly.
2911
2912    If the type of the expression's result is an instance of the target
2913    type, the expectation counts as met.
2914
2915    If not skipped, this function registers an outcome in `ALL_OUTCOMES`.
2916    """
2917    global CHECK_FAILED
2918    context = get_my_context(expectType)
2919    tag = tag_for(context)
2920
2921    # Skip this expectation if necessary
2922    if SKIP_ON_FAILURE == 'all' and CHECK_FAILED:
2923        if DETAIL_LEVEL < 1:
2924            msg = f"~ {tag} (skipped)"
2925        else:
2926            msg = (
2927                f"~ direct expectation at {tag} for skipped because a"
2928                f" prior check failed"
2929            )
2930            print_message(msg, color=msg_color("skipped"))
2931            return None
2932
2933    suppress = SUPPRESS_ON_FAILURE == 'all' and CHECK_FAILED
2934
2935    if type(expr) == typ:
2936        message = f"✓ {tag}"
2937        desc = "the expected type"
2938        msg_cat = "succeeded"
2939        same = True
2940    elif isinstance(expr, typ):
2941        message = f"✓ {tag}"
2942        desc = f"a kind of {typ}"
2943        msg_cat = "succeeded"
2944        same = True
2945    else:
2946        message = f"✗ {tag}"
2947        desc = f"NOT a kind of {typ}"
2948        msg_cat = "failed"
2949        same = False
2950
2951    # Note failed check
2952    if not same:
2953        CHECK_FAILED = True
2954
2955    # Report on the type if the detail level warrants it, and also about
2956    # the test expression
2957    base, extra = expr_details(context)
2958    if (
2959           (same and DETAIL_LEVEL >= 1)
2960        or (not same and not suppress and DETAIL_LEVEL >= 0)
2961    ):
2962        message += f"\n  The result type ({type(expr)}) was {desc}."
2963        message += '\n' + indent(base, 2)
2964
2965    if DETAIL_LEVEL >= 1 and extra:
2966        message += '\n' + indent(extra, 2)
2967
2968    # Print our message
2969    print_message(message, color=msg_color(msg_cat))
2970
2971    # Register our outcome
2972    _register_outcome(same, tag, message)
2973
2974    # Return our result
2975    return same

Works like expect, but establishes an expectation for the type of the result of the expression, not for the exact value. The same rules must be followed as for expect for this to work properly.

If the type of the expression's result is an instance of the target type, the expectation counts as met.

If not skipped, this function registers an outcome in ALL_OUTCOMES.

class ASTMatch: 
2982class ASTMatch:
2983    """
2984    Represents a full, partial, or missing (i.e., non-) match of an
2985    `ASTRequirement` against an abstract syntax tree, ignoring
2986    sub-checks. The `isFull` and `isPartial` fields specify whether the
2987    match is a full match (values `True`, `False`), a partial match
2988    (`False`, `True`) or not a match at all (`False`, `False`).
2989    """
2990    def __init__(self, node, isPartial=None):
2991        """
2992        The matching AST node is required; use None for a non-match. If
2993        a node is given, `isPartial` will be stored to determine whether
2994        it's a partial or full match (when node is set to `None`,
2995        `isPartial` is ignored).
2996        """
2997        self.node = node
2998        if node is None:
2999            self.isFull = False
3000            self.isPartial = False
3001        else:
3002            self.isFull = not isPartial
3003            self.isPartial = isPartial
3004
3005    def __str__(self):
3006        """
3007        Represents the match using the name of the type of node matched,
3008        plus the line number of that node if available.
3009        """
3010        if self.isFull:
3011            result = "Full match: "
3012        elif self.isPartial:
3013            result = "Partial match: "
3014        else:
3015            return "No match found"
3016
3017        name = type(self.node).__name__
3018        if hasattr(self.node, "lineno") and self.node.lineno is not None:
3019            result += f"{name} on line {self.node.lineno}"
3020        else:
3021            result += f"a {name} (unknown location)"
3022
3023        return result

Represents a full, partial, or missing (i.e., non-) match of an ASTRequirement against an abstract syntax tree, ignoring sub-checks. The isFull and isPartial fields specify whether the match is a full match (values True, False), a partial match (False, True) or not a match at all (False, False).

ASTMatch(node, isPartial=None) 
2990    def __init__(self, node, isPartial=None):
2991        """
2992        The matching AST node is required; use None for a non-match. If
2993        a node is given, `isPartial` will be stored to determine whether
2994        it's a partial or full match (when node is set to `None`,
2995        `isPartial` is ignored).
2996        """
2997        self.node = node
2998        if node is None:
2999            self.isFull = False
3000            self.isPartial = False
3001        else:
3002            self.isFull = not isPartial
3003            self.isPartial = isPartial

The matching AST node is required; use None for a non-match. If a node is given, isPartial will be stored to determine whether it's a partial or full match (when node is set to None, isPartial is ignored).

class RuleMatches: 
3026class RuleMatches:
3027    """
3028    Represents how an `ASTRequirement` matches against a syntax tree,
3029    including sub-checks. It can be a full, partial, or non-match, as
3030    dictated by the `isFull` and `isPartial` variables (`True`/`False` →
3031    full match, `False`/`True` → partial match, and `False`/`False` →
3032    non-match).
3033
3034    Stores a list of tuples each containing an `ASTMatch` object for the
3035    check itself, plus a list of `RuleMatches` objects for each
3036    sub-check.
3037
3038    The number of these tuples compared to the min/max match
3039    requirements of the check this `RuleMatches` was derived from
3040    determine if it's a full, partial, or non-match.
3041    """
3042    def __init__(self, check):
3043        """
3044        The check that we're deriving this `RuleMatches` from is
3045        required. An empty structure (set up as a non-match unless the
3046        check's maxMatches or minMatches is 0 in which case it's set up
3047        as a full match) will be created which can be populated using the
3048        `addMatch` method.
3049        """
3050        self.check = check
3051        self.nFull = 0
3052        self.matchPoints = []
3053        self.final = False
3054
3055        if self.check.minMatches == 0 or self.check.maxMatches == 0:
3056            self.isFull = True
3057            self.isPartial = False
3058        else:
3059            self.isFull = False
3060            self.isPartial = False
3061
3062    def __str__(self):
3063        """
3064        Represents the matches by listing them out over multiple lines,
3065        prefaced with a description of whether the whole rule is a
3066        full/partial/non- match.
3067        """
3068        if self.isFull:
3069            category = "fully"
3070        elif self.isPartial:
3071            category = "partially"
3072        else:
3073            category = "not"
3074
3075        # Separate full & partial matches (attending to sub-matches which
3076        # the match objects themselves don't)
3077        full = []
3078        partial = []
3079        for (match, subMatches) in self.matchPoints:
3080            if match.isFull and all(sub.isFull for sub in subMatches):
3081                full.append(str(match).split(':')[-1].strip())
3082            elif match.isFull or match.isPartial:
3083                partial.append(str(match).split(':')[-1].strip())
3084
3085        return (
3086            (
3087                f"Requirement {category} satisfied via {self.nFull} full"
3088                f" and {len(self.matchPoints) - self.nFull} partial"
3089                f" match(es):\n"
3090            )
3091          + '\n'.join(
3092                indent("Full match: " + matchStr, 2)
3093                for matchStr in full
3094            )
3095          + '\n'.join(
3096                indent("Partial match: " + matchStr, 2)
3097                for matchStr in partial
3098            )
3099        )
3100
3101    def addMatch(self, nodeMatch, subMatches):
3102        """
3103        Adds a single matching AST node to this matches suite. The node
3104        at which the match occurs is required (as an `ASTMatch` object),
3105        along with a list of sub-`RuleMatches` objects for each sub-check
3106        of the check. This list is not required if the `nodeMatch` is a
3107        non-match, but in that case the entry will be ignored.
3108
3109        This object's partial/full status will be updated according to
3110        whether or not the count of full matches falls within the
3111        min/max match range after adding the specified match point. Note
3112        that a match point only counts as a full match if the
3113        `nodeMatch` is a full match and each of the `subMatches` are
3114        full matches; if the `nodeMatch` is a non-match, then it doesn't
3115        count at all, and otherwise it's a partial match.
3116
3117        Note that each of the sub-matches provided will be marked as
3118        final, and any attempts to add new matches to them will fail
3119        with a `ValueError`.
3120        """
3121        if self.final:
3122            raise ValueError(
3123                "Attempted to add to a RuleMatches suite after it was"
3124                " used as a sub-suite for another RuleMatches suite"
3125                " (you may not call addMatch after using a RuleMatches"
3126                " as a sub-suite)."
3127            )
3128
3129        # Mark each sub-match as final now that it's being used to
3130        # substantiate a super-match.
3131        for sub in subMatches:
3132            sub.final = True
3133
3134        # IF this isn't actually a match at all, ignore it
3135        if not nodeMatch.isFull and not nodeMatch.isPartial:
3136            return
3137
3138        if len(subMatches) != len(self.check.subChecks):
3139            raise ValueError(
3140                f"One sub-matches object must be supplied for each"
3141                f" sub-check of the rule ({len(self.check.subChecks)}"
3142                f" were required but you supplied {len(subMatches)})."
3143            )
3144
3145        # Add to our list of match points, which includes all full and
3146        # partial matches.
3147        self.matchPoints.append((nodeMatch, subMatches))
3148
3149        # Check if the new match is a full match
3150        if nodeMatch.isFull and all(sub.isFull for sub in subMatches):
3151            self.nFull += 1
3152
3153        # Update our full/partial status depending on the new number
3154        # of full matches
3155        if (
3156            (
3157                self.check.minMatches is None
3158             or self.check.minMatches <= self.nFull
3159            )
3160        and (
3161                self.check.maxMatches is None
3162             or self.check.maxMatches >= self.nFull
3163            )
3164        ):
3165            self.isFull = True
3166            self.isPartial = False
3167        else:
3168            self.isFull = False
3169            self.isPartial = True
3170
3171    def explanation(self):
3172        """
3173        Produces a text explanation of whether or not the associated
3174        check succeeded, and if not, why.
3175        """
3176        # TODO
3177        if self.isFull:
3178            return "check succeeded"
3179        elif self.isPartial:
3180            return "check failed (partial match(es) found)"
3181        else:
3182            return "check failed (no matches)"

Represents how an ASTRequirement matches against a syntax tree, including sub-checks. It can be a full, partial, or non-match, as dictated by the isFull and isPartial variables (True/False → full match, False/True → partial match, and False/False → non-match).

Stores a list of tuples each containing an ASTMatch object for the check itself, plus a list of RuleMatches objects for each sub-check.

The number of these tuples compared to the min/max match requirements of the check this RuleMatches was derived from determine if it's a full, partial, or non-match.

RuleMatches(check) 
3042    def __init__(self, check):
3043        """
3044        The check that we're deriving this `RuleMatches` from is
3045        required. An empty structure (set up as a non-match unless the
3046        check's maxMatches or minMatches is 0 in which case it's set up
3047        as a full match) will be created which can be populated using the
3048        `addMatch` method.
3049        """
3050        self.check = check
3051        self.nFull = 0
3052        self.matchPoints = []
3053        self.final = False
3054
3055        if self.check.minMatches == 0 or self.check.maxMatches == 0:
3056            self.isFull = True
3057            self.isPartial = False
3058        else:
3059            self.isFull = False
3060            self.isPartial = False

The check that we're deriving this RuleMatches from is required. An empty structure (set up as a non-match unless the check's maxMatches or minMatches is 0 in which case it's set up as a full match) will be created which can be populated using the addMatch method.

def addMatch(self, nodeMatch, subMatches) 
3101    def addMatch(self, nodeMatch, subMatches):
3102        """
3103        Adds a single matching AST node to this matches suite. The node
3104        at which the match occurs is required (as an `ASTMatch` object),
3105        along with a list of sub-`RuleMatches` objects for each sub-check
3106        of the check. This list is not required if the `nodeMatch` is a
3107        non-match, but in that case the entry will be ignored.
3108
3109        This object's partial/full status will be updated according to
3110        whether or not the count of full matches falls within the
3111        min/max match range after adding the specified match point. Note
3112        that a match point only counts as a full match if the
3113        `nodeMatch` is a full match and each of the `subMatches` are
3114        full matches; if the `nodeMatch` is a non-match, then it doesn't
3115        count at all, and otherwise it's a partial match.
3116
3117        Note that each of the sub-matches provided will be marked as
3118        final, and any attempts to add new matches to them will fail
3119        with a `ValueError`.
3120        """
3121        if self.final:
3122            raise ValueError(
3123                "Attempted to add to a RuleMatches suite after it was"
3124                " used as a sub-suite for another RuleMatches suite"
3125                " (you may not call addMatch after using a RuleMatches"
3126                " as a sub-suite)."
3127            )
3128
3129        # Mark each sub-match as final now that it's being used to
3130        # substantiate a super-match.
3131        for sub in subMatches:
3132            sub.final = True
3133
3134        # IF this isn't actually a match at all, ignore it
3135        if not nodeMatch.isFull and not nodeMatch.isPartial:
3136            return
3137
3138        if len(subMatches) != len(self.check.subChecks):
3139            raise ValueError(
3140                f"One sub-matches object must be supplied for each"
3141                f" sub-check of the rule ({len(self.check.subChecks)}"
3142                f" were required but you supplied {len(subMatches)})."
3143            )
3144
3145        # Add to our list of match points, which includes all full and
3146        # partial matches.
3147        self.matchPoints.append((nodeMatch, subMatches))
3148
3149        # Check if the new match is a full match
3150        if nodeMatch.isFull and all(sub.isFull for sub in subMatches):
3151            self.nFull += 1
3152
3153        # Update our full/partial status depending on the new number
3154        # of full matches
3155        if (
3156            (
3157                self.check.minMatches is None
3158             or self.check.minMatches <= self.nFull
3159            )
3160        and (
3161                self.check.maxMatches is None
3162             or self.check.maxMatches >= self.nFull
3163            )
3164        ):
3165            self.isFull = True
3166            self.isPartial = False
3167        else:
3168            self.isFull = False
3169            self.isPartial = True

Adds a single matching AST node to this matches suite. The node at which the match occurs is required (as an ASTMatch object), along with a list of sub-RuleMatches objects for each sub-check of the check. This list is not required if the nodeMatch is a non-match, but in that case the entry will be ignored.

This object's partial/full status will be updated according to whether or not the count of full matches falls within the min/max match range after adding the specified match point. Note that a match point only counts as a full match if the nodeMatch is a full match and each of the subMatches are full matches; if the nodeMatch is a non-match, then it doesn't count at all, and otherwise it's a partial match.

Note that each of the sub-matches provided will be marked as final, and any attempts to add new matches to them will fail with a ValueError.

def explanation(self) 
3171    def explanation(self):
3172        """
3173        Produces a text explanation of whether or not the associated
3174        check succeeded, and if not, why.
3175        """
3176        # TODO
3177        if self.isFull:
3178            return "check succeeded"
3179        elif self.isPartial:
3180            return "check failed (partial match(es) found)"
3181        else:
3182            return "check failed (no matches)"

Produces a text explanation of whether or not the associated check succeeded, and if not, why.

class DefaultMin: 
3188class DefaultMin:
3189    """
3190    Represents the default min value (to distinguish from an explicit
3191    value that's the same as the default).
3192    """
3193    pass

Represents the default min value (to distinguish from an explicit value that's the same as the default).

DefaultMin()
class ASTRequirement: 
3196class ASTRequirement:
3197    """
3198    Represents a specific abstract syntax tree structure to check for
3199    within a file, function, or code block (see
3200    `TestManager.checkCodeContains`). This base class is abstract, the
3201    concrete subclasses each check for specific things.
3202    """
3203    def __init__(self, *, min=DefaultMin, max=None, n=None):
3204        """
3205        Creates basic common data structures. The `min`, `max`, and `n`
3206        keyword arguments can be used to specify the number of matches
3207        required: if `n` is set, it overrides both `min` and `max`;
3208        either of those can be set to `None` to eschew an upper/lower
3209        limit. Note that a lower limit of `None` or 0 will mean that the
3210        check isn't required to match, and an upper limit of 0 will mean
3211        that the check will only succeed if the specified structure is
3212        NOT present. If `min` is greater than `max`, the check will never
3213        succeed; a warning will be issued in that case.
3214        """
3215        self.subChecks = []
3216        if min is DefaultMin:
3217            if max == 0:
3218                self.minMatches = 0
3219            else:
3220                self.minMatches = 1
3221        else:
3222            self.minMatches = min
3223
3224        self.maxMatches = max
3225        if n is not None:
3226            self.minMatches = n
3227            self.maxMatches = n
3228
3229        if min is not DefaultMin and not isinstance(min, (int, NoneType)):
3230            raise TypeError(
3231                f"min argument must be an integer or None (got: '{min}'"
3232                f" which is a/an: {type(min)}."
3233            )
3234
3235        if not isinstance(max, (int, NoneType)):
3236            raise TypeError(
3237                f"max argument must be an integer or None (got: '{max}'"
3238                f" which is a/an: {type(max)}."
3239            )
3240
3241        if not isinstance(n, (int, NoneType)):
3242            raise TypeError(
3243                f"n argument must be an integer or None (got: '{n}'"
3244                f" which is a/an: {type(n)}."
3245            )
3246
3247        if (
3248            self.minMatches is not None
3249        and self.maxMatches is not None
3250        and self.minMatches > self.maxMatches
3251        ):
3252            warnings.warn(
3253                "Min matches is larger than max matches for"
3254                " ASTRequirement; it will always fail."
3255            )
3256
3257    def structureString(self):
3258        """
3259        Returns a string expressing the structure that this check is
3260        looking for.
3261        """
3262        raise NotImplementedError(
3263            "ASTRequirement base class is abstract."
3264        )
3265
3266    def howMany(self):
3267        """
3268        Returns a string describing how many are required based on min +
3269        max match values.
3270        """
3271        # Figure out numeric descriptor from min/max
3272        if self.maxMatches is None:
3273            if self.minMatches is None:
3274                return "any number of"
3275            else:
3276                return f"at least {self.minMatches}"
3277        else:
3278            if self.maxMatches == 0:
3279                return "no"
3280            elif self.minMatches is None:
3281                return f"at most {self.maxMatches}"
3282            elif self.minMatches == self.maxMatches:
3283                return str(self.minMatches)
3284            else:
3285                return f"{self.minMatches}-{self.maxMatches}"
3286
3287    def fullStructure(self):
3288        """
3289        The structure string (see `structureString`) plus a list of what
3290        sub-checks are used to constrain contents of those matches, and
3291        text describing how many matches are required.
3292        """
3293        result = f"{self.howMany()} {self.structureString()}"
3294        if len(self.subChecks) > 0:
3295            result += " containing:\n" + '\n'.join(
3296                indent(sub.fullStructure(), 2)
3297                for sub in self.subChecks
3298            )
3299
3300        return result
3301
3302    def _nodesToCheck(self, syntaxTree):
3303        """
3304        Given a syntax tree, yields each node from that tree that should
3305        be checked for subrule matches. These are yielded in tuples
3306        where the second element is True for a full match at that node
3307        and False for a partial match. This is used by `allMatches`.
3308        """
3309        raise NotImplementedError(
3310            "ASTRequirement base class is abstract."
3311        )
3312
3313    def allMatches(self, syntaxTree):
3314        """
3315        Returns a `RuleMatches` object representing all full and partial
3316        matches of this check within the given syntax tree.
3317
3318        Only matches which happen at distinct AST nodes are considered;
3319        this does NOT list out all of the ways a match could happen (per
3320        sub-rule possibilities) for each node that might match.
3321
3322        This object will be finalized and may be used for a sub-result in
3323        another check.
3324        """
3325        result = RuleMatches(self)
3326        for (node, isFull) in self._nodesToCheck(syntaxTree):
3327            subMatchSuites = self._subRuleMatches(node)
3328            result.addMatch(ASTMatch(node, not isFull), subMatchSuites)
3329
3330        return result
3331
3332    def _walkNodesOfType(self, root, nodeTypes):
3333        """
3334        A generator that yields all nodes within the given AST (including
3335        the root node) which match the given node type (or one of the
3336        types in the given node type tuple). The nodes are yielded in
3337        (an approximation of) execution order (see `walk_ast_in_order`).
3338        """
3339        for node in walk_ast_in_order(root):
3340            if isinstance(node, nodeTypes):
3341                yield node
3342
3343    def _subRuleMatches(self, withinNode):
3344        """
3345        Returns a list of one `RuleMatches` object for each sub-check of
3346        this check. These will be finalized and can safely be added as
3347        sub-rule-matches for a entry in a `RuleMatches` suite for this
3348        node.
3349        """
3350        return [
3351            check.allMatches(withinNode)
3352            for check in self.subChecks
3353        ]
3354
3355    def contains(self, *subChecks):
3356        """
3357        Enhances this check with one or more sub-check(s) which must
3358        match (anywhere) within the contents of a basic match for the
3359        whole check to have a full match.
3360
3361        Returns self for chaining.
3362
3363        For example:
3364
3365        >>> import optimism
3366        >>> optimism.messagesAsErrors(False)
3367        >>> optimism.colors(False)
3368        >>> manager = optimism.testBlock('''\\
3369        ... def f():
3370        ...     for i in range(3):
3371        ...         print('A' * i)
3372        ... ''')
3373        >>> manager.checkCodeContains(
3374        ...     optimism.Def().contains(
3375        ...         optimism.Loop().contains(
3376        ...             optimism.Call('print')
3377        ...         )
3378        ...     )
3379        ... ) # doctest: +ELLIPSIS
3380        ✓ ...
3381        True
3382        >>> manager.checkCodeContains(
3383        ...     optimism.Def().contains(
3384        ...         optimism.Call('print')
3385        ...     )
3386        ... ) # doctest: +ELLIPSIS
3387        ✓ ...
3388        True
3389        >>> manager.checkCodeContains(
3390        ...     optimism.Loop().contains(
3391        ...         optimism.Def()
3392        ...     )
3393        ... ) # doctest: +ELLIPSIS
3394        ✗ ...
3395          Code does not contain the expected structure:
3396            at least 1 loop(s) or generator expression(s) containing:
3397              at least 1 function definition(s)
3398          Although it does partially satisfy the requirement:
3399            Requirement partially satisfied via 0 full and 1 partial match(es):
3400              Partial match: For on line 2
3401          checked code from block at ...
3402        False
3403        """
3404        self.subChecks.extend(subChecks)
3405        return self

Represents a specific abstract syntax tree structure to check for within a file, function, or code block (see TestManager.checkCodeContains). This base class is abstract, the concrete subclasses each check for specific things.

ASTRequirement(*, min=<class 'optimism.DefaultMin'>, max=None, n=None) 
3203    def __init__(self, *, min=DefaultMin, max=None, n=None):
3204        """
3205        Creates basic common data structures. The `min`, `max`, and `n`
3206        keyword arguments can be used to specify the number of matches
3207        required: if `n` is set, it overrides both `min` and `max`;
3208        either of those can be set to `None` to eschew an upper/lower
3209        limit. Note that a lower limit of `None` or 0 will mean that the
3210        check isn't required to match, and an upper limit of 0 will mean
3211        that the check will only succeed if the specified structure is
3212        NOT present. If `min` is greater than `max`, the check will never
3213        succeed; a warning will be issued in that case.
3214        """
3215        self.subChecks = []
3216        if min is DefaultMin:
3217            if max == 0:
3218                self.minMatches = 0
3219            else:
3220                self.minMatches = 1
3221        else:
3222            self.minMatches = min
3223
3224        self.maxMatches = max
3225        if n is not None:
3226            self.minMatches = n
3227            self.maxMatches = n
3228
3229        if min is not DefaultMin and not isinstance(min, (int, NoneType)):
3230            raise TypeError(
3231                f"min argument must be an integer or None (got: '{min}'"
3232                f" which is a/an: {type(min)}."
3233            )
3234
3235        if not isinstance(max, (int, NoneType)):
3236            raise TypeError(
3237                f"max argument must be an integer or None (got: '{max}'"
3238                f" which is a/an: {type(max)}."
3239            )
3240
3241        if not isinstance(n, (int, NoneType)):
3242            raise TypeError(
3243                f"n argument must be an integer or None (got: '{n}'"
3244                f" which is a/an: {type(n)}."
3245            )
3246
3247        if (
3248            self.minMatches is not None
3249        and self.maxMatches is not None
3250        and self.minMatches > self.maxMatches
3251        ):
3252            warnings.warn(
3253                "Min matches is larger than max matches for"
3254                " ASTRequirement; it will always fail."
3255            )

Creates basic common data structures. The min, max, and n keyword arguments can be used to specify the number of matches required: if n is set, it overrides both min and max; either of those can be set to None to eschew an upper/lower limit. Note that a lower limit of None or 0 will mean that the check isn't required to match, and an upper limit of 0 will mean that the check will only succeed if the specified structure is NOT present. If min is greater than max, the check will never succeed; a warning will be issued in that case.

def structureString(self) 
3257    def structureString(self):
3258        """
3259        Returns a string expressing the structure that this check is
3260        looking for.
3261        """
3262        raise NotImplementedError(
3263            "ASTRequirement base class is abstract."
3264        )

Returns a string expressing the structure that this check is looking for.

def howMany(self) 
3266    def howMany(self):
3267        """
3268        Returns a string describing how many are required based on min +
3269        max match values.
3270        """
3271        # Figure out numeric descriptor from min/max
3272        if self.maxMatches is None:
3273            if self.minMatches is None:
3274                return "any number of"
3275            else:
3276                return f"at least {self.minMatches}"
3277        else:
3278            if self.maxMatches == 0:
3279                return "no"
3280            elif self.minMatches is None:
3281                return f"at most {self.maxMatches}"
3282            elif self.minMatches == self.maxMatches:
3283                return str(self.minMatches)
3284            else:
3285                return f"{self.minMatches}-{self.maxMatches}"

Returns a string describing how many are required based on min + max match values.

def fullStructure(self) 
3287    def fullStructure(self):
3288        """
3289        The structure string (see `structureString`) plus a list of what
3290        sub-checks are used to constrain contents of those matches, and
3291        text describing how many matches are required.
3292        """
3293        result = f"{self.howMany()} {self.structureString()}"
3294        if len(self.subChecks) > 0:
3295            result += " containing:\n" + '\n'.join(
3296                indent(sub.fullStructure(), 2)
3297                for sub in self.subChecks
3298            )
3299
3300        return result

The structure string (see structureString) plus a list of what sub-checks are used to constrain contents of those matches, and text describing how many matches are required.

def allMatches(self, syntaxTree) 
3313    def allMatches(self, syntaxTree):
3314        """
3315        Returns a `RuleMatches` object representing all full and partial
3316        matches of this check within the given syntax tree.
3317
3318        Only matches which happen at distinct AST nodes are considered;
3319        this does NOT list out all of the ways a match could happen (per
3320        sub-rule possibilities) for each node that might match.
3321
3322        This object will be finalized and may be used for a sub-result in
3323        another check.
3324        """
3325        result = RuleMatches(self)
3326        for (node, isFull) in self._nodesToCheck(syntaxTree):
3327            subMatchSuites = self._subRuleMatches(node)
3328            result.addMatch(ASTMatch(node, not isFull), subMatchSuites)
3329
3330        return result

Returns a RuleMatches object representing all full and partial matches of this check within the given syntax tree.

Only matches which happen at distinct AST nodes are considered; this does NOT list out all of the ways a match could happen (per sub-rule possibilities) for each node that might match.

This object will be finalized and may be used for a sub-result in another check.

def contains(self, *subChecks) 
3355    def contains(self, *subChecks):
3356        """
3357        Enhances this check with one or more sub-check(s) which must
3358        match (anywhere) within the contents of a basic match for the
3359        whole check to have a full match.
3360
3361        Returns self for chaining.
3362
3363        For example:
3364
3365        >>> import optimism
3366        >>> optimism.messagesAsErrors(False)
3367        >>> optimism.colors(False)
3368        >>> manager = optimism.testBlock('''\\
3369        ... def f():
3370        ...     for i in range(3):
3371        ...         print('A' * i)
3372        ... ''')
3373        >>> manager.checkCodeContains(
3374        ...     optimism.Def().contains(
3375        ...         optimism.Loop().contains(
3376        ...             optimism.Call('print')
3377        ...         )
3378        ...     )
3379        ... ) # doctest: +ELLIPSIS
3380        ✓ ...
3381        True
3382        >>> manager.checkCodeContains(
3383        ...     optimism.Def().contains(
3384        ...         optimism.Call('print')
3385        ...     )
3386        ... ) # doctest: +ELLIPSIS
3387        ✓ ...
3388        True
3389        >>> manager.checkCodeContains(
3390        ...     optimism.Loop().contains(
3391        ...         optimism.Def()
3392        ...     )
3393        ... ) # doctest: +ELLIPSIS
3394        ✗ ...
3395          Code does not contain the expected structure:
3396            at least 1 loop(s) or generator expression(s) containing:
3397              at least 1 function definition(s)
3398          Although it does partially satisfy the requirement:
3399            Requirement partially satisfied via 0 full and 1 partial match(es):
3400              Partial match: For on line 2
3401          checked code from block at ...
3402        False
3403        """
3404        self.subChecks.extend(subChecks)
3405        return self

Enhances this check with one or more sub-check(s) which must match (anywhere) within the contents of a basic match for the whole check to have a full match.

Returns self for chaining.

For example:

>>> import optimism
>>> optimism.messagesAsErrors(False)
>>> optimism.colors(False)
>>> manager = optimism.testBlock('''\
... def f():
...     for i in range(3):
...         print('A' * i)
... ''')
>>> manager.checkCodeContains(
...     optimism.Def().contains(
...         optimism.Loop().contains(
...             optimism.Call('print')
...         )
...     )
... ) # doctest: +ELLIPSIS
✓ ...
True
>>> manager.checkCodeContains(
...     optimism.Def().contains(
...         optimism.Call('print')
...     )
... ) # doctest: +ELLIPSIS
✓ ...
True
>>> manager.checkCodeContains(
...     optimism.Loop().contains(
...         optimism.Def()
...     )
... ) # doctest: +ELLIPSIS
✗ ...
  Code does not contain the expected structure:
    at least 1 loop(s) or generator expression(s) containing:
      at least 1 function definition(s)
  Although it does partially satisfy the requirement:
    Requirement partially satisfied via 0 full and 1 partial match(es):
      Partial match: For on line 2
  checked code from block at ...
False
class MatchAny(ASTRequirement): 
3408class MatchAny(ASTRequirement):
3409    """
3410    A special kind of `ASTRequirement` which matches when at least one of
3411    several other checks matches. Allows testing for one of several
3412    different acceptable code structures. For example, the following code
3413    shows how to check that either `with` was used with `open`, or
3414    `try/finally` was used with `open` in the try part and `close` in the
3415    finally part (and that either way, `read` was used):
3416
3417    >>> import optimism
3418    >>> optimism.messagesAsErrors(False)
3419    >>> optimism.colors(False)
3420    >>> manager1 = optimism.testBlock('''\\
3421    ... with open('f') as fileInput:
3422    ...     print(f.read())''')
3423    ...
3424    >>> manager2 = optimism.testBlock('''\\
3425    ... fileInput = None
3426    ... try:
3427    ...     fileInput = open('f')
3428    ...     print(f.read())
3429    ... finally:
3430    ...     close(fileInput)''')
3431    ...
3432    >>> check = optimism.MatchAny(
3433    ...     optimism.With().contains(optimism.Call('open')),
3434    ...     optimism.Try()
3435    ...         .contains(optimism.Call('open'))
3436    ...         .contains(optimism.Call('close'))
3437    ...     # TODO: Implement these
3438    ...     #    .tryContains(optimism.Call('open'))
3439    ...     #    .finallyContains(optimism.call('close'))
3440    ... ).contains(optimism.Call('read', isMethod=True))
3441    ...
3442    >>> manager1.checkCodeContains(check) # doctest: +ELLIPSIS
3443    ✓ ...
3444    True
3445    >>> manager2.checkCodeContains(check) # doctest: +ELLIPSIS
3446    ✓ ...
3447    True
3448    """
3449    def __init__(
3450        self,
3451        *checkers,
3452        min=1,
3453        max=None,
3454        n=None
3455    ):
3456        """
3457        Any number of sub-checks may be supplied. Note that `contains`
3458        will be broadcast to each of these sub-checks if called on the
3459        `MatchAny` object. `min`, `max`, and/or `n` may be specified as
3460        integers to place limits on the number of matches we look for;
3461        `min` must be at least 1, and the default is 1 minimum and no
3462        maximum. The min and max arguments are ignored if a specific
3463        number of required matches is provided.
3464        """
3465        super().__init__(min=min, max=max, n=n)
3466
3467        if self.minMatches <= 0:
3468            raise ValueError(
3469                f"minMatches for a matchAny must be > 0 (got"
3470                f" {self.minMatches})"
3471            )
3472
3473        if len(checkers) == 0:
3474            warnings.warn(
3475                "A MatchAny check without any sub-checks will always"
3476                " fail."
3477            )
3478        self.subChecks = checkers
3479
3480    def structureString(self):
3481        "Lists the full structures of each alternative."
3482        if len(self.subChecks) == 0:
3483            return "zero alternatives"
3484
3485        return "the following:\n" + (
3486            '\n...or...\n'.join(
3487              indent(check.fullStructure(), 2)
3488              for check in self.subChecks
3489          )
3490        )
3491
3492    def fullStructure(self):
3493        "Lists the alternatives."
3494        if len(self.subChecks) == 0:
3495            return "A MatchAny with no alternatives (always fails)"
3496
3497        # Special case 'no' -> 'none of'
3498        n = self.howMany()
3499        if n == "no":
3500            n = "none of"
3501
3502        return f"{n} {self.structureString()}"
3503
3504    def allMatches(self, syntaxTree):
3505        """
3506        Runs each sub-check and returns a `RuleMatches` with just one
3507        `ASTMatch` entry that targets the root of the syntax tree. The
3508        `RuleMatches` sub-entires for this single match point are full
3509        `RuleMatches` for each alternative listed in this `MatchAny`,
3510        which might contain match points on nodes also matched by other
3511        `RuleMatches` of different alternatives.
3512
3513        However, the overall `isFull`/`isPartial` status of the resulting
3514        `RuleMatches` is overridden to be based on the count of distinct
3515        node positions covered by full matches of any of the
3516        alternatives. So if you set min/max on the base `MatchAny`
3517        object, it will apply to the number of node points at which any
3518        sub-rule matches. For example:
3519
3520        >>> import optimism
3521        >>> optimism.messagesAsErrors(False)
3522        >>> optimism.colors(False)
3523        >>> manager = optimism.testBlock('''\\
3524        ... def f(x):
3525        ...     x = max(x, 0)
3526        ...     if x > 5:
3527        ...         print('big')
3528        ...     elif x > 1:
3529        ...         print('medium')
3530        ...     else:
3531        ...         print('small')
3532        ...     return x
3533        ... ''')
3534        ...
3535        >>> check = optimism.MatchAny(
3536        ...    optimism.IfElse(),
3537        ...    optimism.Call('print'),
3538        ...    n=5 # 2 matches for if/else, 3 for call to print
3539        ... )
3540        ...
3541        >>> manager.checkCodeContains(check) # doctest: +ELLIPSIS
3542        ✓ ...
3543        True
3544        >>> check = optimism.MatchAny(
3545        ...    optimism.Call(),
3546        ...    optimism.Call('print'),
3547        ...    n=4 # 4 nodes that match, 3 of which overlap
3548        ... )
3549        ...
3550        >>> manager.checkCodeContains(check) # doctest: +ELLIPSIS
3551        ✓ ...
3552        True
3553        """
3554        result = RuleMatches(self)
3555
3556        if len(self.subChecks) == 0:
3557            return result  # a failure, since minMatches >= 1
3558
3559        # Create a mapping from AST nodes to True/False/None for a
3560        # full/partial/no match at that node from ANY sub-check, since we
3561        # don't want to count multiple match points on the same node. At
3562        # the same time, build a list of sub-matches for each sub-check.
3563        nodeMap = {}
3564        subMatchList = []
3565        for i, check in enumerate(self.subChecks):
3566            # Find sub-matches for this alternative
3567            subMatches = check.allMatches(syntaxTree)
3568
3569            # Record in list + note first full/partial indices
3570            subMatchList.append(subMatches)
3571
3572            # Note per-node best matches
3573            for (match, subSubMatches) in subMatches.matchPoints:
3574                fullPos = (
3575                    match.isFull
3576                and all(subSub.isFull for subSub in subSubMatches)
3577                )
3578                prev = nodeMap.get(match.node, None)
3579                if prev is None or fullPos and prev is False:
3580                    nodeMap[match.node] = fullPos
3581
3582        # We have only a single result, containing the full sub-matches
3583        # for each alternative:
3584        result.addMatch(ASTMatch(syntaxTree), subMatchList)
3585
3586        # Set 'final' on the result so nobody adds more to it
3587        result.final = True
3588
3589        # But we override the counting logic: we don't want to count the
3590        # # of places where a match occurred (there's only ever 1); and
3591        # we don't want to count the # of sub-rules that matched (that
3592        # caps out at the # of subrules, even if they match multiple
3593        # nodes). Instead we want to count the # of distinct nodes
3594        # where full matches were found across all sub-rules.
3595        result.nFull = len([k for k in nodeMap if nodeMap[k]])
3596
3597        # Override isFull/isPartial on result based on new nFull
3598        if (
3599            (
3600                result.check.minMatches is None
3601             or result.check.minMatches <= result.nFull
3602            )
3603        and (
3604                result.check.maxMatches is None
3605             or result.check.maxMatches >= result.nFull
3606            )
3607        ):
3608            result.isFull = True
3609            result.isPartial = False
3610        else:
3611            result.isFull = False
3612            if (
3613                result.nFull == 0
3614            and (
3615                    result.check.maxMatches is None
3616                 or result.check.maxMatches > 0
3617                )
3618            ):
3619                # In this case we consider it to be not a match at all,
3620                # since we found 0 match points for any alternatives and
3621                # the requirement was a positive one where max was > 0.
3622                result.isPartial = False
3623            else:
3624                result.isPartial = True
3625
3626        # And we're done
3627        return result
3628
3629    def contains(self, *subChecks):
3630        """
3631        Broadcasts the call to each sub-check. Note that this can create
3632        a sharing situation where the same `ASTRequirement` object is a
3633        sub-check of multiple other checks.
3634
3635        This function returns the `MatchAny` object for chaining.
3636        """
3637        for check in self.subChecks:
3638            check.contains(*subChecks)
3639
3640        return self

A special kind of ASTRequirement which matches when at least one of several other checks matches. Allows testing for one of several different acceptable code structures. For example, the following code shows how to check that either with was used with open, or try/finally was used with open in the try part and close in the finally part (and that either way, read was used):

>>> import optimism
>>> optimism.messagesAsErrors(False)
>>> optimism.colors(False)
>>> manager1 = optimism.testBlock('''\
... with open('f') as fileInput:
...     print(f.read())''')
...
>>> manager2 = optimism.testBlock('''\
... fileInput = None
... try:
...     fileInput = open('f')
...     print(f.read())
... finally:
...     close(fileInput)''')
...
>>> check = optimism.MatchAny(
...     optimism.With().contains(optimism.Call('open')),
...     optimism.Try()
...         .contains(optimism.Call('open'))
...         .contains(optimism.Call('close'))
...     # TODO: Implement these
...     #    .tryContains(optimism.Call('open'))
...     #    .finallyContains(optimism.call('close'))
... ).contains(optimism.Call('read', isMethod=True))
...
>>> manager1.checkCodeContains(check) # doctest: +ELLIPSIS
✓ ...
True
>>> manager2.checkCodeContains(check) # doctest: +ELLIPSIS
✓ ...
True
MatchAny(*checkers, min=1, max=None, n=None) 
3449    def __init__(
3450        self,
3451        *checkers,
3452        min=1,
3453        max=None,
3454        n=None
3455    ):
3456        """
3457        Any number of sub-checks may be supplied. Note that `contains`
3458        will be broadcast to each of these sub-checks if called on the
3459        `MatchAny` object. `min`, `max`, and/or `n` may be specified as
3460        integers to place limits on the number of matches we look for;
3461        `min` must be at least 1, and the default is 1 minimum and no
3462        maximum. The min and max arguments are ignored if a specific
3463        number of required matches is provided.
3464        """
3465        super().__init__(min=min, max=max, n=n)
3466
3467        if self.minMatches <= 0:
3468            raise ValueError(
3469                f"minMatches for a matchAny must be > 0 (got"
3470                f" {self.minMatches})"
3471            )
3472
3473        if len(checkers) == 0:
3474            warnings.warn(
3475                "A MatchAny check without any sub-checks will always"
3476                " fail."
3477            )
3478        self.subChecks = checkers

Any number of sub-checks may be supplied. Note that contains will be broadcast to each of these sub-checks if called on the MatchAny object. min, max, and/or n may be specified as integers to place limits on the number of matches we look for; min must be at least 1, and the default is 1 minimum and no maximum. The min and max arguments are ignored if a specific number of required matches is provided.

def structureString(self) 
3480    def structureString(self):
3481        "Lists the full structures of each alternative."
3482        if len(self.subChecks) == 0:
3483            return "zero alternatives"
3484
3485        return "the following:\n" + (
3486            '\n...or...\n'.join(
3487              indent(check.fullStructure(), 2)
3488              for check in self.subChecks
3489          )
3490        )

Lists the full structures of each alternative.

def fullStructure(self) 
3492    def fullStructure(self):
3493        "Lists the alternatives."
3494        if len(self.subChecks) == 0:
3495            return "A MatchAny with no alternatives (always fails)"
3496
3497        # Special case 'no' -> 'none of'
3498        n = self.howMany()
3499        if n == "no":
3500            n = "none of"
3501
3502        return f"{n} {self.structureString()}"

Lists the alternatives.

def allMatches(self, syntaxTree) 
3504    def allMatches(self, syntaxTree):
3505        """
3506        Runs each sub-check and returns a `RuleMatches` with just one
3507        `ASTMatch` entry that targets the root of the syntax tree. The
3508        `RuleMatches` sub-entires for this single match point are full
3509        `RuleMatches` for each alternative listed in this `MatchAny`,
3510        which might contain match points on nodes also matched by other
3511        `RuleMatches` of different alternatives.
3512
3513        However, the overall `isFull`/`isPartial` status of the resulting
3514        `RuleMatches` is overridden to be based on the count of distinct
3515        node positions covered by full matches of any of the
3516        alternatives. So if you set min/max on the base `MatchAny`
3517        object, it will apply to the number of node points at which any
3518        sub-rule matches. For example:
3519
3520        >>> import optimism
3521        >>> optimism.messagesAsErrors(False)
3522        >>> optimism.colors(False)
3523        >>> manager = optimism.testBlock('''\\
3524        ... def f(x):
3525        ...     x = max(x, 0)
3526        ...     if x > 5:
3527        ...         print('big')
3528        ...     elif x > 1:
3529        ...         print('medium')
3530        ...     else:
3531        ...         print('small')
3532        ...     return x
3533        ... ''')
3534        ...
3535        >>> check = optimism.MatchAny(
3536        ...    optimism.IfElse(),
3537        ...    optimism.Call('print'),
3538        ...    n=5 # 2 matches for if/else, 3 for call to print
3539        ... )
3540        ...
3541        >>> manager.checkCodeContains(check) # doctest: +ELLIPSIS
3542        ✓ ...
3543        True
3544        >>> check = optimism.MatchAny(
3545        ...    optimism.Call(),
3546        ...    optimism.Call('print'),
3547        ...    n=4 # 4 nodes that match, 3 of which overlap
3548        ... )
3549        ...
3550        >>> manager.checkCodeContains(check) # doctest: +ELLIPSIS
3551        ✓ ...
3552        True
3553        """
3554        result = RuleMatches(self)
3555
3556        if len(self.subChecks) == 0:
3557            return result  # a failure, since minMatches >= 1
3558
3559        # Create a mapping from AST nodes to True/False/None for a
3560        # full/partial/no match at that node from ANY sub-check, since we
3561        # don't want to count multiple match points on the same node. At
3562        # the same time, build a list of sub-matches for each sub-check.
3563        nodeMap = {}
3564        subMatchList = []
3565        for i, check in enumerate(self.subChecks):
3566            # Find sub-matches for this alternative
3567            subMatches = check.allMatches(syntaxTree)
3568
3569            # Record in list + note first full/partial indices
3570            subMatchList.append(subMatches)
3571
3572            # Note per-node best matches
3573            for (match, subSubMatches) in subMatches.matchPoints:
3574                fullPos = (
3575                    match.isFull
3576                and all(subSub.isFull for subSub in subSubMatches)
3577                )
3578                prev = nodeMap.get(match.node, None)
3579                if prev is None or fullPos and prev is False:
3580                    nodeMap[match.node] = fullPos
3581
3582        # We have only a single result, containing the full sub-matches
3583        # for each alternative:
3584        result.addMatch(ASTMatch(syntaxTree), subMatchList)
3585
3586        # Set 'final' on the result so nobody adds more to it
3587        result.final = True
3588
3589        # But we override the counting logic: we don't want to count the
3590        # # of places where a match occurred (there's only ever 1); and
3591        # we don't want to count the # of sub-rules that matched (that
3592        # caps out at the # of subrules, even if they match multiple
3593        # nodes). Instead we want to count the # of distinct nodes
3594        # where full matches were found across all sub-rules.
3595        result.nFull = len([k for k in nodeMap if nodeMap[k]])
3596
3597        # Override isFull/isPartial on result based on new nFull
3598        if (
3599            (
3600                result.check.minMatches is None
3601             or result.check.minMatches <= result.nFull
3602            )
3603        and (
3604                result.check.maxMatches is None
3605             or result.check.maxMatches >= result.nFull
3606            )
3607        ):
3608            result.isFull = True
3609            result.isPartial = False
3610        else:
3611            result.isFull = False
3612            if (
3613                result.nFull == 0
3614            and (
3615                    result.check.maxMatches is None
3616                 or result.check.maxMatches > 0
3617                )
3618            ):
3619                # In this case we consider it to be not a match at all,
3620                # since we found 0 match points for any alternatives and
3621                # the requirement was a positive one where max was > 0.
3622                result.isPartial = False
3623            else:
3624                result.isPartial = True
3625
3626        # And we're done
3627        return result

Runs each sub-check and returns a RuleMatches with just one ASTMatch entry that targets the root of the syntax tree. The RuleMatches sub-entires for this single match point are full RuleMatches for each alternative listed in this MatchAny, which might contain match points on nodes also matched by other RuleMatches of different alternatives.

However, the overall isFull/isPartial status of the resulting RuleMatches is overridden to be based on the count of distinct node positions covered by full matches of any of the alternatives. So if you set min/max on the base MatchAny object, it will apply to the number of node points at which any sub-rule matches. For example:

>>> import optimism
>>> optimism.messagesAsErrors(False)
>>> optimism.colors(False)
>>> manager = optimism.testBlock('''\
... def f(x):
...     x = max(x, 0)
...     if x > 5:
...         print('big')
...     elif x > 1:
...         print('medium')
...     else:
...         print('small')
...     return x
... ''')
...
>>> check = optimism.MatchAny(
...    optimism.IfElse(),
...    optimism.Call('print'),
...    n=5 # 2 matches for if/else, 3 for call to print
... )
...
>>> manager.checkCodeContains(check) # doctest: +ELLIPSIS
✓ ...
True
>>> check = optimism.MatchAny(
...    optimism.Call(),
...    optimism.Call('print'),
...    n=4 # 4 nodes that match, 3 of which overlap
... )
...
>>> manager.checkCodeContains(check) # doctest: +ELLIPSIS
✓ ...
True
def contains(self, *subChecks) 
3629    def contains(self, *subChecks):
3630        """
3631        Broadcasts the call to each sub-check. Note that this can create
3632        a sharing situation where the same `ASTRequirement` object is a
3633        sub-check of multiple other checks.
3634
3635        This function returns the `MatchAny` object for chaining.
3636        """
3637        for check in self.subChecks:
3638            check.contains(*subChecks)
3639
3640        return self

Broadcasts the call to each sub-check. Note that this can create a sharing situation where the same ASTRequirement object is a sub-check of multiple other checks.

This function returns the MatchAny object for chaining.

Inherited Members
ASTRequirement
howMany
class Import(ASTRequirement): 
3643class Import(ASTRequirement):
3644    """
3645    Checks for an `import` statement, possibly with a specific module
3646    name.
3647    """
3648    def __init__(self, moduleName=None, **kwargs):
3649        """
3650        The argument specifies the required module name; leave it as
3651        `None` (the default) to match any `import` statement.
3652        """
3653        super().__init__(**kwargs)
3654        self.name = moduleName
3655
3656    def structureString(self):
3657        if self.name is not None:
3658            return f"import(s) of {self.name}"
3659        else:
3660            return "import statement(s)"
3661
3662    def _nodesToCheck(self, syntaxTree):
3663        # Note that every import statement is a partial match
3664        for node in self._walkNodesOfType(
3665            syntaxTree,
3666            (ast.Import, ast.ImportFrom)
3667        ):
3668            if self.name is None:
3669                yield (node, True)
3670            else:
3671                if isinstance(node, ast.Import):
3672                    if any(
3673                        alias.name == self.name
3674                        for alias in node.names
3675                    ):
3676                        yield (node, True)
3677                    else:
3678                        yield (node, False)
3679                else:  # must be ImportFrom
3680                    if node.module == self.name:
3681                        yield (node, True)
3682                    else:
3683                        yield (node, False)

Checks for an import statement, possibly with a specific module name.

Import(moduleName=None, **kwargs) 
3648    def __init__(self, moduleName=None, **kwargs):
3649        """
3650        The argument specifies the required module name; leave it as
3651        `None` (the default) to match any `import` statement.
3652        """
3653        super().__init__(**kwargs)
3654        self.name = moduleName

The argument specifies the required module name; leave it as None (the default) to match any import statement.

def structureString(self) 
3656    def structureString(self):
3657        if self.name is not None:
3658            return f"import(s) of {self.name}"
3659        else:
3660            return "import statement(s)"

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
class Def(ASTRequirement): 
3686class Def(ASTRequirement):
3687    """
3688    Matches a function definition, possibly with a specific name and/or
3689    number of arguments.
3690    """
3691    def __init__(
3692        self,
3693        name=None,
3694        minArgs=0,
3695        maxArgs=None,
3696        nArgs=None,
3697        **kwargs
3698    ):
3699        """
3700        The first argument specifies the function name. Leave it as
3701        `None` (the default) to allow any function definition to match.
3702
3703        The `minArgs`, `maxArgs`, and `nArgs` arguments specify the
3704        number of arguments the function must accept. Min and max are
3705        ignored if `nArgs` is specified; min or max can be None to eschew
3706        an upper or lower limit. Default is any number of arguments.
3707
3708        A warning is issued if `minArgs` > `maxArgs`.
3709        """
3710        super().__init__(**kwargs)
3711        self.name = name
3712        self.minArgs = minArgs
3713        self.maxArgs = maxArgs
3714        if nArgs is not None:
3715            self.minArgs = nArgs
3716            self.maxArgs = nArgs
3717
3718        if (
3719            self.minArgs is not None
3720        and self.maxArgs is not None
3721        and self.minArgs > self.maxArgs
3722        ):
3723            warnings.warn(
3724                "A def node with minArgs > maxArgs cannot match."
3725            )
3726
3727    def structureString(self):
3728        if self.name is not None:
3729            result = f"definition(s) of {self.name}"
3730        else:
3731            result = "function definition(s)"
3732        if self.minArgs is not None and self.minArgs > 0:
3733            if self.maxArgs is None:
3734                result += f" (with at least {self.minArgs} arguments)"
3735            elif self.maxArgs == self.minArgs:
3736                result += f" (with {self.minArgs} arguments)"
3737            else:
3738                result += f" (with {self.minArgs}-{self.maxArgs} arguments)"
3739        elif self.maxArgs is not None:
3740            result += " (with at most {self.maxArgs} arguments)"
3741        # otherwise no parenthetical is necessary
3742        return result
3743
3744    def _nodesToCheck(self, syntaxTree):
3745        # Note that every def is considered a partial match, but
3746        # definitions whose name matches and whose arguments don't are
3747        # yielded before those whose names don't match.
3748        later = []
3749        for node in self._walkNodesOfType(
3750            syntaxTree,
3751            (ast.FunctionDef, ast.AsyncFunctionDef)
3752        ):
3753            nameMatch = self.name is None or node.name == self.name
3754            nArgs = (
3755                (
3756                    len(node.args.posonlyargs)
3757                    if hasattr(node.args, "posonlyargs")
3758                    else 0
3759                )
3760              + len(node.args.args)
3761              + len(node.args.kwonlyargs)
3762              + (1 if node.args.vararg is not None else 0)
3763              + (1 if node.args.kwarg is not None else 0)
3764            )
3765            argsMatch = (
3766                (self.minArgs is None or self.minArgs <= nArgs)
3767            and (self.maxArgs is None or self.maxArgs >= nArgs)
3768            )
3769            if nameMatch and argsMatch:
3770                yield (node, True)
3771            elif nameMatch:
3772                yield (node, False)
3773            else:
3774                # Order non-name-matched nodes last
3775                later.append(node)
3776
3777        for node in later:
3778            yield (node, False)

Matches a function definition, possibly with a specific name and/or number of arguments.

Def(name=None, minArgs=0, maxArgs=None, nArgs=None, **kwargs) 
3691    def __init__(
3692        self,
3693        name=None,
3694        minArgs=0,
3695        maxArgs=None,
3696        nArgs=None,
3697        **kwargs
3698    ):
3699        """
3700        The first argument specifies the function name. Leave it as
3701        `None` (the default) to allow any function definition to match.
3702
3703        The `minArgs`, `maxArgs`, and `nArgs` arguments specify the
3704        number of arguments the function must accept. Min and max are
3705        ignored if `nArgs` is specified; min or max can be None to eschew
3706        an upper or lower limit. Default is any number of arguments.
3707
3708        A warning is issued if `minArgs` > `maxArgs`.
3709        """
3710        super().__init__(**kwargs)
3711        self.name = name
3712        self.minArgs = minArgs
3713        self.maxArgs = maxArgs
3714        if nArgs is not None:
3715            self.minArgs = nArgs
3716            self.maxArgs = nArgs
3717
3718        if (
3719            self.minArgs is not None
3720        and self.maxArgs is not None
3721        and self.minArgs > self.maxArgs
3722        ):
3723            warnings.warn(
3724                "A def node with minArgs > maxArgs cannot match."
3725            )

The first argument specifies the function name. Leave it as None (the default) to allow any function definition to match.

The minArgs, maxArgs, and nArgs arguments specify the number of arguments the function must accept. Min and max are ignored if nArgs is specified; min or max can be None to eschew an upper or lower limit. Default is any number of arguments.

A warning is issued if minArgs > maxArgs.

def structureString(self) 
3727    def structureString(self):
3728        if self.name is not None:
3729            result = f"definition(s) of {self.name}"
3730        else:
3731            result = "function definition(s)"
3732        if self.minArgs is not None and self.minArgs > 0:
3733            if self.maxArgs is None:
3734                result += f" (with at least {self.minArgs} arguments)"
3735            elif self.maxArgs == self.minArgs:
3736                result += f" (with {self.minArgs} arguments)"
3737            else:
3738                result += f" (with {self.minArgs}-{self.maxArgs} arguments)"
3739        elif self.maxArgs is not None:
3740            result += " (with at most {self.maxArgs} arguments)"
3741        # otherwise no parenthetical is necessary
3742        return result

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
class Call(ASTRequirement): 
3781class Call(ASTRequirement):
3782    """
3783    Matches a function call, possibly with a specific name, and possibly
3784    restricted to only method calls or only non-method calls.
3785    """
3786    def __init__(
3787        self,
3788        name=None,
3789        isMethod=None,
3790        **kwargs
3791    ):
3792        """
3793        The first argument specifies the function name. Leave it as
3794        `None` (the default) to allow any function call to match.
3795
3796        The second argument `isMethod` specifies whether the call must be
3797        a method call, not a method call, or may be either. Note that any
3798        call to an attribute of an object is counted as a "method" call,
3799        including calls that use explicit module names, since it's not
3800        possible to know without running the code whether the attribute's
3801        object is a class or something else. Set this to `True` to
3802        match only method calls, `False` to match only non-method calls,
3803        and any other value (like the default `None`) to match either.
3804
3805        TODO: Support restrictions on arguments used?
3806        """
3807        super().__init__(**kwargs)
3808        self.name = name
3809        self.isMethod = isMethod
3810
3811    def structureString(self):
3812        if self.name is not None:
3813            if self.isMethod is True:
3814                result = f"call(s) to ?.{self.name}"
3815            else:
3816                result = f"call(s) to {self.name}"
3817        else:
3818            if self.isMethod is True:
3819                result = "method call(s)"
3820            elif self.isMethod is False:
3821                result = "non-method function call(s)"
3822            else:
3823                result = "function call(s)"
3824
3825        return result
3826
3827    def _nodesToCheck(self, syntaxTree):
3828        # Note that are calls whose name doesn't match are not considered
3829        # matches at all, while calls which are/aren't methods are still
3830        # considered partial matches even when isMethod indicates they
3831        # should be the opposite. Also note that only calls whose
3832        # function expression is either a Name or an Attribute will match
3833        # if isMethod is specified (one way or the other) or name is not
3834        # None. Things like lambdas, if/else expression results, or
3835        # subscripts won't match because they don't really have a name,
3836        # and they're not really specifically methods or not methods.
3837
3838        # If no specific requirements are present, then we can simply
3839        # yield all of the Call nodes
3840        if (
3841            self.isMethod is not True
3842        and self.isMethod is not False
3843        and self.name is None
3844        ):
3845            for node in self._walkNodesOfType(syntaxTree, ast.Call):
3846                yield (node, True)
3847        else:
3848            # Otherwise only call nodes w/ Name or Attribute expressions
3849            # can match
3850            for node in self._walkNodesOfType(syntaxTree, ast.Call):
3851                # Figure out the name and/or method status of the thing being
3852                # called:
3853                funcExpr = node.func
3854
3855                # Unwrap any := assignments to get at the actual function
3856                # object being used
3857                if HAS_WALRUS:
3858                    while isinstance(funcExpr, ast.NamedExpr):
3859                        funcExpr = funcExpr.value
3860
3861                # Handle name vs. attr nodes
3862                if isinstance(funcExpr, ast.Name):
3863                    name = funcExpr.id
3864                    method = False
3865                elif isinstance(funcExpr, ast.Attribute):
3866                    name = funcExpr.attr
3867                    method = True
3868                else:
3869                    # Only Name and Attribute nodes can actually be checked
3870                    # for details, so other matches are ignored
3871                    continue
3872
3873                if self.name is None or self.name == name:
3874                    # "is not not" is actually correct here...
3875                    yield (node, self.isMethod is not (not method))

Matches a function call, possibly with a specific name, and possibly restricted to only method calls or only non-method calls.

Call(name=None, isMethod=None, **kwargs) 
3786    def __init__(
3787        self,
3788        name=None,
3789        isMethod=None,
3790        **kwargs
3791    ):
3792        """
3793        The first argument specifies the function name. Leave it as
3794        `None` (the default) to allow any function call to match.
3795
3796        The second argument `isMethod` specifies whether the call must be
3797        a method call, not a method call, or may be either. Note that any
3798        call to an attribute of an object is counted as a "method" call,
3799        including calls that use explicit module names, since it's not
3800        possible to know without running the code whether the attribute's
3801        object is a class or something else. Set this to `True` to
3802        match only method calls, `False` to match only non-method calls,
3803        and any other value (like the default `None`) to match either.
3804
3805        TODO: Support restrictions on arguments used?
3806        """
3807        super().__init__(**kwargs)
3808        self.name = name
3809        self.isMethod = isMethod

The first argument specifies the function name. Leave it as None (the default) to allow any function call to match.

The second argument isMethod specifies whether the call must be a method call, not a method call, or may be either. Note that any call to an attribute of an object is counted as a "method" call, including calls that use explicit module names, since it's not possible to know without running the code whether the attribute's object is a class or something else. Set this to True to match only method calls, False to match only non-method calls, and any other value (like the default None) to match either.

TODO: Support restrictions on arguments used?

def structureString(self) 
3811    def structureString(self):
3812        if self.name is not None:
3813            if self.isMethod is True:
3814                result = f"call(s) to ?.{self.name}"
3815            else:
3816                result = f"call(s) to {self.name}"
3817        else:
3818            if self.isMethod is True:
3819                result = "method call(s)"
3820            elif self.isMethod is False:
3821                result = "non-method function call(s)"
3822            else:
3823                result = "function call(s)"
3824
3825        return result

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
def anyNameMatches(nameToMatch, targetsList) 
3878def anyNameMatches(nameToMatch, targetsList):
3879    """
3880    Recursive function for matching assigned names within target
3881    tuple/list AST structures.
3882    """
3883    for target in targetsList:
3884        if isinstance(target, ast.Name) and target.id == nameToMatch:
3885            return True
3886        elif isinstance(target, (ast.List, ast.Tuple)):
3887            if anyNameMatches(nameToMatch, target.elts):
3888                return True
3889        # Any other kind of node is ignored
3890
3891    return False

Recursive function for matching assigned names within target tuple/list AST structures.

class Assign(ASTRequirement): 
3894class Assign(ASTRequirement):
3895    """
3896    Matches an assignment, possibly to a variable with a specific name.
3897    By default augmented assignments and assignments via named
3898    expressions are allowed, but these may be disallowed or required.
3899    Assignments of disallowed types are still counted as partial matches
3900    if their name matches or if no name was specified.
3901
3902    Assignments to things other than variables (like list slots) will not
3903    match when a variable name is specified.
3904
3905    Note that the entire assignment node is matched, so you can use
3906    `contains` to specify checks to apply to the expression (plus the
3907    target, but usually that's fine).
3908
3909    In cases where a tuple assignment is made, if any of the assigned
3910    names matches the required name, the entire tuple assignment is
3911    considered a match, since it may not be possible to pick apart the
3912    right-hand side to find a syntactic node that was assigned to just
3913    that variable. This can lead to some weird matches, for example,
3914
3915    >>> import optimism
3916    >>> optimism.messagesAsErrors(False)
3917    >>> optimism.colors(False)
3918    >>> tester = optimism.testBlock("x, (y, z) = 1, (3, 5)")
3919    >>> tester.checkCodeContains(
3920    ...     optimism.Assign('x').contains(optimism.Constant(5))
3921    ... ) # doctest: +ELLIPSIS
3922    ✓ ...
3923    True
3924    """
3925    def __init__(
3926        self,
3927        name=None,
3928        isAugmented=None,
3929        isNamedExpr=None,
3930        **kwargs
3931    ):
3932        """
3933        The first argument specifies the variable name. Leave it as
3934        `None` (the default) to allow any assignment to match.
3935
3936        `isAugmented` specifies whether augmented assignments (e.g., +=)
3937        are considered matches or not; `False` disallows them, `True`
3938        will only match them, and any other value (like the default
3939        `None`) will allow them and other assignment types.
3940
3941        `isNamedExpr` works the same way for controlling whether named
3942        expressions (:=) are permitted. A `ValueError` will be raised if
3943        both `isAugmented` and `isNamedExpr` are set to true, since named
3944        expressions can't be augmented.
3945
3946        TODO: Allow checking for assignment to fields?
3947        """
3948        if isAugmented is True and isNamedExpr is True:
3949            raise ValueError(
3950                "Both isAugmented and isNamedExpr cannot be set to True"
3951                " at once, since no assignments would match in that"
3952                " case."
3953            )
3954
3955        super().__init__(**kwargs)
3956        self.name = name
3957        self.isAugmented = isAugmented
3958        self.isNamedExpr = isNamedExpr
3959
3960    def structureString(self):
3961        if self.name is None:
3962            if self.isAugmented is True:
3963                result = "augmented assignment statement(s)"
3964            elif self.isNamedExpr is True:
3965                result = "assignment(s) via named expression(s)"
3966            else:
3967                result = "assignment(s)"
3968        else:
3969            if self.isAugmented is True:
3970                result = f"augmented assignment(s) to {self.name}"
3971            elif self.isNamedExpr is True:
3972                result = f"named assignment(s) to {self.name}"
3973            else:
3974                result = f"assignment(s) to {self.name}"
3975
3976        if self.isAugmented is False:
3977            result += " (not augmented)"
3978
3979        if self.isNamedExpr is False:
3980            result += " (not via named expression(s))"
3981
3982        return result
3983
3984    def _nodesToCheck(self, syntaxTree):
3985        # Consider all Assign, AugAssign, AnnAssign, and NamedExpr nodes
3986        matchTypes = (ast.Assign, ast.AugAssign, ast.AnnAssign)
3987        if HAS_WALRUS:
3988            matchTypes += (ast.NamedExpr,)
3989        for node in self._walkNodesOfType(
3990            syntaxTree,
3991            matchTypes
3992        ):
3993            # Figure out the name and/or method status of the thing being
3994            # called:
3995            if self.name is None:
3996                nameMatch = True
3997            else:
3998                if isinstance(node, ast.Assign):
3999                    nameMatch = anyNameMatches(self.name, node.targets)
4000                else:
4001                    nameExpr = node.target
4002                    nameMatch = (
4003                        isinstance(nameExpr, ast.Name)
4004                    and nameExpr.id == self.name
4005                    )
4006
4007            augmented = isinstance(node, ast.AugAssign)
4008            namedExpr = HAS_WALRUS and isinstance(node, ast.NamedExpr)
4009
4010            if (
4011                nameMatch
4012            and self.isAugmented is not (not augmented)
4013            and self.isNamedExpr is not (not namedExpr)
4014            ):
4015                yield (node, True)
4016            elif nameMatch:
4017                yield (node, False)

Matches an assignment, possibly to a variable with a specific name. By default augmented assignments and assignments via named expressions are allowed, but these may be disallowed or required. Assignments of disallowed types are still counted as partial matches if their name matches or if no name was specified.

Assignments to things other than variables (like list slots) will not match when a variable name is specified.

Note that the entire assignment node is matched, so you can use contains to specify checks to apply to the expression (plus the target, but usually that's fine).

In cases where a tuple assignment is made, if any of the assigned names matches the required name, the entire tuple assignment is considered a match, since it may not be possible to pick apart the right-hand side to find a syntactic node that was assigned to just that variable. This can lead to some weird matches, for example,

>>> import optimism
>>> optimism.messagesAsErrors(False)
>>> optimism.colors(False)
>>> tester = optimism.testBlock("x, (y, z) = 1, (3, 5)")
>>> tester.checkCodeContains(
...     optimism.Assign('x').contains(optimism.Constant(5))
... ) # doctest: +ELLIPSIS
✓ ...
True
Assign(name=None, isAugmented=None, isNamedExpr=None, **kwargs) 
3925    def __init__(
3926        self,
3927        name=None,
3928        isAugmented=None,
3929        isNamedExpr=None,
3930        **kwargs
3931    ):
3932        """
3933        The first argument specifies the variable name. Leave it as
3934        `None` (the default) to allow any assignment to match.
3935
3936        `isAugmented` specifies whether augmented assignments (e.g., +=)
3937        are considered matches or not; `False` disallows them, `True`
3938        will only match them, and any other value (like the default
3939        `None`) will allow them and other assignment types.
3940
3941        `isNamedExpr` works the same way for controlling whether named
3942        expressions (:=) are permitted. A `ValueError` will be raised if
3943        both `isAugmented` and `isNamedExpr` are set to true, since named
3944        expressions can't be augmented.
3945
3946        TODO: Allow checking for assignment to fields?
3947        """
3948        if isAugmented is True and isNamedExpr is True:
3949            raise ValueError(
3950                "Both isAugmented and isNamedExpr cannot be set to True"
3951                " at once, since no assignments would match in that"
3952                " case."
3953            )
3954
3955        super().__init__(**kwargs)
3956        self.name = name
3957        self.isAugmented = isAugmented
3958        self.isNamedExpr = isNamedExpr

The first argument specifies the variable name. Leave it as None (the default) to allow any assignment to match.

isAugmented specifies whether augmented assignments (e.g., +=) are considered matches or not; False disallows them, True will only match them, and any other value (like the default None) will allow them and other assignment types.

isNamedExpr works the same way for controlling whether named expressions (:=) are permitted. A ValueError will be raised if both isAugmented and isNamedExpr are set to true, since named expressions can't be augmented.

TODO: Allow checking for assignment to fields?

def structureString(self) 
3960    def structureString(self):
3961        if self.name is None:
3962            if self.isAugmented is True:
3963                result = "augmented assignment statement(s)"
3964            elif self.isNamedExpr is True:
3965                result = "assignment(s) via named expression(s)"
3966            else:
3967                result = "assignment(s)"
3968        else:
3969            if self.isAugmented is True:
3970                result = f"augmented assignment(s) to {self.name}"
3971            elif self.isNamedExpr is True:
3972                result = f"named assignment(s) to {self.name}"
3973            else:
3974                result = f"assignment(s) to {self.name}"
3975
3976        if self.isAugmented is False:
3977            result += " (not augmented)"
3978
3979        if self.isNamedExpr is False:
3980            result += " (not via named expression(s))"
3981
3982        return result

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
class Reference(ASTRequirement): 
4020class Reference(ASTRequirement):
4021    """
4022    Matches a variable reference, possibly to a variable with a specific
4023    name. By default attribute accesses with the given name will also be
4024    matched (e.g., both 'pi' and 'math.pi' will match for the name 'pi').
4025    You may specify that only attributes should match or that attributes
4026    should not match; matches that violate that specification will still
4027    be partial matches.
4028
4029    >>> import optimism
4030    >>> optimism.messagesAsErrors(False)
4031    >>> optimism.colors(False)
4032    >>> tester = optimism.testBlock("x = 5\\ny = x * math.pi")
4033    >>> tester.checkCodeContains(
4034    ...     optimism.Reference('x')
4035    ... ) # doctest: +ELLIPSIS
4036    ✓ ...
4037    True
4038    >>> tester.checkCodeContains(
4039    ...     optimism.Reference('y')
4040    ... ) # doctest: +ELLIPSIS
4041    ✗ ...
4042    False
4043    >>> tester.checkCodeContains(
4044    ...     optimism.Reference('pi')
4045    ... ) # doctest: +ELLIPSIS
4046    ✓ ...
4047    True
4048    >>> tester.checkCodeContains(
4049    ...     optimism.Reference('x', attribute=True)
4050    ... ) # doctest: +ELLIPSIS
4051    ✗ ...
4052    False
4053    >>> tester.checkCodeContains(
4054    ...     optimism.Reference('pi', attribute=True)
4055    ... ) # doctest: +ELLIPSIS
4056    ✓ ...
4057    True
4058    >>> tester.checkCodeContains(
4059    ...     optimism.Reference('pi', attribute=False)
4060    ... ) # doctest: +ELLIPSIS
4061    ✗ ...
4062    False
4063    """
4064    def __init__(
4065        self,
4066        name=None,
4067        attribute=None,
4068        **kwargs
4069    ):
4070        """
4071        The first argument specifies the variable name. Leave it as
4072        `None` (the default) to allow any assignment to match.
4073
4074        The second argument specifies whether the reference must be to
4075        an attribute with that name (if `True`), or to a regular
4076        variable with that name (if `False`). Leave it as the default
4077        `None` to allow matches for either.
4078        """
4079        super().__init__(**kwargs)
4080        self.name = name
4081        self.attribute = attribute
4082
4083    def structureString(self):
4084        if self.name is None:
4085            if self.attribute is True:
4086                result = "attribute reference(s)"
4087            elif self.attribute is False:
4088                result = "non-attribute variable reference(s)"
4089            else:
4090                result = "variable reference(s)"
4091        else:
4092            if self.attribute is True:
4093                result = f"reference(s) to .{self.name}"
4094            else:
4095                result = f"reference(s) to {self.name}"
4096
4097        return result
4098
4099    def _nodesToCheck(self, syntaxTree):
4100        # Consider all Name and Attribute nodes
4101        for node in self._walkNodesOfType(
4102            syntaxTree,
4103            (ast.Name, ast.Attribute)
4104        ):
4105            # Only match references being loaded (use `Assign` for
4106            # variables being assigned).
4107            if not isinstance(node.ctx, ast.Load):
4108                continue
4109
4110            # Figure out whether the name matches:
4111            if self.name is None:
4112                nameMatch = True
4113            else:
4114                if isinstance(node, ast.Name):
4115                    nameMatch = node.id == self.name
4116                else:  # must be en Attribute
4117                    nameMatch = node.attr == self.name
4118
4119            if self.attribute is None:
4120                typeMatch = True
4121            else:
4122                if self.attribute is True:
4123                    typeMatch = isinstance(node, ast.Attribute)
4124                elif self.attribute is False:
4125                    typeMatch = isinstance(node, ast.Name)
4126
4127            if nameMatch and typeMatch:
4128                yield (node, True)
4129            elif nameMatch:
4130                yield (node, False)

Matches a variable reference, possibly to a variable with a specific name. By default attribute accesses with the given name will also be matched (e.g., both 'pi' and 'math.pi' will match for the name 'pi'). You may specify that only attributes should match or that attributes should not match; matches that violate that specification will still be partial matches.

>>> import optimism
>>> optimism.messagesAsErrors(False)
>>> optimism.colors(False)
>>> tester = optimism.testBlock("x = 5\ny = x * math.pi")
>>> tester.checkCodeContains(
...     optimism.Reference('x')
... ) # doctest: +ELLIPSIS
✓ ...
True
>>> tester.checkCodeContains(
...     optimism.Reference('y')
... ) # doctest: +ELLIPSIS
✗ ...
False
>>> tester.checkCodeContains(
...     optimism.Reference('pi')
... ) # doctest: +ELLIPSIS
✓ ...
True
>>> tester.checkCodeContains(
...     optimism.Reference('x', attribute=True)
... ) # doctest: +ELLIPSIS
✗ ...
False
>>> tester.checkCodeContains(
...     optimism.Reference('pi', attribute=True)
... ) # doctest: +ELLIPSIS
✓ ...
True
>>> tester.checkCodeContains(
...     optimism.Reference('pi', attribute=False)
... ) # doctest: +ELLIPSIS
✗ ...
False
Reference(name=None, attribute=None, **kwargs) 
4064    def __init__(
4065        self,
4066        name=None,
4067        attribute=None,
4068        **kwargs
4069    ):
4070        """
4071        The first argument specifies the variable name. Leave it as
4072        `None` (the default) to allow any assignment to match.
4073
4074        The second argument specifies whether the reference must be to
4075        an attribute with that name (if `True`), or to a regular
4076        variable with that name (if `False`). Leave it as the default
4077        `None` to allow matches for either.
4078        """
4079        super().__init__(**kwargs)
4080        self.name = name
4081        self.attribute = attribute

The first argument specifies the variable name. Leave it as None (the default) to allow any assignment to match.

The second argument specifies whether the reference must be to an attribute with that name (if True), or to a regular variable with that name (if False). Leave it as the default None to allow matches for either.

def structureString(self) 
4083    def structureString(self):
4084        if self.name is None:
4085            if self.attribute is True:
4086                result = "attribute reference(s)"
4087            elif self.attribute is False:
4088                result = "non-attribute variable reference(s)"
4089            else:
4090                result = "variable reference(s)"
4091        else:
4092            if self.attribute is True:
4093                result = f"reference(s) to .{self.name}"
4094            else:
4095                result = f"reference(s) to {self.name}"
4096
4097        return result

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
class Class(ASTRequirement): 
4133class Class(ASTRequirement):
4134    """
4135    Matches a class definition, possibly with a specific name.
4136    """
4137    def __init__(self, name=None, **kwargs):
4138        """
4139        A name my be specified; `None` (the default) will match any class
4140        definition.
4141        """
4142        super().__init__(**kwargs)
4143        self.name = name
4144
4145    def structureString(self):
4146        if self.name is not None:
4147            return f"class definition(s) for {self.name}"
4148        else:
4149            return "class definition(s)"
4150
4151    def _nodesToCheck(self, syntaxTree):
4152        # Consider just ClassDef nodes; all such nodes are considered as
4153        # least partial matches.
4154        for node in self._walkNodesOfType(syntaxTree, ast.ClassDef):
4155            yield (node, self.name is None or node.name == self.name)

Matches a class definition, possibly with a specific name.

Class(name=None, **kwargs) 
4137    def __init__(self, name=None, **kwargs):
4138        """
4139        A name my be specified; `None` (the default) will match any class
4140        definition.
4141        """
4142        super().__init__(**kwargs)
4143        self.name = name

A name my be specified; None (the default) will match any class definition.

def structureString(self) 
4145    def structureString(self):
4146        if self.name is not None:
4147            return f"class definition(s) for {self.name}"
4148        else:
4149            return "class definition(s)"

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
class IfElse(ASTRequirement): 
4158class IfElse(ASTRequirement):
4159    """
4160    Matches a single if or elif, possibly with an else attached. In an
4161    if/elif/else construction, it will match on the initial if plus on
4162    each elif, since Python treats them as nested if/else nodes. Also
4163    matches if/else expression nodes, although this can be disabled or
4164    required.
4165    """
4166    def __init__(self, onlyExpr=None, **kwargs):
4167        """
4168        Set `onlyExpr` to `False` to avoid matching if/else expression
4169        nodes; set it to `True` to only match those nodes; set it to
4170        anything else to match both normal and expression if/else.
4171        """
4172        super().__init__(**kwargs)
4173        self.onlyExpr = onlyExpr
4174
4175    def structureString(self):
4176        if self.onlyExpr is True:
4177            return "if/else expression(s)"
4178        elif self.onlyExpr is False:
4179            return "if/else statement(s)"
4180        else:
4181            return "if/else statement(s) or expression(s)"
4182
4183    def _nodesToCheck(self, syntaxTree):
4184        # Consider If and IfExp nodes
4185        for node in self._walkNodesOfType(syntaxTree, (ast.If, ast.IfExp)):
4186            if self.onlyExpr is False:
4187                full = isinstance(node, ast.If)
4188            elif self.onlyExpr is True:
4189                full = isinstance(node, ast.IfExp)
4190            else:
4191                full = True
4192
4193            yield (node, full)

Matches a single if or elif, possibly with an else attached. In an if/elif/else construction, it will match on the initial if plus on each elif, since Python treats them as nested if/else nodes. Also matches if/else expression nodes, although this can be disabled or required.

IfElse(onlyExpr=None, **kwargs) 
4166    def __init__(self, onlyExpr=None, **kwargs):
4167        """
4168        Set `onlyExpr` to `False` to avoid matching if/else expression
4169        nodes; set it to `True` to only match those nodes; set it to
4170        anything else to match both normal and expression if/else.
4171        """
4172        super().__init__(**kwargs)
4173        self.onlyExpr = onlyExpr

Set onlyExpr to False to avoid matching if/else expression nodes; set it to True to only match those nodes; set it to anything else to match both normal and expression if/else.

def structureString(self) 
4175    def structureString(self):
4176        if self.onlyExpr is True:
4177            return "if/else expression(s)"
4178        elif self.onlyExpr is False:
4179            return "if/else statement(s)"
4180        else:
4181            return "if/else statement(s) or expression(s)"

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
class Loop(ASTRequirement): 
4196class Loop(ASTRequirement):
4197    """
4198    Matches for and while loops, asynchronous versions of those loops,
4199    and also generator expressions and list/dict/set comprehensions. Can
4200    be restricted to match only some of those things, although all of
4201    them are always considered at least partial matches.
4202    """
4203    def __init__(self, only=None, **kwargs):
4204        """
4205        The `only` argument can be used to narrow what is matched, it
4206        should be a single string, or a set (or some other iterable) of
4207        strings, from the following list:
4208
4209            - "for" - for loops
4210            - "async for" - asynchronous for loops
4211            - "while" - while loops
4212            - "generator" - generator expressions (NOT in comprehensions)
4213            - "list comp" - list comprehensions
4214            - "dict comp" - dictionary comprehensions
4215            - "set comp" - set comprehensions
4216
4217        A few extra strings can be used as shortcuts for groups from the
4218        list above:
4219
4220            - "any generator" - generator expressions and list/dict/set
4221                comprehensions
4222            - "non-generator" - any non-generator non-comprehension
4223            - "non-async" - any kind except async for
4224
4225        A `ValueError` will be raised if an empty `only` set is provided;
4226        leave it as `None` (the default) to allow any kind of looping
4227        construct to match. A `ValueError` will also be raised if the
4228        `only` set contains any strings not listed above.
4229        """
4230        super().__init__(**kwargs)
4231
4232        if only is not None:
4233            if isinstance(only, str):
4234                only = { only }
4235            else:
4236                only = set(only)
4237
4238            if "any generator" in only:
4239                only.add("generator")
4240                only.add("list comp")
4241                only.add("dict comp")
4242                only.add("set comp")
4243                only.remove("any generator")
4244
4245            if "non-generator" in only:
4246                only.add("for")
4247                only.add("async for")
4248                only.add("while")
4249                only.remove("non-generator")
4250
4251            if "non-async" in only:
4252                only.add("for")
4253                only.add("while")
4254                only.add("generator")
4255                only.add("list comp")
4256                only.add("dict comp")
4257                only.add("set comp")
4258                only.remove("non-async")
4259
4260        self.only = only
4261
4262        if only is not None:
4263            invalid = only - {
4264                "for", "async for", "while", "generator", "list comp",
4265                "dict comp", "set comp"
4266            }
4267            if len(invalid) > 0:
4268                raise ValueError(
4269                    f"One or more invalid loop types was specified for"
4270                    f" 'only': {invalid}"
4271                )
4272
4273            if len(only) == 0:
4274                raise ValueError(
4275                    "At least one type of loop must be specified when"
4276                    " 'only' is used (leave it as None to allow all loop"
4277                    " types."
4278                )
4279
4280    def structureString(self):
4281        if self.only is None:
4282            return "loop(s) or generator expression(s)"
4283        elif self.only == {"for"} or self.only == {"for", "async for"}:
4284            return "for loop(s)"
4285        elif self.only == {"async for"}:
4286            return "async for loop(s)"
4287        elif self.only == {"while"}:
4288            return "while loop(s)"
4289        elif self.only == {"generator"}:
4290            return "generator expression(s)"
4291        elif self.only == {"list comp"}:
4292            return "list comprehension(s)"
4293        elif self.only == {"dict comp"}:
4294            return "dictionary comprehension(s)"
4295        elif self.only == {"set comp"}:
4296            return "set comprehension(s)"
4297        elif len(
4298            self.only - {"for", "async for", "while"}
4299        ) == 0:
4300            return "generator expression(s) or comprehension(s)"
4301        elif len(
4302            self.only - {"generator", "list comp", "dict comp", "set comp"}
4303        ) == 0:
4304            return (
4305                "for or while loop(s) (not generator expression(s) or"
4306                " comprehension(s))"
4307            )
4308
4309    def _nodesToCheck(self, syntaxTree):
4310        allIterationTypes = (
4311            ast.For,
4312            ast.AsyncFor,
4313            ast.While,
4314            ast.GeneratorExp,
4315            ast.ListComp,
4316            ast.DictComp,
4317            ast.SetComp
4318        )
4319        if self.only is not None:
4320            allowed = tuple([
4321                {
4322                    "for": ast.For,
4323                    "async for": ast.AsyncFor,
4324                    "while": ast.While,
4325                    "generator": ast.GeneratorExp,
4326                    "list comp": ast.ListComp,
4327                    "dict comp": ast.DictComp,
4328                    "set comp": ast.SetComp,
4329                }[item]
4330                for item in self.only
4331            ])
4332
4333        for node in self._walkNodesOfType(syntaxTree, allIterationTypes):
4334            if self.only is None or isinstance(node, allowed):
4335                yield (node, True)
4336            else:
4337                # If only some types are required, other types still
4338                # count as partial matches
4339                yield (node, False)

Matches for and while loops, asynchronous versions of those loops, and also generator expressions and list/dict/set comprehensions. Can be restricted to match only some of those things, although all of them are always considered at least partial matches.

Loop(only=None, **kwargs) 
4203    def __init__(self, only=None, **kwargs):
4204        """
4205        The `only` argument can be used to narrow what is matched, it
4206        should be a single string, or a set (or some other iterable) of
4207        strings, from the following list:
4208
4209            - "for" - for loops
4210            - "async for" - asynchronous for loops
4211            - "while" - while loops
4212            - "generator" - generator expressions (NOT in comprehensions)
4213            - "list comp" - list comprehensions
4214            - "dict comp" - dictionary comprehensions
4215            - "set comp" - set comprehensions
4216
4217        A few extra strings can be used as shortcuts for groups from the
4218        list above:
4219
4220            - "any generator" - generator expressions and list/dict/set
4221                comprehensions
4222            - "non-generator" - any non-generator non-comprehension
4223            - "non-async" - any kind except async for
4224
4225        A `ValueError` will be raised if an empty `only` set is provided;
4226        leave it as `None` (the default) to allow any kind of looping
4227        construct to match. A `ValueError` will also be raised if the
4228        `only` set contains any strings not listed above.
4229        """
4230        super().__init__(**kwargs)
4231
4232        if only is not None:
4233            if isinstance(only, str):
4234                only = { only }
4235            else:
4236                only = set(only)
4237
4238            if "any generator" in only:
4239                only.add("generator")
4240                only.add("list comp")
4241                only.add("dict comp")
4242                only.add("set comp")
4243                only.remove("any generator")
4244
4245            if "non-generator" in only:
4246                only.add("for")
4247                only.add("async for")
4248                only.add("while")
4249                only.remove("non-generator")
4250
4251            if "non-async" in only:
4252                only.add("for")
4253                only.add("while")
4254                only.add("generator")
4255                only.add("list comp")
4256                only.add("dict comp")
4257                only.add("set comp")
4258                only.remove("non-async")
4259
4260        self.only = only
4261
4262        if only is not None:
4263            invalid = only - {
4264                "for", "async for", "while", "generator", "list comp",
4265                "dict comp", "set comp"
4266            }
4267            if len(invalid) > 0:
4268                raise ValueError(
4269                    f"One or more invalid loop types was specified for"
4270                    f" 'only': {invalid}"
4271                )
4272
4273            if len(only) == 0:
4274                raise ValueError(
4275                    "At least one type of loop must be specified when"
4276                    " 'only' is used (leave it as None to allow all loop"
4277                    " types."
4278                )

The only argument can be used to narrow what is matched, it should be a single string, or a set (or some other iterable) of strings, from the following list:

- "for" - for loops
- "async for" - asynchronous for loops
- "while" - while loops
- "generator" - generator expressions (NOT in comprehensions)
- "list comp" - list comprehensions
- "dict comp" - dictionary comprehensions
- "set comp" - set comprehensions

A few extra strings can be used as shortcuts for groups from the list above:

- "any generator" - generator expressions and list/dict/set
    comprehensions
- "non-generator" - any non-generator non-comprehension
- "non-async" - any kind except async for

A ValueError will be raised if an empty only set is provided; leave it as None (the default) to allow any kind of looping construct to match. A ValueError will also be raised if the only set contains any strings not listed above.

def structureString(self) 
4280    def structureString(self):
4281        if self.only is None:
4282            return "loop(s) or generator expression(s)"
4283        elif self.only == {"for"} or self.only == {"for", "async for"}:
4284            return "for loop(s)"
4285        elif self.only == {"async for"}:
4286            return "async for loop(s)"
4287        elif self.only == {"while"}:
4288            return "while loop(s)"
4289        elif self.only == {"generator"}:
4290            return "generator expression(s)"
4291        elif self.only == {"list comp"}:
4292            return "list comprehension(s)"
4293        elif self.only == {"dict comp"}:
4294            return "dictionary comprehension(s)"
4295        elif self.only == {"set comp"}:
4296            return "set comprehension(s)"
4297        elif len(
4298            self.only - {"for", "async for", "while"}
4299        ) == 0:
4300            return "generator expression(s) or comprehension(s)"
4301        elif len(
4302            self.only - {"generator", "list comp", "dict comp", "set comp"}
4303        ) == 0:
4304            return (
4305                "for or while loop(s) (not generator expression(s) or"
4306                " comprehension(s))"
4307            )

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
class Return(ASTRequirement): 
4342class Return(ASTRequirement):
4343    """
4344    Matches a return statement. An expression may be required or
4345    forbidden, but by default returns with or without expressions count.
4346    """
4347    def __init__(self, requireExpr=None, **kwargs):
4348        """
4349        `requireExpr` controls whether a return expression is
4350        allowed/required. Set to `True` to require one, or `False` to
4351        forbid one, and any other value (such as the default `None`) to
4352        match returns with or without an expression.
4353        """
4354        super().__init__(**kwargs)
4355        self.requireExpr = requireExpr
4356
4357    def structureString(self):
4358        if self.requireExpr is False:
4359            return "return statement(s) (without expression(s))"
4360        else:
4361            return "return statement(s)"
4362
4363    def _nodesToCheck(self, syntaxTree):
4364        for node in self._walkNodesOfType(syntaxTree, ast.Return):
4365            if self.requireExpr is True:
4366                full = node.value is not None
4367            elif self.requireExpr is False:
4368                full = node.value is None
4369            else:
4370                full = True
4371
4372            yield (node, full)

Matches a return statement. An expression may be required or forbidden, but by default returns with or without expressions count.

Return(requireExpr=None, **kwargs) 
4347    def __init__(self, requireExpr=None, **kwargs):
4348        """
4349        `requireExpr` controls whether a return expression is
4350        allowed/required. Set to `True` to require one, or `False` to
4351        forbid one, and any other value (such as the default `None`) to
4352        match returns with or without an expression.
4353        """
4354        super().__init__(**kwargs)
4355        self.requireExpr = requireExpr

requireExpr controls whether a return expression is allowed/required. Set to True to require one, or False to forbid one, and any other value (such as the default None) to match returns with or without an expression.

def structureString(self) 
4357    def structureString(self):
4358        if self.requireExpr is False:
4359            return "return statement(s) (without expression(s))"
4360        else:
4361            return "return statement(s)"

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
class Try(ASTRequirement): 
4375class Try(ASTRequirement):
4376    """
4377    Matches try/except/finally nodes. The presence of except, finally,
4378    and/or else clauses may be required or forbidden, although all
4379    try/except/finally nodes are counted as at least partial matches.
4380    """
4381    def __init__(
4382        self,
4383        requireExcept=None,
4384        requireFinally=None,
4385        requireElse=None,
4386        **kwargs
4387    ):
4388        """
4389        `requireExcept`, `requireFinally`, and `requireElse` are used to
4390        specify whether those blocks must be present, must not be
4391        present, or are neither required nor forbidden. Use `False` for
4392        to forbid matches with that block and `True` to only match
4393        constructs with that block. Any other value (like the default
4394        `None` will ignore the presence or absence of that block. A
4395        `ValueError` will be raised if both `requireExcept` and
4396        `requireFinally` are set to `False`, as a `try` block must have
4397        at least one or the other to be syntactically valid. Similarly,
4398        if `requireElse` is set to `True`, `requireExcept` must not be
4399        `False` (and syntactically, `else` can only be used when `except`
4400        is present).
4401        """
4402        super().__init__(**kwargs)
4403        if requireExcept is False and requireFinally is False:
4404            raise ValueError(
4405                "Cannot require that neither 'except' nor 'finally' is"
4406                " present on a 'try' statement, as one or the other will"
4407                " always be present."
4408            )
4409
4410        if requireElse is True and requireExcept is False:
4411            raise ValueError(
4412                "Cannot require that 'else' be present on a 'try'"
4413                " statement while also requiring that 'except' not be"
4414                " present, since 'else' cannot be used without 'except'."
4415            )
4416
4417        self.requireExcept = requireExcept
4418        self.requireFinally = requireFinally
4419        self.requireElse = requireElse
4420
4421    def structureString(self):
4422        result = "try statement(s)"
4423        if self.requireExcept is not False:
4424            result += " (with except block(s))"
4425        if self.requireElse is True:
4426            result += " (with else block(s))"
4427        if self.requireFinally is True:
4428            result += " (with finally block(s))"
4429        return result
4430
4431    def _nodesToCheck(self, syntaxTree):
4432        # All try/except/finally statements count as matches, but ones
4433        # missing required clauses or which have forbidden clauses count
4434        # as partial matches.
4435        for node in self._walkNodesOfType(syntaxTree, ast.Try):
4436            full = True
4437            if self.requireExcept is True and len(node.handlers) == 0:
4438                full = False
4439            if self.requireExcept is False and len(node.handlers) > 0:
4440                full = False
4441            if self.requireElse is True and len(node.orelse) == 0:
4442                full = False
4443            if self.requireElse is False and len(node.orelse) > 0:
4444                full = False
4445            if self.requireFinally is True and len(node.finalbody) == 0:
4446                full = False
4447            if self.requireFinally is False and len(node.finalbody) > 0:
4448                full = False
4449
4450            yield (node, full)

Matches try/except/finally nodes. The presence of except, finally, and/or else clauses may be required or forbidden, although all try/except/finally nodes are counted as at least partial matches.

Try(requireExcept=None, requireFinally=None, requireElse=None, **kwargs) 
4381    def __init__(
4382        self,
4383        requireExcept=None,
4384        requireFinally=None,
4385        requireElse=None,
4386        **kwargs
4387    ):
4388        """
4389        `requireExcept`, `requireFinally`, and `requireElse` are used to
4390        specify whether those blocks must be present, must not be
4391        present, or are neither required nor forbidden. Use `False` for
4392        to forbid matches with that block and `True` to only match
4393        constructs with that block. Any other value (like the default
4394        `None` will ignore the presence or absence of that block. A
4395        `ValueError` will be raised if both `requireExcept` and
4396        `requireFinally` are set to `False`, as a `try` block must have
4397        at least one or the other to be syntactically valid. Similarly,
4398        if `requireElse` is set to `True`, `requireExcept` must not be
4399        `False` (and syntactically, `else` can only be used when `except`
4400        is present).
4401        """
4402        super().__init__(**kwargs)
4403        if requireExcept is False and requireFinally is False:
4404            raise ValueError(
4405                "Cannot require that neither 'except' nor 'finally' is"
4406                " present on a 'try' statement, as one or the other will"
4407                " always be present."
4408            )
4409
4410        if requireElse is True and requireExcept is False:
4411            raise ValueError(
4412                "Cannot require that 'else' be present on a 'try'"
4413                " statement while also requiring that 'except' not be"
4414                " present, since 'else' cannot be used without 'except'."
4415            )
4416
4417        self.requireExcept = requireExcept
4418        self.requireFinally = requireFinally
4419        self.requireElse = requireElse

requireExcept, requireFinally, and requireElse are used to specify whether those blocks must be present, must not be present, or are neither required nor forbidden. Use False for to forbid matches with that block and True to only match constructs with that block. Any other value (like the default None will ignore the presence or absence of that block. A ValueError will be raised if both requireExcept and requireFinally are set to False, as a try block must have at least one or the other to be syntactically valid. Similarly, if requireElse is set to True, requireExcept must not be False (and syntactically, else can only be used when except is present).

def structureString(self) 
4421    def structureString(self):
4422        result = "try statement(s)"
4423        if self.requireExcept is not False:
4424            result += " (with except block(s))"
4425        if self.requireElse is True:
4426            result += " (with else block(s))"
4427        if self.requireFinally is True:
4428            result += " (with finally block(s))"
4429        return result

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
class With(ASTRequirement): 
4453class With(ASTRequirement):
4454    """
4455    Matches a `with` or `async with` block. Async may be required or
4456    forbidden, although either form will always be considered at least a
4457    partial match.
4458    """
4459    def __init__(self, onlyAsync=None, **kwargs):
4460        """
4461        `onlyAsync` should be set to `False` to disallow `async with`
4462        blocks, `True` to match only async blocks, and any other value
4463        (like the default `None`) to match both normal and async blocks.
4464        """
4465        super().__init__(**kwargs)
4466        self.onlyAsync = onlyAsync
4467
4468    def structureString(self):
4469        if self.onlyAsync is True:
4470            return "async with statement(s)"
4471        else:
4472            return "with statement(s)"
4473
4474    def _nodesToCheck(self, syntaxTree):
4475        for node in self._walkNodesOfType(
4476            syntaxTree,
4477            (ast.With, ast.AsyncWith)
4478        ):
4479            yield (
4480                node,
4481                self.onlyAsync is not (not isinstance(node, ast.AsyncWith))
4482                # 'not not' is intentional here
4483            )

Matches a with or async with block. Async may be required or forbidden, although either form will always be considered at least a partial match.

With(onlyAsync=None, **kwargs) 
4459    def __init__(self, onlyAsync=None, **kwargs):
4460        """
4461        `onlyAsync` should be set to `False` to disallow `async with`
4462        blocks, `True` to match only async blocks, and any other value
4463        (like the default `None`) to match both normal and async blocks.
4464        """
4465        super().__init__(**kwargs)
4466        self.onlyAsync = onlyAsync

onlyAsync should be set to False to disallow async with blocks, True to match only async blocks, and any other value (like the default None) to match both normal and async blocks.

def structureString(self) 
4468    def structureString(self):
4469        if self.onlyAsync is True:
4470            return "async with statement(s)"
4471        else:
4472            return "with statement(s)"

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
class AnyValue: 
4486class AnyValue:
4487    """
4488    Represents the situation where any value can be accepted for a
4489    node in a `Constant` or `Literal` `ASTRequirement`. Also used to
4490    represent a `getLiteralValue` where we don't know the value.
4491    """
4492    pass

Represents the situation where any value can be accepted for a node in a Constant or Literal ASTRequirement. Also used to represent a getLiteralValue where we don't know the value.

AnyValue()
class AnyType: 
4495class AnyType:
4496    """
4497    Represents the situation where any type can be accepted for a
4498    node in a `Constant` or `Literal` `ASTRequirement`.
4499    """

Represents the situation where any type can be accepted for a node in a Constant or Literal ASTRequirement.

AnyType()
class Constant(ASTRequirement): 
4502class Constant(ASTRequirement):
4503    """
4504    A check for a constant, possibly with a specific value and/or of a
4505    specific type. All constants are considered partial matches.
4506
4507    Note that this cannot match literal tuples, lists, sets,
4508    dictionaries, etc.; only simple constants. Use `Literal` instead for
4509    literal lists, tuples, sets, or dictionaries.
4510    """
4511    def __init__(self, value=AnyValue, types=AnyType, **kwargs):
4512        """
4513        A specific value may be supplied (including `None`) or else any
4514        value will be accepted if the `AnyValue` class (not an instance
4515        of it) is used as the argument (this is the default).
4516
4517        If the value is `AnyValue`, `types` may be specified, and only
4518        constants with that type will match. `type` may be a tuple (but
4519        not list) of types or a single type, as with `isinstance`.
4520
4521        Even if a specific value is specified, the type check is still
4522        applied, since it's possible to create a value that checks equal
4523        to values from more than one type. For example, specifying
4524        `Constant(6)` will match both 6 and 6.0 but `Constant(6, float)`
4525        will only match the latter.
4526        """
4527        super().__init__(**kwargs)
4528        self.value = value
4529        self.types = types
4530
4531        # Allowed types for constants (ignoring doc which claims tuples
4532        # or frozensets can be Constant values)
4533        allowed = (int, float, complex, bool, NoneType, str)
4534
4535        # value-type and type-type checking
4536        if value is not AnyValue and type(value) not in allowed:
4537            raise TypeError(
4538                f"Value {value!r} has type {type(value)} which is not a"
4539                f" type that a Constant can be (did you mean to use a"
4540                f" Literal instead?)."
4541            )
4542
4543        if self.types is not AnyType:
4544            if isinstance(self.types, tuple):
4545                for typ in self.types:
4546                    if typ not in allowed:
4547                        raise TypeError(
4548                            f"Type {typ} has is not a type that a"
4549                            f" Constant can be (did you mean to use a"
4550                            f" Literal instead?)."
4551                        )
4552            else:
4553                if self.types not in allowed:
4554                    raise TypeError(
4555                        f"Type {self.types} has is not a type that a"
4556                        f" Constant can be (did you mean to use a"
4557                        f" Literal instead?)."
4558                    )
4559
4560    def structureString(self):
4561        if self.value == AnyValue:
4562            if self.types == AnyType:
4563                return "constant(s)"
4564            else:
4565                if isinstance(self.types, tuple):
4566                    types = (
4567                        ', '.join(t.__name__ for t in self.types[:-1])
4568                      + ' or ' + self.types[-1].__name__
4569                    )
4570                    return f"{types} constant(s)"
4571                else:
4572                    return f"{self.types.__name__} constant(s)"
4573        else:
4574            return f"constant {repr(self.value)}"
4575
4576    def _nodesToCheck(self, syntaxTree):
4577        # ALL Constants w/ values/types other than what was expected are
4578        # considered partial matches.
4579        if SPLIT_CONSTANTS:
4580            for node in self._walkNodesOfType(
4581                syntaxTree,
4582                (ast.Num, ast.Str, ast.Bytes, ast.NameConstant, ast.Constant)
4583            ):
4584                if isinstance(node, ast.Num):
4585                    val = node.n
4586                elif isinstance(node, (ast.Str, ast.Bytes)):
4587                    val = node.s
4588                elif isinstance(node, (ast.NameConstant, ast.Constant)):
4589                    val = node.value
4590
4591                valMatch = (
4592                    self.value == AnyValue
4593                 or val == self.value
4594                )
4595
4596                typeMatch = (
4597                    self.types == AnyType
4598                 or isinstance(val, self.types)
4599                )
4600
4601                yield (node, valMatch and typeMatch)
4602        else:
4603            for node in self._walkNodesOfType(syntaxTree, ast.Constant):
4604                valMatch = (
4605                    self.value == AnyValue
4606                 or node.value == self.value
4607                )
4608
4609                typeMatch = (
4610                    self.types == AnyType
4611                 or isinstance(node.value, self.types)
4612                )
4613
4614                yield (node, valMatch and typeMatch)

A check for a constant, possibly with a specific value and/or of a specific type. All constants are considered partial matches.

Note that this cannot match literal tuples, lists, sets, dictionaries, etc.; only simple constants. Use Literal instead for literal lists, tuples, sets, or dictionaries.

Constant( value=<class 'optimism.AnyValue'>, types=<class 'optimism.AnyType'>, **kwargs) 
4511    def __init__(self, value=AnyValue, types=AnyType, **kwargs):
4512        """
4513        A specific value may be supplied (including `None`) or else any
4514        value will be accepted if the `AnyValue` class (not an instance
4515        of it) is used as the argument (this is the default).
4516
4517        If the value is `AnyValue`, `types` may be specified, and only
4518        constants with that type will match. `type` may be a tuple (but
4519        not list) of types or a single type, as with `isinstance`.
4520
4521        Even if a specific value is specified, the type check is still
4522        applied, since it's possible to create a value that checks equal
4523        to values from more than one type. For example, specifying
4524        `Constant(6)` will match both 6 and 6.0 but `Constant(6, float)`
4525        will only match the latter.
4526        """
4527        super().__init__(**kwargs)
4528        self.value = value
4529        self.types = types
4530
4531        # Allowed types for constants (ignoring doc which claims tuples
4532        # or frozensets can be Constant values)
4533        allowed = (int, float, complex, bool, NoneType, str)
4534
4535        # value-type and type-type checking
4536        if value is not AnyValue and type(value) not in allowed:
4537            raise TypeError(
4538                f"Value {value!r} has type {type(value)} which is not a"
4539                f" type that a Constant can be (did you mean to use a"
4540                f" Literal instead?)."
4541            )
4542
4543        if self.types is not AnyType:
4544            if isinstance(self.types, tuple):
4545                for typ in self.types:
4546                    if typ not in allowed:
4547                        raise TypeError(
4548                            f"Type {typ} has is not a type that a"
4549                            f" Constant can be (did you mean to use a"
4550                            f" Literal instead?)."
4551                        )
4552            else:
4553                if self.types not in allowed:
4554                    raise TypeError(
4555                        f"Type {self.types} has is not a type that a"
4556                        f" Constant can be (did you mean to use a"
4557                        f" Literal instead?)."
4558                    )

A specific value may be supplied (including None) or else any value will be accepted if the AnyValue class (not an instance of it) is used as the argument (this is the default).

If the value is AnyValue, types may be specified, and only constants with that type will match. type may be a tuple (but not list) of types or a single type, as with isinstance.

Even if a specific value is specified, the type check is still applied, since it's possible to create a value that checks equal to values from more than one type. For example, specifying Constant(6) will match both 6 and 6.0 but Constant(6, float) will only match the latter.

def structureString(self) 
4560    def structureString(self):
4561        if self.value == AnyValue:
4562            if self.types == AnyType:
4563                return "constant(s)"
4564            else:
4565                if isinstance(self.types, tuple):
4566                    types = (
4567                        ', '.join(t.__name__ for t in self.types[:-1])
4568                      + ' or ' + self.types[-1].__name__
4569                    )
4570                    return f"{types} constant(s)"
4571                else:
4572                    return f"{self.types.__name__} constant(s)"
4573        else:
4574            return f"constant {repr(self.value)}"

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
def getLiteralValue(astNode) 
4617def getLiteralValue(astNode):
4618    """
4619    For an AST node that's entirely made up of `Constant` and/or
4620    `Literal` nodes, extracts the value of that node from the AST. For
4621    nodes which have things like variable references in them whose
4622    values are not determined by the AST alone, returns `AnyValue` (the
4623    class itself, not an instance).
4624
4625    Examples:
4626
4627    >>> node = ast.parse('[1, 2, 3]').body[0].value
4628    >>> type(node).__name__
4629    'List'
4630    >>> getLiteralValue(node)
4631    [1, 2, 3]
4632    >>> node = ast.parse("('string', 4, {5: (6, 7)})").body[0].value
4633    >>> getLiteralValue(node)
4634    ('string', 4, {5: (6, 7)})
4635    >>> node = ast.parse("(variable, 4, {5: (6, 7)})").body[0].value
4636    >>> getLiteralValue(node) # can't determine value from AST
4637    <class 'optimism.AnyValue'>
4638    >>> node = ast.parse("[x for x in range(3)]").body[0].value
4639    >>> getLiteralValue(node) # not a literal or constant
4640    <class 'optimism.AnyValue'>
4641    >>> node = ast.parse("[1, 2, 3][0]").body[0].value
4642    >>> getLiteralValue(node) # not a literal or constant
4643    <class 'optimism.AnyValue'>
4644    >>> getLiteralValue(node.value) # the value part is though
4645    [1, 2, 3]
4646    """
4647    # Handle constant node types depending on SPLIT_CONSTANTS
4648    if SPLIT_CONSTANTS:
4649        if isinstance(astNode, ast.Num):
4650            return astNode.n
4651        elif isinstance(astNode, (ast.Str, ast.Bytes)):
4652            return astNode.s
4653        elif isinstance(astNode, (ast.NameConstant, ast.Constant)):
4654            return astNode.value
4655        # Else check literal types below
4656    else:
4657        if isinstance(astNode, ast.Constant):
4658            return astNode.value
4659        # Else check literal types below
4660
4661    if isinstance(astNode, (ast.List, ast.Tuple, ast.Set)):
4662        result = []
4663        for elem in astNode.elts:
4664            subValue = getLiteralValue(elem)
4665            if subValue is AnyValue:
4666                return AnyValue
4667            result.append(subValue)
4668        return {
4669            ast.List: list,
4670            ast.Tuple: tuple,
4671            ast.Set: set
4672        }[type(astNode)](result)
4673
4674    elif isinstance(astNode, ast.Dict):
4675        result = {}
4676        for index in range(len(astNode.keys)):
4677            kv = getLiteralValue(astNode.keys[index])
4678            vv = getLiteralValue(astNode.values[index])
4679            if kv is AnyValue or vv is AnyValue:
4680                return AnyValue
4681            result[kv] = vv
4682        return result
4683
4684    else:
4685        return AnyValue

For an AST node that's entirely made up of Constant and/or Literal nodes, extracts the value of that node from the AST. For nodes which have things like variable references in them whose values are not determined by the AST alone, returns AnyValue (the class itself, not an instance).

Examples:

>>> node = ast.parse('[1, 2, 3]').body[0].value
>>> type(node).__name__
'List'
>>> getLiteralValue(node)
[1, 2, 3]
>>> node = ast.parse("('string', 4, {5: (6, 7)})").body[0].value
>>> getLiteralValue(node)
('string', 4, {5: (6, 7)})
>>> node = ast.parse("(variable, 4, {5: (6, 7)})").body[0].value
>>> getLiteralValue(node) # can't determine value from AST
<class 'optimism.AnyValue'>
>>> node = ast.parse("[x for x in range(3)]").body[0].value
>>> getLiteralValue(node) # not a literal or constant
<class 'optimism.AnyValue'>
>>> node = ast.parse("[1, 2, 3][0]").body[0].value
>>> getLiteralValue(node) # not a literal or constant
<class 'optimism.AnyValue'>
>>> getLiteralValue(node.value) # the value part is though
[1, 2, 3]
class Literal(ASTRequirement): 
4688class Literal(ASTRequirement):
4689    """
4690    A check for a complex literal possibly with a specific value and/or
4691    of a specific type. All literals of the appropriate type(s) are
4692    considered partial matches even when a specific value is supplied,
4693    and list/tuple literals are both considered together for these
4694    partial matches.
4695
4696    Note that this cannot match string, number, or other constants, use
4697    `Constant` for that.
4698    """
4699    def __init__(self, value=AnyValue, types=AnyType, **kwargs):
4700        """
4701        A specific value may be supplied (it must be a list, tuple, set,
4702        or dictionary) or else any value will be accepted if the
4703        `AnyValue` class (not an instance of it) is used as the argument
4704        (that is the default).
4705
4706        If the value is `AnyValue`, one or more `types` may be
4707        specified, and only literals with that type will match. `types`
4708        may be a tuple (but not list) of types or a single type, as with
4709        `isinstance`. Matched nodes will always have a value which is one
4710        of the following types: `list`, `tuple`, `set`, or `dict`.
4711
4712        If both a specific value and a type or tuple of types is
4713        specified, any collection whose members match the members of the
4714        specific value supplied and whose type is one of the listed types
4715        will match. For example, `Literal([1, 2], types=(list, tuple,
4716        set))` will match any of `[1, 2]`, `(1, 2)`, or `{2, 1}` but will
4717        NOT match `[2, 1]`, `(2, 1)`, or any dictionary.
4718
4719        Specifically, the value is converted to match the type of the
4720        node being considered and then a match is checked, so for
4721        example, `Literal([1, 2, 2], types=set)` will match the set `{1,
4722        2}` and the equivalent sets `{2, 1}` and `{1, 1, 2}`.
4723
4724        If a node has elements which aren't constants or literals, it
4725        will never match when a value is provided because we don't
4726        evaluate code during matching. It might still match if only
4727        type(s) are provided, of course.
4728        """
4729        super().__init__(**kwargs)
4730        self.value = value
4731        self.types = types
4732
4733        # Allowed types for literals
4734        allowed = (list, tuple, set, dict)
4735
4736        # value-type and type-type checking
4737        if value is not AnyValue and type(value) not in allowed:
4738            raise TypeError(
4739                f"Value {value!r} has type {type(value)} which is not a"
4740                f" type that a Literal can be (did you mean to use a"
4741                f" Constant instead?)."
4742            )
4743
4744        if self.types is not AnyType:
4745            if isinstance(self.types, tuple):
4746                for typ in self.types:
4747                    if typ not in allowed:
4748                        raise TypeError(
4749                            f"Type {typ} has is not a type that a"
4750                            f" Literal can be (did you mean to use a"
4751                            f" Constant instead?)."
4752                        )
4753            else:
4754                if self.types not in allowed:
4755                    raise TypeError(
4756                        f"Type {self.types} has is not a type that a"
4757                        f" Literal can be (did you mean to use a"
4758                        f" Constant instead?)."
4759                    )
4760
4761    def structureString(self):
4762        if self.value == AnyValue:
4763            if self.types == AnyType:
4764                return "literal(s)"
4765            else:
4766                if isinstance(self.types, tuple):
4767                    types = (
4768                        ', '.join(t.__name__ for t in self.types[:-1])
4769                      + ' or ' + self.types[-1].__name__
4770                    )
4771                    return f"{types} literal(s)"
4772                else:
4773                    return f"{self.types.__name__} literal(s)"
4774        else:
4775            return f"literal {repr(self.value)}"
4776
4777    def _nodesToCheck(self, syntaxTree):
4778        # Some literals might be considered partial matches
4779        for node in self._walkNodesOfType(
4780            syntaxTree,
4781            (ast.List, ast.Tuple, ast.Set, ast.Dict)
4782        ):
4783            # First, get the value of the node. This will be None if
4784            # it's not computable from the AST alone.
4785            value = getLiteralValue(node)
4786
4787            valType = type(value)
4788            if value is None:
4789                valType = {
4790                    ast.List: list,
4791                    ast.Tuple: tuple,
4792                    ast.Set: set,
4793                    ast.Dict: dict,
4794                }[type(node)]
4795
4796            # Next, determine whether we have something that counts as a
4797            # partial match, and if we don't, continue to the next
4798            # potential match.
4799            partial = False
4800            partialTypes = self.types
4801            if partialTypes is AnyType:
4802                if self.value is not AnyValue:
4803                    partialTypes = (type(self.value),)
4804                else:
4805                    partial = True
4806
4807            # Only keep checking if we aren't already sure it's a
4808            # partial match
4809            if not partial:
4810                if not isinstance(partialTypes, tuple):
4811                    partialTypes = (partialTypes,)
4812
4813                # List and tuple imply each other for partials
4814                if list in partialTypes and tuple not in partialTypes:
4815                    partialTypes = partialTypes + (tuple,)
4816                if tuple in partialTypes and list not in partialTypes:
4817                    partialTypes = partialTypes + (list,)
4818
4819                partial = issubclass(valType, partialTypes)
4820
4821            # Skip this match entirely if it doesn't qualify as at least
4822            # a partial match.
4823            if not partial:
4824                continue
4825
4826            # Now check for a value match
4827            if self.value is AnyValue:
4828                valMatch = True
4829            elif value is None:
4830                valMatch = False
4831            elif self.types is AnyType:
4832                valMatch = value == self.value
4833            else:
4834                check = self.types
4835                if not isinstance(check, tuple):
4836                    check = (check,)
4837
4838                valMatch = (
4839                    isinstance(value, check)
4840                and type(value)(self.value) == value
4841                )
4842
4843            typeMatch = (
4844                self.types == AnyType
4845             or issubclass(valType, self.types)
4846            )
4847
4848            # Won't get here unless it's a partial match
4849            yield (node, valMatch and typeMatch)

A check for a complex literal possibly with a specific value and/or of a specific type. All literals of the appropriate type(s) are considered partial matches even when a specific value is supplied, and list/tuple literals are both considered together for these partial matches.

Note that this cannot match string, number, or other constants, use Constant for that.

Literal( value=<class 'optimism.AnyValue'>, types=<class 'optimism.AnyType'>, **kwargs) 
4699    def __init__(self, value=AnyValue, types=AnyType, **kwargs):
4700        """
4701        A specific value may be supplied (it must be a list, tuple, set,
4702        or dictionary) or else any value will be accepted if the
4703        `AnyValue` class (not an instance of it) is used as the argument
4704        (that is the default).
4705
4706        If the value is `AnyValue`, one or more `types` may be
4707        specified, and only literals with that type will match. `types`
4708        may be a tuple (but not list) of types or a single type, as with
4709        `isinstance`. Matched nodes will always have a value which is one
4710        of the following types: `list`, `tuple`, `set`, or `dict`.
4711
4712        If both a specific value and a type or tuple of types is
4713        specified, any collection whose members match the members of the
4714        specific value supplied and whose type is one of the listed types
4715        will match. For example, `Literal([1, 2], types=(list, tuple,
4716        set))` will match any of `[1, 2]`, `(1, 2)`, or `{2, 1}` but will
4717        NOT match `[2, 1]`, `(2, 1)`, or any dictionary.
4718
4719        Specifically, the value is converted to match the type of the
4720        node being considered and then a match is checked, so for
4721        example, `Literal([1, 2, 2], types=set)` will match the set `{1,
4722        2}` and the equivalent sets `{2, 1}` and `{1, 1, 2}`.
4723
4724        If a node has elements which aren't constants or literals, it
4725        will never match when a value is provided because we don't
4726        evaluate code during matching. It might still match if only
4727        type(s) are provided, of course.
4728        """
4729        super().__init__(**kwargs)
4730        self.value = value
4731        self.types = types
4732
4733        # Allowed types for literals
4734        allowed = (list, tuple, set, dict)
4735
4736        # value-type and type-type checking
4737        if value is not AnyValue and type(value) not in allowed:
4738            raise TypeError(
4739                f"Value {value!r} has type {type(value)} which is not a"
4740                f" type that a Literal can be (did you mean to use a"
4741                f" Constant instead?)."
4742            )
4743
4744        if self.types is not AnyType:
4745            if isinstance(self.types, tuple):
4746                for typ in self.types:
4747                    if typ not in allowed:
4748                        raise TypeError(
4749                            f"Type {typ} has is not a type that a"
4750                            f" Literal can be (did you mean to use a"
4751                            f" Constant instead?)."
4752                        )
4753            else:
4754                if self.types not in allowed:
4755                    raise TypeError(
4756                        f"Type {self.types} has is not a type that a"
4757                        f" Literal can be (did you mean to use a"
4758                        f" Constant instead?)."
4759                    )

A specific value may be supplied (it must be a list, tuple, set, or dictionary) or else any value will be accepted if the AnyValue class (not an instance of it) is used as the argument (that is the default).

If the value is AnyValue, one or more types may be specified, and only literals with that type will match. types may be a tuple (but not list) of types or a single type, as with isinstance. Matched nodes will always have a value which is one of the following types: list, tuple, set, or dict.

If both a specific value and a type or tuple of types is specified, any collection whose members match the members of the specific value supplied and whose type is one of the listed types will match. For example, Literal([1, 2], types=(list, tuple, set)) will match any of [1, 2], (1, 2), or {2, 1} but will NOT match [2, 1], (2, 1), or any dictionary.

Specifically, the value is converted to match the type of the node being considered and then a match is checked, so for example, Literal([1, 2, 2], types=set) will match the set {1, 2} and the equivalent sets {2, 1} and {1, 1, 2}.

If a node has elements which aren't constants or literals, it will never match when a value is provided because we don't evaluate code during matching. It might still match if only type(s) are provided, of course.

def structureString(self) 
4761    def structureString(self):
4762        if self.value == AnyValue:
4763            if self.types == AnyType:
4764                return "literal(s)"
4765            else:
4766                if isinstance(self.types, tuple):
4767                    types = (
4768                        ', '.join(t.__name__ for t in self.types[:-1])
4769                      + ' or ' + self.types[-1].__name__
4770                    )
4771                    return f"{types} literal(s)"
4772                else:
4773                    return f"{self.types.__name__} literal(s)"
4774        else:
4775            return f"literal {repr(self.value)}"

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
class Operator(ASTRequirement): 
4852class Operator(ASTRequirement):
4853    """
4854    A check for a unary operator, binary operator, boolean operator, or
4855    comparator. 'Similar' operations will count as partial matches. Note
4856    that 'is' and 'is not' are categorized as the same operator, as are
4857    'in' and 'not in'.
4858    """
4859    def __init__(self, op='+', **kwargs):
4860        """
4861        A specific operator must be specified. Use the text you'd write
4862        in Python to perform that operation (e.g., '//', '<=', or
4863        'and'). The two ambiguous cases are + and - which have both
4864        binary and unary forms. Add a 'u' beforehand to get their unary
4865        forms. Note that 'not in' and 'is not' are both allowed, but they
4866        are treated the same as 'in' and 'is'.
4867        """
4868        super().__init__(**kwargs)
4869        self.op = op
4870        # Determine correct + similar types
4871        typesToMatch = {
4872            'u+': ((ast.UAdd,), ()),
4873            'u-': ((ast.USub,), ()),
4874            'not': ((ast.Not,), ()),
4875            '~': ((ast.Invert,), ()),
4876            '+': ((ast.Add,), (ast.Sub,)),
4877            '-': ((ast.Sub,), (ast.Add,)),
4878            '*': ((ast.Mult,), (ast.Div,)),
4879            '/': ((ast.Div,), (ast.Mult)),
4880            '//': ((ast.FloorDiv,), (ast.Mod, ast.Div,)),
4881            '%': ((ast.Mod,), (ast.Div, ast.FloorDiv,)),
4882            '**': ((ast.Pow,), (ast.Mult,)),
4883            '<<': ((ast.LShift,), (ast.RShift,)),
4884            '>>': ((ast.RShift,), (ast.LShift,)),
4885            '|': ((ast.BitOr,), (ast.BitXor, ast.BitAnd)),
4886            '^': ((ast.BitXor,), (ast.BitOr, ast.BitAnd)),
4887            '&': ((ast.BitAnd,), (ast.BitXor, ast.BitOr)),
4888            '@': ((ast.MatMult,), (ast.Mult,)),
4889            'and': ((ast.And,), (ast.Or,)),
4890            'or': ((ast.Or,), (ast.And,)),
4891            '==': ((ast.Eq,), (ast.NotEq, ast.Is, ast.IsNot)),
4892            '!=': ((ast.NotEq,), (ast.Eq, ast.Is, ast.IsNot)),
4893            '<': ((ast.Lt,), (ast.LtE, ast.Gt, ast.GtE)),
4894            '<=': ((ast.LtE,), (ast.Lt, ast.Gt, ast.GtE)),
4895            '>': ((ast.Gt,), (ast.Lt, ast.LtE, ast.GtE)),
4896            '>=': ((ast.GtE,), (ast.Lt, ast.LtE, ast.Gt)),
4897            'is': ((ast.Is, ast.IsNot), (ast.Eq, ast.NotEq)),
4898            'is not': ((ast.IsNot, ast.Is), (ast.Eq, ast.NotEq)),
4899            'in': ((ast.In, ast.NotIn), ()),
4900            'not in': ((ast.NotIn, ast.In), ()),
4901        }.get(op)
4902
4903        if typesToMatch is None:
4904            raise ValueError(f"Unrecognized operator '{op}'.")
4905
4906        self.opTypes, self.partialTypes = typesToMatch
4907
4908    def structureString(self):
4909        return f"operator '{self.op}'"
4910
4911    def _nodesToCheck(self, syntaxTree):
4912        for node in self._walkNodesOfType(
4913            syntaxTree,
4914            (ast.UnaryOp, ast.BinOp, ast.BoolOp, ast.Compare)
4915        ):
4916            # Determine not/partial/full status of match...
4917            match = False
4918            if isinstance(node, ast.Compare):
4919                if any(
4920                    isinstance(op, self.opTypes)
4921                    for op in node.ops
4922                ):
4923                    match = True
4924                elif match is False and any(
4925                    isinstance(op, self.partialTypes)
4926                    for op in node.ops
4927                ):
4928                    match = "partial"
4929            else:
4930                if isinstance(node.op, self.opTypes):
4931                    match = True
4932                elif (
4933                    match is False
4934                and isinstance(node.op, self.partialTypes)
4935                ):
4936                    match = "partial"
4937
4938            # Yield node if it's a partial or full match
4939            if match:
4940                yield (node, match is True)

A check for a unary operator, binary operator, boolean operator, or comparator. 'Similar' operations will count as partial matches. Note that 'is' and 'is not' are categorized as the same operator, as are 'in' and 'not in'.

Operator(op='+', **kwargs) 
4859    def __init__(self, op='+', **kwargs):
4860        """
4861        A specific operator must be specified. Use the text you'd write
4862        in Python to perform that operation (e.g., '//', '<=', or
4863        'and'). The two ambiguous cases are + and - which have both
4864        binary and unary forms. Add a 'u' beforehand to get their unary
4865        forms. Note that 'not in' and 'is not' are both allowed, but they
4866        are treated the same as 'in' and 'is'.
4867        """
4868        super().__init__(**kwargs)
4869        self.op = op
4870        # Determine correct + similar types
4871        typesToMatch = {
4872            'u+': ((ast.UAdd,), ()),
4873            'u-': ((ast.USub,), ()),
4874            'not': ((ast.Not,), ()),
4875            '~': ((ast.Invert,), ()),
4876            '+': ((ast.Add,), (ast.Sub,)),
4877            '-': ((ast.Sub,), (ast.Add,)),
4878            '*': ((ast.Mult,), (ast.Div,)),
4879            '/': ((ast.Div,), (ast.Mult)),
4880            '//': ((ast.FloorDiv,), (ast.Mod, ast.Div,)),
4881            '%': ((ast.Mod,), (ast.Div, ast.FloorDiv,)),
4882            '**': ((ast.Pow,), (ast.Mult,)),
4883            '<<': ((ast.LShift,), (ast.RShift,)),
4884            '>>': ((ast.RShift,), (ast.LShift,)),
4885            '|': ((ast.BitOr,), (ast.BitXor, ast.BitAnd)),
4886            '^': ((ast.BitXor,), (ast.BitOr, ast.BitAnd)),
4887            '&': ((ast.BitAnd,), (ast.BitXor, ast.BitOr)),
4888            '@': ((ast.MatMult,), (ast.Mult,)),
4889            'and': ((ast.And,), (ast.Or,)),
4890            'or': ((ast.Or,), (ast.And,)),
4891            '==': ((ast.Eq,), (ast.NotEq, ast.Is, ast.IsNot)),
4892            '!=': ((ast.NotEq,), (ast.Eq, ast.Is, ast.IsNot)),
4893            '<': ((ast.Lt,), (ast.LtE, ast.Gt, ast.GtE)),
4894            '<=': ((ast.LtE,), (ast.Lt, ast.Gt, ast.GtE)),
4895            '>': ((ast.Gt,), (ast.Lt, ast.LtE, ast.GtE)),
4896            '>=': ((ast.GtE,), (ast.Lt, ast.LtE, ast.Gt)),
4897            'is': ((ast.Is, ast.IsNot), (ast.Eq, ast.NotEq)),
4898            'is not': ((ast.IsNot, ast.Is), (ast.Eq, ast.NotEq)),
4899            'in': ((ast.In, ast.NotIn), ()),
4900            'not in': ((ast.NotIn, ast.In), ()),
4901        }.get(op)
4902
4903        if typesToMatch is None:
4904            raise ValueError(f"Unrecognized operator '{op}'.")
4905
4906        self.opTypes, self.partialTypes = typesToMatch

A specific operator must be specified. Use the text you'd write in Python to perform that operation (e.g., '//', '<=', or 'and'). The two ambiguous cases are + and - which have both binary and unary forms. Add a 'u' beforehand to get their unary forms. Note that 'not in' and 'is not' are both allowed, but they are treated the same as 'in' and 'is'.

def structureString(self) 
4908    def structureString(self):
4909        return f"operator '{self.op}'"

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
class SpecificNode(ASTRequirement): 
4943class SpecificNode(ASTRequirement):
4944    """
4945    A flexible check where you can simply specify the AST node class(es)
4946    that you're looking for, plus a filter function to determine which
4947    matches are full/partial/non-matches. This does not perform any
4948    complicated sub-checks and doesn't have the cleanest structure
4949    string, so other `ASTRequirement` sub-classes are preferable if one
4950    of them can match what you want.
4951    """
4952    def __init__(self, nodeTypes, filterFunction=None, **kwargs):
4953        """
4954        Either a single AST node class (from the `ast` module, for
4955        example `ast.Break`) or a sequence of such classes is required to
4956        specify what counts as a match. If a sequence is provided, any of
4957        those node types will match; a `ValueError` will be raised if an
4958        empty sequence is provided.
4959
4960        If a filter function is provided, it will be called with an AST
4961        node as the sole argument for each node that has one of the
4962        specified types. If it returns exactly `True`, that node will be
4963        counted as a full match, if it returns exactly `False` that node
4964        will be counted as a partial match, and if it returns any other
4965        value (e.g., `None`) then that node will not be counted as a
4966        match at all.
4967        """
4968        super().__init__(**kwargs)
4969        if issubclass(nodeTypes, ast.AST):
4970            nodeTypes = (nodeTypes,)
4971        else:
4972            nodeTypes = tuple(nodeTypes)
4973            if len(nodeTypes) == 0:
4974                raise ValueError(
4975                    "Cannot specify an empty sequence of node types."
4976                )
4977            wrongTypes = tuple(
4978                [nt for nt in nodeTypes if not issubclass(nt, ast.AST)]
4979            )
4980            if len(wrongTypes) > 0:
4981                raise TypeError(
4982                    (
4983                        "All specified node types must be ast.AST"
4984                        " subclasses, but you provided some node types"
4985                        " that weren't:\n  "
4986                    ) + '\n  '.join(repr(nt) for nt in wrongTypes)
4987                )
4988
4989        self.nodeTypes = nodeTypes
4990        self.filterFunction = filterFunction
4991
4992    def structureString(self):
4993        if isinstance(self.nodeTypes, ast.AST):
4994            result = f"{self.nodeTypes.__name__} node(s)"
4995        elif len(self.nodeTypes) == 1:
4996            result = f"{self.nodeTypes[0].__name__} node(s)"
4997        elif len(self.nodeTypes) == 2:
4998            result = (
4999                f"either {self.nodeTypes[0].__name__} or"
5000                f" {self.nodeTypes[1].__name__} node(s)"
5001            )
5002        elif len(self.nodeTypes) > 2:
5003            result = (
5004                "node(s) that is/are:"
5005              + ', '.join(nt.__name__ for nt in self.nodeTypes[:-1])
5006              + ', or ' + self.nodeTypes[-1].__name__
5007            )
5008
5009        if self.filterFunction is not None:
5010            result += " (with additional criteria)"
5011
5012        return result
5013
5014    def _nodesToCheck(self, syntaxTree):
5015        for node in self._walkNodesOfType(syntaxTree, self.nodeTypes):
5016            if self.filterFunction is None:
5017                yield (node, True)
5018            else:
5019                matchStatus = self.filterFunction(node)
5020                if matchStatus in (True, False):
5021                    yield (node, matchStatus)
5022                # Otherwise (e.g., None) it's a non-match

A flexible check where you can simply specify the AST node class(es) that you're looking for, plus a filter function to determine which matches are full/partial/non-matches. This does not perform any complicated sub-checks and doesn't have the cleanest structure string, so other ASTRequirement sub-classes are preferable if one of them can match what you want.

SpecificNode(nodeTypes, filterFunction=None, **kwargs) 
4952    def __init__(self, nodeTypes, filterFunction=None, **kwargs):
4953        """
4954        Either a single AST node class (from the `ast` module, for
4955        example `ast.Break`) or a sequence of such classes is required to
4956        specify what counts as a match. If a sequence is provided, any of
4957        those node types will match; a `ValueError` will be raised if an
4958        empty sequence is provided.
4959
4960        If a filter function is provided, it will be called with an AST
4961        node as the sole argument for each node that has one of the
4962        specified types. If it returns exactly `True`, that node will be
4963        counted as a full match, if it returns exactly `False` that node
4964        will be counted as a partial match, and if it returns any other
4965        value (e.g., `None`) then that node will not be counted as a
4966        match at all.
4967        """
4968        super().__init__(**kwargs)
4969        if issubclass(nodeTypes, ast.AST):
4970            nodeTypes = (nodeTypes,)
4971        else:
4972            nodeTypes = tuple(nodeTypes)
4973            if len(nodeTypes) == 0:
4974                raise ValueError(
4975                    "Cannot specify an empty sequence of node types."
4976                )
4977            wrongTypes = tuple(
4978                [nt for nt in nodeTypes if not issubclass(nt, ast.AST)]
4979            )
4980            if len(wrongTypes) > 0:
4981                raise TypeError(
4982                    (
4983                        "All specified node types must be ast.AST"
4984                        " subclasses, but you provided some node types"
4985                        " that weren't:\n  "
4986                    ) + '\n  '.join(repr(nt) for nt in wrongTypes)
4987                )
4988
4989        self.nodeTypes = nodeTypes
4990        self.filterFunction = filterFunction

Either a single AST node class (from the ast module, for example ast.Break) or a sequence of such classes is required to specify what counts as a match. If a sequence is provided, any of those node types will match; a ValueError will be raised if an empty sequence is provided.

If a filter function is provided, it will be called with an AST node as the sole argument for each node that has one of the specified types. If it returns exactly True, that node will be counted as a full match, if it returns exactly False that node will be counted as a partial match, and if it returns any other value (e.g., None) then that node will not be counted as a match at all.

def structureString(self) 
4992    def structureString(self):
4993        if isinstance(self.nodeTypes, ast.AST):
4994            result = f"{self.nodeTypes.__name__} node(s)"
4995        elif len(self.nodeTypes) == 1:
4996            result = f"{self.nodeTypes[0].__name__} node(s)"
4997        elif len(self.nodeTypes) == 2:
4998            result = (
4999                f"either {self.nodeTypes[0].__name__} or"
5000                f" {self.nodeTypes[1].__name__} node(s)"
5001            )
5002        elif len(self.nodeTypes) > 2:
5003            result = (
5004                "node(s) that is/are:"
5005              + ', '.join(nt.__name__ for nt in self.nodeTypes[:-1])
5006              + ', or ' + self.nodeTypes[-1].__name__
5007            )
5008
5009        if self.filterFunction is not None:
5010            result += " (with additional criteria)"
5011
5012        return result

Returns a string expressing the structure that this check is looking for.

Inherited Members
ASTRequirement
howMany
fullStructure
allMatches
contains
def indent(msg, level=2) 
5033def indent(msg, level=2):
5034    """
5035    Indents every line of the given message (a string).
5036    """
5037    indent = ' ' * level
5038    return indent + ('\n' + indent).join(msg.splitlines())

Indents every line of the given message (a string).

def ellipsis(string, maxlen=40) 
5041def ellipsis(string, maxlen=40):
5042    """
5043    Returns the provided string as-is, or if it's longer than the given
5044    maximum length, returns the string, truncated, with '...' at the
5045    end, which will, including the ellipsis, be exactly the given
5046    maximum length. The maximum length must be 4 or more.
5047    """
5048    if len(string) > maxlen:
5049        return string[:maxlen - 3] + "..."
5050    else:
5051        return string

Returns the provided string as-is, or if it's longer than the given maximum length, returns the string, truncated, with '...' at the end, which will, including the ellipsis, be exactly the given maximum length. The maximum length must be 4 or more.

def dual_string_repr(string) 
5054def dual_string_repr(string):
5055    """
5056    Returns a pair containing full and truncated representations of the
5057    given string. The formatting of even the full representation depends
5058    on whether it's a multi-line string or not and how long it is.
5059    """
5060    lines = string.split('\n')
5061    if len(repr(string)) < 80 and len(lines) == 1:
5062        full = repr(string)
5063        short = repr(string)
5064    else:
5065        full = '"""\\\n' + string.replace('\r', '\\r') + '"""'
5066        if len(string) < 240 and len(lines) <= 7:
5067            short = full
5068        elif len(lines) > 7:
5069            head = '\n'.join(lines[:7])
5070            short = (
5071                '"""\\\n' + ellipsis(head.replace('\r', '\\r'), 240) + '"""'
5072            )
5073        else:
5074            short = (
5075                '"""\\\n' + ellipsis(string.replace('\r', '\\r'), 240) + '"""'
5076            )
5077
5078    return (full, short)

Returns a pair containing full and truncated representations of the given string. The formatting of even the full representation depends on whether it's a multi-line string or not and how long it is.

def limited_repr(string) 
5081def limited_repr(string):
5082    """
5083    Given a string that might include multiple lines and/or lots of
5084    characters (regardless of lines), returns version cut off by
5085    ellipses either after 5 or so lines, or after 240 characters.
5086    Returns the full string if it's both less than 240 characters and
5087    less than 5 lines.
5088    """
5089    # Split by lines
5090    lines = string.split('\n')
5091
5092    # Already short enough
5093    if len(string) < 240 and len(lines) < 5:
5094        return string
5095
5096    # Try up to 5 lines, cutting them off until we've got a
5097    # short-enough head string
5098    for n in range(min(5, len(lines)), 0, -1):
5099        head = '\n'.join(lines[:n])
5100        if n < len(lines):
5101            head += '\n...'
5102        if len(head) < 240:
5103            break
5104    else:
5105        # If we didn't break, just use first 240 characters
5106        # of the string
5107        head = string[:240] + '...'
5108
5109    # If we cut things too short (e.g., because of initial
5110    # empty lines) use first 240 characters of the string
5111    if len(head) < 12:
5112        head = string[:240] + '...'
5113
5114    return head

Given a string that might include multiple lines and/or lots of characters (regardless of lines), returns version cut off by ellipses either after 5 or so lines, or after 240 characters. Returns the full string if it's both less than 240 characters and less than 5 lines.

def msg_color(category) 
5117def msg_color(category):
5118    """
5119    Returns an ANSI color code for the given category of message (one of
5120    "succeeded", "failed", "skipped", or "reset"), or returns None if
5121    COLORS is disabled or an invalid category is provided.
5122    """
5123    if not COLORS:
5124        return None
5125    else:
5126        return MSG_COLORS.get(category)

Returns an ANSI color code for the given category of message (one of "succeeded", "failed", "skipped", or "reset"), or returns None if COLORS is disabled or an invalid category is provided.

def expr_details(context) 
5162def expr_details(context):
5163    """
5164    Returns a pair of strings containing base and extra details for an
5165    expression as represented by a dictionary returned from
5166    `get_my_context`. The extra message may be an empty string if the
5167    base message contains all relevant information.
5168    """
5169    # Expression that was evaluated
5170    expr = context.get("expr_src", "???")
5171    short_expr = ellipsis(expr, 78)
5172    # Results
5173    msg = ""
5174    extra_msg = ""
5175
5176    # Base message
5177    msg += f"Test expression was:\n{indent(short_expr, 2)}"
5178
5179    # Figure out values to display
5180    vdict = context.get("values", {})
5181    if context.get("relevant") is not None:
5182        show = sorted(
5183            context["relevant"],
5184            key=lambda fragment: (expr.index(fragment), len(fragment))
5185        )
5186    else:
5187        show = sorted(
5188            vdict.keys(),
5189            key=lambda fragment: (expr.index(fragment), len(fragment))
5190        )
5191
5192    if len(show) > 0:
5193        msg += "\nValues were:"
5194
5195    longs = []
5196    for key in show:
5197        if key in vdict:
5198            val = repr(vdict[key])
5199        else:
5200            val = "???"
5201
5202        entry = f"  {key} = {val}"
5203        fits = ellipsis(entry)
5204        msg += '\n' + fits
5205        if fits != entry:
5206            longs.append(entry)
5207
5208    # Extra message
5209    if short_expr != expr:
5210        if extra_msg != "" and not extra_msg.endswith('\n'):
5211            extra_msg += '\n'
5212        extra_msg += f"Full expression:\n{indent(expr, 2)}"
5213    extra_values = sorted(
5214        [
5215            key
5216            for key in vdict.keys()
5217            if key not in context.get("relevant", [])
5218        ],
5219        key=lambda fragment: (expr.index(fragment), len(fragment))
5220    )
5221    if context.get("relevant") is not None and extra_values:
5222        if extra_msg != "" and not extra_msg.endswith('\n'):
5223            extra_msg += '\n'
5224        extra_msg += "Extra values:"
5225        for ev in extra_values:
5226            if ev in vdict:
5227                val = repr(vdict[ev])
5228            else:
5229                val = "???"
5230
5231            entry = f"  {ev} = {val}"
5232            fits = ellipsis(entry, 78)
5233            extra_msg += '\n' + fits
5234            if fits != entry:
5235                longs.append(entry)
5236
5237    if longs:
5238        if extra_msg != "" and not extra_msg.endswith('\n'):
5239            extra_msg += '\n'
5240        extra_msg += "Full values:"
5241        for entry in longs:
5242            extra_msg += '\n' + entry
5243
5244    return msg, extra_msg

Returns a pair of strings containing base and extra details for an expression as represented by a dictionary returned from get_my_context. The extra message may be an empty string if the base message contains all relevant information.

def findFirstDifference(val, ref, comparing=None) 
5251def findFirstDifference(val, ref, comparing=None):
5252    """
5253    Returns a string describing the first point of difference between
5254    `val` and `ref`, or None if the two values are equivalent. If
5255    IGNORE_TRAILING_WHITESPACE is True, trailing whitespace will be
5256    trimmed from each string before looking for differences.
5257
5258    A small amount of difference is ignored between floating point
5259    numbers, including those found in complex structures.
5260
5261    Works for recursive data structures; the `comparing` argument serves
5262    as a memo to avoid infinite recursion, and the `within` argument
5263    indicates where in a complex structure we are; both should normally
5264    be left as their defaults.
5265    """
5266    if comparing is None:
5267        comparing = set()
5268
5269    cmpkey = (id(val), id(ref))
5270    if cmpkey in comparing:
5271        # Either they differ somewhere else, or they're functionally
5272        # identical
5273        # TODO: Does this really ward off all infinite recursion on
5274        # finite structures?
5275        return None
5276
5277    comparing.add(cmpkey)
5278
5279    try:
5280        simple = val == ref
5281    except RecursionError:
5282        simple = False
5283
5284    if simple:
5285        return None
5286
5287    else:  # let's hunt for differences
5288        if (
5289            isinstance(val, (int, float, complex))
5290        and isinstance(ref, (int, float, complex))
5291        ):  # what if they're both numbers?
5292            if cmath.isclose(
5293                val,
5294                ref,
5295                rel_tol=FLOAT_REL_TOLERANCE,
5296                abs_tol=FLOAT_ABS_TOLERANCE
5297            ):
5298                return None
5299            else:
5300                if isinstance(val, complex) and isinstance(ref, complex):
5301                    return f"complex numbers {val} and {ref} are different"
5302                elif isinstance(val, complex) or isinstance(ref, complex):
5303                    return f"numbers {val} and {ref} are different"
5304                elif val > 0 and ref < 0:
5305                    return f"numbers {val} and {ref} have different signs"
5306                else:
5307                    return f"numbers {val} and {ref} are different"
5308
5309        elif type(val) != type(ref):  # different types; not both numbers
5310            svr = ellipsis(repr(val), 8)
5311            srr = ellipsis(repr(ref), 8)
5312            return (
5313                f"values {svr} and {srr} have different types"
5314                f" ({type(val)} and {type(ref)})"
5315            )
5316
5317        elif isinstance(val, str):  # both strings
5318            if '\n' in val or '\n' in ref:
5319                # multi-line strings; look for first different line
5320                # Note: we *don't* use splitlines here because it will
5321                # give multiple line breaks in a \r\r\n situation like
5322                # those caused by csv.DictWriter on windows when opening
5323                # a file without newlines=''. We'd like to instead ignore
5324                # '\r' as a line break (we're not going to work on early
5325                # Macs) and strip it if IGNORE_TRAILING_WHITESPACE is on.
5326                valLines = val.split('\n')
5327                refLines = ref.split('\n')
5328
5329                # First line # where they differ (1-indexed)
5330                firstDiff = None
5331
5332                # Compute point of first difference
5333                i = None
5334                for i in range(min(len(valLines), len(refLines))):
5335                    valLine = valLines[i]
5336                    refLine = refLines[i]
5337                    if IGNORE_TRAILING_WHITESPACE:
5338                        valLine = valLine.rstrip()
5339                        refLine = refLine.rstrip()
5340
5341                    if valLine != refLine:
5342                        firstDiff = i + 1
5343                        break
5344                else:
5345                    if i is not None:
5346                        # if one has more lines
5347                        if len(valLines) != len(refLines):
5348                            # In this case, one of the two is longer...
5349                            # If IGNORE_TRAILING_WHITESPACE is on, and
5350                            # the longer one just has a blank extra line
5351                            # (possibly with some whitespace on it), then
5352                            # the difference is just in the presence or
5353                            # absence of a final newline, which we also
5354                            # count as a "trailing whitespace" difference
5355                            # and ignore. Note that multiple final '\n'
5356                            # characters will be counted as a difference,
5357                            # since they result in multiple final
5358                            # lines...
5359                            if (
5360                                IGNORE_TRAILING_WHITESPACE
5361                            and (
5362                                    (
5363                                        len(valLines) == len(refLines) + 1
5364                                    and valLines[i + 1].strip() == ''
5365                                    )
5366                                 or (
5367                                        len(valLines) + 1 == len(refLines)
5368                                    and refLines[i + 1].strip() == ''
5369                                    )
5370                                )
5371                            ):
5372                                return None
5373                            else:
5374                                # If we're attending trailing whitespace,
5375                                # or if there are multiple extra lines or
5376                                # the single extra line is not blank,
5377                                # then that's where our first difference
5378                                # is.
5379                                firstDiff = i + 2
5380                        else:
5381                            # There is no difference once we trim
5382                            # trailing whitespace...
5383                            return None
5384                    else:
5385                        # Note: this is a line number, NOT a line index
5386                        firstDiff = 1
5387
5388                got = "nothing (string had fewer lines than expected)"
5389                expected = "nothing (string had more lines than expected)"
5390                i = firstDiff - 1
5391                if i < len(valLines):
5392                    got = repr(valLines[i])
5393                if i < len(refLines):
5394                    expected = repr(refLines[i])
5395
5396                limit = 60
5397                shortGot = ellipsis(got, limit)
5398                shortExpected = ellipsis(expected, limit)
5399                while (
5400                    shortGot == shortExpected
5401                and limit < len(got) or limit < len(expected)
5402                and limit < 200
5403                ):
5404                    limit += 10
5405                    shortGot = ellipsis(got, limit)
5406                    shortExpected = ellipsis(expected, limit)
5407
5408                return (
5409                    f"strings differ on line {firstDiff} where we got:"
5410                    f"\n  {shortGot}\nbut we expected:"
5411                    f"\n  {shortExpected}"
5412                )
5413            else:
5414                # Single-line strings: find character pos of difference
5415                if IGNORE_TRAILING_WHITESPACE:
5416                    val = val.rstrip()
5417                    ref = ref.rstrip()
5418                    if val == ref:
5419                        return None
5420
5421                # Find character position of first difference
5422                pos = None
5423                i = None
5424                for i in range(min(len(val), len(ref))):
5425                    if val[i] != ref[i]:
5426                        pos = i
5427                        break
5428                else:
5429                    if i is not None:
5430                        pos = i + 1
5431                    else:
5432                        pos = 0  # one string is empty
5433
5434                vchar = None
5435                rchar = None
5436                if pos < len(val):
5437                    vchar = val[pos]
5438                if pos < len(ref):
5439                    rchar = ref[pos]
5440
5441                if vchar is None:
5442                    missing = ellipsis(repr(ref[pos:]), 20)
5443                    return (
5444                        f"expected text missing from end of string:"
5445                        f" {missing}"
5446                    )
5447                    return f"strings {svr} and {srr} differ at position {pos}"
5448                elif rchar is None:
5449                    extra = ellipsis(repr(val[pos:]), 20)
5450                    return (
5451                        f"extra text at end of string:"
5452                        f" {extra}"
5453                    )
5454                else:
5455                    if pos > 6:
5456                        got = ellipsis(repr(val[pos:]), 14)
5457                        expected = ellipsis(repr(ref[pos:]), 14)
5458                        return (
5459                            f"strings differ from position {pos}: got {got}"
5460                            f" but expected {expected}"
5461                        )
5462                    else:
5463                        got = ellipsis(repr(val), 14)
5464                        expected = ellipsis(repr(ref), 14)
5465                        return (
5466                            f"strings are different: got {got}"
5467                            f" but expected {expected}"
5468                        )
5469
5470        elif isinstance(val, (list, tuple)):  # both lists or tuples
5471            svr = ellipsis(repr(val), 10)
5472            srr = ellipsis(repr(ref), 10)
5473            typ = type(val).__name__
5474            if len(val) != len(ref):
5475                return (
5476                    f"{typ}s {svr} and {srr} have different lengths"
5477                    f" ({len(val)} and {len(ref)})"
5478                )
5479            else:
5480                for i in range(len(val)):
5481                    diff = findFirstDifference(val[i], ref[i], comparing)
5482                    if diff is not None:
5483                        return f"in slot {i} of {typ}, " + diff
5484                return None  # no differences in any slot
5485
5486        elif isinstance(val, (set)):  # both sets
5487            svr = ellipsis(repr(val), 10)
5488            srr = ellipsis(repr(ref), 10)
5489            onlyVal = (val - ref)
5490            onlyRef = (ref - val)
5491            # Sort so we can match up different-but-equivalent
5492            # floating-point items...
5493            try:
5494                sonlyVal = sorted(onlyVal)
5495                sonlyRef = sorted(onlyRef)
5496                diff = findFirstDifference(
5497                    sonlyVal,
5498                    sonlyRef,
5499                    comparing
5500                )
5501            except TypeError:
5502                # not sortable, so not just floating-point diffs
5503                diff = "some"
5504
5505            if diff is None:
5506                return None
5507            else:
5508                nMissing = len(onlyRef)
5509                nExtra = len(onlyVal)
5510                if nExtra == 0:
5511                    firstMissing = ellipsis(repr(list(onlyRef)[0]), 12)
5512                    result = f"in a set, missing element {firstMissing}"
5513                    if nMissing > 1:
5514                        result += f" ({nMissing} missing elements in total)"
5515                    return result
5516                elif nMissing == 0:
5517                    firstExtra = ellipsis(repr(list(onlyVal)[0]), 12)
5518                    return f"in a set, extra element {firstExtra}"
5519                    if nExtra > 1:
5520                        result += f" ({nExtra} extra elements in total)"
5521                    return result
5522                else:
5523                    firstMissing = ellipsis(repr(list(onlyRef)[0]), 8)
5524                    firstExtra = ellipsis(repr(list(onlyVal)[0]), 8)
5525                    result = (
5526                        "in a set, elements are different (extra"
5527                        f" element {firstExtra} and missing element"
5528                        f" {firstMissing}"
5529                    )
5530                    if nMissing > 1 and nExtra > 1:
5531                        result += (
5532                            f" ({nExtra} total extra elements and"
5533                            f" {nMissing} total missing elements"
5534                        )
5535                    elif nMissing == 1:
5536                        if nExtra > 1:
5537                            result += (
5538                                f" (1 missing and {nExtra} total extra"
5539                                f" elements)"
5540                            )
5541                    else:  # nExtra must be 1
5542                        if nMissing > 1:
5543                            result += (
5544                                f" (1 extra and {nExtra} total missing"
5545                                f" elements)"
5546                            )
5547                    return result
5548
5549        elif isinstance(val, dict):  # both dicts
5550            svr = ellipsis(repr(val), 14)
5551            srr = ellipsis(repr(ref), 14)
5552
5553            if len(val) != len(ref):
5554                if len(val) < len(ref):
5555                    ldiff = len(ref) - len(val)
5556                    firstMissing = ellipsis(
5557                        repr(list(set(ref.keys()) - set(val.keys()))[0]),
5558                        30
5559                    )
5560                    return (
5561                        f"dictionary is missing key {firstMissing} (has"
5562                        f" {ldiff} fewer key{'s' if ldiff > 1 else ''}"
5563                        f" than expected)"
5564                    )
5565                else:
5566                    ldiff = len(val) - len(ref)
5567                    firstExtra = ellipsis(
5568                        repr(list(set(val.keys()) - set(ref.keys()))[0]),
5569                        30
5570                    )
5571                    return (
5572                        f"dictionary has extra key {firstExtra} (has"
5573                        f" {ldiff} more key{'s' if ldiff > 1 else ''}"
5574                        f" than expected)"
5575                    )
5576                return (
5577                    f"dictionaries {svr} and {srr} have different sizes"
5578                    f" ({len(val)} and {len(ref)})"
5579                )
5580
5581            vkeys = set(val.keys())
5582            rkeys = set(ref.keys())
5583            try:
5584                onlyVal = sorted(vkeys - rkeys)
5585                onlyRef = sorted(rkeys - vkeys)
5586                keyCorrespondence = {}
5587            except TypeError:  # unsortable...
5588                keyCorrespondence = None
5589
5590            # Check for floating-point equivalence of keys if sets are
5591            # sortable...
5592            if keyCorrespondence is not None:
5593                if findFirstDifference(onlyVal, onlyRef, comparing) is None:
5594                    keyCorrespondence = {
5595                        onlyVal[i]: onlyRef[i]
5596                        for i in range(len(onlyVal))
5597                    }
5598                    # Add pass-through mappings for matching keys
5599                    for k in vkeys & rkeys:
5600                        keyCorrespondence[k] = k
5601                else:
5602                    # No actual mapping is available...
5603                    keyCorrespondence = None
5604
5605            # We couldn't find a correspondence between keys, so we
5606            # return a key-based difference
5607            if keyCorrespondence is None:
5608                onlyVal = vkeys - rkeys
5609                onlyRef = rkeys - vkeys
5610                nExtra = len(onlyVal)
5611                nMissing = len(onlyRef)
5612                if nExtra == 0:
5613                    firstMissing = ellipsis(repr(list(onlyRef)[0]), 10)
5614                    result = f"dictionary is missing key {firstMissing}"
5615                    if nMissing > 1:
5616                        result += f" ({nMissing} missing keys in total)"
5617                    return result
5618                elif nMissing == 0:
5619                    firstExtra = ellipsis(repr(list(onlyVal)[0]), 10)
5620                    result = f"dictionary has extra key {firstExtra}"
5621                    if nExtra > 1:
5622                        result += f" ({nExtra} extra keys in total)"
5623                    return result
5624                else:  # neither is 0
5625                    firstMissing = ellipsis(repr(list(onlyRef)[0]), 10)
5626                    firstExtra = ellipsis(repr(list(onlyVal)[0]), 10)
5627                    result = (
5628                        f"dictionary is missing key {firstMissing} and"
5629                        f" has extra key {firstExtra}"
5630                    )
5631                    if nMissing > 1 and nExtra > 1:
5632                        result += (
5633                            f" ({nMissing} missing and {nExtra} extra"
5634                            f" keys in total)"
5635                        )
5636                    elif nMissing == 1:
5637                        if nExtra > 1:
5638                            result += (
5639                                f" (1 missing and {nExtra} extra keys"
5640                                f" in total)"
5641                            )
5642                    else:  # nExtra must be 1
5643                        if nMissing > 1:
5644                            result += (
5645                                f" (1 extra and {nMissing} missing keys"
5646                                f" in total)"
5647                            )
5648                    return result
5649
5650            # if we reach here, keyCorrespondence maps val keys to
5651            # equivalent (but not necessarily identical) ref keys
5652
5653            for vk in keyCorrespondence:
5654                rk = keyCorrespondence[vk]
5655                vdiff = findFirstDifference(val[vk], ref[rk], comparing)
5656                if vdiff is not None:
5657                    krep = ellipsis(repr(vk), 14)
5658                    return f"in dictionary slot {krep}, " + vdiff
5659
5660            return None
5661
5662        else:  # not sure what kind of thing this is...
5663            if val == ref:
5664                return None
5665            else:
5666                limit = 15
5667                vr = repr(val)
5668                rr = repr(ref)
5669                svr = ellipsis(vr, limit)
5670                srr = ellipsis(rr, limit)
5671                while (
5672                    svr == srr
5673                and (limit < len(vr) or limit < len(rr))
5674                and limit < 100
5675                ):
5676                    limit += 10
5677                    svr = ellipsis(vr, limit)
5678                    srr = ellipsis(rr, limit)
5679                return f" objects {svr} and {srr} are different"

Returns a string describing the first point of difference between val and ref, or None if the two values are equivalent. If IGNORE_TRAILING_WHITESPACE is True, trailing whitespace will be trimmed from each string before looking for differences.

A small amount of difference is ignored between floating point numbers, including those found in complex structures.

Works for recursive data structures; the comparing argument serves as a memo to avoid infinite recursion, and the within argument indicates where in a complex structure we are; both should normally be left as their defaults.

def checkContainment(val1, val2) 
5682def checkContainment(val1, val2):
5683    """
5684    Returns True if val1 is 'contained in' to val2, and False otherwise.
5685    If IGNORE_TRAILING_WHITESPACE is True, will ignore trailing
5686    whitespace in two strings when comparing them for containment.
5687    """
5688    if (not isinstance(val1, str)) or (not isinstance(val2, str)):
5689        return val1 in val2  # use regular containment test
5690    # For two strings, pay attention to IGNORE_TRAILING_WHITESPACE
5691    elif IGNORE_TRAILING_WHITESPACE:
5692        # remove trailing whitespace from both strings (on all lines)
5693        return trimWhitespace(val1) in trimWhitespace(val2)
5694    else:
5695        return val1 in val2  # use regular containment test

Returns True if val1 is 'contained in' to val2, and False otherwise. If IGNORE_TRAILING_WHITESPACE is True, will ignore trailing whitespace in two strings when comparing them for containment.

def trimWhitespace(st, requireNewline=False) 
5698def trimWhitespace(st, requireNewline=False):
5699    """
5700    Assume st a string. Use .rstrip() to remove trailing whitespace from
5701    each line. This has the side effect of replacing complex newlines
5702    with just '\\n'. If requireNewline is set to true, only whitespace
5703    that comes before a newline will be trimmed, and whitespace which
5704    occurs at the end of the string on the last line will be retained if
5705    there is no final newline.
5706    """
5707    if requireNewline:
5708        return re.sub('[ \t\r]*([\r\n])', r'\1', st)
5709    else:
5710        result = '\n'.join(line.rstrip() for line in st.split('\n'))
5711        return result

Assume st a string. Use .rstrip() to remove trailing whitespace from each line. This has the side effect of replacing complex newlines with just '\n'. If requireNewline is set to true, only whitespace that comes before a newline will be trimmed, and whitespace which occurs at the end of the string on the last line will be retained if there is no final newline.

def compare(val, ref) 
5714def compare(val, ref):
5715    """
5716    Comparison function returning a boolean which uses
5717    findFirstDifference under the hood.
5718    """
5719    return findFirstDifference(val, ref) is None

Comparison function returning a boolean which uses findFirstDifference under the hood.

def test_compare() 
5722def test_compare():
5723    "Tests the compare function."
5724    # TODO: test findFirstDifference instead & confirm correct
5725    # messages!!!
5726    # Integers
5727    assert compare(1, 1)
5728    assert compare(1, 2) is False
5729    assert compare(2, 1 + 1)
5730
5731    # Floating point numbers
5732    assert compare(1.1, 1.1)
5733    assert compare(1.1, 1.2) is False
5734    assert compare(1.1, 1.1000000001)
5735    assert compare(1.1, 1.1001) is False
5736
5737    # Complex numbers
5738    assert compare(1.1 + 2.3j, 1.1 + 2.3j)
5739    assert compare(1.1 + 2.3j, 1.1 + 2.4j) is False
5740
5741    # Strings
5742    assert compare('abc', 1.1001) is False
5743    assert compare('abc', 'abc')
5744    assert compare('abc', 'def') is False
5745
5746    # Lists
5747    assert compare([1, 2, 3], [1, 2, 3])
5748    assert compare([1, 2, 3], [1, 2, 4]) is False
5749    assert compare([1, 2, 3], [1, 2, 3.0000000001])
5750
5751    # Tuples
5752    assert compare((1, 2, 3), (1, 2, 3))
5753    assert compare((1, 2, 3), (1, 2, 4)) is False
5754    assert compare((1, 2, 3), (1, 2, 3.0000000001))
5755
5756    # Nested lists + tuples
5757    assert compare(
5758        ['a', 'b', 'cdefg', [1, 2, [3]], (4, 5)],
5759        ['a', 'b', 'cdefg', [1, 2, [3]], (4, 5)]
5760    )
5761    assert compare(
5762        ['a', 'b', 'cdefg', [1, 2, [3]], (4, 5)],
5763        ['a', 'b', 'cdefg', [1, 2, [3]], (4, '5')]
5764    ) is False
5765    assert compare(
5766        ['a', 'b', 'cdefg', [1, 2, [3]], (4, 5)],
5767        ['a', 'b', 'cdefg', [1, 2, [3]], [4, 5]]
5768    ) is False
5769
5770    # Sets
5771    assert compare({1, 2}, {1, 2})
5772    assert compare({1, 2}, {1}) is False
5773    assert compare({1}, {1, 2}) is False
5774    assert compare({1, 2}, {'1', 2}) is False
5775    assert compare({'a', 'b', 'c'}, {'a', 'b', 'c'})
5776    assert compare({'a', 'b', 'c'}, {'a', 'b', 'C'}) is False
5777    # Two tricky cases
5778    assert compare({1, 2}, {1.00000001, 2})
5779    assert compare({(1, 2), 3}, {(1.00000001, 2), 3})
5780
5781    # Dictionaries
5782    assert compare({1: 2, 3: 4}, {1: 2, 3: 4})
5783    assert compare({1: 2, 3: 4}, {1: 2, 3.00000000001: 4})
5784    assert compare({1: 2, 3: 4}, {1: 2, 3: 4.00000000001})
5785    assert compare({1: 2, 3: 4}, {1: 2, 3.1: 4}) is False
5786    assert compare({1: 2, 3: 4}, {1: 2, 3: 4.1}) is False
5787
5788    # Nested dictionaries & lists
5789    assert compare(
5790        {1: {1.1: 2.2}, 2: [2.2, 3.3]},
5791        {1: {1.1: 2.2}, 2: [2.2, 3.3]}
5792    )
5793    assert compare(
5794        {1: {1.1: 2.2}, 2: [2.2, 3.3]},
5795        {1: {1.2: 2.2}, 2: [2.2, 3.3]}
5796    ) is False
5797    assert compare(
5798        {1: {1.1: 2.2}, 2: [2.2, 3.3]},
5799        {1: {1.1: 2.3}, 2: [2.2, 3.3]}
5800    ) is False
5801    assert compare(
5802        {1: {1.1: 2.2}, 2: [2.2, 3.3]},
5803        {1: {1.1: 2.2}, 2: [2.2, 3.4]}
5804    ) is False
5805    assert compare(
5806        {1: {1.1: 2.2}, 2: [2.2, 3.3]},
5807        {1: {1.1: 2.2}, 2: 2.2}
5808    ) is False
5809    assert compare(
5810        {1: {1.1: 2.2}, 2: [2.2, 3.3]},
5811        {1: {1.1: 2.2}, 2: (2.2, 3.3)}
5812    ) is False
5813
5814    # Equivalent infinitely recursive list structures
5815    a = [1, 2, 3]
5816    a.append(a)
5817    a2 = [1, 2, 3]
5818    a2.append(a2)
5819    b = [1, 2, 3, [1, 2, 3]]
5820    b[3].append(b)
5821    c = [1, 2, 3, [1, 2, 3, [1, 2, 3]]]
5822    c[3][3].append(c[3])
5823    d = [1, 2, 3]
5824    d.insert(2, d)
5825
5826    assert compare(a, a)
5827    assert compare(a, a2)
5828    assert compare(a2, a)
5829    assert compare(a, b)
5830    assert compare(a, c)
5831    assert compare(a2, b)
5832    assert compare(b, a)
5833    assert compare(b, a2)
5834    assert compare(c, a)
5835    assert compare(b, c)
5836    assert compare(a, d) is False
5837    assert compare(b, d) is False
5838    assert compare(c, d) is False
5839    assert compare(d, a) is False
5840    assert compare(d, b) is False
5841    assert compare(d, c) is False
5842
5843    # Equivalent infinitely recursive dicitonaries
5844    e = {1: 2}
5845    e[2] = e
5846    e2 = {1: 2}
5847    e2[2] = e2
5848    f = {1: 2}
5849    f2 = {1: 2}
5850    f[2] = f2
5851    f2[2] = f
5852    g = {1: 2, 2: {1: 2.0000000001}}
5853    g[2][2] = g
5854    h = {1: 2, 2: {1: 3}}
5855    h[2][2] = h
5856
5857    assert compare(e, e2)
5858    assert compare(e2, e)
5859    assert compare(f, f2)
5860    assert compare(f2, f)
5861    assert compare(e, f)
5862    assert compare(f, e)
5863    assert compare(e, g)
5864    assert compare(f, g)
5865    assert compare(g, e)
5866    assert compare(g, f)
5867    assert compare(e, h) is False
5868    assert compare(f, h) is False
5869    assert compare(g, h) is False
5870    assert compare(h, e) is False
5871    assert compare(h, f) is False
5872    assert compare(h, g) is False
5873
5874    # Custom types + objects
5875    class T:
5876        pass
5877
5878    assert compare(T, T)
5879    assert compare(T, 1) is False
5880    assert compare(T(), T()) is False
5881
5882    # Custom type w/ a custom __eq__
5883    class E:
5884        def __eq__(self, other):
5885            return isinstance(other, E) or other == 1
5886
5887    assert compare(E, E)
5888    assert compare(E, 1) is False
5889    assert compare(E(), 1)
5890    assert compare(E(), 2) is False
5891    assert compare(E(), E())
5892
5893    # Custom type w/ custom __hash__ and __eq__
5894    class A:
5895        def __init__(self, val):
5896            self.val = val
5897
5898        def __hash__(self):
5899            return 3  # hashes collide
5900
5901        def __eq__(self, other):
5902            return isinstance(other, A) and self.val == other.val
5903
5904    assert compare({A(1), A(2)}, {A(1), A(2)})
5905    assert compare({A(1), A(2)}, {A(1), A(3)}) is False

Tests the compare function.

def detailLevel(level) 
5912def detailLevel(level):
5913    """
5914    Sets the level of detail for printed messages.
5915    The detail levels are:
5916
5917    * -1: Super-minimal output, with no details beyond success/failure.
5918    * 0: Succinct messages indicating success/failure, with minimal
5919        details when failure occurs.
5920    * 1: More verbose success/failure messages, with details about
5921        successes and more details about failures.
5922    """
5923    global DETAIL_LEVEL
5924    DETAIL_LEVEL = level

Sets the level of detail for printed messages. The detail levels are:

  • -1: Super-minimal output, with no details beyond success/failure.
  • 0: Succinct messages indicating success/failure, with minimal details when failure occurs.
  • 1: More verbose success/failure messages, with details about successes and more details about failures.
def attendTrailingWhitespace(on=True) 
5927def attendTrailingWhitespace(on=True):
5928    """
5929    Call this function to force `optimism` to pay attention to
5930    whitespace at the end of lines when checking expectations. By
5931    default, such whitespace is removed both from expected
5932    values/output fragments and from captured outputs/results before
5933    checking expectations. To turn that functionality on again, you
5934    can call this function with False as the argument.
5935    """
5936    global IGNORE_TRAILING_WHITESPACE
5937    IGNORE_TRAILING_WHITESPACE = not on

Call this function to force optimism to pay attention to whitespace at the end of lines when checking expectations. By default, such whitespace is removed both from expected values/output fragments and from captured outputs/results before checking expectations. To turn that functionality on again, you can call this function with False as the argument.

def skipChecksAfterFail(mode='all') 
5940def skipChecksAfterFail(mode="all"):
5941    """
5942    The argument should be either 'case' (the default), 'manager', or
5943    None. In 'manager' mode, when one check fails, any other checks of
5944    cases derived from that manager, including the case where the check
5945    failed, will be skipped. In 'case' mode, once a check fails any
5946    further checks of the same case will be skipped, but checks of other
5947    cases derived from the same manager will not be. In None mode (or if
5948    any other value is provided) no checks will be skipped because of
5949    failed checks (but they might be skipped for other reasons).
5950    """
5951    global SKIP_ON_FAILURE
5952    SKIP_ON_FAILURE = mode

The argument should be either 'case' (the default), 'manager', or None. In 'manager' mode, when one check fails, any other checks of cases derived from that manager, including the case where the check failed, will be skipped. In 'case' mode, once a check fails any further checks of the same case will be skipped, but checks of other cases derived from the same manager will not be. In None mode (or if any other value is provided) no checks will be skipped because of failed checks (but they might be skipped for other reasons).

def suppressErrorDetailsAfterFail(mode='all') 
5955def suppressErrorDetailsAfterFail(mode="all"):
5956    """
5957    The argument should be one of the following values:
5958
5959    - `'case'`: Causes error details to be omitted for failed checks
5960      after the first failed check on each particular test case.
5961    - `'manager'`: Causes error details to be omitted for failed checks
5962      after the first failed check on any test case for a particular
5963      manager.
5964    - `'all'`: Causes error details to be omitted for all failed checks
5965      after any check fails. Reset this with `clearFailure`.
5966    - None (or any other value not listed above): Means that full error
5967      details will always be reported.
5968
5969    The default value is `'all`' if you call this function; see
5970    `SUPPRESS_ON_FAILURE` for the default value when `optimism` is
5971    imported.
5972
5973    Note that detail suppression has no effect if the detail level is set
5974    above 0.
5975    """
5976    global SUPPRESS_ON_FAILURE
5977    SUPPRESS_ON_FAILURE = mode

The argument should be one of the following values:

  • 'case': Causes error details to be omitted for failed checks after the first failed check on each particular test case.
  • 'manager': Causes error details to be omitted for failed checks after the first failed check on any test case for a particular manager.
  • 'all': Causes error details to be omitted for all failed checks after any check fails. Reset this with clearFailure.
  • None (or any other value not listed above): Means that full error details will always be reported.

The default value is 'all' if you call this function; see SUPPRESS_ON_FAILURE for the default value when optimism is imported.

Note that detail suppression has no effect if the detail level is set above 0.

def clearFailure() 
5980def clearFailure():
5981    """
5982    Resets the failure status so that checks will resume when
5983    `SKIP_ON_FAILURE` is set to `'all'`.
5984    """
5985    global CHECK_FAILED
5986    CHECK_FAILED = False

Resets the failure status so that checks will resume when SKIP_ON_FAILURE is set to 'all'.

def showSummary(suiteName=None) 
6005def showSummary(suiteName=None):
6006    """
6007    Shows a summary of the number of checks in the current test suite
6008    (see `currentTestSuite`) that have been met or not. You can also
6009    give an argument to specify the name of the test suite to summarize.
6010    Prints output to sys.stderr.
6011
6012    Note that the results of `expect` checks are not included in the
6013    summary, because they aren't trials.
6014    """
6015    # Flush stdout, stderr, and PRINT_TO to improve ordering
6016    sys.stdout.flush()
6017    sys.stderr.flush()
6018    try:
6019        PRINT_TO.flush()
6020    except Exception:
6021        pass
6022
6023    met = []
6024    unmet = []
6025    for passed, tag, msg in listOutcomesInSuite(suiteName):
6026        if passed:
6027            met.append(tag)
6028        else:
6029            unmet.append(tag)
6030
6031    print('---', file=PRINT_TO)
6032
6033    if len(unmet) == 0:
6034        if len(met) == 0:
6035            print("No expectations were established.", file=PRINT_TO)
6036        else:
6037            print(
6038                f"All {len(met)} expectation(s) were met.",
6039                file=PRINT_TO
6040            )
6041    else:
6042        if len(met) == 0:
6043            print(
6044                f"None of the {len(unmet)} expectation(s) were met!",
6045                file=PRINT_TO
6046            )
6047        else:
6048            print(
6049                (
6050                    f"{len(unmet)} of the {len(met) + len(unmet)}"
6051                    f" expectation(s) were NOT met:"
6052                ),
6053                file=PRINT_TO
6054            )
6055        if COLORS:  # bright red
6056            print("\x1b[1;31m", end="", file=PRINT_TO)
6057        for tag in unmet:
6058            print(f"  ✗ {tag}", file=PRINT_TO)
6059        if COLORS:  # reset
6060            print("\x1b[0m", end="", file=PRINT_TO)
6061    print('---', file=PRINT_TO)
6062
6063    # Flush stdout & stderr to improve ordering
6064    sys.stdout.flush()
6065    sys.stderr.flush()
6066    try:
6067        PRINT_TO.flush()
6068    except Exception:
6069        pass

Shows a summary of the number of checks in the current test suite (see currentTestSuite) that have been met or not. You can also give an argument to specify the name of the test suite to summarize. Prints output to sys.stderr.

Note that the results of expect checks are not included in the summary, because they aren't trials.

def currentTestSuite() 
6072def currentTestSuite():
6073    """
6074    Returns the name of the current test suite (a string).
6075    """
6076    return _CURRENT_SUITE_NAME

Returns the name of the current test suite (a string).

def testSuite(name) 
6079def testSuite(name):
6080    """
6081    Starts a new test suite with the given name, or resumes an old one.
6082    Any cases created subsequently will be registered to that suite.
6083    """
6084    global _CURRENT_SUITE_NAME
6085    if not isinstance(name, str):
6086        raise TypeError(
6087            f"The test suite name must be a string (got: '{repr(name)}'"
6088            f" which is a {type(name)})."
6089        )
6090    _CURRENT_SUITE_NAME = name

Starts a new test suite with the given name, or resumes an old one. Any cases created subsequently will be registered to that suite.

def resetTestSuite(suiteName=None) 
6093def resetTestSuite(suiteName=None):
6094    """
6095    Resets the cases and outcomes recorded in the current test suite (or
6096    the named test suite if an argument is provided).
6097    """
6098    if suiteName is None:
6099        suiteName = currentTestSuite()
6100
6101    ALL_TRIALS[suiteName] = []
6102    ALL_OUTCOMES[suiteName] = []

Resets the cases and outcomes recorded in the current test suite (or the named test suite if an argument is provided).

def freshTestSuite(name) 
6105def freshTestSuite(name):
6106    """
6107    Works like `testSuite`, but calls `resetTestSuite` for that suite
6108    name first, ensuring no old test suite contents will be included.
6109    """
6110    resetTestSuite(name)
6111    testSuite(name)

Works like testSuite, but calls resetTestSuite for that suite name first, ensuring no old test suite contents will be included.

def deleteAllTestSuites() 
6114def deleteAllTestSuites():
6115    """
6116    Deletes all test suites, removing all recorded test cases and
6117    outcomes, and setting the current test suite name back to "default".
6118    """
6119    global ALL_TRIALS, ALL_OUTCOMES, _CURRENT_SUITE_NAME
6120    _CURRENT_SUITE_NAME = "default"
6121    ALL_TRIALS = {}
6122    ALL_OUTCOMES = {}

Deletes all test suites, removing all recorded test cases and outcomes, and setting the current test suite name back to "default".

def listTrialsInSuite(suiteName=None) 
6125def listTrialsInSuite(suiteName=None):
6126    """
6127    Returns a list of trials (`Trial` objects) in the current test suite
6128    (or the named suite if an argument is provided).
6129    """
6130    if suiteName is None:
6131        suiteName = currentTestSuite()
6132
6133    if suiteName not in ALL_TRIALS:
6134        raise ValueError(f"Test suite '{suiteName}' does not exist.")
6135
6136    return ALL_TRIALS[suiteName][:]

Returns a list of trials (Trial objects) in the current test suite (or the named suite if an argument is provided).

def listOutcomesInSuite(suiteName=None) 
6139def listOutcomesInSuite(suiteName=None):
6140    """
6141    Returns a list of all individual expectation outcomes attached to
6142    trials in the given test suite (default: the current test suite).
6143    Includes `expect` and `expectType` outcomes even though those aren't
6144    attached to trials.
6145    """
6146    if suiteName is None:
6147        suiteName = currentTestSuite()
6148
6149    if suiteName not in ALL_OUTCOMES:
6150        raise ValueError(f"Test suite '{suiteName}' does not exit.")
6151
6152    return ALL_OUTCOMES[suiteName][:]

Returns a list of all individual expectation outcomes attached to trials in the given test suite (default: the current test suite). Includes expect and expectType outcomes even though those aren't attached to trials.

def listAllTrials() 
6155def listAllTrials():
6156    """
6157    Returns a list of all registered trials (`Trial` objects) in any
6158    known test suite. Note that if `deleteAllTestSuites` has been called,
6159    this will not include any `Trial` objects created before that point.
6160    """
6161    result = []
6162    for suiteName in ALL_TRIALS:
6163        result.extend(ALL_TRIALS[suiteName])
6164
6165    return result

Returns a list of all registered trials (Trial objects) in any known test suite. Note that if deleteAllTestSuites has been called, this will not include any Trial objects created before that point.

def colors(enable=False) 
6172def colors(enable=False):
6173    """
6174    Enables or disables colors in printed output. If your output does not
6175    support ANSI color codes, the color output will show up as garbage
6176    and you can disable this.
6177    """
6178    global COLORS
6179    COLORS = enable

Enables or disables colors in printed output. If your output does not support ANSI color codes, the color output will show up as garbage and you can disable this.

def trace(expr) 
6186def trace(expr):
6187    """
6188    Given an expression (actually, of course, just a value), returns the
6189    value it was given. But also prints a trace message indicating what
6190    the expression was, what value it had, and the line number of that
6191    line of code.
6192
6193    The file name and overlength results are printed only when the
6194    `detailLevel` is set to 1 or higher.
6195    """
6196    # Flush stdout & stderr to improve ordering
6197    sys.stdout.flush()
6198    sys.stderr.flush()
6199    try:
6200        PRINT_TO.flush()
6201    except Exception:
6202        pass
6203
6204    ctx = get_my_context(trace)
6205    rep = repr(expr)
6206    short = ellipsis(repr(expr))
6207    tag = "{line}".format(**ctx)
6208    if DETAIL_LEVEL >= 1:
6209        tag = "{file}:{line}".format(**ctx)
6210    print(
6211        f"{tag} {ctx.get('expr_src', '???')} ⇒ {short}",
6212        file=PRINT_TO
6213    )
6214    if DETAIL_LEVEL >= 1 and short != rep:
6215        print("  Full result is:\n    " + rep, file=PRINT_TO)
6216
6217    # Flush stdout & stderr to improve ordering
6218    sys.stdout.flush()
6219    sys.stderr.flush()
6220    try:
6221        PRINT_TO.flush()
6222    except Exception:
6223        pass
6224
6225    return expr

Given an expression (actually, of course, just a value), returns the value it was given. But also prints a trace message indicating what the expression was, what value it had, and the line number of that line of code.

The file name and overlength results are printed only when the detailLevel is set to 1 or higher.

def get_src_index(src, lineno, col_offset) 
6232def get_src_index(src, lineno, col_offset):
6233    """
6234    Turns a line number and column offset into an absolute index into
6235    the given source string, assuming length-1 newlines.
6236    """
6237    lines = src.splitlines()
6238    above = lines[:lineno - 1]
6239    return sum(len(line) for line in above) + len(above) + col_offset

Turns a line number and column offset into an absolute index into the given source string, assuming length-1 newlines.

def test_gsr() 
6242def test_gsr():
6243    """Tests for get_src_index."""
6244    s = 'a\nb\nc'
6245    assert get_src_index(s, 1, 0) == 0
6246    assert get_src_index(s, 2, 0) == 2
6247    assert get_src_index(s, 3, 0) == 4
6248    assert s[get_src_index(s, 1, 0)] == 'a'
6249    assert s[get_src_index(s, 2, 0)] == 'b'
6250    assert s[get_src_index(s, 3, 0)] == 'c'

Tests for get_src_index.

def find_identifier_end(code, start_index) 
6253def find_identifier_end(code, start_index):
6254    """
6255    Given a code string and an index in that string which is the start
6256    of an identifier, returns the index of the end of that identifier.
6257    """
6258    at = start_index + 1
6259    while at < len(code):
6260        ch = code[at]
6261        if not ch.isalpha() and not ch.isdigit() and ch != '_':
6262            break
6263        at += 1
6264    return at - 1

Given a code string and an index in that string which is the start of an identifier, returns the index of the end of that identifier.

def test_find_identifier_end() 
6267def test_find_identifier_end():
6268    """Tests for find_identifier_end."""
6269    assert find_identifier_end("abc.xyz", 0) == 2
6270    assert find_identifier_end("abc.xyz", 1) == 2
6271    assert find_identifier_end("abc.xyz", 2) == 2
6272    assert find_identifier_end("abc.xyz", 4) == 6
6273    assert find_identifier_end("abc.xyz", 5) == 6
6274    assert find_identifier_end("abc.xyz", 6) == 6
6275    assert find_identifier_end("abc_xyz123", 0) == 9
6276    assert find_identifier_end("abc xyz123", 0) == 2
6277    assert find_identifier_end("abc xyz123", 4) == 9
6278    assert find_identifier_end("x", 0) == 0
6279    assert find_identifier_end("  x", 2) == 2
6280    assert find_identifier_end("  xyz1", 2) == 5
6281    s = "def abc(def):\n  print(xyz)\n"
6282    assert find_identifier_end(s, 0) == 2
6283    assert find_identifier_end(s, 4) == 6
6284    assert find_identifier_end(s, 8) == 10
6285    assert find_identifier_end(s, 16) == 20
6286    assert find_identifier_end(s, 22) == 24

Tests for find_identifier_end.

def unquoted_enumerate(src, start_index) 
6289def unquoted_enumerate(src, start_index):
6290    """
6291    A generator that yields index, character pairs from the given code
6292    string, skipping quotation marks and the strings that they delimit,
6293    including triple-quotes and respecting backslash-escapes within
6294    strings.
6295    """
6296    quote = None
6297    at = start_index
6298
6299    while at < len(src):
6300        char = src[at]
6301
6302        # skip escaped characters in quoted strings
6303        if quote and char == '\\':
6304            # (thank goodness I don't have to worry about r-strings)
6305            at += 2
6306            continue
6307
6308        # handle quoted strings
6309        elif char == '"' or char == "'":
6310            if quote == char:
6311                quote = None  # single end quote
6312                at += 1
6313                continue
6314            elif src[at:at + 3] in ('"""', "'''"):
6315                tq = src[at:at + 3]
6316                at += 3  # going to skip these no matter what
6317                if tq == quote or tq[0] == quote:
6318                    # Ending triple-quote, or matching triple-quote at
6319                    # end of single-quoted string = ending quote +
6320                    # empty string
6321                    quote = None
6322                    continue
6323                else:
6324                    if quote:
6325                        # triple quote of other kind inside single or
6326                        # triple quoted string
6327                        continue
6328                    else:
6329                        quote = tq
6330                        continue
6331            elif quote is None:
6332                # opening single quote
6333                quote = char
6334                at += 1
6335                continue
6336            else:
6337                # single quote inside other quotes
6338                at += 1
6339                continue
6340
6341        # Non-quote characters in quoted strings
6342        elif quote:
6343            at += 1
6344            continue
6345
6346        else:
6347            yield (at, char)
6348            at += 1
6349            continue

A generator that yields index, character pairs from the given code string, skipping quotation marks and the strings that they delimit, including triple-quotes and respecting backslash-escapes within strings.

def test_unquoted_enumerate() 
6352def test_unquoted_enumerate():
6353    """Tests for unquoted_enumerate."""
6354    uqe = unquoted_enumerate
6355    assert list(uqe("abc'123'", 0)) == list(zip(range(3), "abc"))
6356    assert list(uqe("'abc'123", 0)) == list(zip(range(5, 8), "123"))
6357    assert list(uqe("'abc'123''", 0)) == list(zip(range(5, 8), "123"))
6358    assert list(uqe("'abc'123''", 1)) == [(1, 'a'), (2, 'b'), (3, 'c')]
6359    mls = "'''\na\nb\nc'''\ndef"
6360    assert list(uqe(mls, 0)) == list(zip(range(12, 16), "\ndef"))
6361    tqs = '"""\'\'\'ab\'\'\'\'""" cd'
6362    assert list(uqe(tqs, 0)) == [(15, ' '), (16, 'c'), (17, 'd')]
6363    rqs = "a'b'''c\"\"\"'''\"d\"''''\"\"\"e'''\"\"\"f\"\"\"'''"
6364    assert list(uqe(rqs, 0)) == [(0, 'a'), (6, 'c'), (23, 'e')]
6365    assert list(uqe(rqs, 6)) == [(6, 'c'), (23, 'e')]
6366    bss = "a'\\'b\\''c"
6367    assert list(uqe(bss, 0)) == [(0, 'a'), (8, 'c')]
6368    mqs = "'\"a'b\""
6369    assert list(uqe(mqs, 0)) == [(4, 'b')]

Tests for unquoted_enumerate.

def find_nth_attribute_period(code, start_index, n) 
6372def find_nth_attribute_period(code, start_index, n):
6373    """
6374    Given a string of Python code and a start index within that string,
6375    finds the nth period character (counting from first = zero) after
6376    that start point, but only considers periods which are used for
6377    attribute access, i.e., periods outside of quoted strings and which
6378    are not part of ellipses. Returns the index within the string of the
6379    period that it found. A period at the start index (if there is one)
6380    will be counted. Returns None if there are not enough periods in the
6381    code. If the start index is inside a quoted string, things will get
6382    weird, and the results will probably be wrong.
6383    """
6384    for (at, char) in unquoted_enumerate(code, start_index):
6385        if char == '.':
6386            if code[at - 1:at] == '.' or code[at + 1:at + 2] == '.':
6387                # part of an ellipsis, so ignore it
6388                continue
6389            else:
6390                n -= 1
6391                if n < 0:
6392                    break
6393
6394    # Did we hit the end of the string before counting below 0?
6395    if n < 0:
6396        return at
6397    else:
6398        return None

Given a string of Python code and a start index within that string, finds the nth period character (counting from first = zero) after that start point, but only considers periods which are used for attribute access, i.e., periods outside of quoted strings and which are not part of ellipses. Returns the index within the string of the period that it found. A period at the start index (if there is one) will be counted. Returns None if there are not enough periods in the code. If the start index is inside a quoted string, things will get weird, and the results will probably be wrong.

def test_find_nth_attribute_period() 
6401def test_find_nth_attribute_period():
6402    """Tests for find_nth_attribute_period."""
6403    assert find_nth_attribute_period("a.b", 0, 0) == 1
6404    assert find_nth_attribute_period("a.b", 0, 1) is None
6405    assert find_nth_attribute_period("a.b", 0, 100) is None
6406    assert find_nth_attribute_period("a.b.c", 0, 1) == 3
6407    assert find_nth_attribute_period("a.b.cde.f", 0, 1) == 3
6408    assert find_nth_attribute_period("a.b.cde.f", 0, 2) == 7
6409    s = "a.b, c.d, 'e.f', g.h"
6410    assert find_nth_attribute_period(s, 0, 0) == 1
6411    assert find_nth_attribute_period(s, 0, 1) == 6
6412    assert find_nth_attribute_period(s, 0, 2) == 18
6413    assert find_nth_attribute_period(s, 0, 3) is None
6414    assert find_nth_attribute_period(s, 0, 3) is None
6415    assert find_nth_attribute_period(s, 1, 0) == 1
6416    assert find_nth_attribute_period(s, 2, 0) == 6
6417    assert find_nth_attribute_period(s, 6, 0) == 6
6418    assert find_nth_attribute_period(s, 7, 0) == 18
6419    assert find_nth_attribute_period(s, 15, 0) == 18

Tests for find_nth_attribute_period.

def find_closing_item(code, start_index, openclose='()') 
6422def find_closing_item(code, start_index, openclose='()'):
6423    """
6424    Given a string of Python code, a starting index where there's an
6425    open paren, bracket, etc., and a 2-character string containing the
6426    opening and closing delimiters of interest (parentheses by default),
6427    returns the index of the matching closing delimiter, or None if the
6428    opening delimiter is unclosed. Note that the given code must not
6429    contain syntax errors, or the behavior will be undefined.
6430
6431    Does NOT work with quotation marks (single or double).
6432    """
6433    level = 1
6434    open_delim = openclose[0]
6435    close_delim = openclose[1]
6436    for at, char in unquoted_enumerate(code, start_index + 1):
6437        # Non-quoted open delimiters
6438        if char == open_delim:
6439            level += 1
6440
6441        # Non-quoted close delimiters
6442        elif char == close_delim:
6443            level -= 1
6444            if level < 1:
6445                break
6446
6447        # Everything else: ignore it
6448
6449    if level == 0:
6450        return at
6451    else:
6452        return None

Given a string of Python code, a starting index where there's an open paren, bracket, etc., and a 2-character string containing the opening and closing delimiters of interest (parentheses by default), returns the index of the matching closing delimiter, or None if the opening delimiter is unclosed. Note that the given code must not contain syntax errors, or the behavior will be undefined.

Does NOT work with quotation marks (single or double).

def test_find_closing_item() 
6455def test_find_closing_item():
6456    """Tests for find_closing_item."""
6457    assert find_closing_item('()', 0, '()') == 1
6458    assert find_closing_item('()', 0) == 1
6459    assert find_closing_item('(())', 0, '()') == 3
6460    assert find_closing_item('(())', 1, '()') == 2
6461    assert find_closing_item('((word))', 0, '()') == 7
6462    assert find_closing_item('((word))', 1, '()') == 6
6463    assert find_closing_item('(("(("))', 0, '()') == 7
6464    assert find_closing_item('(("(("))', 1, '()') == 6
6465    assert find_closing_item('(("))"))', 0, '()') == 7
6466    assert find_closing_item('(("))"))', 1, '()') == 6
6467    assert find_closing_item('(()())', 0, '()') == 5
6468    assert find_closing_item('(()())', 1, '()') == 2
6469    assert find_closing_item('(()())', 3, '()') == 4
6470    assert find_closing_item('(""")(\n""")', 0, '()') == 10
6471    assert find_closing_item("\"abc(\" + ('''def''')", 9, '()') == 19
6472    assert find_closing_item("\"abc(\" + ('''def''')", 0, '()') is None
6473    assert find_closing_item("\"abc(\" + ('''def''')", 4, '()') is None
6474    assert find_closing_item("(()", 0, '()') is None
6475    assert find_closing_item("(()", 1, '()') == 2
6476    assert find_closing_item("()(", 0, '()') == 1
6477    assert find_closing_item("()(", 2, '()') is None
6478    assert find_closing_item("[]", 0, '[]') == 1
6479    assert find_closing_item("[]", 0) is None
6480    assert find_closing_item("{}", 0, '{}') == 1
6481    assert find_closing_item("aabb", 0, 'ab') == 3

Tests for find_closing_item.

def find_unbracketed_comma(code, start_index) 
6484def find_unbracketed_comma(code, start_index):
6485    """
6486    Given a string of Python code and a starting index, finds the next
6487    comma at or after that index which isn't surrounded by brackets of
6488    any kind that start at or after that index and which isn't in a
6489    quoted string. Returns the index of the matching comma, or None if
6490    there is none. Stops and returns None if it finds an unmatched
6491    closing bracket. Note that the given code must not contain syntax
6492    errors, or the behavior will be undefined.
6493    """
6494    seeking = []
6495    delims = {
6496        '(': ')',
6497        '[': ']',
6498        '{': '}'
6499    }
6500    closing = delims.values()
6501    for at, char in unquoted_enumerate(code, start_index):
6502        # Non-quoted open delimiter
6503        if char in delims:
6504            seeking.append(delims[char])
6505
6506        # Non-quoted matching close delimiter
6507        elif len(seeking) > 0 and char == seeking[-1]:
6508            seeking.pop()
6509
6510        # Non-quoted non-matching close delimiter
6511        elif char in closing:
6512            return None
6513
6514        # A non-quoted comma
6515        elif char == ',' and len(seeking) == 0:
6516            return at
6517
6518        # Everything else: ignore it
6519
6520    # Got to the end
6521    return None

Given a string of Python code and a starting index, finds the next comma at or after that index which isn't surrounded by brackets of any kind that start at or after that index and which isn't in a quoted string. Returns the index of the matching comma, or None if there is none. Stops and returns None if it finds an unmatched closing bracket. Note that the given code must not contain syntax errors, or the behavior will be undefined.

def test_find_unbracketed_comma() 
6524def test_find_unbracketed_comma():
6525    """Tests for find_unbracketed_comma."""
6526    assert find_unbracketed_comma('()', 0) is None
6527    assert find_unbracketed_comma('(),', 0) == 2
6528    assert find_unbracketed_comma('((,),)', 0) is None
6529    assert find_unbracketed_comma('((,),),', 0) == 6
6530    assert find_unbracketed_comma('((,),),', 1) == 4
6531    assert find_unbracketed_comma(',,,', 1) == 1
6532    assert find_unbracketed_comma('",,",","', 0) == 4
6533    assert find_unbracketed_comma('"""\n,,\n""","""\n,,\n"""', 0) == 10
6534    assert find_unbracketed_comma('"""\n,,\n""","""\n,,\n"""', 4) == 4
6535    assert find_unbracketed_comma('"""\n,,\n"""+"""\n,,\n"""', 0) is None
6536    assert find_unbracketed_comma('\n\n,\n', 0) == 2

Tests for find_unbracketed_comma.

def get_expr_src(src, call_node) 
6539def get_expr_src(src, call_node):
6540    """
6541    Gets the string containing the source code for the expression passed
6542    as the first argument to a function call, given the string source of
6543    the file that defines the function and the AST node for the function
6544    call.
6545    """
6546    # Find the child node for the first (and only) argument
6547    arg_expr = call_node.args[0]
6548
6549    # If get_source_segment is available, use that
6550    if hasattr(ast, "get_source_segment"):
6551        return textwrap.dedent(
6552            ast.get_source_segment(src, arg_expr)
6553        ).strip()
6554    else:
6555        # We're going to have to do this ourself: find the start of the
6556        # expression and state-machine to find a matching paren
6557        start = get_src_index(src, call_node.lineno, call_node.col_offset)
6558        open_paren = src.index('(', start)
6559        end = find_closing_item(src, open_paren, '()')
6560        # Note: can't be None because that would have been a SyntaxError
6561        first_comma = find_unbracketed_comma(src, open_paren + 1)
6562        # Could be None if it's a 1-argument function
6563        if first_comma is not None:
6564            end = min(end, first_comma)
6565        return textwrap.dedent(src[open_paren + 1:end]).strip()

Gets the string containing the source code for the expression passed as the first argument to a function call, given the string source of the file that defines the function and the AST node for the function call.

def get_ref_src(src, node) 
6568def get_ref_src(src, node):
6569    """
6570    Gets the string containing the source code for a variable reference,
6571    attribute, or subscript.
6572    """
6573    # Use get_source_segment if it's available
6574    if hasattr(ast, "get_source_segment"):
6575        return ast.get_source_segment(src, node)
6576    else:
6577        # We're going to have to do this ourself: find the start of the
6578        # expression and state-machine to find its end
6579        start = get_src_index(src, node.lineno, node.col_offset)
6580
6581        # Figure out the end point
6582        if isinstance(node, ast.Attribute):
6583            # Find sub-attributes so we can count syntactic periods to
6584            # figure out where the name part begins to get the span
6585            inner_period_count = 0
6586            for node in ast.walk(node):
6587                if isinstance(node, ast.Attribute):
6588                    inner_period_count += 1
6589            inner_period_count -= 1  # for the node itself
6590            dot = find_nth_attribute_period(src, start, inner_period_count)
6591            end = find_identifier_end(src, dot + 1)
6592
6593        elif isinstance(node, ast.Name):
6594            # It's just an identifier so we can find the end
6595            end = find_identifier_end(src, start)
6596
6597        elif isinstance(node, ast.Subscript):
6598            # Find start of sub-expression so we can find opening brace
6599            # and then match it to find the end
6600            inner = node.slice
6601            if isinstance(inner, ast.Slice):
6602                pass
6603            elif hasattr(ast, "Index") and isinstance(inner, ast.Index):
6604                # 3.7 Index has a "value"
6605                inner = inner.value
6606            elif hasattr(ast, "ExtSlice") and isinstance(inner, ast.ExtSlice):
6607                # 3.7 ExtSlice has "dims"
6608                inner = inner.dims[0]
6609            else:
6610                raise TypeError(
6611                    f"Unexpected subscript slice type {type(inner)} for"
6612                    f" node:\n{ast.dump(node)}"
6613                )
6614            sub_start = get_src_index(src, inner.lineno, inner.col_offset)
6615            end = find_closing_item(src, sub_start - 1, "[]")
6616
6617        return src[start:end + 1]

Gets the string containing the source code for a variable reference, attribute, or subscript.

def deepish_copy(obj, memo=None) 
6620def deepish_copy(obj, memo=None):
6621    """
6622    Returns the deepest possible copy of the given object, using
6623    copy.deepcopy wherever possible and making shallower copies
6624    elsewhere. Basically a middle-ground between copy.deepcopy and
6625    copy.copy.
6626    """
6627    if memo is None:
6628        memo = {}
6629    if id(obj) in memo:
6630        return memo[id(obj)]
6631
6632    try:
6633        result = copy.deepcopy(obj)  # not sure about memo dict compatibility
6634        memo[id(obj)] = result
6635        return result
6636
6637    except Exception:
6638        if isinstance(obj, list):
6639            result = []
6640            memo[id(obj)] = result
6641            result.extend(deepish_copy(item, memo) for item in obj)
6642            return result
6643        elif isinstance(obj, tuple):
6644            # Note: no way to pre-populate the memo, but also no way to
6645            # construct an infinitely-recursive tuple without having
6646            # some mutable structure at some layer...
6647            result = (deepish_copy(item, memo) for item in obj)
6648            memo[id(obj)] = result
6649            return result
6650        elif isinstance(obj, dict):
6651            result = {}
6652            memo[id(obj)] = result
6653            result.update(
6654                {
6655                    deepish_copy(key, memo): deepish_copy(value, memo)
6656                    for key, value in obj.items()
6657                }
6658            )
6659            return result
6660        elif isinstance(obj, set):
6661            result = set()
6662            memo[id(obj)] = result
6663            result |= set(deepish_copy(item, memo) for item in obj)
6664            return result
6665        else:
6666            # Can't go deeper I guess
6667            try:
6668                result = copy.copy(obj)
6669                memo[id(obj)] = result
6670                return result
6671            except Exception:
6672                # Can't even copy (e.g., a module)
6673                result = obj
6674                memo[id(obj)] = result
6675                return result

Returns the deepest possible copy of the given object, using copy.deepcopy wherever possible and making shallower copies elsewhere. Basically a middle-ground between copy.deepcopy and copy.copy.

def get_external_calling_frame() 
6678def get_external_calling_frame():
6679    """
6680    Uses the inspect module to get a reference to the stack frame which
6681    called into the `optimism` module. Returns None if it can't find an
6682    appropriate call frame in the current stack.
6683
6684    Remember to del the result after you're done with it, so that
6685    garbage doesn't pile up.
6686    """
6687    myname = __name__
6688    cf = inspect.currentframe()
6689    while (
6690        hasattr(cf, "f_back")
6691    and cf.f_globals.get("__name__") == myname
6692    ):
6693        cf = cf.f_back
6694
6695    return cf

Uses the inspect module to get a reference to the stack frame which called into the optimism module. Returns None if it can't find an appropriate call frame in the current stack.

Remember to del the result after you're done with it, so that garbage doesn't pile up.

def get_module(stack_frame) 
6698def get_module(stack_frame):
6699    """
6700    Given a stack frame, returns a reference to the module where the
6701    code from that frame was defined.
6702
6703    Returns None if it can't figure that out.
6704    """
6705    other_name = stack_frame.f_globals.get("__name__", None)
6706    return sys.modules.get(other_name)

Given a stack frame, returns a reference to the module where the code from that frame was defined.

Returns None if it can't figure that out.

def get_filename(stack_frame, speculate_filename=True) 
6709def get_filename(stack_frame, speculate_filename=True):
6710    """
6711    Given a stack frame, returns the filename of the file in which the
6712    code which created that stack frame was defined. Returns None if
6713    that information isn't available via a __file__ global, or if
6714    speculate_filename is True (the default), uses the value of the
6715    frame's f_code.co_filename, which may not always be a real file on
6716    disk, or which is weird circumstances could be the name of a file on
6717    disk which is *not* where the code came from.
6718    """
6719    filename = stack_frame.f_globals.get("__file__")
6720    if filename is None and speculate_filename:
6721        filename = stack_frame.f_code.co_filename
6722    return filename

Given a stack frame, returns the filename of the file in which the code which created that stack frame was defined. Returns None if that information isn't available via a __file__ global, or if speculate_filename is True (the default), uses the value of the frame's f_code.co_filename, which may not always be a real file on disk, or which is weird circumstances could be the name of a file on disk which is not where the code came from.

def get_code_line(stack_frame) 
6725def get_code_line(stack_frame):
6726    """
6727    Given a stack frame, returns
6728    """
6729    return stack_frame.f_lineno

Given a stack frame, returns

def evaluate_in_context(node, stack_frame) 
6732def evaluate_in_context(node, stack_frame):
6733    """
6734    Given an AST node which is an expression, returns the value of that
6735    expression as evaluated in the context of the given stack frame.
6736
6737    Shallow copies of the stack frame's locals and globals are made in
6738    an attempt to prevent the code being evaluated from having any
6739    impact on the stack frame's values, but of course there's still some
6740    possibility of side effects...
6741    """
6742    expr = ast.Expression(node)
6743    code = compile(
6744        expr,
6745        stack_frame.f_globals.get("__file__", "__unknown__"),
6746        'eval'
6747    )
6748    return eval(
6749        code,
6750        copy.copy(stack_frame.f_globals),
6751        copy.copy(stack_frame.f_locals)
6752    )

Given an AST node which is an expression, returns the value of that expression as evaluated in the context of the given stack frame.

Shallow copies of the stack frame's locals and globals are made in an attempt to prevent the code being evaluated from having any impact on the stack frame's values, but of course there's still some possibility of side effects...

def walk_ast_in_order(node) 
6755def walk_ast_in_order(node):
6756    """
6757    Yields all of the descendants of the given node (or list of nodes)
6758    in execution order. Note that this has its limits, for example, if
6759    we run it on the code:
6760
6761    ```py
6762    x = [A for y in C if D]
6763    ```
6764
6765    It will yield the nodes for C, then y, then D, then A, and finally
6766    x, but in actual execution the nodes for D and A may be executed
6767    multiple times before x is assigned.
6768    """
6769    if node is None:
6770        pass  # empty iterator
6771    elif isinstance(node, (list, tuple)):
6772        for child in node:
6773            yield from walk_ast_in_order(child)
6774    else:  # must be an ast.something
6775        # Note: the node itself will be yielded LAST
6776        if isinstance(node, (ast.Module, ast.Interactive, ast.Expression)):
6777            yield from walk_ast_in_order(node.body)
6778        elif (
6779            hasattr(ast, "FunctionType")
6780        and isinstance(node, ast.FunctionType)
6781        ):
6782            yield from walk_ast_in_order(node.argtypes)
6783            yield from walk_ast_in_order(node.returns)
6784        elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
6785            yield from walk_ast_in_order(node.args)
6786            yield from walk_ast_in_order(node.returns)
6787            yield from walk_ast_in_order(reversed(node.decorator_list))
6788            yield from walk_ast_in_order(node.body)
6789        elif isinstance(node, ast.ClassDef):
6790            yield from walk_ast_in_order(node.bases)
6791            yield from walk_ast_in_order(node.keywords)
6792            yield from walk_ast_in_order(reversed(node.decorator_list))
6793            yield from walk_ast_in_order(node.body)
6794        elif isinstance(node, ast.Return):
6795            yield from walk_ast_in_order(node.value)
6796        elif isinstance(node, ast.Delete):
6797            yield from walk_ast_in_order(node.targets)
6798        elif isinstance(node, ast.Assign):
6799            yield from walk_ast_in_order(node.value)
6800            yield from walk_ast_in_order(node.targets)
6801        elif isinstance(node, ast.AugAssign):
6802            yield from walk_ast_in_order(node.value)
6803            yield from walk_ast_in_order(node.target)
6804        elif isinstance(node, ast.AnnAssign):
6805            yield from walk_ast_in_order(node.value)
6806            yield from walk_ast_in_order(node.annotation)
6807            yield from walk_ast_in_order(node.target)
6808        elif isinstance(node, (ast.For, ast.AsyncFor)):
6809            yield from walk_ast_in_order(node.iter)
6810            yield from walk_ast_in_order(node.target)
6811            yield from walk_ast_in_order(node.body)
6812            yield from walk_ast_in_order(node.orelse)
6813        elif isinstance(node, (ast.While, ast.If, ast.IfExp)):
6814            yield from walk_ast_in_order(node.test)
6815            yield from walk_ast_in_order(node.body)
6816            yield from walk_ast_in_order(node.orelse)
6817        elif isinstance(node, (ast.With, ast.AsyncWith)):
6818            yield from walk_ast_in_order(node.items)
6819            yield from walk_ast_in_order(node.body)
6820        elif isinstance(node, ast.Raise):
6821            yield from walk_ast_in_order(node.cause)
6822            yield from walk_ast_in_order(node.exc)
6823        elif isinstance(node, ast.Try):
6824            yield from walk_ast_in_order(node.body)
6825            yield from walk_ast_in_order(node.handlers)
6826            yield from walk_ast_in_order(node.orelse)
6827            yield from walk_ast_in_order(node.finalbody)
6828        elif isinstance(node, ast.Assert):
6829            yield from walk_ast_in_order(node.test)
6830            yield from walk_ast_in_order(node.msg)
6831        elif isinstance(node, ast.Expr):
6832            yield from walk_ast_in_order(node.value)
6833        # Import, ImportFrom, Global, Nonlocal, Pass, Break, and
6834        # Continue each have no executable content, so we'll yield them
6835        # but not any children
6836
6837        elif isinstance(node, ast.BoolOp):
6838            yield from walk_ast_in_order(node.values)
6839        elif HAS_WALRUS and isinstance(node, ast.NamedExpr):
6840            yield from walk_ast_in_order(node.value)
6841            yield from walk_ast_in_order(node.target)
6842        elif isinstance(node, ast.BinOp):
6843            yield from walk_ast_in_order(node.left)
6844            yield from walk_ast_in_order(node.right)
6845        elif isinstance(node, ast.UnaryOp):
6846            yield from walk_ast_in_order(node.operand)
6847        elif isinstance(node, ast.Lambda):
6848            yield from walk_ast_in_order(node.args)
6849            yield from walk_ast_in_order(node.body)
6850        elif isinstance(node, ast.Dict):
6851            for i in range(len(node.keys)):
6852                yield from walk_ast_in_order(node.keys[i])
6853                yield from walk_ast_in_order(node.values[i])
6854        elif isinstance(node, (ast.Tuple, ast.List, ast.Set)):
6855            yield from walk_ast_in_order(node.elts)
6856        elif isinstance(node, (ast.ListComp, ast.SetComp, ast.GeneratorExp)):
6857            yield from walk_ast_in_order(node.generators)
6858            yield from walk_ast_in_order(node.elt)
6859        elif isinstance(node, ast.DictComp):
6860            yield from walk_ast_in_order(node.generators)
6861            yield from walk_ast_in_order(node.key)
6862            yield from walk_ast_in_order(node.value)
6863        elif isinstance(node, (ast.Await, ast.Yield, ast.YieldFrom)):
6864            yield from walk_ast_in_order(node.value)
6865        elif isinstance(node, ast.Compare):
6866            yield from walk_ast_in_order(node.left)
6867            yield from walk_ast_in_order(node.comparators)
6868        elif isinstance(node, ast.Call):
6869            yield from walk_ast_in_order(node.func)
6870            yield from walk_ast_in_order(node.args)
6871            yield from walk_ast_in_order(node.keywords)
6872        elif isinstance(node, ast.FormattedValue):
6873            yield from walk_ast_in_order(node.value)
6874            yield from walk_ast_in_order(node.format_spec)
6875        elif isinstance(node, ast.JoinedStr):
6876            yield from walk_ast_in_order(node.values)
6877        elif isinstance(node, (ast.Attribute, ast.Starred)):
6878            yield from walk_ast_in_order(node.value)
6879        elif isinstance(node, ast.Subscript):
6880            yield from walk_ast_in_order(node.value)
6881            yield from walk_ast_in_order(node.slice)
6882        elif isinstance(node, ast.Slice):
6883            yield from walk_ast_in_order(node.lower)
6884            yield from walk_ast_in_order(node.upper)
6885            yield from walk_ast_in_order(node.step)
6886        # Constant and Name nodes don't have executable contents
6887
6888        elif isinstance(node, ast.comprehension):
6889            yield from walk_ast_in_order(node.iter)
6890            yield from walk_ast_in_order(node.ifs)
6891            yield from walk_ast_in_order(node.target)
6892        elif isinstance(node, ast.ExceptHandler):
6893            yield from walk_ast_in_order(node.type)
6894            yield from walk_ast_in_order(node.body)
6895        elif isinstance(node, ast.arguments):
6896            yield from walk_ast_in_order(node.defaults)
6897            yield from walk_ast_in_order(node.kw_defaults)
6898            if hasattr(node, "posonlyargs"):
6899                yield from walk_ast_in_order(node.posonlyargs)
6900            yield from walk_ast_in_order(node.args)
6901            yield from walk_ast_in_order(node.vararg)
6902            yield from walk_ast_in_order(node.kwonlyargs)
6903            yield from walk_ast_in_order(node.kwarg)
6904        elif isinstance(node, ast.arg):
6905            yield from walk_ast_in_order(node.annotation)
6906        elif isinstance(node, ast.keyword):
6907            yield from walk_ast_in_order(node.value)
6908        elif isinstance(node, ast.withitem):
6909            yield from walk_ast_in_order(node.context_expr)
6910            yield from walk_ast_in_order(node.optional_vars)
6911        # alias and typeignore have no executable members
6912
6913        # Finally, yield this node itself
6914        yield node

Yields all of the descendants of the given node (or list of nodes) in execution order. Note that this has its limits, for example, if we run it on the code:

x = [A for y in C if D]

It will yield the nodes for C, then y, then D, then A, and finally x, but in actual execution the nodes for D and A may be executed multiple times before x is assigned.

def find_call_nodes_on_line(node, frame, function, lineno) 
6917def find_call_nodes_on_line(node, frame, function, lineno):
6918    """
6919    Given an AST node, a stack frame, a function object, and a line
6920    number, looks for all function calls which occur on the given line
6921    number and which are calls to the given function (as evaluated in
6922    the given stack frame).
6923
6924    Note that calls to functions defined as part of the given AST cannot
6925    be found in this manner, because the objects being called are newly
6926    created and one could not possibly pass a reference to one of them
6927    into this function. For that reason, if the function argument is a
6928    string, any function call whose call part matches the given string
6929    will be matched. Normally only Name nodes can match this way, but if
6930    ast.unparse is available, the string will also attempt to match
6931    (exactly) against the unparsed call expression.
6932
6933    Calls that start on the given line number will match, but if there
6934    are no such calls, then a call on a preceding line whose expression
6935    includes the target line will be looked for and may match.
6936
6937    The return value will be a list of ast.Call nodes, and they will be
6938    ordered in the same order that those nodes would be executed when
6939    the line of code is executed.
6940    """
6941    def call_matches(call_node):
6942        """
6943        Locally-defined matching predicate.
6944        """
6945        nonlocal function
6946        call_expr = call_node.func
6947        return (
6948            (
6949                isinstance(function, str)
6950            and (
6951                    (
6952                        isinstance(call_expr, ast.Name)
6953                    and call_expr.id == function
6954                    )
6955                 or (
6956                        isinstance(call_expr, ast.Attribute)
6957                    and call_expr.attr == function
6958                    )
6959                 or (
6960                        hasattr(ast, "unparse")
6961                    and ast.unparse(call_expr) == function
6962                    )
6963                )
6964            )
6965         or (
6966                not isinstance(function, str)
6967            and evaluate_in_context(call_expr, frame) is function
6968            )
6969        )
6970
6971    result = []
6972    all_on_line = []
6973    for child in walk_ast_in_order(node):
6974        # only consider call nodes on the target line
6975        if (
6976            hasattr(child, "lineno")
6977        and child.lineno == lineno
6978        ):
6979            all_on_line.append(child)
6980            if isinstance(child, ast.Call) and call_matches(child):
6981                result.append(child)
6982
6983    # If we didn't find any candidates, look outwards from ast nodes on
6984    # the target line to find a Call that encompasses them...
6985    if len(result) == 0:
6986        for on_line in all_on_line:
6987            here = getattr(on_line, "parent", None)
6988            while (
6989                here is not None
6990            and not isinstance(
6991                    here,
6992                    # Call (what we're looking for) plus most nodes that
6993                    # indicate there couldn't be a call grandparent:
6994                    (
6995                        ast.Call,
6996                        ast.Module, ast.Interactive, ast.Expression,
6997                        ast.FunctionDef, ast.AsyncFunctionDef,
6998                        ast.ClassDef,
6999                        ast.Return,
7000                        ast.Delete,
7001                        ast.Assign, ast.AugAssign, ast.AnnAssign,
7002                        ast.For, ast.AsyncFor,
7003                        ast.While,
7004                        ast.If,
7005                        ast.With, ast.AsyncWith,
7006                        ast.Raise,
7007                        ast.Try,
7008                        ast.Assert,
7009                        ast.Assert,
7010                        ast.Assert,
7011                        ast.Assert,
7012                        ast.Assert,
7013                        ast.Assert,
7014                        ast.Assert,
7015                        ast.Assert,
7016                    )
7017                )
7018            ):
7019                here = getattr(here, "parent", None)
7020
7021            # If we found a Call that includes the target line as one
7022            # of its children...
7023            if isinstance(here, ast.Call) and call_matches(here):
7024                result.append(here)
7025
7026    return result

Given an AST node, a stack frame, a function object, and a line number, looks for all function calls which occur on the given line number and which are calls to the given function (as evaluated in the given stack frame).

Note that calls to functions defined as part of the given AST cannot be found in this manner, because the objects being called are newly created and one could not possibly pass a reference to one of them into this function. For that reason, if the function argument is a string, any function call whose call part matches the given string will be matched. Normally only Name nodes can match this way, but if ast.unparse is available, the string will also attempt to match (exactly) against the unparsed call expression.

Calls that start on the given line number will match, but if there are no such calls, then a call on a preceding line whose expression includes the target line will be looked for and may match.

The return value will be a list of ast.Call nodes, and they will be ordered in the same order that those nodes would be executed when the line of code is executed.

def assign_parents(root) 
7029def assign_parents(root):
7030    """
7031    Given an AST node, assigns "parent" attributes to each sub-node
7032    indicating their parent AST node. Assigns None as the value of the
7033    parent attribute of the root node.
7034    """
7035    for node in ast.walk(root):
7036        for child in ast.iter_child_nodes(node):
7037            child.parent = node
7038
7039    root.parent = None

Given an AST node, assigns "parent" attributes to each sub-node indicating their parent AST node. Assigns None as the value of the parent attribute of the root node.

def is_inside_call_func(node) 
7042def is_inside_call_func(node):
7043    """
7044    Given an AST node which has a parent attribute, traverses parents to
7045    see if this node is part of the func attribute of a Call node.
7046    """
7047    if not hasattr(node, "parent") or node.parent is None:
7048        return False
7049    if isinstance(node.parent, ast.Call) and node.parent.func is node:
7050        return True
7051    else:
7052        return is_inside_call_func(node.parent)

Given an AST node which has a parent attribute, traverses parents to see if this node is part of the func attribute of a Call node.

def tag_for(located) 
7055def tag_for(located):
7056    """
7057    Given a dictionary which has 'file' and 'line' slots, returns a
7058    string to be used as the tag for a test with 'filename:line' as the
7059    format. Unless the `DETAIL_LEVEL` is 2 or higher, the filename will
7060    be shown without the full path.
7061    """
7062    filename = located.get('file', '???')
7063    if DETAIL_LEVEL < 2:
7064        filename = os.path.basename(filename)
7065    line = located.get('line', '?')
7066    return f"{filename}:{line}"

Given a dictionary which has 'file' and 'line' slots, returns a string to be used as the tag for a test with 'filename:line' as the format. Unless the DETAIL_LEVEL is 2 or higher, the filename will be shown without the full path.

def get_my_location(speculate_filename=True) 
7069def get_my_location(speculate_filename=True):
7070    """
7071    Fetches the filename and line number of the external module whose
7072    call into this module ended up invoking this function. Returns a
7073    dictionary with "file" and "line" keys.
7074
7075    If speculate_filename is False, then the filename will be set to
7076    None in cases where a __file__ global cannot be found, instead of
7077    using f_code.co_filename as a backup. In some cases, this is useful
7078    because f_code.co_filename may not be a valid file.
7079    """
7080    frame = get_external_calling_frame()
7081    try:
7082        filename = get_filename(frame, speculate_filename)
7083        lineno = get_code_line(frame)
7084    finally:
7085        del frame
7086
7087    return { "file": filename, "line": lineno }

Fetches the filename and line number of the external module whose call into this module ended up invoking this function. Returns a dictionary with "file" and "line" keys.

If speculate_filename is False, then the filename will be set to None in cases where a __file__ global cannot be found, instead of using f_code.co_filename as a backup. In some cases, this is useful because f_code.co_filename may not be a valid file.

def get_my_context(function_or_name) 
7090def get_my_context(function_or_name):
7091    """
7092    Returns a dictionary indicating the context of a function call,
7093    assuming that this function is called from within a function with the
7094    given name (or from within the given function), and that that
7095    function is being called from within a different module. The result
7096    has the following keys:
7097
7098    - file: The filename of the calling module
7099    - line: The line number on which the call to the function occurred
7100    - src: The source code string of the calling module
7101    - expr: An AST node storing the expression passed as the first
7102        argument to the function
7103    - expr_src: The source code string of the expression passed as the
7104        first argument to the function
7105    - values: A dictionary mapping source code fragments to their
7106        values, for each variable reference in the test expression. These
7107        are deepish copies of the values encountered.
7108    - relevant: A list of source code fragments which appear in the
7109        values dictionary which are judged to be most-relevant to the
7110        result of the test.
7111
7112    Currently, the relevant list just lists any fragments which aren't
7113    found in the func slot of Call nodes, under the assumption that we
7114    don't care as much about the values of the functions we're calling.
7115
7116    Prints a warning and returns a dictionary with just "file" and
7117    "line" entries if the other context info is unavailable.
7118    """
7119    if isinstance(function_or_name, types.FunctionType):
7120        function_name = function_or_name.__name__
7121    else:
7122        function_name = function_or_name
7123
7124    frame = get_external_calling_frame()
7125    try:
7126        filename = get_filename(frame)
7127        lineno = get_code_line(frame)
7128        if filename is None:
7129            src = None
7130        else:
7131            try:
7132                with open(filename, 'r') as fin:
7133                    src = fin.read()
7134            except Exception:
7135                # Try to get contents from the linecache as a backup...
7136                try:
7137                    src = ''.join(linecache.getlines(filename))
7138                except Exception:
7139                    # We'll assume here that the source is something like
7140                    # an interactive shell so we won't warn unless the
7141                    # detail level is turned up.
7142                    if DETAIL_LEVEL >= 2:
7143                        print(
7144                            "Warning: unable to get calling code's source.",
7145                            file=PRINT_TO
7146                        )
7147                        print(
7148                            (
7149                                "Call is on line {} of module {} from file"
7150                                " '{}'"
7151                            ).format(
7152                                lineno,
7153                                frame.f_globals.get("__name__"),
7154                                filename
7155                            ),
7156                            file=PRINT_TO
7157                        )
7158                    src = None
7159
7160        if src is None:
7161            return {
7162                "file": filename,
7163                "line": lineno
7164            }
7165
7166        src_node = ast.parse(src, filename=filename, mode='exec')
7167        assign_parents(src_node)
7168        candidates = find_call_nodes_on_line(
7169            src_node,
7170            frame,
7171            function_or_name,
7172            lineno
7173        )
7174
7175        # What if there are zero candidates?
7176        if len(candidates) == 0:
7177            print(
7178                f"Warning: unable to find call node for {function_name}"
7179                f" on line {lineno} of file {filename}.",
7180                file=PRINT_TO
7181            )
7182            return {
7183                "file": filename,
7184                "line": lineno
7185            }
7186
7187        # Figure out how many calls to get_my_context have happened
7188        # referencing this line before, so that we know which call on
7189        # this line we might be
7190        prev_this_line = COMPLETED_PER_LINE\
7191            .setdefault(function_name, {})\
7192            .setdefault((filename, lineno), 0)
7193        match = candidates[prev_this_line % len(candidates)]
7194
7195        # Record this call so the next one will grab the subsequent
7196        # candidate
7197        COMPLETED_PER_LINE[function_name][(filename, lineno)] += 1
7198
7199        arg_expr = match.args[0]
7200
7201        # Add .parent attributes
7202        assign_parents(arg_expr)
7203
7204        # Source code for the expression
7205        expr_src = get_expr_src(src, match)
7206
7207        # Prepare our result dictionary
7208        result = {
7209            "file": filename,
7210            "line": lineno,
7211            "src": src,
7212            "expr": arg_expr,
7213            "expr_src": expr_src,
7214            "values": {},
7215            "relevant": set()
7216        }
7217
7218        # Walk expression to find values for each variable
7219        for node in ast.walk(arg_expr):
7220            # If it's potentially a reference to a variable...
7221            if isinstance(
7222                node,
7223                (ast.Attribute, ast.Subscript, ast.Name)
7224            ):
7225                key = get_ref_src(src, node)
7226                if key not in result["values"]:
7227                    # Don't re-evaluate multiply-reference expressions
7228                    # Note: we assume they won't take on multiple
7229                    # values; if they did, even our first evaluation
7230                    # would probably be inaccurate.
7231                    val = deepish_copy(evaluate_in_context(node, frame))
7232                    result["values"][key] = val
7233                    if not is_inside_call_func(node):
7234                        result["relevant"].add(key)
7235
7236        return result
7237
7238    finally:
7239        del frame

Returns a dictionary indicating the context of a function call, assuming that this function is called from within a function with the given name (or from within the given function), and that that function is being called from within a different module. The result has the following keys:

  • file: The filename of the calling module
  • line: The line number on which the call to the function occurred
  • src: The source code string of the calling module
  • expr: An AST node storing the expression passed as the first argument to the function
  • expr_src: The source code string of the expression passed as the first argument to the function
  • values: A dictionary mapping source code fragments to their values, for each variable reference in the test expression. These are deepish copies of the values encountered.
  • relevant: A list of source code fragments which appear in the values dictionary which are judged to be most-relevant to the result of the test.

Currently, the relevant list just lists any fragments which aren't found in the func slot of Call nodes, under the assumption that we don't care as much about the values of the functions we're calling.

Prints a warning and returns a dictionary with just "file" and "line" entries if the other context info is unavailable.

def messagesAsErrors(activate=True) 
7246def messagesAsErrors(activate=True):
7247    """
7248    Sets `PRINT_TO` to `sys.stderr` so that messages from optimism will
7249    appear as error messages, rather than as normal printed output. This
7250    is the default behavior, but you can pass `False` as the argument to
7251    set it to `sys.stdout` instead, causing messages to appear as normal
7252    output.
7253    """
7254    global PRINT_TO
7255    if activate:
7256        PRINT_TO = sys.stderr
7257    else:
7258        PRINT_TO = sys.stdout

Sets PRINT_TO to sys.stderr so that messages from optimism will appear as error messages, rather than as normal printed output. This is the default behavior, but you can pass False as the argument to set it to sys.stdout instead, causing messages to appear as normal output.