doctest.py revision a6b68327b2aab767197f513c18abb5db496241d7
1# Module doctest. 2# Released to the public domain 16-Jan-2001, by Tim Peters (tim@python.org). 3# Major enhancements and refactoring by: 4# Jim Fulton 5# Edward Loper 6 7# Provided as-is; use at your own risk; no warranty; no promises; enjoy! 8 9r"""Module doctest -- a framework for running examples in docstrings. 10 11NORMAL USAGE 12 13In simplest use, end each module M to be tested with: 14 15def _test(): 16 import doctest 17 return doctest.testmod() 18 19if __name__ == "__main__": 20 _test() 21 22Then running the module as a script will cause the examples in the 23docstrings to get executed and verified: 24 25python M.py 26 27This won't display anything unless an example fails, in which case the 28failing example(s) and the cause(s) of the failure(s) are printed to stdout 29(why not stderr? because stderr is a lame hack <0.2 wink>), and the final 30line of output is "Test failed.". 31 32Run it with the -v switch instead: 33 34python M.py -v 35 36and a detailed report of all examples tried is printed to stdout, along 37with assorted summaries at the end. 38 39You can force verbose mode by passing "verbose=True" to testmod, or prohibit 40it by passing "verbose=False". In either of those cases, sys.argv is not 41examined by testmod. 42 43In any case, testmod returns a 2-tuple of ints (f, t), where f is the 44number of docstring examples that failed and t is the total number of 45docstring examples attempted. 46 47There are a variety of other ways to run doctests, including integration 48with the unittest framework, and support for running non-Python text 49files containing doctests. There are also many ways to override parts 50of doctest's default behaviors. See the Library Reference Manual for 51details. 52 53 54WHICH DOCSTRINGS ARE EXAMINED? 55 56+ M.__doc__. 57 58+ f.__doc__ for all functions f in M.__dict__.values(), except those 59 defined in other modules. 60 61+ C.__doc__ for all classes C in M.__dict__.values(), except those 62 defined in other modules. 63 64+ If M.__test__ exists and "is true", it must be a dict, and 65 each entry maps a (string) name to a function object, class object, or 66 string. Function and class object docstrings found from M.__test__ 67 are searched, and strings are searched directly as if they were docstrings. 68 In output, a key K in M.__test__ appears with name 69 <name of M>.__test__.K 70 71Any classes found are recursively searched similarly, to test docstrings in 72their contained methods and nested classes. 73 74 75WHAT'S THE EXECUTION CONTEXT? 76 77By default, each time testmod finds a docstring to test, it uses a *copy* 78of M's globals (so that running tests on a module doesn't change the 79module's real globals, and so that one test in M can't leave behind crumbs 80that accidentally allow another test to work). This means examples can 81freely use any names defined at top-level in M. It also means that sloppy 82imports (see above) can cause examples in external docstrings to use 83globals inappropriate for them. 84 85You can force use of your own dict as the execution context by passing 86"globs=your_dict" to testmod instead. Presumably this would be a copy of 87M.__dict__ merged with the globals from other imported modules. 88 89 90WHAT ABOUT EXCEPTIONS? 91 92No problem, as long as the only output generated by the example is the 93traceback itself. For example: 94 95 >>> [1, 2, 3].remove(42) 96 Traceback (most recent call last): 97 File "<stdin>", line 1, in ? 98 ValueError: list.remove(x): x not in list 99 >>> 100 101Note that only the exception type and value are compared. 102 103 104SO WHAT DOES A DOCTEST EXAMPLE LOOK LIKE ALREADY!? 105 106Oh ya. It's easy! In most cases a copy-and-paste of an interactive 107console session works fine -- just make sure the leading whitespace is 108rigidly consistent (you can mix tabs and spaces if you're too lazy to do it 109right, but doctest is not in the business of guessing what you think a tab 110means). 111 112 >>> # comments are ignored 113 >>> x = 12 114 >>> x 115 12 116 >>> if x == 13: 117 ... print "yes" 118 ... else: 119 ... print "no" 120 ... print "NO" 121 ... print "NO!!!" 122 ... 123 no 124 NO 125 NO!!! 126 >>> 127 128Any expected output must immediately follow the final ">>>" or "..." line 129containing the code, and the expected output (if any) extends to the next 130">>>" or all-whitespace line. That's it. 131 132Bummers: 133 134+ Output to stdout is captured, but not output to stderr (exception 135 tracebacks are captured via a different means). 136 137+ If you continue a line via backslashing in an interactive session, 138 or for any other reason use a backslash, you should use a raw 139 docstring, which will preserve your backslahses exactly as you type 140 them: 141 142 >>> def f(x): 143 ... r'''Backslashes in a raw docstring: m\n''' 144 >>> print f.__doc__ 145 Backslashes in a raw docstring: m\n 146 147 Otherwise, the backslash will be interpreted as part of the string. 148 E.g., the "\n" above would be interpreted as a newline character. 149 Alternatively, you can double each backslash in the doctest version 150 (and not use a raw string): 151 152 >>> def f(x): 153 ... '''Backslashes in a raw docstring: m\\n''' 154 >>> print f.__doc__ 155 Backslashes in a raw docstring: m\n 156 157The starting column doesn't matter: 158 159>>> assert "Easy!" 160 >>> import math 161 >>> math.floor(1.9) 162 1.0 163 164and as many leading whitespace characters are stripped from the expected 165output as appeared in the initial ">>>" line that triggered it. 166 167If you execute this very file, the examples above will be found and 168executed. 169""" 170__docformat__ = 'reStructuredText en' 171 172__all__ = [ 173 # 0, Option Flags 174 'register_optionflag', 175 'DONT_ACCEPT_TRUE_FOR_1', 176 'DONT_ACCEPT_BLANKLINE', 177 'NORMALIZE_WHITESPACE', 178 'ELLIPSIS', 179 'UNIFIED_DIFF', 180 'CONTEXT_DIFF', 181 'NDIFF_DIFF', 182 # 1. Utility Functions 183 'is_private', 184 # 2. Example & DocTest 185 'Example', 186 'DocTest', 187 # 3. Doctest Parser 188 'DocTestParser', 189 # 4. Doctest Finder 190 'DocTestFinder', 191 # 5. Doctest Runner 192 'DocTestRunner', 193 'OutputChecker', 194 'DocTestFailure', 195 'UnexpectedException', 196 'DebugRunner', 197 # 6. Test Functions 198 'testmod', 199 'run_docstring_examples', 200 # 7. Tester 201 'Tester', 202 # 8. Unittest Support 203 'DocTestCase', 204 'DocTestSuite', 205 'DocFileCase', 206 'DocFileTest', 207 'DocFileSuite', 208 # 9. Debugging Support 209 'script_from_examples', 210 'testsource', 211 'debug_src', 212 'debug_script', 213 'debug', 214] 215 216import __future__ 217 218import sys, traceback, inspect, linecache, os, re, types 219import unittest, difflib, pdb, tempfile 220import warnings 221from StringIO import StringIO 222 223# Don't whine about the deprecated is_private function in this 224# module's tests. 225warnings.filterwarnings("ignore", "is_private", DeprecationWarning, 226 __name__, 0) 227 228real_pdb_set_trace = pdb.set_trace 229 230# There are 4 basic classes: 231# - Example: a <source, want> pair, plus an intra-docstring line number. 232# - DocTest: a collection of examples, parsed from a docstring, plus 233# info about where the docstring came from (name, filename, lineno). 234# - DocTestFinder: extracts DocTests from a given object's docstring and 235# its contained objects' docstrings. 236# - DocTestRunner: runs DocTest cases, and accumulates statistics. 237# 238# So the basic picture is: 239# 240# list of: 241# +------+ +---------+ +-------+ 242# |object| --DocTestFinder-> | DocTest | --DocTestRunner-> |results| 243# +------+ +---------+ +-------+ 244# | Example | 245# | ... | 246# | Example | 247# +---------+ 248 249# Option constants. 250OPTIONFLAGS_BY_NAME = {} 251def register_optionflag(name): 252 flag = 1 << len(OPTIONFLAGS_BY_NAME) 253 OPTIONFLAGS_BY_NAME[name] = flag 254 return flag 255 256DONT_ACCEPT_TRUE_FOR_1 = register_optionflag('DONT_ACCEPT_TRUE_FOR_1') 257DONT_ACCEPT_BLANKLINE = register_optionflag('DONT_ACCEPT_BLANKLINE') 258NORMALIZE_WHITESPACE = register_optionflag('NORMALIZE_WHITESPACE') 259ELLIPSIS = register_optionflag('ELLIPSIS') 260UNIFIED_DIFF = register_optionflag('UNIFIED_DIFF') 261CONTEXT_DIFF = register_optionflag('CONTEXT_DIFF') 262NDIFF_DIFF = register_optionflag('NDIFF_DIFF') 263 264# Special string markers for use in `want` strings: 265BLANKLINE_MARKER = '<BLANKLINE>' 266ELLIPSIS_MARKER = '...' 267 268###################################################################### 269## Table of Contents 270###################################################################### 271# 1. Utility Functions 272# 2. Example & DocTest -- store test cases 273# 3. DocTest Parser -- extracts examples from strings 274# 4. DocTest Finder -- extracts test cases from objects 275# 5. DocTest Runner -- runs test cases 276# 6. Test Functions -- convenient wrappers for testing 277# 7. Tester Class -- for backwards compatibility 278# 8. Unittest Support 279# 9. Debugging Support 280# 10. Example Usage 281 282###################################################################### 283## 1. Utility Functions 284###################################################################### 285 286def is_private(prefix, base): 287 """prefix, base -> true iff name prefix + "." + base is "private". 288 289 Prefix may be an empty string, and base does not contain a period. 290 Prefix is ignored (although functions you write conforming to this 291 protocol may make use of it). 292 Return true iff base begins with an (at least one) underscore, but 293 does not both begin and end with (at least) two underscores. 294 295 >>> is_private("a.b", "my_func") 296 False 297 >>> is_private("____", "_my_func") 298 True 299 >>> is_private("someclass", "__init__") 300 False 301 >>> is_private("sometypo", "__init_") 302 True 303 >>> is_private("x.y.z", "_") 304 True 305 >>> is_private("_x.y.z", "__") 306 False 307 >>> is_private("", "") # senseless but consistent 308 False 309 """ 310 warnings.warn("is_private is deprecated; it wasn't useful; " 311 "examine DocTestFinder.find() lists instead", 312 DeprecationWarning, stacklevel=2) 313 return base[:1] == "_" and not base[:2] == "__" == base[-2:] 314 315def _extract_future_flags(globs): 316 """ 317 Return the compiler-flags associated with the future features that 318 have been imported into the given namespace (globs). 319 """ 320 flags = 0 321 for fname in __future__.all_feature_names: 322 feature = globs.get(fname, None) 323 if feature is getattr(__future__, fname): 324 flags |= feature.compiler_flag 325 return flags 326 327def _normalize_module(module, depth=2): 328 """ 329 Return the module specified by `module`. In particular: 330 - If `module` is a module, then return module. 331 - If `module` is a string, then import and return the 332 module with that name. 333 - If `module` is None, then return the calling module. 334 The calling module is assumed to be the module of 335 the stack frame at the given depth in the call stack. 336 """ 337 if inspect.ismodule(module): 338 return module 339 elif isinstance(module, (str, unicode)): 340 return __import__(module, globals(), locals(), ["*"]) 341 elif module is None: 342 return sys.modules[sys._getframe(depth).f_globals['__name__']] 343 else: 344 raise TypeError("Expected a module, string, or None") 345 346def _tag_msg(tag, msg, indent=' '): 347 """ 348 Return a string that displays a tag-and-message pair nicely, 349 keeping the tag and its message on the same line when that 350 makes sense. If the message is displayed on separate lines, 351 then `indent` is added to the beginning of each line. 352 """ 353 # If the message doesn't end in a newline, then add one. 354 if msg[-1:] != '\n': 355 msg += '\n' 356 # If the message is short enough, and contains no internal 357 # newlines, then display it on the same line as the tag. 358 # Otherwise, display the tag on its own line. 359 if (len(tag) + len(msg) < 75 and 360 msg.find('\n', 0, len(msg)-1) == -1): 361 return '%s: %s' % (tag, msg) 362 else: 363 msg = '\n'.join([indent+l for l in msg[:-1].split('\n')]) 364 return '%s:\n%s\n' % (tag, msg) 365 366def _exception_traceback(exc_info): 367 """ 368 Return a string containing a traceback message for the given 369 exc_info tuple (as returned by sys.exc_info()). 370 """ 371 # Get a traceback message. 372 excout = StringIO() 373 exc_type, exc_val, exc_tb = exc_info 374 traceback.print_exception(exc_type, exc_val, exc_tb, file=excout) 375 return excout.getvalue() 376 377# Override some StringIO methods. 378class _SpoofOut(StringIO): 379 def getvalue(self): 380 result = StringIO.getvalue(self) 381 # If anything at all was written, make sure there's a trailing 382 # newline. There's no way for the expected output to indicate 383 # that a trailing newline is missing. 384 if result and not result.endswith("\n"): 385 result += "\n" 386 # Prevent softspace from screwing up the next test case, in 387 # case they used print with a trailing comma in an example. 388 if hasattr(self, "softspace"): 389 del self.softspace 390 return result 391 392 def truncate(self, size=None): 393 StringIO.truncate(self, size) 394 if hasattr(self, "softspace"): 395 del self.softspace 396 397# Worst-case linear-time ellipsis matching. 398def _ellipsis_match(want, got): 399 """ 400 Essentially the only subtle case: 401 >>> _ellipsis_match('aa...aa', 'aaa') 402 False 403 """ 404 if ELLIPSIS_MARKER not in want: 405 return want == got 406 407 # Find "the real" strings. 408 ws = want.split(ELLIPSIS_MARKER) 409 assert len(ws) >= 2 410 411 # Deal with exact matches possibly needed at one or both ends. 412 startpos, endpos = 0, len(got) 413 w = ws[0] 414 if w: # starts with exact match 415 if got.startswith(w): 416 startpos = len(w) 417 del ws[0] 418 else: 419 return False 420 w = ws[-1] 421 if w: # ends with exact match 422 if got.endswith(w): 423 endpos -= len(w) 424 del ws[-1] 425 else: 426 return False 427 428 if startpos > endpos: 429 # Exact end matches required more characters than we have, as in 430 # _ellipsis_match('aa...aa', 'aaa') 431 return False 432 433 # For the rest, we only need to find the leftmost non-overlapping 434 # match for each piece. If there's no overall match that way alone, 435 # there's no overall match period. 436 for w in ws: 437 # w may be '' at times, if there are consecutive ellipses, or 438 # due to an ellipsis at the start or end of `want`. That's OK. 439 # Search for an empty string succeeds, and doesn't change startpos. 440 startpos = got.find(w, startpos, endpos) 441 if startpos < 0: 442 return False 443 startpos += len(w) 444 445 return True 446 447###################################################################### 448## 2. Example & DocTest 449###################################################################### 450## - An "example" is a <source, want> pair, where "source" is a 451## fragment of source code, and "want" is the expected output for 452## "source." The Example class also includes information about 453## where the example was extracted from. 454## 455## - A "doctest" is a collection of examples, typically extracted from 456## a string (such as an object's docstring). The DocTest class also 457## includes information about where the string was extracted from. 458 459class Example: 460 """ 461 A single doctest example, consisting of source code and expected 462 output. `Example` defines the following attributes: 463 464 - source: A single Python statement, always ending with a newline. 465 The constructor adds a newline if needed. 466 467 - want: The expected output from running the source code (either 468 from stdout, or a traceback in case of exception). `want` ends 469 with a newline unless it's empty, in which case it's an empty 470 string. The constructor adds a newline if needed. 471 472 - exc_msg: The exception message generated by the example, if 473 the example is expected to generate an exception; or `None` if 474 it is not expected to generate an exception. This exception 475 message is compared against the return value of 476 `traceback.format_exception_only()`. `exc_msg` ends with a 477 newline unless it's `None`. The constructor adds a newline 478 if needed. 479 480 - lineno: The line number within the DocTest string containing 481 this Example where the Example begins. This line number is 482 zero-based, with respect to the beginning of the DocTest. 483 484 - indent: The example's indentation in the DocTest string. 485 I.e., the number of space characters that preceed the 486 example's first prompt. 487 488 - options: A dictionary mapping from option flags to True or 489 False, which is used to override default options for this 490 example. Any option flags not contained in this dictionary 491 are left at their default value (as specified by the 492 DocTestRunner's optionflags). By default, no options are set. 493 """ 494 def __init__(self, source, want, exc_msg=None, lineno=0, indent=0, 495 options=None): 496 # Normalize inputs. 497 if not source.endswith('\n'): 498 source += '\n' 499 if want and not want.endswith('\n'): 500 want += '\n' 501 if exc_msg is not None and not exc_msg.endswith('\n'): 502 exc_msg += '\n' 503 # Store properties. 504 self.source = source 505 self.want = want 506 self.lineno = lineno 507 self.indent = indent 508 if options is None: options = {} 509 self.options = options 510 self.exc_msg = exc_msg 511 512class DocTest: 513 """ 514 A collection of doctest examples that should be run in a single 515 namespace. Each `DocTest` defines the following attributes: 516 517 - examples: the list of examples. 518 519 - globs: The namespace (aka globals) that the examples should 520 be run in. 521 522 - name: A name identifying the DocTest (typically, the name of 523 the object whose docstring this DocTest was extracted from). 524 525 - filename: The name of the file that this DocTest was extracted 526 from, or `None` if the filename is unknown. 527 528 - lineno: The line number within filename where this DocTest 529 begins, or `None` if the line number is unavailable. This 530 line number is zero-based, with respect to the beginning of 531 the file. 532 533 - docstring: The string that the examples were extracted from, 534 or `None` if the string is unavailable. 535 """ 536 def __init__(self, examples, globs, name, filename, lineno, docstring): 537 """ 538 Create a new DocTest containing the given examples. The 539 DocTest's globals are initialized with a copy of `globs`. 540 """ 541 assert not isinstance(examples, basestring), \ 542 "DocTest no longer accepts str; use DocTestParser instead" 543 self.examples = examples 544 self.docstring = docstring 545 self.globs = globs.copy() 546 self.name = name 547 self.filename = filename 548 self.lineno = lineno 549 550 def __repr__(self): 551 if len(self.examples) == 0: 552 examples = 'no examples' 553 elif len(self.examples) == 1: 554 examples = '1 example' 555 else: 556 examples = '%d examples' % len(self.examples) 557 return ('<DocTest %s from %s:%s (%s)>' % 558 (self.name, self.filename, self.lineno, examples)) 559 560 561 # This lets us sort tests by name: 562 def __cmp__(self, other): 563 if not isinstance(other, DocTest): 564 return -1 565 return cmp((self.name, self.filename, self.lineno, id(self)), 566 (other.name, other.filename, other.lineno, id(other))) 567 568###################################################################### 569## 3. DocTestParser 570###################################################################### 571 572class DocTestParser: 573 """ 574 A class used to parse strings containing doctest examples. 575 """ 576 # This regular expression is used to find doctest examples in a 577 # string. It defines three groups: `source` is the source code 578 # (including leading indentation and prompts); `indent` is the 579 # indentation of the first (PS1) line of the source code; and 580 # `want` is the expected output (including leading indentation). 581 _EXAMPLE_RE = re.compile(r''' 582 # Source consists of a PS1 line followed by zero or more PS2 lines. 583 (?P<source> 584 (?:^(?P<indent> [ ]*) >>> .*) # PS1 line 585 (?:\n [ ]* \.\.\. .*)*) # PS2 lines 586 \n? 587 # Want consists of any non-blank lines that do not start with PS1. 588 (?P<want> (?:(?![ ]*$) # Not a blank line 589 (?![ ]*>>>) # Not a line starting with PS1 590 .*$\n? # But any other line 591 )*) 592 ''', re.MULTILINE | re.VERBOSE) 593 594 # A regular expression for handling `want` strings that contain 595 # expected exceptions. It divides `want` into three pieces: 596 # - the traceback header line (`hdr`) 597 # - the traceback stack (`stack`) 598 # - the exception message (`msg`), as generated by 599 # traceback.format_exception_only() 600 # `msg` may have multiple lines. We assume/require that the 601 # exception message is the first non-indented line starting with a word 602 # character following the traceback header line. 603 _EXCEPTION_RE = re.compile(r""" 604 # Grab the traceback header. Different versions of Python have 605 # said different things on the first traceback line. 606 ^(?P<hdr> Traceback\ \( 607 (?: most\ recent\ call\ last 608 | innermost\ last 609 ) \) : 610 ) 611 \s* $ # toss trailing whitespace on the header. 612 (?P<stack> .*?) # don't blink: absorb stuff until... 613 ^ (?P<msg> \w+ .*) # a line *starts* with alphanum. 614 """, re.VERBOSE | re.MULTILINE | re.DOTALL) 615 616 # A callable returning a true value iff its argument is a blank line 617 # or contains a single comment. 618 _IS_BLANK_OR_COMMENT = re.compile(r'^[ ]*(#.*)?$').match 619 620 def get_doctest(self, string, globs, name, filename, lineno): 621 """ 622 Extract all doctest examples from the given string, and 623 collect them into a `DocTest` object. 624 625 `globs`, `name`, `filename`, and `lineno` are attributes for 626 the new `DocTest` object. See the documentation for `DocTest` 627 for more information. 628 """ 629 return DocTest(self.get_examples(string, name), globs, 630 name, filename, lineno, string) 631 632 def get_examples(self, string, name='<string>'): 633 """ 634 Extract all doctest examples from the given string, and return 635 them as a list of `Example` objects. Line numbers are 636 0-based, because it's most common in doctests that nothing 637 interesting appears on the same line as opening triple-quote, 638 and so the first interesting line is called \"line 1\" then. 639 640 The optional argument `name` is a name identifying this 641 string, and is only used for error messages. 642 643 >>> text = ''' 644 ... >>> x, y = 2, 3 # no output expected 645 ... >>> if 1: 646 ... ... print x 647 ... ... print y 648 ... 2 649 ... 3 650 ... 651 ... Some text. 652 ... >>> x+y 653 ... 5 654 ... ''' 655 >>> for x in DocTestParser().get_examples(text): 656 ... print (x.source, x.want, x.lineno) 657 ('x, y = 2, 3 # no output expected\\n', '', 1) 658 ('if 1:\\n print x\\n print y\\n', '2\\n3\\n', 2) 659 ('x+y\\n', '5\\n', 9) 660 """ 661 examples = [] 662 charno, lineno = 0, 0 663 # Find all doctest examples in the string: 664 for m in self._EXAMPLE_RE.finditer(string.expandtabs()): 665 # Update lineno (lines before this example) 666 lineno += string.count('\n', charno, m.start()) 667 # Extract source/want from the regexp match. 668 (source, want, exc_msg) = self._parse_example(m, name, lineno) 669 # Extract extra options from the source. 670 options = self._find_options(source, name, lineno) 671 # Create an Example, and add it to the list. 672 if not self._IS_BLANK_OR_COMMENT(source): 673 examples.append( Example(source, want, exc_msg, 674 lineno=lineno, 675 indent=len(m.group('indent')), 676 options=options) ) 677 # Update lineno (lines inside this example) 678 lineno += string.count('\n', m.start(), m.end()) 679 # Update charno. 680 charno = m.end() 681 return examples 682 683 def get_program(self, string, name="<string>"): 684 """ 685 Return an executable program from the given string, as a string. 686 687 The format of this isn't rigidly defined. In general, doctest 688 examples become the executable statements in the result, and 689 their expected outputs become comments, preceded by an \"#Expected:\" 690 comment. Everything else (text, comments, everything not part of 691 a doctest test) is also placed in comments. 692 693 The optional argument `name` is a name identifying this 694 string, and is only used for error messages. 695 696 >>> text = ''' 697 ... >>> x, y = 2, 3 # no output expected 698 ... >>> if 1: 699 ... ... print x 700 ... ... print y 701 ... 2 702 ... 3 703 ... 704 ... Some text. 705 ... >>> x+y 706 ... 5 707 ... ''' 708 >>> print DocTestParser().get_program(text) 709 x, y = 2, 3 # no output expected 710 if 1: 711 print x 712 print y 713 # Expected: 714 ## 2 715 ## 3 716 # 717 # Some text. 718 x+y 719 # Expected: 720 ## 5 721 """ 722 string = string.expandtabs() 723 # If all lines begin with the same indentation, then strip it. 724 min_indent = self._min_indent(string) 725 if min_indent > 0: 726 string = '\n'.join([l[min_indent:] for l in string.split('\n')]) 727 728 output = [] 729 charnum, lineno = 0, 0 730 # Find all doctest examples in the string: 731 for m in self._EXAMPLE_RE.finditer(string.expandtabs()): 732 # Add any text before this example, as a comment. 733 if m.start() > charnum: 734 lines = string[charnum:m.start()-1].split('\n') 735 output.extend([self._comment_line(l) for l in lines]) 736 lineno += len(lines) 737 738 # Extract source/want from the regexp match. 739 (source, want, exc_msg) = self._parse_example(m, name, lineno) 740 # Display the source 741 output.append(source) 742 # Display the expected output, if any 743 if want: 744 output.append('# Expected:') 745 output.extend(['## '+l for l in want.split('\n')]) 746 747 # Update the line number & char number. 748 lineno += string.count('\n', m.start(), m.end()) 749 charnum = m.end() 750 # Add any remaining text, as comments. 751 output.extend([self._comment_line(l) 752 for l in string[charnum:].split('\n')]) 753 # Trim junk on both ends. 754 while output and output[-1] == '#': 755 output.pop() 756 while output and output[0] == '#': 757 output.pop(0) 758 # Combine the output, and return it. 759 return '\n'.join(output) 760 761 def _parse_example(self, m, name, lineno): 762 """ 763 Given a regular expression match from `_EXAMPLE_RE` (`m`), 764 return a pair `(source, want)`, where `source` is the matched 765 example's source code (with prompts and indentation stripped); 766 and `want` is the example's expected output (with indentation 767 stripped). 768 769 `name` is the string's name, and `lineno` is the line number 770 where the example starts; both are used for error messages. 771 """ 772 # Get the example's indentation level. 773 indent = len(m.group('indent')) 774 775 # Divide source into lines; check that they're properly 776 # indented; and then strip their indentation & prompts. 777 source_lines = m.group('source').split('\n') 778 self._check_prompt_blank(source_lines, indent, name, lineno) 779 self._check_prefix(source_lines[1:], ' '*indent + '.', name, lineno) 780 source = '\n'.join([sl[indent+4:] for sl in source_lines]) 781 782 # Divide want into lines; check that it's properly indented; and 783 # then strip the indentation. Spaces before the last newline should 784 # be preserved, so plain rstrip() isn't good enough. 785 want = m.group('want') 786 want_lines = want.split('\n') 787 if len(want_lines) > 1 and re.match(r' *$', want_lines[-1]): 788 del want_lines[-1] # forget final newline & spaces after it 789 self._check_prefix(want_lines, ' '*indent, name, 790 lineno + len(source_lines)) 791 want = '\n'.join([wl[indent:] for wl in want_lines]) 792 793 # If `want` contains a traceback message, then extract it. 794 m = self._EXCEPTION_RE.match(want) 795 if m: 796 exc_msg = m.group('msg') 797 else: 798 exc_msg = None 799 800 return source, want, exc_msg 801 802 # This regular expression looks for option directives in the 803 # source code of an example. Option directives are comments 804 # starting with "doctest:". Warning: this may give false 805 # positives for string-literals that contain the string 806 # "#doctest:". Eliminating these false positives would require 807 # actually parsing the string; but we limit them by ignoring any 808 # line containing "#doctest:" that is *followed* by a quote mark. 809 _OPTION_DIRECTIVE_RE = re.compile(r'#\s*doctest:\s*([^\n\'"]*)$', 810 re.MULTILINE) 811 812 def _find_options(self, source, name, lineno): 813 """ 814 Return a dictionary containing option overrides extracted from 815 option directives in the given source string. 816 817 `name` is the string's name, and `lineno` is the line number 818 where the example starts; both are used for error messages. 819 """ 820 options = {} 821 # (note: with the current regexp, this will match at most once:) 822 for m in self._OPTION_DIRECTIVE_RE.finditer(source): 823 option_strings = m.group(1).replace(',', ' ').split() 824 for option in option_strings: 825 if (option[0] not in '+-' or 826 option[1:] not in OPTIONFLAGS_BY_NAME): 827 raise ValueError('line %r of the doctest for %s ' 828 'has an invalid option: %r' % 829 (lineno+1, name, option)) 830 flag = OPTIONFLAGS_BY_NAME[option[1:]] 831 options[flag] = (option[0] == '+') 832 if options and self._IS_BLANK_OR_COMMENT(source): 833 raise ValueError('line %r of the doctest for %s has an option ' 834 'directive on a line with no example: %r' % 835 (lineno, name, source)) 836 return options 837 838 # This regular expression finds the indentation of every non-blank 839 # line in a string. 840 _INDENT_RE = re.compile('^([ ]+)(?=\S)', re.MULTILINE) 841 842 def _min_indent(self, s): 843 "Return the minimum indentation of any non-blank line in `s`" 844 return min([len(indent) for indent in self._INDENT_RE.findall(s)]) 845 846 def _comment_line(self, line): 847 "Return a commented form of the given line" 848 line = line.rstrip() 849 if line: 850 return '# '+line 851 else: 852 return '#' 853 854 def _check_prompt_blank(self, lines, indent, name, lineno): 855 """ 856 Given the lines of a source string (including prompts and 857 leading indentation), check to make sure that every prompt is 858 followed by a space character. If any line is not followed by 859 a space character, then raise ValueError. 860 """ 861 for i, line in enumerate(lines): 862 if len(line) >= indent+4 and line[indent+3] != ' ': 863 raise ValueError('line %r of the docstring for %s ' 864 'lacks blank after %s: %r' % 865 (lineno+i+1, name, 866 line[indent:indent+3], line)) 867 868 def _check_prefix(self, lines, prefix, name, lineno): 869 """ 870 Check that every line in the given list starts with the given 871 prefix; if any line does not, then raise a ValueError. 872 """ 873 for i, line in enumerate(lines): 874 if line and not line.startswith(prefix): 875 raise ValueError('line %r of the docstring for %s has ' 876 'inconsistent leading whitespace: %r' % 877 (lineno+i+1, name, line)) 878 879 880###################################################################### 881## 4. DocTest Finder 882###################################################################### 883 884class DocTestFinder: 885 """ 886 A class used to extract the DocTests that are relevant to a given 887 object, from its docstring and the docstrings of its contained 888 objects. Doctests can currently be extracted from the following 889 object types: modules, functions, classes, methods, staticmethods, 890 classmethods, and properties. 891 """ 892 893 def __init__(self, verbose=False, parser=DocTestParser(), 894 recurse=True, _namefilter=None): 895 """ 896 Create a new doctest finder. 897 898 The optional argument `parser` specifies a class or 899 function that should be used to create new DocTest objects (or 900 objects that implement the same interface as DocTest). The 901 signature for this factory function should match the signature 902 of the DocTest constructor. 903 904 If the optional argument `recurse` is false, then `find` will 905 only examine the given object, and not any contained objects. 906 """ 907 self._parser = parser 908 self._verbose = verbose 909 self._recurse = recurse 910 # _namefilter is undocumented, and exists only for temporary backward- 911 # compatibility support of testmod's deprecated isprivate mess. 912 self._namefilter = _namefilter 913 914 def find(self, obj, name=None, module=None, globs=None, 915 extraglobs=None): 916 """ 917 Return a list of the DocTests that are defined by the given 918 object's docstring, or by any of its contained objects' 919 docstrings. 920 921 The optional parameter `module` is the module that contains 922 the given object. If the module is not specified or is None, then 923 the test finder will attempt to automatically determine the 924 correct module. The object's module is used: 925 926 - As a default namespace, if `globs` is not specified. 927 - To prevent the DocTestFinder from extracting DocTests 928 from objects that are imported from other modules. 929 - To find the name of the file containing the object. 930 - To help find the line number of the object within its 931 file. 932 933 Contained objects whose module does not match `module` are ignored. 934 935 If `module` is False, no attempt to find the module will be made. 936 This is obscure, of use mostly in tests: if `module` is False, or 937 is None but cannot be found automatically, then all objects are 938 considered to belong to the (non-existent) module, so all contained 939 objects will (recursively) be searched for doctests. 940 941 The globals for each DocTest is formed by combining `globs` 942 and `extraglobs` (bindings in `extraglobs` override bindings 943 in `globs`). A new copy of the globals dictionary is created 944 for each DocTest. If `globs` is not specified, then it 945 defaults to the module's `__dict__`, if specified, or {} 946 otherwise. If `extraglobs` is not specified, then it defaults 947 to {}. 948 949 """ 950 # If name was not specified, then extract it from the object. 951 if name is None: 952 name = getattr(obj, '__name__', None) 953 if name is None: 954 raise ValueError("DocTestFinder.find: name must be given " 955 "when obj.__name__ doesn't exist: %r" % 956 (type(obj),)) 957 958 # Find the module that contains the given object (if obj is 959 # a module, then module=obj.). Note: this may fail, in which 960 # case module will be None. 961 if module is False: 962 module = None 963 elif module is None: 964 module = inspect.getmodule(obj) 965 966 # Read the module's source code. This is used by 967 # DocTestFinder._find_lineno to find the line number for a 968 # given object's docstring. 969 try: 970 file = inspect.getsourcefile(obj) or inspect.getfile(obj) 971 source_lines = linecache.getlines(file) 972 if not source_lines: 973 source_lines = None 974 except TypeError: 975 source_lines = None 976 977 # Initialize globals, and merge in extraglobs. 978 if globs is None: 979 if module is None: 980 globs = {} 981 else: 982 globs = module.__dict__.copy() 983 else: 984 globs = globs.copy() 985 if extraglobs is not None: 986 globs.update(extraglobs) 987 988 # Recursively expore `obj`, extracting DocTests. 989 tests = [] 990 self._find(tests, obj, name, module, source_lines, globs, {}) 991 return tests 992 993 def _filter(self, obj, prefix, base): 994 """ 995 Return true if the given object should not be examined. 996 """ 997 return (self._namefilter is not None and 998 self._namefilter(prefix, base)) 999 1000 def _from_module(self, module, object): 1001 """ 1002 Return true if the given object is defined in the given 1003 module. 1004 """ 1005 if module is None: 1006 return True 1007 elif inspect.isfunction(object): 1008 return module.__dict__ is object.func_globals 1009 elif inspect.isclass(object): 1010 return module.__name__ == object.__module__ 1011 elif inspect.getmodule(object) is not None: 1012 return module is inspect.getmodule(object) 1013 elif hasattr(object, '__module__'): 1014 return module.__name__ == object.__module__ 1015 elif isinstance(object, property): 1016 return True # [XX] no way not be sure. 1017 else: 1018 raise ValueError("object must be a class or function") 1019 1020 def _find(self, tests, obj, name, module, source_lines, globs, seen): 1021 """ 1022 Find tests for the given object and any contained objects, and 1023 add them to `tests`. 1024 """ 1025 if self._verbose: 1026 print 'Finding tests in %s' % name 1027 1028 # If we've already processed this object, then ignore it. 1029 if id(obj) in seen: 1030 return 1031 seen[id(obj)] = 1 1032 1033 # Find a test for this object, and add it to the list of tests. 1034 test = self._get_test(obj, name, module, globs, source_lines) 1035 if test is not None: 1036 tests.append(test) 1037 1038 # Look for tests in a module's contained objects. 1039 if inspect.ismodule(obj) and self._recurse: 1040 for valname, val in obj.__dict__.items(): 1041 # Check if this contained object should be ignored. 1042 if self._filter(val, name, valname): 1043 continue 1044 valname = '%s.%s' % (name, valname) 1045 # Recurse to functions & classes. 1046 if ((inspect.isfunction(val) or inspect.isclass(val)) and 1047 self._from_module(module, val)): 1048 self._find(tests, val, valname, module, source_lines, 1049 globs, seen) 1050 1051 # Look for tests in a module's __test__ dictionary. 1052 if inspect.ismodule(obj) and self._recurse: 1053 for valname, val in getattr(obj, '__test__', {}).items(): 1054 if not isinstance(valname, basestring): 1055 raise ValueError("DocTestFinder.find: __test__ keys " 1056 "must be strings: %r" % 1057 (type(valname),)) 1058 if not (inspect.isfunction(val) or inspect.isclass(val) or 1059 inspect.ismethod(val) or inspect.ismodule(val) or 1060 isinstance(val, basestring)): 1061 raise ValueError("DocTestFinder.find: __test__ values " 1062 "must be strings, functions, methods, " 1063 "classes, or modules: %r" % 1064 (type(val),)) 1065 valname = '%s.%s' % (name, valname) 1066 self._find(tests, val, valname, module, source_lines, 1067 globs, seen) 1068 1069 # Look for tests in a class's contained objects. 1070 if inspect.isclass(obj) and self._recurse: 1071 for valname, val in obj.__dict__.items(): 1072 # Check if this contained object should be ignored. 1073 if self._filter(val, name, valname): 1074 continue 1075 # Special handling for staticmethod/classmethod. 1076 if isinstance(val, staticmethod): 1077 val = getattr(obj, valname) 1078 if isinstance(val, classmethod): 1079 val = getattr(obj, valname).im_func 1080 1081 # Recurse to methods, properties, and nested classes. 1082 if ((inspect.isfunction(val) or inspect.isclass(val) or 1083 isinstance(val, property)) and 1084 self._from_module(module, val)): 1085 valname = '%s.%s' % (name, valname) 1086 self._find(tests, val, valname, module, source_lines, 1087 globs, seen) 1088 1089 def _get_test(self, obj, name, module, globs, source_lines): 1090 """ 1091 Return a DocTest for the given object, if it defines a docstring; 1092 otherwise, return None. 1093 """ 1094 # Extract the object's docstring. If it doesn't have one, 1095 # then return None (no test for this object). 1096 if isinstance(obj, basestring): 1097 docstring = obj 1098 else: 1099 try: 1100 if obj.__doc__ is None: 1101 return None 1102 docstring = str(obj.__doc__) 1103 except (TypeError, AttributeError): 1104 return None 1105 1106 # Don't bother if the docstring is empty. 1107 if not docstring: 1108 return None 1109 1110 # Find the docstring's location in the file. 1111 lineno = self._find_lineno(obj, source_lines) 1112 1113 # Return a DocTest for this object. 1114 if module is None: 1115 filename = None 1116 else: 1117 filename = getattr(module, '__file__', module.__name__) 1118 if filename[-4:] in (".pyc", ".pyo"): 1119 filename = filename[:-1] 1120 return self._parser.get_doctest(docstring, globs, name, 1121 filename, lineno) 1122 1123 def _find_lineno(self, obj, source_lines): 1124 """ 1125 Return a line number of the given object's docstring. Note: 1126 this method assumes that the object has a docstring. 1127 """ 1128 lineno = None 1129 1130 # Find the line number for modules. 1131 if inspect.ismodule(obj): 1132 lineno = 0 1133 1134 # Find the line number for classes. 1135 # Note: this could be fooled if a class is defined multiple 1136 # times in a single file. 1137 if inspect.isclass(obj): 1138 if source_lines is None: 1139 return None 1140 pat = re.compile(r'^\s*class\s*%s\b' % 1141 getattr(obj, '__name__', '-')) 1142 for i, line in enumerate(source_lines): 1143 if pat.match(line): 1144 lineno = i 1145 break 1146 1147 # Find the line number for functions & methods. 1148 if inspect.ismethod(obj): obj = obj.im_func 1149 if inspect.isfunction(obj): obj = obj.func_code 1150 if inspect.istraceback(obj): obj = obj.tb_frame 1151 if inspect.isframe(obj): obj = obj.f_code 1152 if inspect.iscode(obj): 1153 lineno = getattr(obj, 'co_firstlineno', None)-1 1154 1155 # Find the line number where the docstring starts. Assume 1156 # that it's the first line that begins with a quote mark. 1157 # Note: this could be fooled by a multiline function 1158 # signature, where a continuation line begins with a quote 1159 # mark. 1160 if lineno is not None: 1161 if source_lines is None: 1162 return lineno+1 1163 pat = re.compile('(^|.*:)\s*\w*("|\')') 1164 for lineno in range(lineno, len(source_lines)): 1165 if pat.match(source_lines[lineno]): 1166 return lineno 1167 1168 # We couldn't find the line number. 1169 return None 1170 1171###################################################################### 1172## 5. DocTest Runner 1173###################################################################### 1174 1175class DocTestRunner: 1176 """ 1177 A class used to run DocTest test cases, and accumulate statistics. 1178 The `run` method is used to process a single DocTest case. It 1179 returns a tuple `(f, t)`, where `t` is the number of test cases 1180 tried, and `f` is the number of test cases that failed. 1181 1182 >>> tests = DocTestFinder().find(_TestClass) 1183 >>> runner = DocTestRunner(verbose=False) 1184 >>> for test in tests: 1185 ... print runner.run(test) 1186 (0, 2) 1187 (0, 1) 1188 (0, 2) 1189 (0, 2) 1190 1191 The `summarize` method prints a summary of all the test cases that 1192 have been run by the runner, and returns an aggregated `(f, t)` 1193 tuple: 1194 1195 >>> runner.summarize(verbose=1) 1196 4 items passed all tests: 1197 2 tests in _TestClass 1198 2 tests in _TestClass.__init__ 1199 2 tests in _TestClass.get 1200 1 tests in _TestClass.square 1201 7 tests in 4 items. 1202 7 passed and 0 failed. 1203 Test passed. 1204 (0, 7) 1205 1206 The aggregated number of tried examples and failed examples is 1207 also available via the `tries` and `failures` attributes: 1208 1209 >>> runner.tries 1210 7 1211 >>> runner.failures 1212 0 1213 1214 The comparison between expected outputs and actual outputs is done 1215 by an `OutputChecker`. This comparison may be customized with a 1216 number of option flags; see the documentation for `testmod` for 1217 more information. If the option flags are insufficient, then the 1218 comparison may also be customized by passing a subclass of 1219 `OutputChecker` to the constructor. 1220 1221 The test runner's display output can be controlled in two ways. 1222 First, an output function (`out) can be passed to 1223 `TestRunner.run`; this function will be called with strings that 1224 should be displayed. It defaults to `sys.stdout.write`. If 1225 capturing the output is not sufficient, then the display output 1226 can be also customized by subclassing DocTestRunner, and 1227 overriding the methods `report_start`, `report_success`, 1228 `report_unexpected_exception`, and `report_failure`. 1229 """ 1230 # This divider string is used to separate failure messages, and to 1231 # separate sections of the summary. 1232 DIVIDER = "*" * 70 1233 1234 def __init__(self, checker=None, verbose=None, optionflags=0): 1235 """ 1236 Create a new test runner. 1237 1238 Optional keyword arg `checker` is the `OutputChecker` that 1239 should be used to compare the expected outputs and actual 1240 outputs of doctest examples. 1241 1242 Optional keyword arg 'verbose' prints lots of stuff if true, 1243 only failures if false; by default, it's true iff '-v' is in 1244 sys.argv. 1245 1246 Optional argument `optionflags` can be used to control how the 1247 test runner compares expected output to actual output, and how 1248 it displays failures. See the documentation for `testmod` for 1249 more information. 1250 """ 1251 self._checker = checker or OutputChecker() 1252 if verbose is None: 1253 verbose = '-v' in sys.argv 1254 self._verbose = verbose 1255 self.optionflags = optionflags 1256 self.original_optionflags = optionflags 1257 1258 # Keep track of the examples we've run. 1259 self.tries = 0 1260 self.failures = 0 1261 self._name2ft = {} 1262 1263 # Create a fake output target for capturing doctest output. 1264 self._fakeout = _SpoofOut() 1265 1266 #///////////////////////////////////////////////////////////////// 1267 # Reporting methods 1268 #///////////////////////////////////////////////////////////////// 1269 1270 def report_start(self, out, test, example): 1271 """ 1272 Report that the test runner is about to process the given 1273 example. (Only displays a message if verbose=True) 1274 """ 1275 if self._verbose: 1276 out(_tag_msg("Trying", example.source) + 1277 _tag_msg("Expecting", example.want or "nothing")) 1278 1279 def report_success(self, out, test, example, got): 1280 """ 1281 Report that the given example ran successfully. (Only 1282 displays a message if verbose=True) 1283 """ 1284 if self._verbose: 1285 out("ok\n") 1286 1287 def report_failure(self, out, test, example, got): 1288 """ 1289 Report that the given example failed. 1290 """ 1291 # Print an error message. 1292 out(self._failure_header(test, example) + 1293 self._checker.output_difference(example.want, got, 1294 self.optionflags)) 1295 1296 def report_unexpected_exception(self, out, test, example, exc_info): 1297 """ 1298 Report that the given example raised an unexpected exception. 1299 """ 1300 out(self._failure_header(test, example) + 1301 _tag_msg("Exception raised", _exception_traceback(exc_info))) 1302 1303 def _failure_header(self, test, example): 1304 out = [self.DIVIDER] 1305 if test.filename: 1306 if test.lineno is not None and example.lineno is not None: 1307 lineno = test.lineno + example.lineno + 1 1308 else: 1309 lineno = '?' 1310 out.append('File "%s", line %s, in %s' % 1311 (test.filename, lineno, test.name)) 1312 else: 1313 out.append('Line %s, in %s' % (example.lineno+1, test.name)) 1314 out.append('Failed example:') 1315 source = example.source 1316 if source.endswith('\n'): 1317 source = source[:-1] 1318 out.append(' ' + '\n '.join(source.split('\n'))) 1319 return '\n'.join(out)+'\n' 1320 1321 #///////////////////////////////////////////////////////////////// 1322 # DocTest Running 1323 #///////////////////////////////////////////////////////////////// 1324 1325 def __run(self, test, compileflags, out): 1326 """ 1327 Run the examples in `test`. Write the outcome of each example 1328 with one of the `DocTestRunner.report_*` methods, using the 1329 writer function `out`. `compileflags` is the set of compiler 1330 flags that should be used to execute examples. Return a tuple 1331 `(f, t)`, where `t` is the number of examples tried, and `f` 1332 is the number of examples that failed. The examples are run 1333 in the namespace `test.globs`. 1334 """ 1335 # Keep track of the number of failures and tries. 1336 failures = tries = 0 1337 1338 # Save the option flags (since option directives can be used 1339 # to modify them). 1340 original_optionflags = self.optionflags 1341 1342 # Process each example. 1343 for example in test.examples: 1344 # Merge in the example's options. 1345 self.optionflags = original_optionflags 1346 if example.options: 1347 for (optionflag, val) in example.options.items(): 1348 if val: 1349 self.optionflags |= optionflag 1350 else: 1351 self.optionflags &= ~optionflag 1352 1353 # Record that we started this example. 1354 tries += 1 1355 self.report_start(out, test, example) 1356 1357 # Run the example in the given context (globs), and record 1358 # any exception that gets raised. (But don't intercept 1359 # keyboard interrupts.) 1360 try: 1361 # Don't blink! This is where the user's code gets run. 1362 exec compile(example.source, "<string>", "single", 1363 compileflags, 1) in test.globs 1364 exception = None 1365 except KeyboardInterrupt: 1366 raise 1367 except: 1368 exception = sys.exc_info() 1369 1370 got = self._fakeout.getvalue() # the actual output 1371 self._fakeout.truncate(0) 1372 1373 # If the example executed without raising any exceptions, 1374 # then verify its output and report its outcome. 1375 if exception is None: 1376 if self._checker.check_output(example.want, got, 1377 self.optionflags): 1378 self.report_success(out, test, example, got) 1379 else: 1380 self.report_failure(out, test, example, got) 1381 failures += 1 1382 1383 # If the example raised an exception, then check if it was 1384 # expected. 1385 else: 1386 exc_info = sys.exc_info() 1387 exc_msg = traceback.format_exception_only(*exc_info[:2])[-1] 1388 1389 # If `example.exc_msg` is None, then we weren't 1390 # expecting an exception. 1391 if example.exc_msg is None: 1392 self.report_unexpected_exception(out, test, example, 1393 exc_info) 1394 failures += 1 1395 # If `example.exc_msg` matches the actual exception 1396 # message (`exc_msg`), then the example succeeds. 1397 elif (self._checker.check_output(example.exc_msg, exc_msg, 1398 self.optionflags)): 1399 self.report_success(out, test, example, 1400 got + _exception_traceback(exc_info)) 1401 # Otherwise, the example fails. 1402 else: 1403 self.report_failure(out, test, example, 1404 got + _exception_traceback(exc_info)) 1405 failures += 1 1406 1407 # Restore the option flags (in case they were modified) 1408 self.optionflags = original_optionflags 1409 1410 # Record and return the number of failures and tries. 1411 self.__record_outcome(test, failures, tries) 1412 return failures, tries 1413 1414 def __record_outcome(self, test, f, t): 1415 """ 1416 Record the fact that the given DocTest (`test`) generated `f` 1417 failures out of `t` tried examples. 1418 """ 1419 f2, t2 = self._name2ft.get(test.name, (0,0)) 1420 self._name2ft[test.name] = (f+f2, t+t2) 1421 self.failures += f 1422 self.tries += t 1423 1424 def run(self, test, compileflags=None, out=None, clear_globs=True): 1425 """ 1426 Run the examples in `test`, and display the results using the 1427 writer function `out`. 1428 1429 The examples are run in the namespace `test.globs`. If 1430 `clear_globs` is true (the default), then this namespace will 1431 be cleared after the test runs, to help with garbage 1432 collection. If you would like to examine the namespace after 1433 the test completes, then use `clear_globs=False`. 1434 1435 `compileflags` gives the set of flags that should be used by 1436 the Python compiler when running the examples. If not 1437 specified, then it will default to the set of future-import 1438 flags that apply to `globs`. 1439 1440 The output of each example is checked using 1441 `DocTestRunner.check_output`, and the results are formatted by 1442 the `DocTestRunner.report_*` methods. 1443 """ 1444 if compileflags is None: 1445 compileflags = _extract_future_flags(test.globs) 1446 1447 save_stdout = sys.stdout 1448 if out is None: 1449 out = save_stdout.write 1450 sys.stdout = self._fakeout 1451 1452 # Patch pdb.set_trace to restore sys.stdout, so that interactive 1453 # debugging output is visible (not still redirected to self._fakeout). 1454 # Note that we run "the real" pdb.set_trace (captured at doctest 1455 # import time) in our replacement. Because the current run() may 1456 # run another doctest (and so on), the current pdb.set_trace may be 1457 # our set_trace function, which changes sys.stdout. If we called 1458 # a chain of those, we wouldn't be left with the save_stdout 1459 # *this* run() invocation wants. 1460 def set_trace(): 1461 sys.stdout = save_stdout 1462 real_pdb_set_trace() 1463 1464 save_set_trace = pdb.set_trace 1465 pdb.set_trace = set_trace 1466 try: 1467 return self.__run(test, compileflags, out) 1468 finally: 1469 sys.stdout = save_stdout 1470 pdb.set_trace = save_set_trace 1471 if clear_globs: 1472 test.globs.clear() 1473 1474 #///////////////////////////////////////////////////////////////// 1475 # Summarization 1476 #///////////////////////////////////////////////////////////////// 1477 def summarize(self, verbose=None): 1478 """ 1479 Print a summary of all the test cases that have been run by 1480 this DocTestRunner, and return a tuple `(f, t)`, where `f` is 1481 the total number of failed examples, and `t` is the total 1482 number of tried examples. 1483 1484 The optional `verbose` argument controls how detailed the 1485 summary is. If the verbosity is not specified, then the 1486 DocTestRunner's verbosity is used. 1487 """ 1488 if verbose is None: 1489 verbose = self._verbose 1490 notests = [] 1491 passed = [] 1492 failed = [] 1493 totalt = totalf = 0 1494 for x in self._name2ft.items(): 1495 name, (f, t) = x 1496 assert f <= t 1497 totalt += t 1498 totalf += f 1499 if t == 0: 1500 notests.append(name) 1501 elif f == 0: 1502 passed.append( (name, t) ) 1503 else: 1504 failed.append(x) 1505 if verbose: 1506 if notests: 1507 print len(notests), "items had no tests:" 1508 notests.sort() 1509 for thing in notests: 1510 print " ", thing 1511 if passed: 1512 print len(passed), "items passed all tests:" 1513 passed.sort() 1514 for thing, count in passed: 1515 print " %3d tests in %s" % (count, thing) 1516 if failed: 1517 print self.DIVIDER 1518 print len(failed), "items had failures:" 1519 failed.sort() 1520 for thing, (f, t) in failed: 1521 print " %3d of %3d in %s" % (f, t, thing) 1522 if verbose: 1523 print totalt, "tests in", len(self._name2ft), "items." 1524 print totalt - totalf, "passed and", totalf, "failed." 1525 if totalf: 1526 print "***Test Failed***", totalf, "failures." 1527 elif verbose: 1528 print "Test passed." 1529 return totalf, totalt 1530 1531class OutputChecker: 1532 """ 1533 A class used to check the whether the actual output from a doctest 1534 example matches the expected output. `OutputChecker` defines two 1535 methods: `check_output`, which compares a given pair of outputs, 1536 and returns true if they match; and `output_difference`, which 1537 returns a string describing the differences between two outputs. 1538 """ 1539 def check_output(self, want, got, optionflags): 1540 """ 1541 Return True iff the actual output from an example (`got`) 1542 matches the expected output (`want`). These strings are 1543 always considered to match if they are identical; but 1544 depending on what option flags the test runner is using, 1545 several non-exact match types are also possible. See the 1546 documentation for `TestRunner` for more information about 1547 option flags. 1548 """ 1549 # Handle the common case first, for efficiency: 1550 # if they're string-identical, always return true. 1551 if got == want: 1552 return True 1553 1554 # The values True and False replaced 1 and 0 as the return 1555 # value for boolean comparisons in Python 2.3. 1556 if not (optionflags & DONT_ACCEPT_TRUE_FOR_1): 1557 if (got,want) == ("True\n", "1\n"): 1558 return True 1559 if (got,want) == ("False\n", "0\n"): 1560 return True 1561 1562 # <BLANKLINE> can be used as a special sequence to signify a 1563 # blank line, unless the DONT_ACCEPT_BLANKLINE flag is used. 1564 if not (optionflags & DONT_ACCEPT_BLANKLINE): 1565 # Replace <BLANKLINE> in want with a blank line. 1566 want = re.sub('(?m)^%s\s*?$' % re.escape(BLANKLINE_MARKER), 1567 '', want) 1568 # If a line in got contains only spaces, then remove the 1569 # spaces. 1570 got = re.sub('(?m)^\s*?$', '', got) 1571 if got == want: 1572 return True 1573 1574 # This flag causes doctest to ignore any differences in the 1575 # contents of whitespace strings. Note that this can be used 1576 # in conjunction with the ELLIPSIS flag. 1577 if optionflags & NORMALIZE_WHITESPACE: 1578 got = ' '.join(got.split()) 1579 want = ' '.join(want.split()) 1580 if got == want: 1581 return True 1582 1583 # The ELLIPSIS flag says to let the sequence "..." in `want` 1584 # match any substring in `got`. 1585 if optionflags & ELLIPSIS: 1586 if _ellipsis_match(want, got): 1587 return True 1588 1589 # We didn't find any match; return false. 1590 return False 1591 1592 # Should we do a fancy diff? 1593 def _do_a_fancy_diff(self, want, got, optionflags): 1594 # Not unless they asked for a fancy diff. 1595 if not optionflags & (UNIFIED_DIFF | 1596 CONTEXT_DIFF | 1597 NDIFF_DIFF): 1598 return False 1599 # If expected output uses ellipsis, a meaningful fancy diff is 1600 # too hard. 1601 if optionflags & ELLIPSIS and ELLIPSIS_MARKER in want: 1602 return False 1603 # ndiff does intraline difference marking, so can be useful even 1604 # for 1-line inputs. 1605 if optionflags & NDIFF_DIFF: 1606 return True 1607 # The other diff types need at least a few lines to be helpful. 1608 return want.count('\n') > 2 and got.count('\n') > 2 1609 1610 def output_difference(self, want, got, optionflags): 1611 """ 1612 Return a string describing the differences between the 1613 expected output for an example (`want`) and the actual output 1614 (`got`). `optionflags` is the set of option flags used to 1615 compare `want` and `got`. `indent` is the indentation of the 1616 original example. 1617 """ 1618 1619 # If <BLANKLINE>s are being used, then replace blank lines 1620 # with <BLANKLINE> in the actual output string. 1621 if not (optionflags & DONT_ACCEPT_BLANKLINE): 1622 got = re.sub('(?m)^[ ]*(?=\n)', BLANKLINE_MARKER, got) 1623 1624 # Check if we should use diff. Don't use diff if the actual 1625 # or expected outputs are too short, or if the expected output 1626 # contains an ellipsis marker. 1627 if self._do_a_fancy_diff(want, got, optionflags): 1628 # Split want & got into lines. 1629 want_lines = [l+'\n' for l in want.split('\n')] 1630 got_lines = [l+'\n' for l in got.split('\n')] 1631 # Use difflib to find their differences. 1632 if optionflags & UNIFIED_DIFF: 1633 diff = difflib.unified_diff(want_lines, got_lines, n=2, 1634 fromfile='Expected', tofile='Got') 1635 kind = 'unified diff' 1636 elif optionflags & CONTEXT_DIFF: 1637 diff = difflib.context_diff(want_lines, got_lines, n=2, 1638 fromfile='Expected', tofile='Got') 1639 kind = 'context diff' 1640 elif optionflags & NDIFF_DIFF: 1641 engine = difflib.Differ(charjunk=difflib.IS_CHARACTER_JUNK) 1642 diff = list(engine.compare(want_lines, got_lines)) 1643 kind = 'ndiff with -expected +actual' 1644 else: 1645 assert 0, 'Bad diff option' 1646 # Remove trailing whitespace on diff output. 1647 diff = [line.rstrip() + '\n' for line in diff] 1648 return _tag_msg("Differences (" + kind + ")", 1649 ''.join(diff)) 1650 1651 # If we're not using diff, then simply list the expected 1652 # output followed by the actual output. 1653 if want.endswith('\n'): 1654 want = want[:-1] 1655 want = ' ' + '\n '.join(want.split('\n')) 1656 if got.endswith('\n'): 1657 got = got[:-1] 1658 got = ' ' + '\n '.join(got.split('\n')) 1659 return "Expected:\n%s\nGot:\n%s\n" % (want, got) 1660 1661class DocTestFailure(Exception): 1662 """A DocTest example has failed in debugging mode. 1663 1664 The exception instance has variables: 1665 1666 - test: the DocTest object being run 1667 1668 - excample: the Example object that failed 1669 1670 - got: the actual output 1671 """ 1672 def __init__(self, test, example, got): 1673 self.test = test 1674 self.example = example 1675 self.got = got 1676 1677 def __str__(self): 1678 return str(self.test) 1679 1680class UnexpectedException(Exception): 1681 """A DocTest example has encountered an unexpected exception 1682 1683 The exception instance has variables: 1684 1685 - test: the DocTest object being run 1686 1687 - excample: the Example object that failed 1688 1689 - exc_info: the exception info 1690 """ 1691 def __init__(self, test, example, exc_info): 1692 self.test = test 1693 self.example = example 1694 self.exc_info = exc_info 1695 1696 def __str__(self): 1697 return str(self.test) 1698 1699class DebugRunner(DocTestRunner): 1700 r"""Run doc tests but raise an exception as soon as there is a failure. 1701 1702 If an unexpected exception occurs, an UnexpectedException is raised. 1703 It contains the test, the example, and the original exception: 1704 1705 >>> runner = DebugRunner(verbose=False) 1706 >>> test = DocTestParser().get_doctest('>>> raise KeyError\n42', 1707 ... {}, 'foo', 'foo.py', 0) 1708 >>> try: 1709 ... runner.run(test) 1710 ... except UnexpectedException, failure: 1711 ... pass 1712 1713 >>> failure.test is test 1714 True 1715 1716 >>> failure.example.want 1717 '42\n' 1718 1719 >>> exc_info = failure.exc_info 1720 >>> raise exc_info[0], exc_info[1], exc_info[2] 1721 Traceback (most recent call last): 1722 ... 1723 KeyError 1724 1725 We wrap the original exception to give the calling application 1726 access to the test and example information. 1727 1728 If the output doesn't match, then a DocTestFailure is raised: 1729 1730 >>> test = DocTestParser().get_doctest(''' 1731 ... >>> x = 1 1732 ... >>> x 1733 ... 2 1734 ... ''', {}, 'foo', 'foo.py', 0) 1735 1736 >>> try: 1737 ... runner.run(test) 1738 ... except DocTestFailure, failure: 1739 ... pass 1740 1741 DocTestFailure objects provide access to the test: 1742 1743 >>> failure.test is test 1744 True 1745 1746 As well as to the example: 1747 1748 >>> failure.example.want 1749 '2\n' 1750 1751 and the actual output: 1752 1753 >>> failure.got 1754 '1\n' 1755 1756 If a failure or error occurs, the globals are left intact: 1757 1758 >>> del test.globs['__builtins__'] 1759 >>> test.globs 1760 {'x': 1} 1761 1762 >>> test = DocTestParser().get_doctest(''' 1763 ... >>> x = 2 1764 ... >>> raise KeyError 1765 ... ''', {}, 'foo', 'foo.py', 0) 1766 1767 >>> runner.run(test) 1768 Traceback (most recent call last): 1769 ... 1770 UnexpectedException: <DocTest foo from foo.py:0 (2 examples)> 1771 1772 >>> del test.globs['__builtins__'] 1773 >>> test.globs 1774 {'x': 2} 1775 1776 But the globals are cleared if there is no error: 1777 1778 >>> test = DocTestParser().get_doctest(''' 1779 ... >>> x = 2 1780 ... ''', {}, 'foo', 'foo.py', 0) 1781 1782 >>> runner.run(test) 1783 (0, 1) 1784 1785 >>> test.globs 1786 {} 1787 1788 """ 1789 1790 def run(self, test, compileflags=None, out=None, clear_globs=True): 1791 r = DocTestRunner.run(self, test, compileflags, out, False) 1792 if clear_globs: 1793 test.globs.clear() 1794 return r 1795 1796 def report_unexpected_exception(self, out, test, example, exc_info): 1797 raise UnexpectedException(test, example, exc_info) 1798 1799 def report_failure(self, out, test, example, got): 1800 raise DocTestFailure(test, example, got) 1801 1802###################################################################### 1803## 6. Test Functions 1804###################################################################### 1805# These should be backwards compatible. 1806 1807def testmod(m=None, name=None, globs=None, verbose=None, isprivate=None, 1808 report=True, optionflags=0, extraglobs=None, 1809 raise_on_error=False): 1810 """m=None, name=None, globs=None, verbose=None, isprivate=None, 1811 report=True, optionflags=0, extraglobs=None 1812 1813 Test examples in docstrings in functions and classes reachable 1814 from module m (or the current module if m is not supplied), starting 1815 with m.__doc__. Unless isprivate is specified, private names 1816 are not skipped. 1817 1818 Also test examples reachable from dict m.__test__ if it exists and is 1819 not None. m.__test__ maps names to functions, classes and strings; 1820 function and class docstrings are tested even if the name is private; 1821 strings are tested directly, as if they were docstrings. 1822 1823 Return (#failures, #tests). 1824 1825 See doctest.__doc__ for an overview. 1826 1827 Optional keyword arg "name" gives the name of the module; by default 1828 use m.__name__. 1829 1830 Optional keyword arg "globs" gives a dict to be used as the globals 1831 when executing examples; by default, use m.__dict__. A copy of this 1832 dict is actually used for each docstring, so that each docstring's 1833 examples start with a clean slate. 1834 1835 Optional keyword arg "extraglobs" gives a dictionary that should be 1836 merged into the globals that are used to execute examples. By 1837 default, no extra globals are used. This is new in 2.4. 1838 1839 Optional keyword arg "verbose" prints lots of stuff if true, prints 1840 only failures if false; by default, it's true iff "-v" is in sys.argv. 1841 1842 Optional keyword arg "report" prints a summary at the end when true, 1843 else prints nothing at the end. In verbose mode, the summary is 1844 detailed, else very brief (in fact, empty if all tests passed). 1845 1846 Optional keyword arg "optionflags" or's together module constants, 1847 and defaults to 0. This is new in 2.3. Possible values (see the 1848 docs for details): 1849 1850 DONT_ACCEPT_TRUE_FOR_1 1851 DONT_ACCEPT_BLANKLINE 1852 NORMALIZE_WHITESPACE 1853 ELLIPSIS 1854 UNIFIED_DIFF 1855 CONTEXT_DIFF 1856 NDIFF_DIFF 1857 1858 Optional keyword arg "raise_on_error" raises an exception on the 1859 first unexpected exception or failure. This allows failures to be 1860 post-mortem debugged. 1861 1862 Deprecated in Python 2.4: 1863 Optional keyword arg "isprivate" specifies a function used to 1864 determine whether a name is private. The default function is 1865 treat all functions as public. Optionally, "isprivate" can be 1866 set to doctest.is_private to skip over functions marked as private 1867 using the underscore naming convention; see its docs for details. 1868 """ 1869 1870 """ [XX] This is no longer true: 1871 Advanced tomfoolery: testmod runs methods of a local instance of 1872 class doctest.Tester, then merges the results into (or creates) 1873 global Tester instance doctest.master. Methods of doctest.master 1874 can be called directly too, if you want to do something unusual. 1875 Passing report=0 to testmod is especially useful then, to delay 1876 displaying a summary. Invoke doctest.master.summarize(verbose) 1877 when you're done fiddling. 1878 """ 1879 if isprivate is not None: 1880 warnings.warn("the isprivate argument is deprecated; " 1881 "examine DocTestFinder.find() lists instead", 1882 DeprecationWarning) 1883 1884 # If no module was given, then use __main__. 1885 if m is None: 1886 # DWA - m will still be None if this wasn't invoked from the command 1887 # line, in which case the following TypeError is about as good an error 1888 # as we should expect 1889 m = sys.modules.get('__main__') 1890 1891 # Check that we were actually given a module. 1892 if not inspect.ismodule(m): 1893 raise TypeError("testmod: module required; %r" % (m,)) 1894 1895 # If no name was given, then use the module's name. 1896 if name is None: 1897 name = m.__name__ 1898 1899 # Find, parse, and run all tests in the given module. 1900 finder = DocTestFinder(_namefilter=isprivate) 1901 1902 if raise_on_error: 1903 runner = DebugRunner(verbose=verbose, optionflags=optionflags) 1904 else: 1905 runner = DocTestRunner(verbose=verbose, optionflags=optionflags) 1906 1907 for test in finder.find(m, name, globs=globs, extraglobs=extraglobs): 1908 runner.run(test) 1909 1910 if report: 1911 runner.summarize() 1912 1913 return runner.failures, runner.tries 1914 1915def run_docstring_examples(f, globs, verbose=False, name="NoName", 1916 compileflags=None, optionflags=0): 1917 """ 1918 Test examples in the given object's docstring (`f`), using `globs` 1919 as globals. Optional argument `name` is used in failure messages. 1920 If the optional argument `verbose` is true, then generate output 1921 even if there are no failures. 1922 1923 `compileflags` gives the set of flags that should be used by the 1924 Python compiler when running the examples. If not specified, then 1925 it will default to the set of future-import flags that apply to 1926 `globs`. 1927 1928 Optional keyword arg `optionflags` specifies options for the 1929 testing and output. See the documentation for `testmod` for more 1930 information. 1931 """ 1932 # Find, parse, and run all tests in the given module. 1933 finder = DocTestFinder(verbose=verbose, recurse=False) 1934 runner = DocTestRunner(verbose=verbose, optionflags=optionflags) 1935 for test in finder.find(f, name, globs=globs): 1936 runner.run(test, compileflags=compileflags) 1937 1938###################################################################### 1939## 7. Tester 1940###################################################################### 1941# This is provided only for backwards compatibility. It's not 1942# actually used in any way. 1943 1944class Tester: 1945 def __init__(self, mod=None, globs=None, verbose=None, 1946 isprivate=None, optionflags=0): 1947 1948 warnings.warn("class Tester is deprecated; " 1949 "use class doctest.DocTestRunner instead", 1950 DeprecationWarning, stacklevel=2) 1951 if mod is None and globs is None: 1952 raise TypeError("Tester.__init__: must specify mod or globs") 1953 if mod is not None and not _ismodule(mod): 1954 raise TypeError("Tester.__init__: mod must be a module; %r" % 1955 (mod,)) 1956 if globs is None: 1957 globs = mod.__dict__ 1958 self.globs = globs 1959 1960 self.verbose = verbose 1961 self.isprivate = isprivate 1962 self.optionflags = optionflags 1963 self.testfinder = DocTestFinder(_namefilter=isprivate) 1964 self.testrunner = DocTestRunner(verbose=verbose, 1965 optionflags=optionflags) 1966 1967 def runstring(self, s, name): 1968 test = DocTestParser().get_doctest(s, self.globs, name, None, None) 1969 if self.verbose: 1970 print "Running string", name 1971 (f,t) = self.testrunner.run(test) 1972 if self.verbose: 1973 print f, "of", t, "examples failed in string", name 1974 return (f,t) 1975 1976 def rundoc(self, object, name=None, module=None): 1977 f = t = 0 1978 tests = self.testfinder.find(object, name, module=module, 1979 globs=self.globs) 1980 for test in tests: 1981 (f2, t2) = self.testrunner.run(test) 1982 (f,t) = (f+f2, t+t2) 1983 return (f,t) 1984 1985 def rundict(self, d, name, module=None): 1986 import new 1987 m = new.module(name) 1988 m.__dict__.update(d) 1989 if module is None: 1990 module = False 1991 return self.rundoc(m, name, module) 1992 1993 def run__test__(self, d, name): 1994 import new 1995 m = new.module(name) 1996 m.__test__ = d 1997 return self.rundoc(m, name, module) 1998 1999 def summarize(self, verbose=None): 2000 return self.testrunner.summarize(verbose) 2001 2002 def merge(self, other): 2003 d = self.testrunner._name2ft 2004 for name, (f, t) in other.testrunner._name2ft.items(): 2005 if name in d: 2006 print "*** Tester.merge: '" + name + "' in both" \ 2007 " testers; summing outcomes." 2008 f2, t2 = d[name] 2009 f = f + f2 2010 t = t + t2 2011 d[name] = f, t 2012 2013###################################################################### 2014## 8. Unittest Support 2015###################################################################### 2016 2017class DocTestCase(unittest.TestCase): 2018 2019 def __init__(self, test, optionflags=0, setUp=None, tearDown=None, 2020 checker=None): 2021 2022 unittest.TestCase.__init__(self) 2023 self._dt_optionflags = optionflags 2024 self._dt_checker = checker 2025 self._dt_test = test 2026 self._dt_setUp = setUp 2027 self._dt_tearDown = tearDown 2028 2029 def setUp(self): 2030 if self._dt_setUp is not None: 2031 self._dt_setUp() 2032 2033 def tearDown(self): 2034 if self._dt_tearDown is not None: 2035 self._dt_tearDown() 2036 2037 def runTest(self): 2038 test = self._dt_test 2039 old = sys.stdout 2040 new = StringIO() 2041 runner = DocTestRunner(optionflags=self._dt_optionflags, 2042 checker=self._dt_checker, verbose=False) 2043 2044 try: 2045 runner.DIVIDER = "-"*70 2046 failures, tries = runner.run(test, out=new.write) 2047 finally: 2048 sys.stdout = old 2049 2050 if failures: 2051 raise self.failureException(self.format_failure(new.getvalue())) 2052 2053 def format_failure(self, err): 2054 test = self._dt_test 2055 if test.lineno is None: 2056 lineno = 'unknown line number' 2057 else: 2058 lineno = '%s' % test.lineno 2059 lname = '.'.join(test.name.split('.')[-1:]) 2060 return ('Failed doctest test for %s\n' 2061 ' File "%s", line %s, in %s\n\n%s' 2062 % (test.name, test.filename, lineno, lname, err) 2063 ) 2064 2065 def debug(self): 2066 r"""Run the test case without results and without catching exceptions 2067 2068 The unit test framework includes a debug method on test cases 2069 and test suites to support post-mortem debugging. The test code 2070 is run in such a way that errors are not caught. This way a 2071 caller can catch the errors and initiate post-mortem debugging. 2072 2073 The DocTestCase provides a debug method that raises 2074 UnexpectedException errors if there is an unexepcted 2075 exception: 2076 2077 >>> test = DocTestParser().get_doctest('>>> raise KeyError\n42', 2078 ... {}, 'foo', 'foo.py', 0) 2079 >>> case = DocTestCase(test) 2080 >>> try: 2081 ... case.debug() 2082 ... except UnexpectedException, failure: 2083 ... pass 2084 2085 The UnexpectedException contains the test, the example, and 2086 the original exception: 2087 2088 >>> failure.test is test 2089 True 2090 2091 >>> failure.example.want 2092 '42\n' 2093 2094 >>> exc_info = failure.exc_info 2095 >>> raise exc_info[0], exc_info[1], exc_info[2] 2096 Traceback (most recent call last): 2097 ... 2098 KeyError 2099 2100 If the output doesn't match, then a DocTestFailure is raised: 2101 2102 >>> test = DocTestParser().get_doctest(''' 2103 ... >>> x = 1 2104 ... >>> x 2105 ... 2 2106 ... ''', {}, 'foo', 'foo.py', 0) 2107 >>> case = DocTestCase(test) 2108 2109 >>> try: 2110 ... case.debug() 2111 ... except DocTestFailure, failure: 2112 ... pass 2113 2114 DocTestFailure objects provide access to the test: 2115 2116 >>> failure.test is test 2117 True 2118 2119 As well as to the example: 2120 2121 >>> failure.example.want 2122 '2\n' 2123 2124 and the actual output: 2125 2126 >>> failure.got 2127 '1\n' 2128 2129 """ 2130 2131 runner = DebugRunner(optionflags=self._dt_optionflags, 2132 checker=self._dt_checker, verbose=False) 2133 runner.run(self._dt_test) 2134 2135 def id(self): 2136 return self._dt_test.name 2137 2138 def __repr__(self): 2139 name = self._dt_test.name.split('.') 2140 return "%s (%s)" % (name[-1], '.'.join(name[:-1])) 2141 2142 __str__ = __repr__ 2143 2144 def shortDescription(self): 2145 return "Doctest: " + self._dt_test.name 2146 2147def DocTestSuite(module=None, globs=None, extraglobs=None, 2148 optionflags=0, test_finder=None, 2149 setUp=lambda: None, tearDown=lambda: None, 2150 checker=None): 2151 """ 2152 Convert doctest tests for a module to a unittest test suite. 2153 2154 This converts each documentation string in a module that 2155 contains doctest tests to a unittest test case. If any of the 2156 tests in a doc string fail, then the test case fails. An exception 2157 is raised showing the name of the file containing the test and a 2158 (sometimes approximate) line number. 2159 2160 The `module` argument provides the module to be tested. The argument 2161 can be either a module or a module name. 2162 2163 If no argument is given, the calling module is used. 2164 """ 2165 2166 if test_finder is None: 2167 test_finder = DocTestFinder() 2168 2169 module = _normalize_module(module) 2170 tests = test_finder.find(module, globs=globs, extraglobs=extraglobs) 2171 if globs is None: 2172 globs = module.__dict__ 2173 if not tests: # [XX] why do we want to do this? 2174 raise ValueError(module, "has no tests") 2175 2176 tests.sort() 2177 suite = unittest.TestSuite() 2178 for test in tests: 2179 if len(test.examples) == 0: 2180 continue 2181 if not test.filename: 2182 filename = module.__file__ 2183 if filename[-4:] in (".pyc", ".pyo"): 2184 filename = filename[:-1] 2185 test.filename = filename 2186 suite.addTest(DocTestCase(test, optionflags, setUp, tearDown, 2187 checker)) 2188 2189 return suite 2190 2191class DocFileCase(DocTestCase): 2192 2193 def id(self): 2194 return '_'.join(self._dt_test.name.split('.')) 2195 2196 def __repr__(self): 2197 return self._dt_test.filename 2198 __str__ = __repr__ 2199 2200 def format_failure(self, err): 2201 return ('Failed doctest test for %s\n File "%s", line 0\n\n%s' 2202 % (self._dt_test.name, self._dt_test.filename, err) 2203 ) 2204 2205def DocFileTest(path, package=None, globs=None, 2206 setUp=None, tearDown=None, 2207 optionflags=0): 2208 package = _normalize_module(package) 2209 name = path.split('/')[-1] 2210 dir = os.path.split(package.__file__)[0] 2211 path = os.path.join(dir, *(path.split('/'))) 2212 doc = open(path).read() 2213 2214 if globs is None: 2215 globs = {} 2216 2217 test = DocTestParser().get_doctest(doc, globs, name, path, 0) 2218 2219 return DocFileCase(test, optionflags, setUp, tearDown) 2220 2221def DocFileSuite(*paths, **kw): 2222 """Creates a suite of doctest files. 2223 2224 One or more text file paths are given as strings. These should 2225 use "/" characters to separate path segments. Paths are relative 2226 to the directory of the calling module, or relative to the package 2227 passed as a keyword argument. 2228 2229 A number of options may be provided as keyword arguments: 2230 2231 package 2232 The name of a Python package. Text-file paths will be 2233 interpreted relative to the directory containing this package. 2234 The package may be supplied as a package object or as a dotted 2235 package name. 2236 2237 setUp 2238 The name of a set-up function. This is called before running the 2239 tests in each file. 2240 2241 tearDown 2242 The name of a tear-down function. This is called after running the 2243 tests in each file. 2244 2245 globs 2246 A dictionary containing initial global variables for the tests. 2247 """ 2248 suite = unittest.TestSuite() 2249 2250 # We do this here so that _normalize_module is called at the right 2251 # level. If it were called in DocFileTest, then this function 2252 # would be the caller and we might guess the package incorrectly. 2253 kw['package'] = _normalize_module(kw.get('package')) 2254 2255 for path in paths: 2256 suite.addTest(DocFileTest(path, **kw)) 2257 2258 return suite 2259 2260###################################################################### 2261## 9. Debugging Support 2262###################################################################### 2263 2264def script_from_examples(s): 2265 r"""Extract script from text with examples. 2266 2267 Converts text with examples to a Python script. Example input is 2268 converted to regular code. Example output and all other words 2269 are converted to comments: 2270 2271 >>> text = ''' 2272 ... Here are examples of simple math. 2273 ... 2274 ... Python has super accurate integer addition 2275 ... 2276 ... >>> 2 + 2 2277 ... 5 2278 ... 2279 ... And very friendly error messages: 2280 ... 2281 ... >>> 1/0 2282 ... To Infinity 2283 ... And 2284 ... Beyond 2285 ... 2286 ... You can use logic if you want: 2287 ... 2288 ... >>> if 0: 2289 ... ... blah 2290 ... ... blah 2291 ... ... 2292 ... 2293 ... Ho hum 2294 ... ''' 2295 2296 >>> print script_from_examples(text) 2297 # Here are examples of simple math. 2298 # 2299 # Python has super accurate integer addition 2300 # 2301 2 + 2 2302 # Expected: 2303 ## 5 2304 # 2305 # And very friendly error messages: 2306 # 2307 1/0 2308 # Expected: 2309 ## To Infinity 2310 ## And 2311 ## Beyond 2312 # 2313 # You can use logic if you want: 2314 # 2315 if 0: 2316 blah 2317 blah 2318 <BLANKLINE> 2319 # 2320 # Ho hum 2321 """ 2322 2323 return DocTestParser().get_program(s) 2324 2325def _want_comment(example): 2326 """ 2327 Return a comment containing the expected output for the given example. 2328 """ 2329 # Return the expected output, if any 2330 want = example.want 2331 if want: 2332 if want[-1] == '\n': 2333 want = want[:-1] 2334 want = "\n# ".join(want.split("\n")) 2335 want = "\n# Expected:\n# %s" % want 2336 return want 2337 2338def testsource(module, name): 2339 """Extract the test sources from a doctest docstring as a script. 2340 2341 Provide the module (or dotted name of the module) containing the 2342 test to be debugged and the name (within the module) of the object 2343 with the doc string with tests to be debugged. 2344 """ 2345 module = _normalize_module(module) 2346 tests = DocTestFinder().find(module) 2347 test = [t for t in tests if t.name == name] 2348 if not test: 2349 raise ValueError(name, "not found in tests") 2350 test = test[0] 2351 testsrc = script_from_examples(test.docstring) 2352 return testsrc 2353 2354def debug_src(src, pm=False, globs=None): 2355 """Debug a single doctest docstring, in argument `src`'""" 2356 testsrc = script_from_examples(src) 2357 debug_script(testsrc, pm, globs) 2358 2359def debug_script(src, pm=False, globs=None): 2360 "Debug a test script. `src` is the script, as a string." 2361 import pdb 2362 2363 # Note that tempfile.NameTemporaryFile() cannot be used. As the 2364 # docs say, a file so created cannot be opened by name a second time 2365 # on modern Windows boxes, and execfile() needs to open it. 2366 srcfilename = tempfile.mktemp(".py", "doctestdebug") 2367 f = open(srcfilename, 'w') 2368 f.write(src) 2369 f.close() 2370 2371 try: 2372 if globs: 2373 globs = globs.copy() 2374 else: 2375 globs = {} 2376 2377 if pm: 2378 try: 2379 execfile(srcfilename, globs, globs) 2380 except: 2381 print sys.exc_info()[1] 2382 pdb.post_mortem(sys.exc_info()[2]) 2383 else: 2384 # Note that %r is vital here. '%s' instead can, e.g., cause 2385 # backslashes to get treated as metacharacters on Windows. 2386 pdb.run("execfile(%r)" % srcfilename, globs, globs) 2387 2388 finally: 2389 os.remove(srcfilename) 2390 2391def debug(module, name, pm=False): 2392 """Debug a single doctest docstring. 2393 2394 Provide the module (or dotted name of the module) containing the 2395 test to be debugged and the name (within the module) of the object 2396 with the docstring with tests to be debugged. 2397 """ 2398 module = _normalize_module(module) 2399 testsrc = testsource(module, name) 2400 debug_script(testsrc, pm, module.__dict__) 2401 2402###################################################################### 2403## 10. Example Usage 2404###################################################################### 2405class _TestClass: 2406 """ 2407 A pointless class, for sanity-checking of docstring testing. 2408 2409 Methods: 2410 square() 2411 get() 2412 2413 >>> _TestClass(13).get() + _TestClass(-12).get() 2414 1 2415 >>> hex(_TestClass(13).square().get()) 2416 '0xa9' 2417 """ 2418 2419 def __init__(self, val): 2420 """val -> _TestClass object with associated value val. 2421 2422 >>> t = _TestClass(123) 2423 >>> print t.get() 2424 123 2425 """ 2426 2427 self.val = val 2428 2429 def square(self): 2430 """square() -> square TestClass's associated value 2431 2432 >>> _TestClass(13).square().get() 2433 169 2434 """ 2435 2436 self.val = self.val ** 2 2437 return self 2438 2439 def get(self): 2440 """get() -> return TestClass's associated value. 2441 2442 >>> x = _TestClass(-42) 2443 >>> print x.get() 2444 -42 2445 """ 2446 2447 return self.val 2448 2449__test__ = {"_TestClass": _TestClass, 2450 "string": r""" 2451 Example of a string object, searched as-is. 2452 >>> x = 1; y = 2 2453 >>> x + y, x * y 2454 (3, 2) 2455 """, 2456 2457 "bool-int equivalence": r""" 2458 In 2.2, boolean expressions displayed 2459 0 or 1. By default, we still accept 2460 them. This can be disabled by passing 2461 DONT_ACCEPT_TRUE_FOR_1 to the new 2462 optionflags argument. 2463 >>> 4 == 4 2464 1 2465 >>> 4 == 4 2466 True 2467 >>> 4 > 4 2468 0 2469 >>> 4 > 4 2470 False 2471 """, 2472 2473 "blank lines": r""" 2474 Blank lines can be marked with <BLANKLINE>: 2475 >>> print 'foo\n\nbar\n' 2476 foo 2477 <BLANKLINE> 2478 bar 2479 <BLANKLINE> 2480 """, 2481 2482 "ellipsis": r""" 2483 If the ellipsis flag is used, then '...' can be used to 2484 elide substrings in the desired output: 2485 >>> print range(1000) #doctest: +ELLIPSIS 2486 [0, 1, 2, ..., 999] 2487 """, 2488 2489 "whitespace normalization": r""" 2490 If the whitespace normalization flag is used, then 2491 differences in whitespace are ignored. 2492 >>> print range(30) #doctest: +NORMALIZE_WHITESPACE 2493 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 2494 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 2495 27, 28, 29] 2496 """, 2497 } 2498 2499def _test(): 2500 r = unittest.TextTestRunner() 2501 r.run(DocTestSuite()) 2502 2503if __name__ == "__main__": 2504 _test() 2505