doctest.py revision 3fa8c2079a164254b037a975a811aae946845815
1# Module doctest. 2# Released to the public domain 16-Jan-2001, by Tim Peters (tim@python.org). 3# Major enhancements and refactoring by: 4# Jim Fulton 5# Edward Loper 6 7# Provided as-is; use at your own risk; no warranty; no promises; enjoy! 8 9r"""Module doctest -- a framework for running examples in docstrings. 10 11NORMAL USAGE 12 13In simplest use, end each module M to be tested with: 14 15def _test(): 16 import doctest 17 return doctest.testmod() 18 19if __name__ == "__main__": 20 _test() 21 22Then running the module as a script will cause the examples in the 23docstrings to get executed and verified: 24 25python M.py 26 27This won't display anything unless an example fails, in which case the 28failing example(s) and the cause(s) of the failure(s) are printed to stdout 29(why not stderr? because stderr is a lame hack <0.2 wink>), and the final 30line of output is "Test failed.". 31 32Run it with the -v switch instead: 33 34python M.py -v 35 36and a detailed report of all examples tried is printed to stdout, along 37with assorted summaries at the end. 38 39You can force verbose mode by passing "verbose=True" to testmod, or prohibit 40it by passing "verbose=False". In either of those cases, sys.argv is not 41examined by testmod. 42 43In any case, testmod returns a 2-tuple of ints (f, t), where f is the 44number of docstring examples that failed and t is the total number of 45docstring examples attempted. 46 47There are a variety of other ways to run doctests, including integration 48with the unittest framework, and support for running non-Python text 49files containing doctests. There are also many ways to override parts 50of doctest's default behaviors. See the Library Reference Manual for 51details. 52 53 54WHICH DOCSTRINGS ARE EXAMINED? 55 56+ M.__doc__. 57 58+ f.__doc__ for all functions f in M.__dict__.values(), except those 59 defined in other modules. 60 61+ C.__doc__ for all classes C in M.__dict__.values(), except those 62 defined in other modules. 63 64+ If M.__test__ exists and "is true", it must be a dict, and 65 each entry maps a (string) name to a function object, class object, or 66 string. Function and class object docstrings found from M.__test__ 67 are searched, and strings are searched directly as if they were docstrings. 68 In output, a key K in M.__test__ appears with name 69 <name of M>.__test__.K 70 71Any classes found are recursively searched similarly, to test docstrings in 72their contained methods and nested classes. 73 74 75WHAT'S THE EXECUTION CONTEXT? 76 77By default, each time testmod finds a docstring to test, it uses a *copy* 78of M's globals (so that running tests on a module doesn't change the 79module's real globals, and so that one test in M can't leave behind crumbs 80that accidentally allow another test to work). This means examples can 81freely use any names defined at top-level in M. It also means that sloppy 82imports (see above) can cause examples in external docstrings to use 83globals inappropriate for them. 84 85You can force use of your own dict as the execution context by passing 86"globs=your_dict" to testmod instead. Presumably this would be a copy of 87M.__dict__ merged with the globals from other imported modules. 88 89 90WHAT ABOUT EXCEPTIONS? 91 92No problem, as long as the only output generated by the example is the 93traceback itself. For example: 94 95 >>> [1, 2, 3].remove(42) 96 Traceback (most recent call last): 97 File "<stdin>", line 1, in ? 98 ValueError: list.remove(x): x not in list 99 >>> 100 101Note that only the exception type and value are compared. 102 103 104SO WHAT DOES A DOCTEST EXAMPLE LOOK LIKE ALREADY!? 105 106Oh ya. It's easy! In most cases a copy-and-paste of an interactive 107console session works fine -- just make sure the leading whitespace is 108rigidly consistent (you can mix tabs and spaces if you're too lazy to do it 109right, but doctest is not in the business of guessing what you think a tab 110means). 111 112 >>> # comments are ignored 113 >>> x = 12 114 >>> x 115 12 116 >>> if x == 13: 117 ... print "yes" 118 ... else: 119 ... print "no" 120 ... print "NO" 121 ... print "NO!!!" 122 ... 123 no 124 NO 125 NO!!! 126 >>> 127 128Any expected output must immediately follow the final ">>>" or "..." line 129containing the code, and the expected output (if any) extends to the next 130">>>" or all-whitespace line. That's it. 131 132Bummers: 133 134+ Output to stdout is captured, but not output to stderr (exception 135 tracebacks are captured via a different means). 136 137+ If you continue a line via backslashing in an interactive session, 138 or for any other reason use a backslash, you should use a raw 139 docstring, which will preserve your backslahses exactly as you type 140 them: 141 142 >>> def f(x): 143 ... r'''Backslashes in a raw docstring: m\n''' 144 >>> print f.__doc__ 145 Backslashes in a raw docstring: m\n 146 147 Otherwise, the backslash will be interpreted as part of the string. 148 E.g., the "\n" above would be interpreted as a newline character. 149 Alternatively, you can double each backslash in the doctest version 150 (and not use a raw string): 151 152 >>> def f(x): 153 ... '''Backslashes in a raw docstring: m\\n''' 154 >>> print f.__doc__ 155 Backslashes in a raw docstring: m\n 156 157The starting column doesn't matter: 158 159>>> assert "Easy!" 160 >>> import math 161 >>> math.floor(1.9) 162 1.0 163 164and as many leading whitespace characters are stripped from the expected 165output as appeared in the initial ">>>" line that triggered it. 166 167If you execute this very file, the examples above will be found and 168executed. 169""" 170__docformat__ = 'reStructuredText en' 171 172__all__ = [ 173 # 0, Option Flags 174 'register_optionflag', 175 'DONT_ACCEPT_TRUE_FOR_1', 176 'DONT_ACCEPT_BLANKLINE', 177 'NORMALIZE_WHITESPACE', 178 'ELLIPSIS', 179 'UNIFIED_DIFF', 180 'CONTEXT_DIFF', 181 'NDIFF_DIFF', 182 # 1. Utility Functions 183 'is_private', 184 # 2. Example & DocTest 185 'Example', 186 'DocTest', 187 # 3. Doctest Parser 188 'DocTestParser', 189 # 4. Doctest Finder 190 'DocTestFinder', 191 # 5. Doctest Runner 192 'DocTestRunner', 193 'OutputChecker', 194 'DocTestFailure', 195 'UnexpectedException', 196 'DebugRunner', 197 # 6. Test Functions 198 'testmod', 199 'run_docstring_examples', 200 # 7. Tester 201 'Tester', 202 # 8. Unittest Support 203 'DocTestCase', 204 'DocTestSuite', 205 'DocFileCase', 206 'DocFileTest', 207 'DocFileSuite', 208 # 9. Debugging Support 209 'script_from_examples', 210 'testsource', 211 'debug_src', 212 'debug_script', 213 'debug', 214] 215 216import __future__ 217 218import sys, traceback, inspect, linecache, os, re, types 219import unittest, difflib, pdb, tempfile 220import warnings 221from StringIO import StringIO 222 223real_pdb_set_trace = pdb.set_trace 224 225# There are 4 basic classes: 226# - Example: a <source, want> pair, plus an intra-docstring line number. 227# - DocTest: a collection of examples, parsed from a docstring, plus 228# info about where the docstring came from (name, filename, lineno). 229# - DocTestFinder: extracts DocTests from a given object's docstring and 230# its contained objects' docstrings. 231# - DocTestRunner: runs DocTest cases, and accumulates statistics. 232# 233# So the basic picture is: 234# 235# list of: 236# +------+ +---------+ +-------+ 237# |object| --DocTestFinder-> | DocTest | --DocTestRunner-> |results| 238# +------+ +---------+ +-------+ 239# | Example | 240# | ... | 241# | Example | 242# +---------+ 243 244# Option constants. 245OPTIONFLAGS_BY_NAME = {} 246def register_optionflag(name): 247 flag = 1 << len(OPTIONFLAGS_BY_NAME) 248 OPTIONFLAGS_BY_NAME[name] = flag 249 return flag 250 251DONT_ACCEPT_TRUE_FOR_1 = register_optionflag('DONT_ACCEPT_TRUE_FOR_1') 252DONT_ACCEPT_BLANKLINE = register_optionflag('DONT_ACCEPT_BLANKLINE') 253NORMALIZE_WHITESPACE = register_optionflag('NORMALIZE_WHITESPACE') 254ELLIPSIS = register_optionflag('ELLIPSIS') 255UNIFIED_DIFF = register_optionflag('UNIFIED_DIFF') 256CONTEXT_DIFF = register_optionflag('CONTEXT_DIFF') 257NDIFF_DIFF = register_optionflag('NDIFF_DIFF') 258 259# Special string markers for use in `want` strings: 260BLANKLINE_MARKER = '<BLANKLINE>' 261ELLIPSIS_MARKER = '...' 262 263###################################################################### 264## Table of Contents 265###################################################################### 266# 1. Utility Functions 267# 2. Example & DocTest -- store test cases 268# 3. DocTest Parser -- extracts examples from strings 269# 4. DocTest Finder -- extracts test cases from objects 270# 5. DocTest Runner -- runs test cases 271# 6. Test Functions -- convenient wrappers for testing 272# 7. Tester Class -- for backwards compatibility 273# 8. Unittest Support 274# 9. Debugging Support 275# 10. Example Usage 276 277###################################################################### 278## 1. Utility Functions 279###################################################################### 280 281def is_private(prefix, base): 282 """prefix, base -> true iff name prefix + "." + base is "private". 283 284 Prefix may be an empty string, and base does not contain a period. 285 Prefix is ignored (although functions you write conforming to this 286 protocol may make use of it). 287 Return true iff base begins with an (at least one) underscore, but 288 does not both begin and end with (at least) two underscores. 289 290 >>> warnings.filterwarnings("ignore", "is_private", DeprecationWarning, 291 ... "doctest", 0) 292 >>> is_private("a.b", "my_func") 293 False 294 >>> is_private("____", "_my_func") 295 True 296 >>> is_private("someclass", "__init__") 297 False 298 >>> is_private("sometypo", "__init_") 299 True 300 >>> is_private("x.y.z", "_") 301 True 302 >>> is_private("_x.y.z", "__") 303 False 304 >>> is_private("", "") # senseless but consistent 305 False 306 """ 307 warnings.warn("is_private is deprecated; it wasn't useful; " 308 "examine DocTestFinder.find() lists instead", 309 DeprecationWarning, stacklevel=2) 310 return base[:1] == "_" and not base[:2] == "__" == base[-2:] 311 312def _extract_future_flags(globs): 313 """ 314 Return the compiler-flags associated with the future features that 315 have been imported into the given namespace (globs). 316 """ 317 flags = 0 318 for fname in __future__.all_feature_names: 319 feature = globs.get(fname, None) 320 if feature is getattr(__future__, fname): 321 flags |= feature.compiler_flag 322 return flags 323 324def _normalize_module(module, depth=2): 325 """ 326 Return the module specified by `module`. In particular: 327 - If `module` is a module, then return module. 328 - If `module` is a string, then import and return the 329 module with that name. 330 - If `module` is None, then return the calling module. 331 The calling module is assumed to be the module of 332 the stack frame at the given depth in the call stack. 333 """ 334 if inspect.ismodule(module): 335 return module 336 elif isinstance(module, (str, unicode)): 337 return __import__(module, globals(), locals(), ["*"]) 338 elif module is None: 339 return sys.modules[sys._getframe(depth).f_globals['__name__']] 340 else: 341 raise TypeError("Expected a module, string, or None") 342 343def _tag_msg(tag, msg, indent=' '): 344 """ 345 Return a string that displays a tag-and-message pair nicely, 346 keeping the tag and its message on the same line when that 347 makes sense. If the message is displayed on separate lines, 348 then `indent` is added to the beginning of each line. 349 """ 350 # If the message doesn't end in a newline, then add one. 351 if msg[-1:] != '\n': 352 msg += '\n' 353 # If the message is short enough, and contains no internal 354 # newlines, then display it on the same line as the tag. 355 # Otherwise, display the tag on its own line. 356 if (len(tag) + len(msg) < 75 and 357 msg.find('\n', 0, len(msg)-1) == -1): 358 return '%s: %s' % (tag, msg) 359 else: 360 msg = '\n'.join([indent+l for l in msg[:-1].split('\n')]) 361 return '%s:\n%s\n' % (tag, msg) 362 363def _exception_traceback(exc_info): 364 """ 365 Return a string containing a traceback message for the given 366 exc_info tuple (as returned by sys.exc_info()). 367 """ 368 # Get a traceback message. 369 excout = StringIO() 370 exc_type, exc_val, exc_tb = exc_info 371 traceback.print_exception(exc_type, exc_val, exc_tb, file=excout) 372 return excout.getvalue() 373 374# Override some StringIO methods. 375class _SpoofOut(StringIO): 376 def getvalue(self): 377 result = StringIO.getvalue(self) 378 # If anything at all was written, make sure there's a trailing 379 # newline. There's no way for the expected output to indicate 380 # that a trailing newline is missing. 381 if result and not result.endswith("\n"): 382 result += "\n" 383 # Prevent softspace from screwing up the next test case, in 384 # case they used print with a trailing comma in an example. 385 if hasattr(self, "softspace"): 386 del self.softspace 387 return result 388 389 def truncate(self, size=None): 390 StringIO.truncate(self, size) 391 if hasattr(self, "softspace"): 392 del self.softspace 393 394# Worst-case linear-time ellipsis matching. 395def _ellipsis_match(want, got): 396 """ 397 Essentially the only subtle case: 398 >>> _ellipsis_match('aa...aa', 'aaa') 399 False 400 """ 401 if ELLIPSIS_MARKER not in want: 402 return want == got 403 404 # Find "the real" strings. 405 ws = want.split(ELLIPSIS_MARKER) 406 assert len(ws) >= 2 407 408 # Deal with exact matches possibly needed at one or both ends. 409 startpos, endpos = 0, len(got) 410 w = ws[0] 411 if w: # starts with exact match 412 if got.startswith(w): 413 startpos = len(w) 414 del ws[0] 415 else: 416 return False 417 w = ws[-1] 418 if w: # ends with exact match 419 if got.endswith(w): 420 endpos -= len(w) 421 del ws[-1] 422 else: 423 return False 424 425 if startpos > endpos: 426 # Exact end matches required more characters than we have, as in 427 # _ellipsis_match('aa...aa', 'aaa') 428 return False 429 430 # For the rest, we only need to find the leftmost non-overlapping 431 # match for each piece. If there's no overall match that way alone, 432 # there's no overall match period. 433 for w in ws: 434 # w may be '' at times, if there are consecutive ellipses, or 435 # due to an ellipsis at the start or end of `want`. That's OK. 436 # Search for an empty string succeeds, and doesn't change startpos. 437 startpos = got.find(w, startpos, endpos) 438 if startpos < 0: 439 return False 440 startpos += len(w) 441 442 return True 443 444###################################################################### 445## 2. Example & DocTest 446###################################################################### 447## - An "example" is a <source, want> pair, where "source" is a 448## fragment of source code, and "want" is the expected output for 449## "source." The Example class also includes information about 450## where the example was extracted from. 451## 452## - A "doctest" is a collection of examples, typically extracted from 453## a string (such as an object's docstring). The DocTest class also 454## includes information about where the string was extracted from. 455 456class Example: 457 """ 458 A single doctest example, consisting of source code and expected 459 output. `Example` defines the following attributes: 460 461 - source: A single Python statement, always ending with a newline. 462 The constructor adds a newline if needed. 463 464 - want: The expected output from running the source code (either 465 from stdout, or a traceback in case of exception). `want` ends 466 with a newline unless it's empty, in which case it's an empty 467 string. The constructor adds a newline if needed. 468 469 - lineno: The line number within the DocTest string containing 470 this Example where the Example begins. This line number is 471 zero-based, with respect to the beginning of the DocTest. 472 473 - indent: The example's indentation in the DocTest string. 474 I.e., the number of space characters that preceed the 475 example's first prompt. 476 477 - options: A dictionary mapping from option flags to True or 478 False, which is used to override default options for this 479 example. Any option flags not contained in this dictionary 480 are left at their default value (as specified by the 481 DocTestRunner's optionflags). By default, no options are set. 482 """ 483 def __init__(self, source, want, lineno, indent=0, options=None): 484 # Normalize inputs. 485 if not source.endswith('\n'): 486 source += '\n' 487 if want and not want.endswith('\n'): 488 want += '\n' 489 # Store properties. 490 self.source = source 491 self.want = want 492 self.lineno = lineno 493 self.indent = indent 494 if options is None: options = {} 495 self.options = options 496 497class DocTest: 498 """ 499 A collection of doctest examples that should be run in a single 500 namespace. Each `DocTest` defines the following attributes: 501 502 - examples: the list of examples. 503 504 - globs: The namespace (aka globals) that the examples should 505 be run in. 506 507 - name: A name identifying the DocTest (typically, the name of 508 the object whose docstring this DocTest was extracted from). 509 510 - filename: The name of the file that this DocTest was extracted 511 from, or `None` if the filename is unknown. 512 513 - lineno: The line number within filename where this DocTest 514 begins, or `None` if the line number is unavailable. This 515 line number is zero-based, with respect to the beginning of 516 the file. 517 518 - docstring: The string that the examples were extracted from, 519 or `None` if the string is unavailable. 520 """ 521 def __init__(self, examples, globs, name, filename, lineno, docstring): 522 """ 523 Create a new DocTest containing the given examples. The 524 DocTest's globals are initialized with a copy of `globs`. 525 """ 526 assert not isinstance(examples, basestring), \ 527 "DocTest no longer accepts str; use DocTestParser instead" 528 self.examples = examples 529 self.docstring = docstring 530 self.globs = globs.copy() 531 self.name = name 532 self.filename = filename 533 self.lineno = lineno 534 535 def __repr__(self): 536 if len(self.examples) == 0: 537 examples = 'no examples' 538 elif len(self.examples) == 1: 539 examples = '1 example' 540 else: 541 examples = '%d examples' % len(self.examples) 542 return ('<DocTest %s from %s:%s (%s)>' % 543 (self.name, self.filename, self.lineno, examples)) 544 545 546 # This lets us sort tests by name: 547 def __cmp__(self, other): 548 if not isinstance(other, DocTest): 549 return -1 550 return cmp((self.name, self.filename, self.lineno, id(self)), 551 (other.name, other.filename, other.lineno, id(other))) 552 553###################################################################### 554## 3. DocTestParser 555###################################################################### 556 557class DocTestParser: 558 """ 559 A class used to parse strings containing doctest examples. 560 """ 561 # This regular expression is used to find doctest examples in a 562 # string. It defines three groups: `source` is the source code 563 # (including leading indentation and prompts); `indent` is the 564 # indentation of the first (PS1) line of the source code; and 565 # `want` is the expected output (including leading indentation). 566 _EXAMPLE_RE = re.compile(r''' 567 # Source consists of a PS1 line followed by zero or more PS2 lines. 568 (?P<source> 569 (?:^(?P<indent> [ ]*) >>> .*) # PS1 line 570 (?:\n [ ]* \.\.\. .*)*) # PS2 lines 571 \n? 572 # Want consists of any non-blank lines that do not start with PS1. 573 (?P<want> (?:(?![ ]*$) # Not a blank line 574 (?![ ]*>>>) # Not a line starting with PS1 575 .*$\n? # But any other line 576 )*) 577 ''', re.MULTILINE | re.VERBOSE) 578 579 # A callable returning a true value iff its argument is a blank line 580 # or contains a single comment. 581 _IS_BLANK_OR_COMMENT = re.compile(r'^[ ]*(#.*)?$').match 582 583 def get_doctest(self, string, globs, name, filename, lineno): 584 """ 585 Extract all doctest examples from the given string, and 586 collect them into a `DocTest` object. 587 588 `globs`, `name`, `filename`, and `lineno` are attributes for 589 the new `DocTest` object. See the documentation for `DocTest` 590 for more information. 591 """ 592 return DocTest(self.get_examples(string, name), globs, 593 name, filename, lineno, string) 594 595 def get_examples(self, string, name='<string>'): 596 """ 597 Extract all doctest examples from the given string, and return 598 them as a list of `Example` objects. Line numbers are 599 0-based, because it's most common in doctests that nothing 600 interesting appears on the same line as opening triple-quote, 601 and so the first interesting line is called \"line 1\" then. 602 603 The optional argument `name` is a name identifying this 604 string, and is only used for error messages. 605 606 >>> text = ''' 607 ... >>> x, y = 2, 3 # no output expected 608 ... >>> if 1: 609 ... ... print x 610 ... ... print y 611 ... 2 612 ... 3 613 ... 614 ... Some text. 615 ... >>> x+y 616 ... 5 617 ... ''' 618 >>> for x in DocTestParser().get_examples(text): 619 ... print (x.source, x.want, x.lineno) 620 ('x, y = 2, 3 # no output expected\\n', '', 1) 621 ('if 1:\\n print x\\n print y\\n', '2\\n3\\n', 2) 622 ('x+y\\n', '5\\n', 9) 623 """ 624 examples = [] 625 charno, lineno = 0, 0 626 # Find all doctest examples in the string: 627 for m in self._EXAMPLE_RE.finditer(string.expandtabs()): 628 # Update lineno (lines before this example) 629 lineno += string.count('\n', charno, m.start()) 630 # Extract source/want from the regexp match. 631 (source, want) = self._parse_example(m, name, lineno) 632 # Extract extra options from the source. 633 options = self._find_options(source, name, lineno) 634 # Create an Example, and add it to the list. 635 if not self._IS_BLANK_OR_COMMENT(source): 636 examples.append( Example(source, want, lineno, 637 len(m.group('indent')), options) ) 638 # Update lineno (lines inside this example) 639 lineno += string.count('\n', m.start(), m.end()) 640 # Update charno. 641 charno = m.end() 642 return examples 643 644 def get_program(self, string, name="<string>"): 645 """ 646 Return an executable program from the given string, as a string. 647 648 The format of this isn't rigidly defined. In general, doctest 649 examples become the executable statements in the result, and 650 their expected outputs become comments, preceded by an \"#Expected:\" 651 comment. Everything else (text, comments, everything not part of 652 a doctest test) is also placed in comments. 653 654 The optional argument `name` is a name identifying this 655 string, and is only used for error messages. 656 657 >>> text = ''' 658 ... >>> x, y = 2, 3 # no output expected 659 ... >>> if 1: 660 ... ... print x 661 ... ... print y 662 ... 2 663 ... 3 664 ... 665 ... Some text. 666 ... >>> x+y 667 ... 5 668 ... ''' 669 >>> print DocTestParser().get_program(text) 670 x, y = 2, 3 # no output expected 671 if 1: 672 print x 673 print y 674 # Expected: 675 ## 2 676 ## 3 677 # 678 # Some text. 679 x+y 680 # Expected: 681 ## 5 682 """ 683 string = string.expandtabs() 684 # If all lines begin with the same indentation, then strip it. 685 min_indent = self._min_indent(string) 686 if min_indent > 0: 687 string = '\n'.join([l[min_indent:] for l in string.split('\n')]) 688 689 output = [] 690 charnum, lineno = 0, 0 691 # Find all doctest examples in the string: 692 for m in self._EXAMPLE_RE.finditer(string.expandtabs()): 693 # Add any text before this example, as a comment. 694 if m.start() > charnum: 695 lines = string[charnum:m.start()-1].split('\n') 696 output.extend([self._comment_line(l) for l in lines]) 697 lineno += len(lines) 698 699 # Extract source/want from the regexp match. 700 (source, want) = self._parse_example(m, name, lineno) 701 # Display the source 702 output.append(source) 703 # Display the expected output, if any 704 if want: 705 output.append('# Expected:') 706 output.extend(['## '+l for l in want.split('\n')]) 707 708 # Update the line number & char number. 709 lineno += string.count('\n', m.start(), m.end()) 710 charnum = m.end() 711 # Add any remaining text, as comments. 712 output.extend([self._comment_line(l) 713 for l in string[charnum:].split('\n')]) 714 # Trim junk on both ends. 715 while output and output[-1] == '#': 716 output.pop() 717 while output and output[0] == '#': 718 output.pop(0) 719 # Combine the output, and return it. 720 return '\n'.join(output) 721 722 def _parse_example(self, m, name, lineno): 723 """ 724 Given a regular expression match from `_EXAMPLE_RE` (`m`), 725 return a pair `(source, want)`, where `source` is the matched 726 example's source code (with prompts and indentation stripped); 727 and `want` is the example's expected output (with indentation 728 stripped). 729 730 `name` is the string's name, and `lineno` is the line number 731 where the example starts; both are used for error messages. 732 """ 733 # Get the example's indentation level. 734 indent = len(m.group('indent')) 735 736 # Divide source into lines; check that they're properly 737 # indented; and then strip their indentation & prompts. 738 source_lines = m.group('source').split('\n') 739 self._check_prompt_blank(source_lines, indent, name, lineno) 740 self._check_prefix(source_lines[1:], ' '*indent + '.', name, lineno) 741 source = '\n'.join([sl[indent+4:] for sl in source_lines]) 742 743 # Divide want into lines; check that it's properly indented; and 744 # then strip the indentation. Spaces before the last newline should 745 # be preserved, so plain rstrip() isn't good enough. 746 want = m.group('want') 747 want_lines = want.split('\n') 748 if len(want_lines) > 1 and re.match(r' *$', want_lines[-1]): 749 del want_lines[-1] # forget final newline & spaces after it 750 self._check_prefix(want_lines, ' '*indent, name, 751 lineno + len(source_lines)) 752 want = '\n'.join([wl[indent:] for wl in want_lines]) 753 754 return source, want 755 756 # This regular expression looks for option directives in the 757 # source code of an example. Option directives are comments 758 # starting with "doctest:". Warning: this may give false 759 # positives for string-literals that contain the string 760 # "#doctest:". Eliminating these false positives would require 761 # actually parsing the string; but we limit them by ignoring any 762 # line containing "#doctest:" that is *followed* by a quote mark. 763 _OPTION_DIRECTIVE_RE = re.compile(r'#\s*doctest:\s*([^\n\'"]*)$', 764 re.MULTILINE) 765 766 def _find_options(self, source, name, lineno): 767 """ 768 Return a dictionary containing option overrides extracted from 769 option directives in the given source string. 770 771 `name` is the string's name, and `lineno` is the line number 772 where the example starts; both are used for error messages. 773 """ 774 options = {} 775 # (note: with the current regexp, this will match at most once:) 776 for m in self._OPTION_DIRECTIVE_RE.finditer(source): 777 option_strings = m.group(1).replace(',', ' ').split() 778 for option in option_strings: 779 if (option[0] not in '+-' or 780 option[1:] not in OPTIONFLAGS_BY_NAME): 781 raise ValueError('line %r of the doctest for %s ' 782 'has an invalid option: %r' % 783 (lineno+1, name, option)) 784 flag = OPTIONFLAGS_BY_NAME[option[1:]] 785 options[flag] = (option[0] == '+') 786 if options and self._IS_BLANK_OR_COMMENT(source): 787 raise ValueError('line %r of the doctest for %s has an option ' 788 'directive on a line with no example: %r' % 789 (lineno, name, source)) 790 return options 791 792 # This regular expression finds the indentation of every non-blank 793 # line in a string. 794 _INDENT_RE = re.compile('^([ ]+)(?=\S)', re.MULTILINE) 795 796 def _min_indent(self, s): 797 "Return the minimum indentation of any non-blank line in `s`" 798 return min([len(indent) for indent in self._INDENT_RE.findall(s)]) 799 800 def _comment_line(self, line): 801 "Return a commented form of the given line" 802 line = line.rstrip() 803 if line: 804 return '# '+line 805 else: 806 return '#' 807 808 def _check_prompt_blank(self, lines, indent, name, lineno): 809 """ 810 Given the lines of a source string (including prompts and 811 leading indentation), check to make sure that every prompt is 812 followed by a space character. If any line is not followed by 813 a space character, then raise ValueError. 814 """ 815 for i, line in enumerate(lines): 816 if len(line) >= indent+4 and line[indent+3] != ' ': 817 raise ValueError('line %r of the docstring for %s ' 818 'lacks blank after %s: %r' % 819 (lineno+i+1, name, 820 line[indent:indent+3], line)) 821 822 def _check_prefix(self, lines, prefix, name, lineno): 823 """ 824 Check that every line in the given list starts with the given 825 prefix; if any line does not, then raise a ValueError. 826 """ 827 for i, line in enumerate(lines): 828 if line and not line.startswith(prefix): 829 raise ValueError('line %r of the docstring for %s has ' 830 'inconsistent leading whitespace: %r' % 831 (lineno+i+1, name, line)) 832 833 834###################################################################### 835## 4. DocTest Finder 836###################################################################### 837 838class DocTestFinder: 839 """ 840 A class used to extract the DocTests that are relevant to a given 841 object, from its docstring and the docstrings of its contained 842 objects. Doctests can currently be extracted from the following 843 object types: modules, functions, classes, methods, staticmethods, 844 classmethods, and properties. 845 """ 846 847 def __init__(self, verbose=False, parser=DocTestParser(), 848 recurse=True, _namefilter=None): 849 """ 850 Create a new doctest finder. 851 852 The optional argument `parser` specifies a class or 853 function that should be used to create new DocTest objects (or 854 objects that implement the same interface as DocTest). The 855 signature for this factory function should match the signature 856 of the DocTest constructor. 857 858 If the optional argument `recurse` is false, then `find` will 859 only examine the given object, and not any contained objects. 860 """ 861 self._parser = parser 862 self._verbose = verbose 863 self._recurse = recurse 864 # _namefilter is undocumented, and exists only for temporary backward- 865 # compatibility support of testmod's deprecated isprivate mess. 866 self._namefilter = _namefilter 867 868 def find(self, obj, name=None, module=None, globs=None, 869 extraglobs=None): 870 """ 871 Return a list of the DocTests that are defined by the given 872 object's docstring, or by any of its contained objects' 873 docstrings. 874 875 The optional parameter `module` is the module that contains 876 the given object. If the module is not specified or is None, then 877 the test finder will attempt to automatically determine the 878 correct module. The object's module is used: 879 880 - As a default namespace, if `globs` is not specified. 881 - To prevent the DocTestFinder from extracting DocTests 882 from objects that are imported from other modules. 883 - To find the name of the file containing the object. 884 - To help find the line number of the object within its 885 file. 886 887 Contained objects whose module does not match `module` are ignored. 888 889 If `module` is False, no attempt to find the module will be made. 890 This is obscure, of use mostly in tests: if `module` is False, or 891 is None but cannot be found automatically, then all objects are 892 considered to belong to the (non-existent) module, so all contained 893 objects will (recursively) be searched for doctests. 894 895 The globals for each DocTest is formed by combining `globs` 896 and `extraglobs` (bindings in `extraglobs` override bindings 897 in `globs`). A new copy of the globals dictionary is created 898 for each DocTest. If `globs` is not specified, then it 899 defaults to the module's `__dict__`, if specified, or {} 900 otherwise. If `extraglobs` is not specified, then it defaults 901 to {}. 902 903 """ 904 # If name was not specified, then extract it from the object. 905 if name is None: 906 name = getattr(obj, '__name__', None) 907 if name is None: 908 raise ValueError("DocTestFinder.find: name must be given " 909 "when obj.__name__ doesn't exist: %r" % 910 (type(obj),)) 911 912 # Find the module that contains the given object (if obj is 913 # a module, then module=obj.). Note: this may fail, in which 914 # case module will be None. 915 if module is False: 916 module = None 917 elif module is None: 918 module = inspect.getmodule(obj) 919 920 # Read the module's source code. This is used by 921 # DocTestFinder._find_lineno to find the line number for a 922 # given object's docstring. 923 try: 924 file = inspect.getsourcefile(obj) or inspect.getfile(obj) 925 source_lines = linecache.getlines(file) 926 if not source_lines: 927 source_lines = None 928 except TypeError: 929 source_lines = None 930 931 # Initialize globals, and merge in extraglobs. 932 if globs is None: 933 if module is None: 934 globs = {} 935 else: 936 globs = module.__dict__.copy() 937 else: 938 globs = globs.copy() 939 if extraglobs is not None: 940 globs.update(extraglobs) 941 942 # Recursively expore `obj`, extracting DocTests. 943 tests = [] 944 self._find(tests, obj, name, module, source_lines, globs, {}) 945 return tests 946 947 def _filter(self, obj, prefix, base): 948 """ 949 Return true if the given object should not be examined. 950 """ 951 return (self._namefilter is not None and 952 self._namefilter(prefix, base)) 953 954 def _from_module(self, module, object): 955 """ 956 Return true if the given object is defined in the given 957 module. 958 """ 959 if module is None: 960 return True 961 elif inspect.isfunction(object): 962 return module.__dict__ is object.func_globals 963 elif inspect.isclass(object): 964 return module.__name__ == object.__module__ 965 elif inspect.getmodule(object) is not None: 966 return module is inspect.getmodule(object) 967 elif hasattr(object, '__module__'): 968 return module.__name__ == object.__module__ 969 elif isinstance(object, property): 970 return True # [XX] no way not be sure. 971 else: 972 raise ValueError("object must be a class or function") 973 974 def _find(self, tests, obj, name, module, source_lines, globs, seen): 975 """ 976 Find tests for the given object and any contained objects, and 977 add them to `tests`. 978 """ 979 if self._verbose: 980 print 'Finding tests in %s' % name 981 982 # If we've already processed this object, then ignore it. 983 if id(obj) in seen: 984 return 985 seen[id(obj)] = 1 986 987 # Find a test for this object, and add it to the list of tests. 988 test = self._get_test(obj, name, module, globs, source_lines) 989 if test is not None: 990 tests.append(test) 991 992 # Look for tests in a module's contained objects. 993 if inspect.ismodule(obj) and self._recurse: 994 for valname, val in obj.__dict__.items(): 995 # Check if this contained object should be ignored. 996 if self._filter(val, name, valname): 997 continue 998 valname = '%s.%s' % (name, valname) 999 # Recurse to functions & classes. 1000 if ((inspect.isfunction(val) or inspect.isclass(val)) and 1001 self._from_module(module, val)): 1002 self._find(tests, val, valname, module, source_lines, 1003 globs, seen) 1004 1005 # Look for tests in a module's __test__ dictionary. 1006 if inspect.ismodule(obj) and self._recurse: 1007 for valname, val in getattr(obj, '__test__', {}).items(): 1008 if not isinstance(valname, basestring): 1009 raise ValueError("DocTestFinder.find: __test__ keys " 1010 "must be strings: %r" % 1011 (type(valname),)) 1012 if not (inspect.isfunction(val) or inspect.isclass(val) or 1013 inspect.ismethod(val) or inspect.ismodule(val) or 1014 isinstance(val, basestring)): 1015 raise ValueError("DocTestFinder.find: __test__ values " 1016 "must be strings, functions, methods, " 1017 "classes, or modules: %r" % 1018 (type(val),)) 1019 valname = '%s.%s' % (name, valname) 1020 self._find(tests, val, valname, module, source_lines, 1021 globs, seen) 1022 1023 # Look for tests in a class's contained objects. 1024 if inspect.isclass(obj) and self._recurse: 1025 for valname, val in obj.__dict__.items(): 1026 # Check if this contained object should be ignored. 1027 if self._filter(val, name, valname): 1028 continue 1029 # Special handling for staticmethod/classmethod. 1030 if isinstance(val, staticmethod): 1031 val = getattr(obj, valname) 1032 if isinstance(val, classmethod): 1033 val = getattr(obj, valname).im_func 1034 1035 # Recurse to methods, properties, and nested classes. 1036 if ((inspect.isfunction(val) or inspect.isclass(val) or 1037 isinstance(val, property)) and 1038 self._from_module(module, val)): 1039 valname = '%s.%s' % (name, valname) 1040 self._find(tests, val, valname, module, source_lines, 1041 globs, seen) 1042 1043 def _get_test(self, obj, name, module, globs, source_lines): 1044 """ 1045 Return a DocTest for the given object, if it defines a docstring; 1046 otherwise, return None. 1047 """ 1048 # Extract the object's docstring. If it doesn't have one, 1049 # then return None (no test for this object). 1050 if isinstance(obj, basestring): 1051 docstring = obj 1052 else: 1053 try: 1054 if obj.__doc__ is None: 1055 return None 1056 docstring = str(obj.__doc__) 1057 except (TypeError, AttributeError): 1058 return None 1059 1060 # Don't bother if the docstring is empty. 1061 if not docstring: 1062 return None 1063 1064 # Find the docstring's location in the file. 1065 lineno = self._find_lineno(obj, source_lines) 1066 1067 # Return a DocTest for this object. 1068 if module is None: 1069 filename = None 1070 else: 1071 filename = getattr(module, '__file__', module.__name__) 1072 if filename[-4:] in (".pyc", ".pyo"): 1073 filename = filename[:-1] 1074 return self._parser.get_doctest(docstring, globs, name, 1075 filename, lineno) 1076 1077 def _find_lineno(self, obj, source_lines): 1078 """ 1079 Return a line number of the given object's docstring. Note: 1080 this method assumes that the object has a docstring. 1081 """ 1082 lineno = None 1083 1084 # Find the line number for modules. 1085 if inspect.ismodule(obj): 1086 lineno = 0 1087 1088 # Find the line number for classes. 1089 # Note: this could be fooled if a class is defined multiple 1090 # times in a single file. 1091 if inspect.isclass(obj): 1092 if source_lines is None: 1093 return None 1094 pat = re.compile(r'^\s*class\s*%s\b' % 1095 getattr(obj, '__name__', '-')) 1096 for i, line in enumerate(source_lines): 1097 if pat.match(line): 1098 lineno = i 1099 break 1100 1101 # Find the line number for functions & methods. 1102 if inspect.ismethod(obj): obj = obj.im_func 1103 if inspect.isfunction(obj): obj = obj.func_code 1104 if inspect.istraceback(obj): obj = obj.tb_frame 1105 if inspect.isframe(obj): obj = obj.f_code 1106 if inspect.iscode(obj): 1107 lineno = getattr(obj, 'co_firstlineno', None)-1 1108 1109 # Find the line number where the docstring starts. Assume 1110 # that it's the first line that begins with a quote mark. 1111 # Note: this could be fooled by a multiline function 1112 # signature, where a continuation line begins with a quote 1113 # mark. 1114 if lineno is not None: 1115 if source_lines is None: 1116 return lineno+1 1117 pat = re.compile('(^|.*:)\s*\w*("|\')') 1118 for lineno in range(lineno, len(source_lines)): 1119 if pat.match(source_lines[lineno]): 1120 return lineno 1121 1122 # We couldn't find the line number. 1123 return None 1124 1125###################################################################### 1126## 5. DocTest Runner 1127###################################################################### 1128 1129class DocTestRunner: 1130 """ 1131 A class used to run DocTest test cases, and accumulate statistics. 1132 The `run` method is used to process a single DocTest case. It 1133 returns a tuple `(f, t)`, where `t` is the number of test cases 1134 tried, and `f` is the number of test cases that failed. 1135 1136 >>> tests = DocTestFinder().find(_TestClass) 1137 >>> runner = DocTestRunner(verbose=False) 1138 >>> for test in tests: 1139 ... print runner.run(test) 1140 (0, 2) 1141 (0, 1) 1142 (0, 2) 1143 (0, 2) 1144 1145 The `summarize` method prints a summary of all the test cases that 1146 have been run by the runner, and returns an aggregated `(f, t)` 1147 tuple: 1148 1149 >>> runner.summarize(verbose=1) 1150 4 items passed all tests: 1151 2 tests in _TestClass 1152 2 tests in _TestClass.__init__ 1153 2 tests in _TestClass.get 1154 1 tests in _TestClass.square 1155 7 tests in 4 items. 1156 7 passed and 0 failed. 1157 Test passed. 1158 (0, 7) 1159 1160 The aggregated number of tried examples and failed examples is 1161 also available via the `tries` and `failures` attributes: 1162 1163 >>> runner.tries 1164 7 1165 >>> runner.failures 1166 0 1167 1168 The comparison between expected outputs and actual outputs is done 1169 by an `OutputChecker`. This comparison may be customized with a 1170 number of option flags; see the documentation for `testmod` for 1171 more information. If the option flags are insufficient, then the 1172 comparison may also be customized by passing a subclass of 1173 `OutputChecker` to the constructor. 1174 1175 The test runner's display output can be controlled in two ways. 1176 First, an output function (`out) can be passed to 1177 `TestRunner.run`; this function will be called with strings that 1178 should be displayed. It defaults to `sys.stdout.write`. If 1179 capturing the output is not sufficient, then the display output 1180 can be also customized by subclassing DocTestRunner, and 1181 overriding the methods `report_start`, `report_success`, 1182 `report_unexpected_exception`, and `report_failure`. 1183 """ 1184 # This divider string is used to separate failure messages, and to 1185 # separate sections of the summary. 1186 DIVIDER = "*" * 70 1187 1188 def __init__(self, checker=None, verbose=None, optionflags=0): 1189 """ 1190 Create a new test runner. 1191 1192 Optional keyword arg `checker` is the `OutputChecker` that 1193 should be used to compare the expected outputs and actual 1194 outputs of doctest examples. 1195 1196 Optional keyword arg 'verbose' prints lots of stuff if true, 1197 only failures if false; by default, it's true iff '-v' is in 1198 sys.argv. 1199 1200 Optional argument `optionflags` can be used to control how the 1201 test runner compares expected output to actual output, and how 1202 it displays failures. See the documentation for `testmod` for 1203 more information. 1204 """ 1205 self._checker = checker or OutputChecker() 1206 if verbose is None: 1207 verbose = '-v' in sys.argv 1208 self._verbose = verbose 1209 self.optionflags = optionflags 1210 self.original_optionflags = optionflags 1211 1212 # Keep track of the examples we've run. 1213 self.tries = 0 1214 self.failures = 0 1215 self._name2ft = {} 1216 1217 # Create a fake output target for capturing doctest output. 1218 self._fakeout = _SpoofOut() 1219 1220 #///////////////////////////////////////////////////////////////// 1221 # Reporting methods 1222 #///////////////////////////////////////////////////////////////// 1223 1224 def report_start(self, out, test, example): 1225 """ 1226 Report that the test runner is about to process the given 1227 example. (Only displays a message if verbose=True) 1228 """ 1229 if self._verbose: 1230 out(_tag_msg("Trying", example.source) + 1231 _tag_msg("Expecting", example.want or "nothing")) 1232 1233 def report_success(self, out, test, example, got): 1234 """ 1235 Report that the given example ran successfully. (Only 1236 displays a message if verbose=True) 1237 """ 1238 if self._verbose: 1239 out("ok\n") 1240 1241 def report_failure(self, out, test, example, got): 1242 """ 1243 Report that the given example failed. 1244 """ 1245 # Print an error message. 1246 out(self._failure_header(test, example) + 1247 self._checker.output_difference(example.want, got, 1248 self.optionflags)) 1249 1250 def report_unexpected_exception(self, out, test, example, exc_info): 1251 """ 1252 Report that the given example raised an unexpected exception. 1253 """ 1254 out(self._failure_header(test, example) + 1255 _tag_msg("Exception raised", _exception_traceback(exc_info))) 1256 1257 def _failure_header(self, test, example): 1258 out = [self.DIVIDER] 1259 if test.filename: 1260 if test.lineno is not None and example.lineno is not None: 1261 lineno = test.lineno + example.lineno + 1 1262 else: 1263 lineno = '?' 1264 out.append('File "%s", line %s, in %s' % 1265 (test.filename, lineno, test.name)) 1266 else: 1267 out.append('Line %s, in %s' % (example.lineno+1, test.name)) 1268 out.append('Failed example:') 1269 source = example.source 1270 if source.endswith('\n'): 1271 source = source[:-1] 1272 out.append(' ' + '\n '.join(source.split('\n'))) 1273 return '\n'.join(out)+'\n' 1274 1275 #///////////////////////////////////////////////////////////////// 1276 # DocTest Running 1277 #///////////////////////////////////////////////////////////////// 1278 1279 # A regular expression for handling `want` strings that contain 1280 # expected exceptions. It divides `want` into three pieces: 1281 # - the pre-exception output (`want`) 1282 # - the traceback header line (`hdr`) 1283 # - the exception message (`msg`), as generated by 1284 # traceback.format_exception_only() 1285 # `msg` may have multiple lines. We assume/require that the 1286 # exception message is the first non-indented line starting with a word 1287 # character following the traceback header line. 1288 _EXCEPTION_RE = re.compile(r""" 1289 (?P<want> .*?) # suck up everything until traceback header 1290 # Grab the traceback header. Different versions of Python have 1291 # said different things on the first traceback line. 1292 ^(?P<hdr> Traceback\ \( 1293 (?: most\ recent\ call\ last 1294 | innermost\ last 1295 ) \) : 1296 ) 1297 \s* $ # toss trailing whitespace on traceback header 1298 .*? # don't blink: absorb stuff until a line *starts* with \w 1299 ^ (?P<msg> \w+ .*) 1300 """, re.VERBOSE | re.MULTILINE | re.DOTALL) 1301 1302 def __run(self, test, compileflags, out): 1303 """ 1304 Run the examples in `test`. Write the outcome of each example 1305 with one of the `DocTestRunner.report_*` methods, using the 1306 writer function `out`. `compileflags` is the set of compiler 1307 flags that should be used to execute examples. Return a tuple 1308 `(f, t)`, where `t` is the number of examples tried, and `f` 1309 is the number of examples that failed. The examples are run 1310 in the namespace `test.globs`. 1311 """ 1312 # Keep track of the number of failures and tries. 1313 failures = tries = 0 1314 1315 # Save the option flags (since option directives can be used 1316 # to modify them). 1317 original_optionflags = self.optionflags 1318 1319 # Process each example. 1320 for example in test.examples: 1321 # Merge in the example's options. 1322 self.optionflags = original_optionflags 1323 if example.options: 1324 for (optionflag, val) in example.options.items(): 1325 if val: 1326 self.optionflags |= optionflag 1327 else: 1328 self.optionflags &= ~optionflag 1329 1330 # Record that we started this example. 1331 tries += 1 1332 self.report_start(out, test, example) 1333 1334 # Run the example in the given context (globs), and record 1335 # any exception that gets raised. (But don't intercept 1336 # keyboard interrupts.) 1337 try: 1338 # Don't blink! This is where the user's code gets run. 1339 exec compile(example.source, "<string>", "single", 1340 compileflags, 1) in test.globs 1341 exception = None 1342 except KeyboardInterrupt: 1343 raise 1344 except: 1345 exception = sys.exc_info() 1346 1347 got = self._fakeout.getvalue() # the actual output 1348 self._fakeout.truncate(0) 1349 1350 # If the example executed without raising any exceptions, 1351 # then verify its output and report its outcome. 1352 if exception is None: 1353 if self._checker.check_output(example.want, got, 1354 self.optionflags): 1355 self.report_success(out, test, example, got) 1356 else: 1357 self.report_failure(out, test, example, got) 1358 failures += 1 1359 1360 # If the example raised an exception, then check if it was 1361 # expected. 1362 else: 1363 exc_info = sys.exc_info() 1364 exc_msg = traceback.format_exception_only(*exc_info[:2])[-1] 1365 1366 # Search the `want` string for an exception. If we don't 1367 # find one, then report an unexpected exception. 1368 m = self._EXCEPTION_RE.match(example.want) 1369 if m is None: 1370 self.report_unexpected_exception(out, test, example, 1371 exc_info) 1372 failures += 1 1373 else: 1374 e_want, e_msg = m.group('want', 'msg') 1375 # The test passes iff the pre-exception output and 1376 # the exception description match the values given 1377 # in `want`. 1378 if (self._checker.check_output(e_want, got, 1379 self.optionflags) and 1380 self._checker.check_output(e_msg, exc_msg, 1381 self.optionflags)): 1382 self.report_success(out, test, example, 1383 got + _exception_traceback(exc_info)) 1384 else: 1385 self.report_failure(out, test, example, 1386 got + _exception_traceback(exc_info)) 1387 failures += 1 1388 1389 # Restore the option flags (in case they were modified) 1390 self.optionflags = original_optionflags 1391 1392 # Record and return the number of failures and tries. 1393 self.__record_outcome(test, failures, tries) 1394 return failures, tries 1395 1396 def __record_outcome(self, test, f, t): 1397 """ 1398 Record the fact that the given DocTest (`test`) generated `f` 1399 failures out of `t` tried examples. 1400 """ 1401 f2, t2 = self._name2ft.get(test.name, (0,0)) 1402 self._name2ft[test.name] = (f+f2, t+t2) 1403 self.failures += f 1404 self.tries += t 1405 1406 def run(self, test, compileflags=None, out=None, clear_globs=True): 1407 """ 1408 Run the examples in `test`, and display the results using the 1409 writer function `out`. 1410 1411 The examples are run in the namespace `test.globs`. If 1412 `clear_globs` is true (the default), then this namespace will 1413 be cleared after the test runs, to help with garbage 1414 collection. If you would like to examine the namespace after 1415 the test completes, then use `clear_globs=False`. 1416 1417 `compileflags` gives the set of flags that should be used by 1418 the Python compiler when running the examples. If not 1419 specified, then it will default to the set of future-import 1420 flags that apply to `globs`. 1421 1422 The output of each example is checked using 1423 `DocTestRunner.check_output`, and the results are formatted by 1424 the `DocTestRunner.report_*` methods. 1425 """ 1426 if compileflags is None: 1427 compileflags = _extract_future_flags(test.globs) 1428 1429 save_stdout = sys.stdout 1430 if out is None: 1431 out = save_stdout.write 1432 sys.stdout = self._fakeout 1433 1434 # Patch pdb.set_trace to restore sys.stdout, so that interactive 1435 # debugging output is visible (not still redirected to self._fakeout). 1436 # Note that we run "the real" pdb.set_trace (captured at doctest 1437 # import time) in our replacement. Because the current run() may 1438 # run another doctest (and so on), the current pdb.set_trace may be 1439 # our set_trace function, which changes sys.stdout. If we called 1440 # a chain of those, we wouldn't be left with the save_stdout 1441 # *this* run() invocation wants. 1442 def set_trace(): 1443 sys.stdout = save_stdout 1444 real_pdb_set_trace() 1445 1446 save_set_trace = pdb.set_trace 1447 pdb.set_trace = set_trace 1448 try: 1449 return self.__run(test, compileflags, out) 1450 finally: 1451 sys.stdout = save_stdout 1452 pdb.set_trace = save_set_trace 1453 if clear_globs: 1454 test.globs.clear() 1455 1456 #///////////////////////////////////////////////////////////////// 1457 # Summarization 1458 #///////////////////////////////////////////////////////////////// 1459 def summarize(self, verbose=None): 1460 """ 1461 Print a summary of all the test cases that have been run by 1462 this DocTestRunner, and return a tuple `(f, t)`, where `f` is 1463 the total number of failed examples, and `t` is the total 1464 number of tried examples. 1465 1466 The optional `verbose` argument controls how detailed the 1467 summary is. If the verbosity is not specified, then the 1468 DocTestRunner's verbosity is used. 1469 """ 1470 if verbose is None: 1471 verbose = self._verbose 1472 notests = [] 1473 passed = [] 1474 failed = [] 1475 totalt = totalf = 0 1476 for x in self._name2ft.items(): 1477 name, (f, t) = x 1478 assert f <= t 1479 totalt += t 1480 totalf += f 1481 if t == 0: 1482 notests.append(name) 1483 elif f == 0: 1484 passed.append( (name, t) ) 1485 else: 1486 failed.append(x) 1487 if verbose: 1488 if notests: 1489 print len(notests), "items had no tests:" 1490 notests.sort() 1491 for thing in notests: 1492 print " ", thing 1493 if passed: 1494 print len(passed), "items passed all tests:" 1495 passed.sort() 1496 for thing, count in passed: 1497 print " %3d tests in %s" % (count, thing) 1498 if failed: 1499 print self.DIVIDER 1500 print len(failed), "items had failures:" 1501 failed.sort() 1502 for thing, (f, t) in failed: 1503 print " %3d of %3d in %s" % (f, t, thing) 1504 if verbose: 1505 print totalt, "tests in", len(self._name2ft), "items." 1506 print totalt - totalf, "passed and", totalf, "failed." 1507 if totalf: 1508 print "***Test Failed***", totalf, "failures." 1509 elif verbose: 1510 print "Test passed." 1511 return totalf, totalt 1512 1513class OutputChecker: 1514 """ 1515 A class used to check the whether the actual output from a doctest 1516 example matches the expected output. `OutputChecker` defines two 1517 methods: `check_output`, which compares a given pair of outputs, 1518 and returns true if they match; and `output_difference`, which 1519 returns a string describing the differences between two outputs. 1520 """ 1521 def check_output(self, want, got, optionflags): 1522 """ 1523 Return True iff the actual output from an example (`got`) 1524 matches the expected output (`want`). These strings are 1525 always considered to match if they are identical; but 1526 depending on what option flags the test runner is using, 1527 several non-exact match types are also possible. See the 1528 documentation for `TestRunner` for more information about 1529 option flags. 1530 """ 1531 # Handle the common case first, for efficiency: 1532 # if they're string-identical, always return true. 1533 if got == want: 1534 return True 1535 1536 # The values True and False replaced 1 and 0 as the return 1537 # value for boolean comparisons in Python 2.3. 1538 if not (optionflags & DONT_ACCEPT_TRUE_FOR_1): 1539 if (got,want) == ("True\n", "1\n"): 1540 return True 1541 if (got,want) == ("False\n", "0\n"): 1542 return True 1543 1544 # <BLANKLINE> can be used as a special sequence to signify a 1545 # blank line, unless the DONT_ACCEPT_BLANKLINE flag is used. 1546 if not (optionflags & DONT_ACCEPT_BLANKLINE): 1547 # Replace <BLANKLINE> in want with a blank line. 1548 want = re.sub('(?m)^%s\s*?$' % re.escape(BLANKLINE_MARKER), 1549 '', want) 1550 # If a line in got contains only spaces, then remove the 1551 # spaces. 1552 got = re.sub('(?m)^\s*?$', '', got) 1553 if got == want: 1554 return True 1555 1556 # This flag causes doctest to ignore any differences in the 1557 # contents of whitespace strings. Note that this can be used 1558 # in conjunction with the ELLIPSIS flag. 1559 if optionflags & NORMALIZE_WHITESPACE: 1560 got = ' '.join(got.split()) 1561 want = ' '.join(want.split()) 1562 if got == want: 1563 return True 1564 1565 # The ELLIPSIS flag says to let the sequence "..." in `want` 1566 # match any substring in `got`. 1567 if optionflags & ELLIPSIS: 1568 if _ellipsis_match(want, got): 1569 return True 1570 1571 # We didn't find any match; return false. 1572 return False 1573 1574 # Should we do a fancy diff? 1575 def _do_a_fancy_diff(self, want, got, optionflags): 1576 # Not unless they asked for a fancy diff. 1577 if not optionflags & (UNIFIED_DIFF | 1578 CONTEXT_DIFF | 1579 NDIFF_DIFF): 1580 return False 1581 # If expected output uses ellipsis, a meaningful fancy diff is 1582 # too hard. 1583 if optionflags & ELLIPSIS and ELLIPSIS_MARKER in want: 1584 return False 1585 # ndiff does intraline difference marking, so can be useful even 1586 # for 1-line inputs. 1587 if optionflags & NDIFF_DIFF: 1588 return True 1589 # The other diff types need at least a few lines to be helpful. 1590 return want.count('\n') > 2 and got.count('\n') > 2 1591 1592 def output_difference(self, want, got, optionflags): 1593 """ 1594 Return a string describing the differences between the 1595 expected output for an example (`want`) and the actual output 1596 (`got`). `optionflags` is the set of option flags used to 1597 compare `want` and `got`. `indent` is the indentation of the 1598 original example. 1599 """ 1600 1601 # If <BLANKLINE>s are being used, then replace blank lines 1602 # with <BLANKLINE> in the actual output string. 1603 if not (optionflags & DONT_ACCEPT_BLANKLINE): 1604 got = re.sub('(?m)^[ ]*(?=\n)', BLANKLINE_MARKER, got) 1605 1606 # Check if we should use diff. Don't use diff if the actual 1607 # or expected outputs are too short, or if the expected output 1608 # contains an ellipsis marker. 1609 if self._do_a_fancy_diff(want, got, optionflags): 1610 # Split want & got into lines. 1611 want_lines = [l+'\n' for l in want.split('\n')] 1612 got_lines = [l+'\n' for l in got.split('\n')] 1613 # Use difflib to find their differences. 1614 if optionflags & UNIFIED_DIFF: 1615 diff = difflib.unified_diff(want_lines, got_lines, n=2, 1616 fromfile='Expected', tofile='Got') 1617 kind = 'unified diff' 1618 elif optionflags & CONTEXT_DIFF: 1619 diff = difflib.context_diff(want_lines, got_lines, n=2, 1620 fromfile='Expected', tofile='Got') 1621 kind = 'context diff' 1622 elif optionflags & NDIFF_DIFF: 1623 engine = difflib.Differ(charjunk=difflib.IS_CHARACTER_JUNK) 1624 diff = list(engine.compare(want_lines, got_lines)) 1625 kind = 'ndiff with -expected +actual' 1626 else: 1627 assert 0, 'Bad diff option' 1628 # Remove trailing whitespace on diff output. 1629 diff = [line.rstrip() + '\n' for line in diff] 1630 return _tag_msg("Differences (" + kind + ")", 1631 ''.join(diff)) 1632 1633 # If we're not using diff, then simply list the expected 1634 # output followed by the actual output. 1635 if want.endswith('\n'): 1636 want = want[:-1] 1637 want = ' ' + '\n '.join(want.split('\n')) 1638 if got.endswith('\n'): 1639 got = got[:-1] 1640 got = ' ' + '\n '.join(got.split('\n')) 1641 return "Expected:\n%s\nGot:\n%s\n" % (want, got) 1642 1643class DocTestFailure(Exception): 1644 """A DocTest example has failed in debugging mode. 1645 1646 The exception instance has variables: 1647 1648 - test: the DocTest object being run 1649 1650 - excample: the Example object that failed 1651 1652 - got: the actual output 1653 """ 1654 def __init__(self, test, example, got): 1655 self.test = test 1656 self.example = example 1657 self.got = got 1658 1659 def __str__(self): 1660 return str(self.test) 1661 1662class UnexpectedException(Exception): 1663 """A DocTest example has encountered an unexpected exception 1664 1665 The exception instance has variables: 1666 1667 - test: the DocTest object being run 1668 1669 - excample: the Example object that failed 1670 1671 - exc_info: the exception info 1672 """ 1673 def __init__(self, test, example, exc_info): 1674 self.test = test 1675 self.example = example 1676 self.exc_info = exc_info 1677 1678 def __str__(self): 1679 return str(self.test) 1680 1681class DebugRunner(DocTestRunner): 1682 r"""Run doc tests but raise an exception as soon as there is a failure. 1683 1684 If an unexpected exception occurs, an UnexpectedException is raised. 1685 It contains the test, the example, and the original exception: 1686 1687 >>> runner = DebugRunner(verbose=False) 1688 >>> test = DocTestParser().get_doctest('>>> raise KeyError\n42', 1689 ... {}, 'foo', 'foo.py', 0) 1690 >>> try: 1691 ... runner.run(test) 1692 ... except UnexpectedException, failure: 1693 ... pass 1694 1695 >>> failure.test is test 1696 True 1697 1698 >>> failure.example.want 1699 '42\n' 1700 1701 >>> exc_info = failure.exc_info 1702 >>> raise exc_info[0], exc_info[1], exc_info[2] 1703 Traceback (most recent call last): 1704 ... 1705 KeyError 1706 1707 We wrap the original exception to give the calling application 1708 access to the test and example information. 1709 1710 If the output doesn't match, then a DocTestFailure is raised: 1711 1712 >>> test = DocTestParser().get_doctest(''' 1713 ... >>> x = 1 1714 ... >>> x 1715 ... 2 1716 ... ''', {}, 'foo', 'foo.py', 0) 1717 1718 >>> try: 1719 ... runner.run(test) 1720 ... except DocTestFailure, failure: 1721 ... pass 1722 1723 DocTestFailure objects provide access to the test: 1724 1725 >>> failure.test is test 1726 True 1727 1728 As well as to the example: 1729 1730 >>> failure.example.want 1731 '2\n' 1732 1733 and the actual output: 1734 1735 >>> failure.got 1736 '1\n' 1737 1738 If a failure or error occurs, the globals are left intact: 1739 1740 >>> del test.globs['__builtins__'] 1741 >>> test.globs 1742 {'x': 1} 1743 1744 >>> test = DocTestParser().get_doctest(''' 1745 ... >>> x = 2 1746 ... >>> raise KeyError 1747 ... ''', {}, 'foo', 'foo.py', 0) 1748 1749 >>> runner.run(test) 1750 Traceback (most recent call last): 1751 ... 1752 UnexpectedException: <DocTest foo from foo.py:0 (2 examples)> 1753 1754 >>> del test.globs['__builtins__'] 1755 >>> test.globs 1756 {'x': 2} 1757 1758 But the globals are cleared if there is no error: 1759 1760 >>> test = DocTestParser().get_doctest(''' 1761 ... >>> x = 2 1762 ... ''', {}, 'foo', 'foo.py', 0) 1763 1764 >>> runner.run(test) 1765 (0, 1) 1766 1767 >>> test.globs 1768 {} 1769 1770 """ 1771 1772 def run(self, test, compileflags=None, out=None, clear_globs=True): 1773 r = DocTestRunner.run(self, test, compileflags, out, False) 1774 if clear_globs: 1775 test.globs.clear() 1776 return r 1777 1778 def report_unexpected_exception(self, out, test, example, exc_info): 1779 raise UnexpectedException(test, example, exc_info) 1780 1781 def report_failure(self, out, test, example, got): 1782 raise DocTestFailure(test, example, got) 1783 1784###################################################################### 1785## 6. Test Functions 1786###################################################################### 1787# These should be backwards compatible. 1788 1789def testmod(m=None, name=None, globs=None, verbose=None, isprivate=None, 1790 report=True, optionflags=0, extraglobs=None, 1791 raise_on_error=False): 1792 """m=None, name=None, globs=None, verbose=None, isprivate=None, 1793 report=True, optionflags=0, extraglobs=None 1794 1795 Test examples in docstrings in functions and classes reachable 1796 from module m (or the current module if m is not supplied), starting 1797 with m.__doc__. Unless isprivate is specified, private names 1798 are not skipped. 1799 1800 Also test examples reachable from dict m.__test__ if it exists and is 1801 not None. m.__test__ maps names to functions, classes and strings; 1802 function and class docstrings are tested even if the name is private; 1803 strings are tested directly, as if they were docstrings. 1804 1805 Return (#failures, #tests). 1806 1807 See doctest.__doc__ for an overview. 1808 1809 Optional keyword arg "name" gives the name of the module; by default 1810 use m.__name__. 1811 1812 Optional keyword arg "globs" gives a dict to be used as the globals 1813 when executing examples; by default, use m.__dict__. A copy of this 1814 dict is actually used for each docstring, so that each docstring's 1815 examples start with a clean slate. 1816 1817 Optional keyword arg "extraglobs" gives a dictionary that should be 1818 merged into the globals that are used to execute examples. By 1819 default, no extra globals are used. This is new in 2.4. 1820 1821 Optional keyword arg "verbose" prints lots of stuff if true, prints 1822 only failures if false; by default, it's true iff "-v" is in sys.argv. 1823 1824 Optional keyword arg "report" prints a summary at the end when true, 1825 else prints nothing at the end. In verbose mode, the summary is 1826 detailed, else very brief (in fact, empty if all tests passed). 1827 1828 Optional keyword arg "optionflags" or's together module constants, 1829 and defaults to 0. This is new in 2.3. Possible values (see the 1830 docs for details): 1831 1832 DONT_ACCEPT_TRUE_FOR_1 1833 DONT_ACCEPT_BLANKLINE 1834 NORMALIZE_WHITESPACE 1835 ELLIPSIS 1836 UNIFIED_DIFF 1837 CONTEXT_DIFF 1838 NDIFF_DIFF 1839 1840 Optional keyword arg "raise_on_error" raises an exception on the 1841 first unexpected exception or failure. This allows failures to be 1842 post-mortem debugged. 1843 1844 Deprecated in Python 2.4: 1845 Optional keyword arg "isprivate" specifies a function used to 1846 determine whether a name is private. The default function is 1847 treat all functions as public. Optionally, "isprivate" can be 1848 set to doctest.is_private to skip over functions marked as private 1849 using the underscore naming convention; see its docs for details. 1850 """ 1851 1852 """ [XX] This is no longer true: 1853 Advanced tomfoolery: testmod runs methods of a local instance of 1854 class doctest.Tester, then merges the results into (or creates) 1855 global Tester instance doctest.master. Methods of doctest.master 1856 can be called directly too, if you want to do something unusual. 1857 Passing report=0 to testmod is especially useful then, to delay 1858 displaying a summary. Invoke doctest.master.summarize(verbose) 1859 when you're done fiddling. 1860 """ 1861 if isprivate is not None: 1862 warnings.warn("the isprivate argument is deprecated; " 1863 "examine DocTestFinder.find() lists instead", 1864 DeprecationWarning) 1865 1866 # If no module was given, then use __main__. 1867 if m is None: 1868 # DWA - m will still be None if this wasn't invoked from the command 1869 # line, in which case the following TypeError is about as good an error 1870 # as we should expect 1871 m = sys.modules.get('__main__') 1872 1873 # Check that we were actually given a module. 1874 if not inspect.ismodule(m): 1875 raise TypeError("testmod: module required; %r" % (m,)) 1876 1877 # If no name was given, then use the module's name. 1878 if name is None: 1879 name = m.__name__ 1880 1881 # Find, parse, and run all tests in the given module. 1882 finder = DocTestFinder(_namefilter=isprivate) 1883 1884 if raise_on_error: 1885 runner = DebugRunner(verbose=verbose, optionflags=optionflags) 1886 else: 1887 runner = DocTestRunner(verbose=verbose, optionflags=optionflags) 1888 1889 for test in finder.find(m, name, globs=globs, extraglobs=extraglobs): 1890 runner.run(test) 1891 1892 if report: 1893 runner.summarize() 1894 1895 return runner.failures, runner.tries 1896 1897def run_docstring_examples(f, globs, verbose=False, name="NoName", 1898 compileflags=None, optionflags=0): 1899 """ 1900 Test examples in the given object's docstring (`f`), using `globs` 1901 as globals. Optional argument `name` is used in failure messages. 1902 If the optional argument `verbose` is true, then generate output 1903 even if there are no failures. 1904 1905 `compileflags` gives the set of flags that should be used by the 1906 Python compiler when running the examples. If not specified, then 1907 it will default to the set of future-import flags that apply to 1908 `globs`. 1909 1910 Optional keyword arg `optionflags` specifies options for the 1911 testing and output. See the documentation for `testmod` for more 1912 information. 1913 """ 1914 # Find, parse, and run all tests in the given module. 1915 finder = DocTestFinder(verbose=verbose, recurse=False) 1916 runner = DocTestRunner(verbose=verbose, optionflags=optionflags) 1917 for test in finder.find(f, name, globs=globs): 1918 runner.run(test, compileflags=compileflags) 1919 1920###################################################################### 1921## 7. Tester 1922###################################################################### 1923# This is provided only for backwards compatibility. It's not 1924# actually used in any way. 1925 1926class Tester: 1927 def __init__(self, mod=None, globs=None, verbose=None, 1928 isprivate=None, optionflags=0): 1929 1930 warnings.warn("class Tester is deprecated; " 1931 "use class doctest.DocTestRunner instead", 1932 DeprecationWarning, stacklevel=2) 1933 if mod is None and globs is None: 1934 raise TypeError("Tester.__init__: must specify mod or globs") 1935 if mod is not None and not _ismodule(mod): 1936 raise TypeError("Tester.__init__: mod must be a module; %r" % 1937 (mod,)) 1938 if globs is None: 1939 globs = mod.__dict__ 1940 self.globs = globs 1941 1942 self.verbose = verbose 1943 self.isprivate = isprivate 1944 self.optionflags = optionflags 1945 self.testfinder = DocTestFinder(_namefilter=isprivate) 1946 self.testrunner = DocTestRunner(verbose=verbose, 1947 optionflags=optionflags) 1948 1949 def runstring(self, s, name): 1950 test = DocTestParser().get_doctest(s, self.globs, name, None, None) 1951 if self.verbose: 1952 print "Running string", name 1953 (f,t) = self.testrunner.run(test) 1954 if self.verbose: 1955 print f, "of", t, "examples failed in string", name 1956 return (f,t) 1957 1958 def rundoc(self, object, name=None, module=None): 1959 f = t = 0 1960 tests = self.testfinder.find(object, name, module=module, 1961 globs=self.globs) 1962 for test in tests: 1963 (f2, t2) = self.testrunner.run(test) 1964 (f,t) = (f+f2, t+t2) 1965 return (f,t) 1966 1967 def rundict(self, d, name, module=None): 1968 import new 1969 m = new.module(name) 1970 m.__dict__.update(d) 1971 if module is None: 1972 module = False 1973 return self.rundoc(m, name, module) 1974 1975 def run__test__(self, d, name): 1976 import new 1977 m = new.module(name) 1978 m.__test__ = d 1979 return self.rundoc(m, name, module) 1980 1981 def summarize(self, verbose=None): 1982 return self.testrunner.summarize(verbose) 1983 1984 def merge(self, other): 1985 d = self.testrunner._name2ft 1986 for name, (f, t) in other.testrunner._name2ft.items(): 1987 if name in d: 1988 print "*** Tester.merge: '" + name + "' in both" \ 1989 " testers; summing outcomes." 1990 f2, t2 = d[name] 1991 f = f + f2 1992 t = t + t2 1993 d[name] = f, t 1994 1995###################################################################### 1996## 8. Unittest Support 1997###################################################################### 1998 1999class DocTestCase(unittest.TestCase): 2000 2001 def __init__(self, test, optionflags=0, setUp=None, tearDown=None, 2002 checker=None): 2003 2004 unittest.TestCase.__init__(self) 2005 self._dt_optionflags = optionflags 2006 self._dt_checker = checker 2007 self._dt_test = test 2008 self._dt_setUp = setUp 2009 self._dt_tearDown = tearDown 2010 2011 def setUp(self): 2012 if self._dt_setUp is not None: 2013 self._dt_setUp() 2014 2015 def tearDown(self): 2016 if self._dt_tearDown is not None: 2017 self._dt_tearDown() 2018 2019 def runTest(self): 2020 test = self._dt_test 2021 old = sys.stdout 2022 new = StringIO() 2023 runner = DocTestRunner(optionflags=self._dt_optionflags, 2024 checker=self._dt_checker, verbose=False) 2025 2026 try: 2027 runner.DIVIDER = "-"*70 2028 failures, tries = runner.run(test, out=new.write) 2029 finally: 2030 sys.stdout = old 2031 2032 if failures: 2033 raise self.failureException(self.format_failure(new.getvalue())) 2034 2035 def format_failure(self, err): 2036 test = self._dt_test 2037 if test.lineno is None: 2038 lineno = 'unknown line number' 2039 else: 2040 lineno = '%s' % test.lineno 2041 lname = '.'.join(test.name.split('.')[-1:]) 2042 return ('Failed doctest test for %s\n' 2043 ' File "%s", line %s, in %s\n\n%s' 2044 % (test.name, test.filename, lineno, lname, err) 2045 ) 2046 2047 def debug(self): 2048 r"""Run the test case without results and without catching exceptions 2049 2050 The unit test framework includes a debug method on test cases 2051 and test suites to support post-mortem debugging. The test code 2052 is run in such a way that errors are not caught. This way a 2053 caller can catch the errors and initiate post-mortem debugging. 2054 2055 The DocTestCase provides a debug method that raises 2056 UnexpectedException errors if there is an unexepcted 2057 exception: 2058 2059 >>> test = DocTestParser().get_doctest('>>> raise KeyError\n42', 2060 ... {}, 'foo', 'foo.py', 0) 2061 >>> case = DocTestCase(test) 2062 >>> try: 2063 ... case.debug() 2064 ... except UnexpectedException, failure: 2065 ... pass 2066 2067 The UnexpectedException contains the test, the example, and 2068 the original exception: 2069 2070 >>> failure.test is test 2071 True 2072 2073 >>> failure.example.want 2074 '42\n' 2075 2076 >>> exc_info = failure.exc_info 2077 >>> raise exc_info[0], exc_info[1], exc_info[2] 2078 Traceback (most recent call last): 2079 ... 2080 KeyError 2081 2082 If the output doesn't match, then a DocTestFailure is raised: 2083 2084 >>> test = DocTestParser().get_doctest(''' 2085 ... >>> x = 1 2086 ... >>> x 2087 ... 2 2088 ... ''', {}, 'foo', 'foo.py', 0) 2089 >>> case = DocTestCase(test) 2090 2091 >>> try: 2092 ... case.debug() 2093 ... except DocTestFailure, failure: 2094 ... pass 2095 2096 DocTestFailure objects provide access to the test: 2097 2098 >>> failure.test is test 2099 True 2100 2101 As well as to the example: 2102 2103 >>> failure.example.want 2104 '2\n' 2105 2106 and the actual output: 2107 2108 >>> failure.got 2109 '1\n' 2110 2111 """ 2112 2113 runner = DebugRunner(optionflags=self._dt_optionflags, 2114 checker=self._dt_checker, verbose=False) 2115 runner.run(self._dt_test) 2116 2117 def id(self): 2118 return self._dt_test.name 2119 2120 def __repr__(self): 2121 name = self._dt_test.name.split('.') 2122 return "%s (%s)" % (name[-1], '.'.join(name[:-1])) 2123 2124 __str__ = __repr__ 2125 2126 def shortDescription(self): 2127 return "Doctest: " + self._dt_test.name 2128 2129def DocTestSuite(module=None, globs=None, extraglobs=None, 2130 optionflags=0, test_finder=None, 2131 setUp=lambda: None, tearDown=lambda: None, 2132 checker=None): 2133 """ 2134 Convert doctest tests for a module to a unittest test suite. 2135 2136 This converts each documentation string in a module that 2137 contains doctest tests to a unittest test case. If any of the 2138 tests in a doc string fail, then the test case fails. An exception 2139 is raised showing the name of the file containing the test and a 2140 (sometimes approximate) line number. 2141 2142 The `module` argument provides the module to be tested. The argument 2143 can be either a module or a module name. 2144 2145 If no argument is given, the calling module is used. 2146 """ 2147 2148 if test_finder is None: 2149 test_finder = DocTestFinder() 2150 2151 module = _normalize_module(module) 2152 tests = test_finder.find(module, globs=globs, extraglobs=extraglobs) 2153 if globs is None: 2154 globs = module.__dict__ 2155 if not tests: # [XX] why do we want to do this? 2156 raise ValueError(module, "has no tests") 2157 2158 tests.sort() 2159 suite = unittest.TestSuite() 2160 for test in tests: 2161 if len(test.examples) == 0: 2162 continue 2163 if not test.filename: 2164 filename = module.__file__ 2165 if filename[-4:] in (".pyc", ".pyo"): 2166 filename = filename[:-1] 2167 test.filename = filename 2168 suite.addTest(DocTestCase(test, optionflags, setUp, tearDown, 2169 checker)) 2170 2171 return suite 2172 2173class DocFileCase(DocTestCase): 2174 2175 def id(self): 2176 return '_'.join(self._dt_test.name.split('.')) 2177 2178 def __repr__(self): 2179 return self._dt_test.filename 2180 __str__ = __repr__ 2181 2182 def format_failure(self, err): 2183 return ('Failed doctest test for %s\n File "%s", line 0\n\n%s' 2184 % (self._dt_test.name, self._dt_test.filename, err) 2185 ) 2186 2187def DocFileTest(path, package=None, globs=None, 2188 setUp=None, tearDown=None, 2189 optionflags=0): 2190 package = _normalize_module(package) 2191 name = path.split('/')[-1] 2192 dir = os.path.split(package.__file__)[0] 2193 path = os.path.join(dir, *(path.split('/'))) 2194 doc = open(path).read() 2195 2196 if globs is None: 2197 globs = {} 2198 2199 test = DocTestParser().get_doctest(doc, globs, name, path, 0) 2200 2201 return DocFileCase(test, optionflags, setUp, tearDown) 2202 2203def DocFileSuite(*paths, **kw): 2204 """Creates a suite of doctest files. 2205 2206 One or more text file paths are given as strings. These should 2207 use "/" characters to separate path segments. Paths are relative 2208 to the directory of the calling module, or relative to the package 2209 passed as a keyword argument. 2210 2211 A number of options may be provided as keyword arguments: 2212 2213 package 2214 The name of a Python package. Text-file paths will be 2215 interpreted relative to the directory containing this package. 2216 The package may be supplied as a package object or as a dotted 2217 package name. 2218 2219 setUp 2220 The name of a set-up function. This is called before running the 2221 tests in each file. 2222 2223 tearDown 2224 The name of a tear-down function. This is called after running the 2225 tests in each file. 2226 2227 globs 2228 A dictionary containing initial global variables for the tests. 2229 """ 2230 suite = unittest.TestSuite() 2231 2232 # We do this here so that _normalize_module is called at the right 2233 # level. If it were called in DocFileTest, then this function 2234 # would be the caller and we might guess the package incorrectly. 2235 kw['package'] = _normalize_module(kw.get('package')) 2236 2237 for path in paths: 2238 suite.addTest(DocFileTest(path, **kw)) 2239 2240 return suite 2241 2242###################################################################### 2243## 9. Debugging Support 2244###################################################################### 2245 2246def script_from_examples(s): 2247 r"""Extract script from text with examples. 2248 2249 Converts text with examples to a Python script. Example input is 2250 converted to regular code. Example output and all other words 2251 are converted to comments: 2252 2253 >>> text = ''' 2254 ... Here are examples of simple math. 2255 ... 2256 ... Python has super accurate integer addition 2257 ... 2258 ... >>> 2 + 2 2259 ... 5 2260 ... 2261 ... And very friendly error messages: 2262 ... 2263 ... >>> 1/0 2264 ... To Infinity 2265 ... And 2266 ... Beyond 2267 ... 2268 ... You can use logic if you want: 2269 ... 2270 ... >>> if 0: 2271 ... ... blah 2272 ... ... blah 2273 ... ... 2274 ... 2275 ... Ho hum 2276 ... ''' 2277 2278 >>> print script_from_examples(text) 2279 # Here are examples of simple math. 2280 # 2281 # Python has super accurate integer addition 2282 # 2283 2 + 2 2284 # Expected: 2285 ## 5 2286 # 2287 # And very friendly error messages: 2288 # 2289 1/0 2290 # Expected: 2291 ## To Infinity 2292 ## And 2293 ## Beyond 2294 # 2295 # You can use logic if you want: 2296 # 2297 if 0: 2298 blah 2299 blah 2300 <BLANKLINE> 2301 # 2302 # Ho hum 2303 """ 2304 2305 return DocTestParser().get_program(s) 2306 2307def _want_comment(example): 2308 """ 2309 Return a comment containing the expected output for the given example. 2310 """ 2311 # Return the expected output, if any 2312 want = example.want 2313 if want: 2314 if want[-1] == '\n': 2315 want = want[:-1] 2316 want = "\n# ".join(want.split("\n")) 2317 want = "\n# Expected:\n# %s" % want 2318 return want 2319 2320def testsource(module, name): 2321 """Extract the test sources from a doctest docstring as a script. 2322 2323 Provide the module (or dotted name of the module) containing the 2324 test to be debugged and the name (within the module) of the object 2325 with the doc string with tests to be debugged. 2326 """ 2327 module = _normalize_module(module) 2328 tests = DocTestFinder().find(module) 2329 test = [t for t in tests if t.name == name] 2330 if not test: 2331 raise ValueError(name, "not found in tests") 2332 test = test[0] 2333 testsrc = script_from_examples(test.docstring) 2334 return testsrc 2335 2336def debug_src(src, pm=False, globs=None): 2337 """Debug a single doctest docstring, in argument `src`'""" 2338 testsrc = script_from_examples(src) 2339 debug_script(testsrc, pm, globs) 2340 2341def debug_script(src, pm=False, globs=None): 2342 "Debug a test script. `src` is the script, as a string." 2343 import pdb 2344 2345 # Note that tempfile.NameTemporaryFile() cannot be used. As the 2346 # docs say, a file so created cannot be opened by name a second time 2347 # on modern Windows boxes, and execfile() needs to open it. 2348 srcfilename = tempfile.mktemp(".py", "doctestdebug") 2349 f = open(srcfilename, 'w') 2350 f.write(src) 2351 f.close() 2352 2353 try: 2354 if globs: 2355 globs = globs.copy() 2356 else: 2357 globs = {} 2358 2359 if pm: 2360 try: 2361 execfile(srcfilename, globs, globs) 2362 except: 2363 print sys.exc_info()[1] 2364 pdb.post_mortem(sys.exc_info()[2]) 2365 else: 2366 # Note that %r is vital here. '%s' instead can, e.g., cause 2367 # backslashes to get treated as metacharacters on Windows. 2368 pdb.run("execfile(%r)" % srcfilename, globs, globs) 2369 2370 finally: 2371 os.remove(srcfilename) 2372 2373def debug(module, name, pm=False): 2374 """Debug a single doctest docstring. 2375 2376 Provide the module (or dotted name of the module) containing the 2377 test to be debugged and the name (within the module) of the object 2378 with the docstring with tests to be debugged. 2379 """ 2380 module = _normalize_module(module) 2381 testsrc = testsource(module, name) 2382 debug_script(testsrc, pm, module.__dict__) 2383 2384###################################################################### 2385## 10. Example Usage 2386###################################################################### 2387class _TestClass: 2388 """ 2389 A pointless class, for sanity-checking of docstring testing. 2390 2391 Methods: 2392 square() 2393 get() 2394 2395 >>> _TestClass(13).get() + _TestClass(-12).get() 2396 1 2397 >>> hex(_TestClass(13).square().get()) 2398 '0xa9' 2399 """ 2400 2401 def __init__(self, val): 2402 """val -> _TestClass object with associated value val. 2403 2404 >>> t = _TestClass(123) 2405 >>> print t.get() 2406 123 2407 """ 2408 2409 self.val = val 2410 2411 def square(self): 2412 """square() -> square TestClass's associated value 2413 2414 >>> _TestClass(13).square().get() 2415 169 2416 """ 2417 2418 self.val = self.val ** 2 2419 return self 2420 2421 def get(self): 2422 """get() -> return TestClass's associated value. 2423 2424 >>> x = _TestClass(-42) 2425 >>> print x.get() 2426 -42 2427 """ 2428 2429 return self.val 2430 2431__test__ = {"_TestClass": _TestClass, 2432 "string": r""" 2433 Example of a string object, searched as-is. 2434 >>> x = 1; y = 2 2435 >>> x + y, x * y 2436 (3, 2) 2437 """, 2438 2439 "bool-int equivalence": r""" 2440 In 2.2, boolean expressions displayed 2441 0 or 1. By default, we still accept 2442 them. This can be disabled by passing 2443 DONT_ACCEPT_TRUE_FOR_1 to the new 2444 optionflags argument. 2445 >>> 4 == 4 2446 1 2447 >>> 4 == 4 2448 True 2449 >>> 4 > 4 2450 0 2451 >>> 4 > 4 2452 False 2453 """, 2454 2455 "blank lines": r""" 2456 Blank lines can be marked with <BLANKLINE>: 2457 >>> print 'foo\n\nbar\n' 2458 foo 2459 <BLANKLINE> 2460 bar 2461 <BLANKLINE> 2462 """, 2463 2464 "ellipsis": r""" 2465 If the ellipsis flag is used, then '...' can be used to 2466 elide substrings in the desired output: 2467 >>> print range(1000) #doctest: +ELLIPSIS 2468 [0, 1, 2, ..., 999] 2469 """, 2470 2471 "whitespace normalization": r""" 2472 If the whitespace normalization flag is used, then 2473 differences in whitespace are ignored. 2474 >>> print range(30) #doctest: +NORMALIZE_WHITESPACE 2475 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 2476 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 2477 27, 28, 29] 2478 """, 2479 } 2480 2481def test1(): r""" 2482>>> warnings.filterwarnings("ignore", "class Tester", DeprecationWarning, 2483... "doctest", 0) 2484>>> from doctest import Tester 2485>>> t = Tester(globs={'x': 42}, verbose=0) 2486>>> t.runstring(r''' 2487... >>> x = x * 2 2488... >>> print x 2489... 42 2490... ''', 'XYZ') 2491********************************************************************** 2492Line 3, in XYZ 2493Failed example: 2494 print x 2495Expected: 2496 42 2497Got: 2498 84 2499(1, 2) 2500>>> t.runstring(">>> x = x * 2\n>>> print x\n84\n", 'example2') 2501(0, 2) 2502>>> t.summarize() 2503********************************************************************** 25041 items had failures: 2505 1 of 2 in XYZ 2506***Test Failed*** 1 failures. 2507(1, 4) 2508>>> t.summarize(verbose=1) 25091 items passed all tests: 2510 2 tests in example2 2511********************************************************************** 25121 items had failures: 2513 1 of 2 in XYZ 25144 tests in 2 items. 25153 passed and 1 failed. 2516***Test Failed*** 1 failures. 2517(1, 4) 2518""" 2519 2520def test2(): r""" 2521 >>> warnings.filterwarnings("ignore", "class Tester", 2522 ... DeprecationWarning, "doctest", 0) 2523 >>> t = Tester(globs={}, verbose=1) 2524 >>> test = r''' 2525 ... # just an example 2526 ... >>> x = 1 + 2 2527 ... >>> x 2528 ... 3 2529 ... ''' 2530 >>> t.runstring(test, "Example") 2531 Running string Example 2532 Trying: x = 1 + 2 2533 Expecting: nothing 2534 ok 2535 Trying: x 2536 Expecting: 3 2537 ok 2538 0 of 2 examples failed in string Example 2539 (0, 2) 2540""" 2541def test3(): r""" 2542 >>> warnings.filterwarnings("ignore", "class Tester", 2543 ... DeprecationWarning, "doctest", 0) 2544 >>> t = Tester(globs={}, verbose=0) 2545 >>> def _f(): 2546 ... '''Trivial docstring example. 2547 ... >>> assert 2 == 2 2548 ... ''' 2549 ... return 32 2550 ... 2551 >>> t.rundoc(_f) # expect 0 failures in 1 example 2552 (0, 1) 2553""" 2554def test4(): """ 2555 >>> import new 2556 >>> m1 = new.module('_m1') 2557 >>> m2 = new.module('_m2') 2558 >>> test_data = \""" 2559 ... def _f(): 2560 ... '''>>> assert 1 == 1 2561 ... ''' 2562 ... def g(): 2563 ... '''>>> assert 2 != 1 2564 ... ''' 2565 ... class H: 2566 ... '''>>> assert 2 > 1 2567 ... ''' 2568 ... def bar(self): 2569 ... '''>>> assert 1 < 2 2570 ... ''' 2571 ... \""" 2572 >>> exec test_data in m1.__dict__ 2573 >>> exec test_data in m2.__dict__ 2574 >>> m1.__dict__.update({"f2": m2._f, "g2": m2.g, "h2": m2.H}) 2575 2576 Tests that objects outside m1 are excluded: 2577 2578 >>> warnings.filterwarnings("ignore", "class Tester", 2579 ... DeprecationWarning, "doctest", 0) 2580 >>> t = Tester(globs={}, verbose=0) 2581 >>> t.rundict(m1.__dict__, "rundict_test", m1) # f2 and g2 and h2 skipped 2582 (0, 4) 2583 2584 Once more, not excluding stuff outside m1: 2585 2586 >>> t = Tester(globs={}, verbose=0) 2587 >>> t.rundict(m1.__dict__, "rundict_test_pvt") # None are skipped. 2588 (0, 8) 2589 2590 The exclusion of objects from outside the designated module is 2591 meant to be invoked automagically by testmod. 2592 2593 >>> testmod(m1, verbose=False) 2594 (0, 4) 2595""" 2596 2597def _test(): 2598 r = unittest.TextTestRunner() 2599 r.run(DocTestSuite()) 2600 2601if __name__ == "__main__": 2602 _test() 2603