doctest.py revision dd0e4752977b807ad02e0d47123ace3f2ef89edc
1# Module doctest. 2# Released to the public domain 16-Jan-2001, by Tim Peters (tim@python.org). 3# Major enhancements and refactoring by: 4# Jim Fulton 5# Edward Loper 6 7# Provided as-is; use at your own risk; no warranty; no promises; enjoy! 8 9# [XX] This docstring is out-of-date: 10r"""Module doctest -- a framework for running examples in docstrings. 11 12NORMAL USAGE 13 14In normal use, end each module M with: 15 16def _test(): 17 import doctest, M # replace M with your module's name 18 return doctest.testmod(M) # ditto 19 20if __name__ == "__main__": 21 _test() 22 23Then running the module as a script will cause the examples in the 24docstrings to get executed and verified: 25 26python M.py 27 28This won't display anything unless an example fails, in which case the 29failing example(s) and the cause(s) of the failure(s) are printed to stdout 30(why not stderr? because stderr is a lame hack <0.2 wink>), and the final 31line of output is "Test failed.". 32 33Run it with the -v switch instead: 34 35python M.py -v 36 37and a detailed report of all examples tried is printed to stdout, along 38with assorted summaries at the end. 39 40You can force verbose mode by passing "verbose=1" to testmod, or prohibit 41it by passing "verbose=0". In either of those cases, sys.argv is not 42examined by testmod. 43 44In any case, testmod returns a 2-tuple of ints (f, t), where f is the 45number of docstring examples that failed and t is the total number of 46docstring examples attempted. 47 48 49WHICH DOCSTRINGS ARE EXAMINED? 50 51+ M.__doc__. 52 53+ f.__doc__ for all functions f in M.__dict__.values(), except those 54 defined in other modules. 55 56+ C.__doc__ for all classes C in M.__dict__.values(), except those 57 defined in other modules. 58 59+ If M.__test__ exists and "is true", it must be a dict, and 60 each entry maps a (string) name to a function object, class object, or 61 string. Function and class object docstrings found from M.__test__ 62 are searched even if the name is private, and strings are searched 63 directly as if they were docstrings. In output, a key K in M.__test__ 64 appears with name 65 <name of M>.__test__.K 66 67Any classes found are recursively searched similarly, to test docstrings in 68their contained methods and nested classes. All names reached from 69M.__test__ are searched. 70 71Optionally, functions with private names can be skipped (unless listed in 72M.__test__) by supplying a function to the "isprivate" argument that will 73identify private functions. For convenience, one such function is 74supplied. docttest.is_private considers a name to be private if it begins 75with an underscore (like "_my_func") but doesn't both begin and end with 76(at least) two underscores (like "__init__"). By supplying this function 77or your own "isprivate" function to testmod, the behavior can be customized. 78 79If you want to test docstrings in objects with private names too, stuff 80them into an M.__test__ dict, or see ADVANCED USAGE below (e.g., pass your 81own isprivate function to Tester's constructor, or call the rundoc method 82of a Tester instance). 83 84WHAT'S THE EXECUTION CONTEXT? 85 86By default, each time testmod finds a docstring to test, it uses a *copy* 87of M's globals (so that running tests on a module doesn't change the 88module's real globals, and so that one test in M can't leave behind crumbs 89that accidentally allow another test to work). This means examples can 90freely use any names defined at top-level in M. It also means that sloppy 91imports (see above) can cause examples in external docstrings to use 92globals inappropriate for them. 93 94You can force use of your own dict as the execution context by passing 95"globs=your_dict" to testmod instead. Presumably this would be a copy of 96M.__dict__ merged with the globals from other imported modules. 97 98 99WHAT IF I WANT TO TEST A WHOLE PACKAGE? 100 101Piece o' cake, provided the modules do their testing from docstrings. 102Here's the test.py I use for the world's most elaborate Rational/ 103floating-base-conversion pkg (which I'll distribute some day): 104 105from Rational import Cvt 106from Rational import Format 107from Rational import machprec 108from Rational import Rat 109from Rational import Round 110from Rational import utils 111 112modules = (Cvt, 113 Format, 114 machprec, 115 Rat, 116 Round, 117 utils) 118 119def _test(): 120 import doctest 121 import sys 122 verbose = "-v" in sys.argv 123 for mod in modules: 124 doctest.testmod(mod, verbose=verbose, report=0) 125 doctest.master.summarize() 126 127if __name__ == "__main__": 128 _test() 129 130IOW, it just runs testmod on all the pkg modules. testmod remembers the 131names and outcomes (# of failures, # of tries) for each item it's seen, and 132passing "report=0" prevents it from printing a summary in verbose mode. 133Instead, the summary is delayed until all modules have been tested, and 134then "doctest.master.summarize()" forces the summary at the end. 135 136So this is very nice in practice: each module can be tested individually 137with almost no work beyond writing up docstring examples, and collections 138of modules can be tested too as a unit with no more work than the above. 139 140 141WHAT ABOUT EXCEPTIONS? 142 143No problem, as long as the only output generated by the example is the 144traceback itself. For example: 145 146 >>> [1, 2, 3].remove(42) 147 Traceback (most recent call last): 148 File "<stdin>", line 1, in ? 149 ValueError: list.remove(x): x not in list 150 >>> 151 152Note that only the exception type and value are compared (specifically, 153only the last line in the traceback). 154 155 156ADVANCED USAGE 157 158doctest.testmod() captures the testing policy I find most useful most 159often. You may want other policies. 160 161testmod() actually creates a local instance of class doctest.Tester, runs 162appropriate methods of that class, and merges the results into global 163Tester instance doctest.master. 164 165You can create your own instances of doctest.Tester, and so build your own 166policies, or even run methods of doctest.master directly. See 167doctest.Tester.__doc__ for details. 168 169 170SO WHAT DOES A DOCSTRING EXAMPLE LOOK LIKE ALREADY!? 171 172Oh ya. It's easy! In most cases a copy-and-paste of an interactive 173console session works fine -- just make sure the leading whitespace is 174rigidly consistent (you can mix tabs and spaces if you're too lazy to do it 175right, but doctest is not in the business of guessing what you think a tab 176means). 177 178 >>> # comments are ignored 179 >>> x = 12 180 >>> x 181 12 182 >>> if x == 13: 183 ... print "yes" 184 ... else: 185 ... print "no" 186 ... print "NO" 187 ... print "NO!!!" 188 ... 189 no 190 NO 191 NO!!! 192 >>> 193 194Any expected output must immediately follow the final ">>>" or "..." line 195containing the code, and the expected output (if any) extends to the next 196">>>" or all-whitespace line. That's it. 197 198Bummers: 199 200+ Expected output cannot contain an all-whitespace line, since such a line 201 is taken to signal the end of expected output. 202 203+ Output to stdout is captured, but not output to stderr (exception 204 tracebacks are captured via a different means). 205 206+ If you continue a line via backslashing in an interactive session, 207 or for any other reason use a backslash, you should use a raw 208 docstring, which will preserve your backslahses exactly as you type 209 them: 210 211 >>> def f(x): 212 ... r'''Backslashes in a raw docstring: m\n''' 213 >>> print f.__doc__ 214 Backslashes in a raw docstring: m\n 215 216 Otherwise, the backslash will be interpreted as part of the string. 217 E.g., the "\n" above would be interpreted as a newline character. 218 Alternatively, you can double each backslash in the doctest version 219 (and not use a raw string): 220 221 >>> def f(x): 222 ... '''Backslashes in a raw docstring: m\\n''' 223 >>> print f.__doc__ 224 Backslashes in a raw docstring: m\n 225 226The starting column doesn't matter: 227 228>>> assert "Easy!" 229 >>> import math 230 >>> math.floor(1.9) 231 1.0 232 233and as many leading whitespace characters are stripped from the expected 234output as appeared in the initial ">>>" line that triggered it. 235 236If you execute this very file, the examples above will be found and 237executed, leading to this output in verbose mode: 238 239Running doctest.__doc__ 240Trying: [1, 2, 3].remove(42) 241Expecting: 242Traceback (most recent call last): 243 File "<stdin>", line 1, in ? 244ValueError: list.remove(x): x not in list 245ok 246Trying: x = 12 247Expecting: nothing 248ok 249Trying: x 250Expecting: 12 251ok 252Trying: 253if x == 13: 254 print "yes" 255else: 256 print "no" 257 print "NO" 258 print "NO!!!" 259Expecting: 260no 261NO 262NO!!! 263ok 264... and a bunch more like that, with this summary at the end: 265 2665 items had no tests: 267 doctest.Tester.__init__ 268 doctest.Tester.run__test__ 269 doctest.Tester.summarize 270 doctest.run_docstring_examples 271 doctest.testmod 27212 items passed all tests: 273 8 tests in doctest 274 6 tests in doctest.Tester 275 10 tests in doctest.Tester.merge 276 14 tests in doctest.Tester.rundict 277 3 tests in doctest.Tester.rundoc 278 3 tests in doctest.Tester.runstring 279 2 tests in doctest.__test__._TestClass 280 2 tests in doctest.__test__._TestClass.__init__ 281 2 tests in doctest.__test__._TestClass.get 282 1 tests in doctest.__test__._TestClass.square 283 2 tests in doctest.__test__.string 284 7 tests in doctest.is_private 28560 tests in 17 items. 28660 passed and 0 failed. 287Test passed. 288""" 289 290__all__ = [ 291 'is_private', 292 'Example', 293 'DocTest', 294 'DocTestFinder', 295 'DocTestRunner', 296 'testmod', 297 'run_docstring_examples', 298 'Tester', 299 'DocTestCase', 300 'DocTestSuite', 301 'testsource', 302 'debug', 303# 'master', 304] 305 306import __future__ 307 308import sys, traceback, inspect, linecache, os, re, types 309import unittest, difflib, tempfile 310import warnings 311from StringIO import StringIO 312 313# Option constants. 314DONT_ACCEPT_TRUE_FOR_1 = 1 << 0 315DONT_ACCEPT_BLANKLINE = 1 << 1 316NORMALIZE_WHITESPACE = 1 << 2 317ELLIPSIS = 1 << 3 318UNIFIED_DIFF = 1 << 4 319CONTEXT_DIFF = 1 << 5 320 321OPTIONFLAGS_BY_NAME = { 322 'DONT_ACCEPT_TRUE_FOR_1': DONT_ACCEPT_TRUE_FOR_1, 323 'DONT_ACCEPT_BLANKLINE': DONT_ACCEPT_BLANKLINE, 324 'NORMALIZE_WHITESPACE': NORMALIZE_WHITESPACE, 325 'ELLIPSIS': ELLIPSIS, 326 'UNIFIED_DIFF': UNIFIED_DIFF, 327 'CONTEXT_DIFF': CONTEXT_DIFF, 328 } 329 330# Special string markers for use in `want` strings: 331BLANKLINE_MARKER = '<BLANKLINE>' 332ELLIPSIS_MARKER = '...' 333 334 335# There are 4 basic classes: 336# - Example: a <source, want> pair, plus an intra-docstring line number. 337# - DocTest: a collection of examples, parsed from a docstring, plus 338# info about where the docstring came from (name, filename, lineno). 339# - DocTestFinder: extracts DocTests from a given object's docstring and 340# its contained objects' docstrings. 341# - DocTestRunner: runs DocTest cases, and accumulates statistics. 342# 343# So the basic picture is: 344# 345# list of: 346# +------+ +---------+ +-------+ 347# |object| --DocTestFinder-> | DocTest | --DocTestRunner-> |results| 348# +------+ +---------+ +-------+ 349# | Example | 350# | ... | 351# | Example | 352# +---------+ 353 354###################################################################### 355## Table of Contents 356###################################################################### 357# 1. Utility Functions 358# 2. Example & DocTest -- store test cases 359# 3. DocTest Parser -- extracts examples from strings 360# 4. DocTest Finder -- extracts test cases from objects 361# 5. DocTest Runner -- runs test cases 362# 6. Test Functions -- convenient wrappers for testing 363# 7. Tester Class -- for backwards compatibility 364# 8. Unittest Support 365# 9. Debugging Support 366# 10. Example Usage 367 368###################################################################### 369## 1. Utility Functions 370###################################################################### 371 372def is_private(prefix, base): 373 """prefix, base -> true iff name prefix + "." + base is "private". 374 375 Prefix may be an empty string, and base does not contain a period. 376 Prefix is ignored (although functions you write conforming to this 377 protocol may make use of it). 378 Return true iff base begins with an (at least one) underscore, but 379 does not both begin and end with (at least) two underscores. 380 381 >>> warnings.filterwarnings("ignore", "is_private", DeprecationWarning, 382 ... "doctest", 0) 383 >>> is_private("a.b", "my_func") 384 False 385 >>> is_private("____", "_my_func") 386 True 387 >>> is_private("someclass", "__init__") 388 False 389 >>> is_private("sometypo", "__init_") 390 True 391 >>> is_private("x.y.z", "_") 392 True 393 >>> is_private("_x.y.z", "__") 394 False 395 >>> is_private("", "") # senseless but consistent 396 False 397 """ 398 warnings.warn("is_private is deprecated; it wasn't useful; " 399 "examine DocTestFinder.find() lists instead", 400 DeprecationWarning, stacklevel=2) 401 return base[:1] == "_" and not base[:2] == "__" == base[-2:] 402 403def _extract_future_flags(globs): 404 """ 405 Return the compiler-flags associated with the future features that 406 have been imported into the given namespace (globs). 407 """ 408 flags = 0 409 for fname in __future__.all_feature_names: 410 feature = globs.get(fname, None) 411 if feature is getattr(__future__, fname): 412 flags |= feature.compiler_flag 413 return flags 414 415def _normalize_module(module, depth=2): 416 """ 417 Return the module specified by `module`. In particular: 418 - If `module` is a module, then return module. 419 - If `module` is a string, then import and return the 420 module with that name. 421 - If `module` is None, then return the calling module. 422 The calling module is assumed to be the module of 423 the stack frame at the given depth in the call stack. 424 """ 425 if inspect.ismodule(module): 426 return module 427 elif isinstance(module, (str, unicode)): 428 return __import__(module, globals(), locals(), ["*"]) 429 elif module is None: 430 return sys.modules[sys._getframe(depth).f_globals['__name__']] 431 else: 432 raise TypeError("Expected a module, string, or None") 433 434def _tag_msg(tag, msg, indent_msg=True): 435 """ 436 Return a string that displays a tag-and-message pair nicely, 437 keeping the tag and its message on the same line when that 438 makes sense. If `indent_msg` is true, then messages that are 439 put on separate lines will be indented. 440 """ 441 # What string should we use to indent contents? 442 INDENT = ' ' 443 444 # If the message doesn't end in a newline, then add one. 445 if msg[-1:] != '\n': 446 msg += '\n' 447 # If the message is short enough, and contains no internal 448 # newlines, then display it on the same line as the tag. 449 # Otherwise, display the tag on its own line. 450 if (len(tag) + len(msg) < 75 and 451 msg.find('\n', 0, len(msg)-1) == -1): 452 return '%s: %s' % (tag, msg) 453 else: 454 if indent_msg: 455 msg = '\n'.join([INDENT+l for l in msg.split('\n')]) 456 msg = msg[:-len(INDENT)] 457 return '%s:\n%s' % (tag, msg) 458 459# Override some StringIO methods. 460class _SpoofOut(StringIO): 461 def getvalue(self): 462 result = StringIO.getvalue(self) 463 # If anything at all was written, make sure there's a trailing 464 # newline. There's no way for the expected output to indicate 465 # that a trailing newline is missing. 466 if result and not result.endswith("\n"): 467 result += "\n" 468 # Prevent softspace from screwing up the next test case, in 469 # case they used print with a trailing comma in an example. 470 if hasattr(self, "softspace"): 471 del self.softspace 472 return result 473 474 def truncate(self, size=None): 475 StringIO.truncate(self, size) 476 if hasattr(self, "softspace"): 477 del self.softspace 478 479###################################################################### 480## 2. Example & DocTest 481###################################################################### 482## - An "example" is a <source, want> pair, where "source" is a 483## fragment of source code, and "want" is the expected output for 484## "source." The Example class also includes information about 485## where the example was extracted from. 486## 487## - A "doctest" is a collection of examples extracted from a string 488## (such as an object's docstring). The DocTest class also includes 489## information about where the string was extracted from. 490 491class Example: 492 """ 493 A single doctest example, consisting of source code and expected 494 output. Example defines the following attributes: 495 496 - source: A single python statement, ending in a newline iff the 497 statement spans more than one line. 498 499 - want: The expected output from running the source code (either 500 from stdout, or a traceback in case of exception). `want` 501 should always end with a newline, unless no output is expected, 502 503 - lineno: The line number within the DocTest string containing 504 this Example where the Example begins. This line number is 505 zero-based, with respect to the beginning of the DocTest. 506 """ 507 def __init__(self, source, want, lineno): 508 # Check invariants. 509 if (source[-1:] == '\n') != ('\n' in source[:-1]): 510 raise AssertionError("source must end with newline iff " 511 "source contains more than one line") 512 if want and want[-1] != '\n': 513 raise AssertionError("non-empty want must end with newline") 514 # Store properties. 515 self.source = source 516 self.want = want 517 self.lineno = lineno 518 519class DocTest: 520 """ 521 A collection of doctest examples that should be run in a single 522 namespace. Each DocTest defines the following attributes: 523 524 - examples: the list of examples. 525 526 - globs: The namespace (aka globals) that the examples should 527 be run in. 528 529 - name: A name identifying the DocTest (typically, the name of 530 the object whose docstring this DocTest was extracted from). 531 532 - docstring: The docstring being tested 533 534 - filename: The name of the file that this DocTest was extracted 535 from. 536 537 - lineno: The line number within filename where this DocTest 538 begins. This line number is zero-based, with respect to the 539 beginning of the file. 540 """ 541 def __init__(self, docstring, globs, name, filename, lineno): 542 """ 543 Create a new DocTest, by extracting examples from `docstring`. 544 The DocTest's globals are initialized with a copy of `globs`. 545 """ 546 # Store a copy of the globals 547 self.globs = globs.copy() 548 # Store identifying information 549 self.name = name 550 self.filename = filename 551 self.lineno = lineno 552 # Parse the docstring. 553 self.docstring = docstring 554 self.examples = Parser(name, docstring).get_examples() 555 556 def __repr__(self): 557 if len(self.examples) == 0: 558 examples = 'no examples' 559 elif len(self.examples) == 1: 560 examples = '1 example' 561 else: 562 examples = '%d examples' % len(self.examples) 563 return ('<DocTest %s from %s:%s (%s)>' % 564 (self.name, self.filename, self.lineno, examples)) 565 566 567 # This lets us sort tests by name: 568 def __cmp__(self, other): 569 if not isinstance(other, DocTest): 570 return -1 571 return cmp((self.name, self.filename, self.lineno, id(self)), 572 (other.name, other.filename, other.lineno, id(other))) 573 574###################################################################### 575## 2. Example Parser 576###################################################################### 577 578class Parser: 579 """ 580 Extract doctests from a string. 581 """ 582 def __init__(self, name, string): 583 """ 584 Prepare to extract doctests from string `string`. 585 586 `name` is an arbitrary (string) name associated with the string, 587 and is used only in error messages. 588 """ 589 self.name = name 590 self.string = string.expandtabs() 591 592 _EXAMPLE_RE = re.compile(r''' 593 # Source consists of a PS1 line followed by zero or more PS2 lines. 594 (?P<source> 595 (?:^(?P<indent> [ ]*) >>> .*) # PS1 line 596 (?:\n [ ]* \.\.\. .*)*) # PS2 lines 597 \n? 598 # Want consists of any non-blank lines that do not start with PS1. 599 (?P<want> (?:(?![ ]*$) # Not a blank line 600 (?![ ]*>>>) # Not a line starting with PS1 601 .*$\n? # But any other line 602 )*) 603 ''', re.MULTILINE | re.VERBOSE) 604 _IS_BLANK_OR_COMMENT = re.compile('^[ ]*(#.*)?$').match 605 606 def get_examples(self): 607 """ 608 Extract all doctest examples, from the string, and return them 609 as a list of `Example` objects. Line numbers are 0-based, 610 because it's most common in doctests that nothing interesting 611 appears on the same line as opening triple-quote, and so the 612 first interesting line is called \"line 1\" then. 613 614 >>> text = ''' 615 ... >>> x, y = 2, 3 # no output expected 616 ... >>> if 1: 617 ... ... print x 618 ... ... print y 619 ... 2 620 ... 3 621 ... 622 ... Some text. 623 ... >>> x+y 624 ... 5 625 ... ''' 626 >>> for x in Parser('<string>', text).get_examples(): 627 ... print (x.source, x.want, x.lineno) 628 ('x, y = 2, 3 # no output expected', '', 1) 629 ('if 1:\\n print x\\n print y\\n', '2\\n3\\n', 2) 630 ('x+y', '5\\n', 9) 631 """ 632 examples = [] 633 charno, lineno = 0, 0 634 # Find all doctest examples in the string: 635 for m in self._EXAMPLE_RE.finditer(self.string): 636 # Update lineno (lines before this example) 637 lineno += self.string.count('\n', charno, m.start()) 638 639 # Extract source/want from the regexp match. 640 (source, want) = self._parse_example(m, lineno) 641 if self._IS_BLANK_OR_COMMENT(source): 642 continue 643 examples.append( Example(source, want, lineno) ) 644 645 # Update lineno (lines inside this example) 646 lineno += self.string.count('\n', m.start(), m.end()) 647 # Update charno. 648 charno = m.end() 649 return examples 650 651 def get_program(self): 652 """ 653 Return an executable program from the string, as a string. 654 655 The format of this isn't rigidly defined. In general, doctest 656 examples become the executable statements in the result, and 657 their expected outputs become comments, preceded by an \"#Expected:\" 658 comment. Everything else (text, comments, everything not part of 659 a doctest test) is also placed in comments. 660 661 >>> text = ''' 662 ... >>> x, y = 2, 3 # no output expected 663 ... >>> if 1: 664 ... ... print x 665 ... ... print y 666 ... 2 667 ... 3 668 ... 669 ... Some text. 670 ... >>> x+y 671 ... 5 672 ... ''' 673 >>> print Parser('<string>', text).get_program() 674 x, y = 2, 3 # no output expected 675 if 1: 676 print x 677 print y 678 # Expected: 679 # 2 680 # 3 681 # 682 # Some text. 683 x+y 684 # Expected: 685 # 5 686 """ 687 output = [] 688 charnum, lineno = 0, 0 689 # Find all doctest examples in the string: 690 for m in self._EXAMPLE_RE.finditer(self.string): 691 # Add any text before this example, as a comment. 692 if m.start() > charnum: 693 lines = self.string[charnum:m.start()-1].split('\n') 694 output.extend([self._comment_line(l) for l in lines]) 695 lineno += len(lines) 696 697 # Extract source/want from the regexp match. 698 (source, want) = self._parse_example(m, lineno, False) 699 # Display the source 700 output.append(source) 701 # Display the expected output, if any 702 if want: 703 output.append('# Expected:') 704 output.extend(['# '+l for l in want.split('\n')]) 705 706 # Update the line number & char number. 707 lineno += self.string.count('\n', m.start(), m.end()) 708 charnum = m.end() 709 # Add any remaining text, as comments. 710 output.extend([self._comment_line(l) 711 for l in self.string[charnum:].split('\n')]) 712 # Trim junk on both ends. 713 while output and output[-1] == '#': 714 output.pop() 715 while output and output[0] == '#': 716 output.pop(0) 717 # Combine the output, and return it. 718 return '\n'.join(output) 719 720 def _parse_example(self, m, lineno, add_newlines=True): 721 # Get the example's indentation level. 722 indent = len(m.group('indent')) 723 724 # Divide source into lines; check that they're properly 725 # indented; and then strip their indentation & prompts. 726 source_lines = m.group('source').split('\n') 727 self._check_prompt_blank(source_lines, indent, lineno) 728 self._check_prefix(source_lines[1:], ' '*indent+'.', lineno) 729 source = '\n'.join([sl[indent+4:] for sl in source_lines]) 730 if len(source_lines) > 1 and add_newlines: 731 source += '\n' 732 733 # Divide want into lines; check that it's properly 734 # indented; and then strip the indentation. 735 want_lines = m.group('want').rstrip().split('\n') 736 self._check_prefix(want_lines, ' '*indent, 737 lineno+len(source_lines)) 738 want = '\n'.join([wl[indent:] for wl in want_lines]) 739 if len(want) > 0 and add_newlines: 740 want += '\n' 741 742 return source, want 743 744 def _comment_line(self, line): 745 line = line.rstrip() 746 if line: 747 return '# '+line 748 else: 749 return '#' 750 751 def _check_prompt_blank(self, lines, indent, lineno): 752 for i, line in enumerate(lines): 753 if len(line) >= indent+4 and line[indent+3] != ' ': 754 raise ValueError('line %r of the docstring for %s ' 755 'lacks blank after %s: %r' % 756 (lineno+i+1, self.name, 757 line[indent:indent+3], line)) 758 759 def _check_prefix(self, lines, prefix, lineno): 760 for i, line in enumerate(lines): 761 if line and not line.startswith(prefix): 762 raise ValueError('line %r of the docstring for %s has ' 763 'inconsistent leading whitespace: %r' % 764 (lineno+i+1, self.name, line)) 765 766 767###################################################################### 768## 4. DocTest Finder 769###################################################################### 770 771class DocTestFinder: 772 """ 773 A class used to extract the DocTests that are relevant to a given 774 object, from its docstring and the docstrings of its contained 775 objects. Doctests can currently be extracted from the following 776 object types: modules, functions, classes, methods, staticmethods, 777 classmethods, and properties. 778 """ 779 780 def __init__(self, verbose=False, doctest_factory=DocTest, 781 recurse=True, _namefilter=None): 782 """ 783 Create a new doctest finder. 784 785 The optional argument `doctest_factory` specifies a class or 786 function that should be used to create new DocTest objects (or 787 objects that implement the same interface as DocTest). The 788 signature for this factory function should match the signature 789 of the DocTest constructor. 790 791 If the optional argument `recurse` is false, then `find` will 792 only examine the given object, and not any contained objects. 793 """ 794 self._doctest_factory = doctest_factory 795 self._verbose = verbose 796 self._recurse = recurse 797 # _namefilter is undocumented, and exists only for temporary backward- 798 # compatibility support of testmod's deprecated isprivate mess. 799 self._namefilter = _namefilter 800 801 def find(self, obj, name=None, module=None, globs=None, 802 extraglobs=None): 803 """ 804 Return a list of the DocTests that are defined by the given 805 object's docstring, or by any of its contained objects' 806 docstrings. 807 808 The optional parameter `module` is the module that contains 809 the given object. If the module is not specified or is None, then 810 the test finder will attempt to automatically determine the 811 correct module. The object's module is used: 812 813 - As a default namespace, if `globs` is not specified. 814 - To prevent the DocTestFinder from extracting DocTests 815 from objects that are imported from other modules. 816 - To find the name of the file containing the object. 817 - To help find the line number of the object within its 818 file. 819 820 Contained objects whose module does not match `module` are ignored. 821 822 If `module` is False, no attempt to find the module will be made. 823 This is obscure, of use mostly in tests: if `module` is False, or 824 is None but cannot be found automatically, then all objects are 825 considered to belong to the (non-existent) module, so all contained 826 objects will (recursively) be searched for doctests. 827 828 The globals for each DocTest is formed by combining `globs` 829 and `extraglobs` (bindings in `extraglobs` override bindings 830 in `globs`). A new copy of the globals dictionary is created 831 for each DocTest. If `globs` is not specified, then it 832 defaults to the module's `__dict__`, if specified, or {} 833 otherwise. If `extraglobs` is not specified, then it defaults 834 to {}. 835 836 """ 837 # If name was not specified, then extract it from the object. 838 if name is None: 839 name = getattr(obj, '__name__', None) 840 if name is None: 841 raise ValueError("DocTestFinder.find: name must be given " 842 "when obj.__name__ doesn't exist: %r" % 843 (type(obj),)) 844 845 # Find the module that contains the given object (if obj is 846 # a module, then module=obj.). Note: this may fail, in which 847 # case module will be None. 848 if module is False: 849 module = None 850 elif module is None: 851 module = inspect.getmodule(obj) 852 853 # Read the module's source code. This is used by 854 # DocTestFinder._find_lineno to find the line number for a 855 # given object's docstring. 856 try: 857 file = inspect.getsourcefile(obj) or inspect.getfile(obj) 858 source_lines = linecache.getlines(file) 859 if not source_lines: 860 source_lines = None 861 except TypeError: 862 source_lines = None 863 864 # Initialize globals, and merge in extraglobs. 865 if globs is None: 866 if module is None: 867 globs = {} 868 else: 869 globs = module.__dict__.copy() 870 else: 871 globs = globs.copy() 872 if extraglobs is not None: 873 globs.update(extraglobs) 874 875 # Recursively expore `obj`, extracting DocTests. 876 tests = [] 877 self._find(tests, obj, name, module, source_lines, globs, {}) 878 return tests 879 880 def _filter(self, obj, prefix, base): 881 """ 882 Return true if the given object should not be examined. 883 """ 884 return (self._namefilter is not None and 885 self._namefilter(prefix, base)) 886 887 def _from_module(self, module, object): 888 """ 889 Return true if the given object is defined in the given 890 module. 891 """ 892 if module is None: 893 return True 894 elif inspect.isfunction(object): 895 return module.__dict__ is object.func_globals 896 elif inspect.isclass(object): 897 return module.__name__ == object.__module__ 898 elif inspect.getmodule(object) is not None: 899 return module is inspect.getmodule(object) 900 elif hasattr(object, '__module__'): 901 return module.__name__ == object.__module__ 902 elif isinstance(object, property): 903 return True # [XX] no way not be sure. 904 else: 905 raise ValueError("object must be a class or function") 906 907 def _find(self, tests, obj, name, module, source_lines, globs, seen): 908 """ 909 Find tests for the given object and any contained objects, and 910 add them to `tests`. 911 """ 912 if self._verbose: 913 print 'Finding tests in %s' % name 914 915 # If we've already processed this object, then ignore it. 916 if id(obj) in seen: 917 return 918 seen[id(obj)] = 1 919 920 # Find a test for this object, and add it to the list of tests. 921 test = self._get_test(obj, name, module, globs, source_lines) 922 if test is not None: 923 tests.append(test) 924 925 # Look for tests in a module's contained objects. 926 if inspect.ismodule(obj) and self._recurse: 927 for valname, val in obj.__dict__.items(): 928 # Check if this contained object should be ignored. 929 if self._filter(val, name, valname): 930 continue 931 valname = '%s.%s' % (name, valname) 932 # Recurse to functions & classes. 933 if ((inspect.isfunction(val) or inspect.isclass(val)) and 934 self._from_module(module, val)): 935 self._find(tests, val, valname, module, source_lines, 936 globs, seen) 937 938 # Look for tests in a module's __test__ dictionary. 939 if inspect.ismodule(obj) and self._recurse: 940 for valname, val in getattr(obj, '__test__', {}).items(): 941 if not isinstance(valname, basestring): 942 raise ValueError("DocTestFinder.find: __test__ keys " 943 "must be strings: %r" % 944 (type(valname),)) 945 if not (inspect.isfunction(val) or inspect.isclass(val) or 946 inspect.ismethod(val) or inspect.ismodule(val) or 947 isinstance(val, basestring)): 948 raise ValueError("DocTestFinder.find: __test__ values " 949 "must be strings, functions, methods, " 950 "classes, or modules: %r" % 951 (type(val),)) 952 valname = '%s.%s' % (name, valname) 953 self._find(tests, val, valname, module, source_lines, 954 globs, seen) 955 956 # Look for tests in a class's contained objects. 957 if inspect.isclass(obj) and self._recurse: 958 for valname, val in obj.__dict__.items(): 959 # Check if this contained object should be ignored. 960 if self._filter(val, name, valname): 961 continue 962 # Special handling for staticmethod/classmethod. 963 if isinstance(val, staticmethod): 964 val = getattr(obj, valname) 965 if isinstance(val, classmethod): 966 val = getattr(obj, valname).im_func 967 968 # Recurse to methods, properties, and nested classes. 969 if ((inspect.isfunction(val) or inspect.isclass(val) or 970 isinstance(val, property)) and 971 self._from_module(module, val)): 972 valname = '%s.%s' % (name, valname) 973 self._find(tests, val, valname, module, source_lines, 974 globs, seen) 975 976 def _get_test(self, obj, name, module, globs, source_lines): 977 """ 978 Return a DocTest for the given object, if it defines a docstring; 979 otherwise, return None. 980 """ 981 # Extract the object's docstring. If it doesn't have one, 982 # then return None (no test for this object). 983 if isinstance(obj, basestring): 984 docstring = obj 985 else: 986 try: 987 if obj.__doc__ is None: 988 return None 989 docstring = str(obj.__doc__) 990 except (TypeError, AttributeError): 991 return None 992 993 # Don't bother if the docstring is empty. 994 if not docstring: 995 return None 996 997 # Find the docstring's location in the file. 998 lineno = self._find_lineno(obj, source_lines) 999 1000 # Return a DocTest for this object. 1001 if module is None: 1002 filename = None 1003 else: 1004 filename = getattr(module, '__file__', module.__name__) 1005 return self._doctest_factory(docstring, globs, name, filename, lineno) 1006 1007 def _find_lineno(self, obj, source_lines): 1008 """ 1009 Return a line number of the given object's docstring. Note: 1010 this method assumes that the object has a docstring. 1011 """ 1012 lineno = None 1013 1014 # Find the line number for modules. 1015 if inspect.ismodule(obj): 1016 lineno = 0 1017 1018 # Find the line number for classes. 1019 # Note: this could be fooled if a class is defined multiple 1020 # times in a single file. 1021 if inspect.isclass(obj): 1022 if source_lines is None: 1023 return None 1024 pat = re.compile(r'^\s*class\s*%s\b' % 1025 getattr(obj, '__name__', '-')) 1026 for i, line in enumerate(source_lines): 1027 if pat.match(line): 1028 lineno = i 1029 break 1030 1031 # Find the line number for functions & methods. 1032 if inspect.ismethod(obj): obj = obj.im_func 1033 if inspect.isfunction(obj): obj = obj.func_code 1034 if inspect.istraceback(obj): obj = obj.tb_frame 1035 if inspect.isframe(obj): obj = obj.f_code 1036 if inspect.iscode(obj): 1037 lineno = getattr(obj, 'co_firstlineno', None)-1 1038 1039 # Find the line number where the docstring starts. Assume 1040 # that it's the first line that begins with a quote mark. 1041 # Note: this could be fooled by a multiline function 1042 # signature, where a continuation line begins with a quote 1043 # mark. 1044 if lineno is not None: 1045 if source_lines is None: 1046 return lineno+1 1047 pat = re.compile('(^|.*:)\s*\w*("|\')') 1048 for lineno in range(lineno, len(source_lines)): 1049 if pat.match(source_lines[lineno]): 1050 return lineno 1051 1052 # We couldn't find the line number. 1053 return None 1054 1055###################################################################### 1056## 5. DocTest Runner 1057###################################################################### 1058 1059class DocTestRunner: 1060 """ 1061 A class used to run DocTest test cases, and accumulate statistics. 1062 The `run` method is used to process a single DocTest case. It 1063 returns a tuple `(f, t)`, where `t` is the number of test cases 1064 tried, and `f` is the number of test cases that failed. 1065 1066 >>> tests = DocTestFinder().find(_TestClass) 1067 >>> runner = DocTestRunner(verbose=False) 1068 >>> for test in tests: 1069 ... print runner.run(test) 1070 (0, 2) 1071 (0, 1) 1072 (0, 2) 1073 (0, 2) 1074 1075 The `summarize` method prints a summary of all the test cases that 1076 have been run by the runner, and returns an aggregated `(f, t)` 1077 tuple: 1078 1079 >>> runner.summarize(verbose=1) 1080 4 items passed all tests: 1081 2 tests in _TestClass 1082 2 tests in _TestClass.__init__ 1083 2 tests in _TestClass.get 1084 1 tests in _TestClass.square 1085 7 tests in 4 items. 1086 7 passed and 0 failed. 1087 Test passed. 1088 (0, 7) 1089 1090 The aggregated number of tried examples and failed examples is 1091 also available via the `tries` and `failures` attributes: 1092 1093 >>> runner.tries 1094 7 1095 >>> runner.failures 1096 0 1097 1098 The comparison between expected outputs and actual outputs is done 1099 by an `OutputChecker`. This comparison may be customized with a 1100 number of option flags; see the documentation for `testmod` for 1101 more information. If the option flags are insufficient, then the 1102 comparison may also be customized by passing a subclass of 1103 `OutputChecker` to the constructor. 1104 1105 The test runner's display output can be controlled in two ways. 1106 First, an output function (`out) can be passed to 1107 `TestRunner.run`; this function will be called with strings that 1108 should be displayed. It defaults to `sys.stdout.write`. If 1109 capturing the output is not sufficient, then the display output 1110 can be also customized by subclassing DocTestRunner, and 1111 overriding the methods `report_start`, `report_success`, 1112 `report_unexpected_exception`, and `report_failure`. 1113 """ 1114 # This divider string is used to separate failure messages, and to 1115 # separate sections of the summary. 1116 DIVIDER = "*" * 70 1117 1118 def __init__(self, checker=None, verbose=None, optionflags=0): 1119 """ 1120 Create a new test runner. 1121 1122 Optional keyword arg `checker` is the `OutputChecker` that 1123 should be used to compare the expected outputs and actual 1124 outputs of doctest examples. 1125 1126 Optional keyword arg 'verbose' prints lots of stuff if true, 1127 only failures if false; by default, it's true iff '-v' is in 1128 sys.argv. 1129 1130 Optional argument `optionflags` can be used to control how the 1131 test runner compares expected output to actual output, and how 1132 it displays failures. See the documentation for `testmod` for 1133 more information. 1134 """ 1135 self._checker = checker or OutputChecker() 1136 if verbose is None: 1137 verbose = '-v' in sys.argv 1138 self._verbose = verbose 1139 self.optionflags = optionflags 1140 1141 # Keep track of the examples we've run. 1142 self.tries = 0 1143 self.failures = 0 1144 self._name2ft = {} 1145 1146 # Create a fake output target for capturing doctest output. 1147 self._fakeout = _SpoofOut() 1148 1149 #///////////////////////////////////////////////////////////////// 1150 # Reporting methods 1151 #///////////////////////////////////////////////////////////////// 1152 1153 def report_start(self, out, test, example): 1154 """ 1155 Report that the test runner is about to process the given 1156 example. (Only displays a message if verbose=True) 1157 """ 1158 if self._verbose: 1159 out(_tag_msg("Trying", example.source) + 1160 _tag_msg("Expecting", example.want or "nothing")) 1161 1162 def report_success(self, out, test, example, got): 1163 """ 1164 Report that the given example ran successfully. (Only 1165 displays a message if verbose=True) 1166 """ 1167 if self._verbose: 1168 out("ok\n") 1169 1170 def report_failure(self, out, test, example, got): 1171 """ 1172 Report that the given example failed. 1173 """ 1174 # Print an error message. 1175 out(self.__failure_header(test, example) + 1176 self._checker.output_difference(example.want, got, 1177 self.optionflags)) 1178 1179 def report_unexpected_exception(self, out, test, example, exc_info): 1180 """ 1181 Report that the given example raised an unexpected exception. 1182 """ 1183 # Get a traceback message. 1184 excout = StringIO() 1185 exc_type, exc_val, exc_tb = exc_info 1186 traceback.print_exception(exc_type, exc_val, exc_tb, file=excout) 1187 exception_tb = excout.getvalue() 1188 # Print an error message. 1189 out(self.__failure_header(test, example) + 1190 _tag_msg("Exception raised", exception_tb)) 1191 1192 def __failure_header(self, test, example): 1193 s = (self.DIVIDER + "\n" + 1194 _tag_msg("Failure in example", example.source)) 1195 if test.filename is None: 1196 # [XX] I'm not putting +1 here, to give the same output 1197 # as the old version. But I think it *should* go here. 1198 return s + ("from line #%s of %s\n" % 1199 (example.lineno, test.name)) 1200 elif test.lineno is None: 1201 return s + ("from line #%s of %s in %s\n" % 1202 (example.lineno+1, test.name, test.filename)) 1203 else: 1204 lineno = test.lineno+example.lineno+1 1205 return s + ("from line #%s of %s (%s)\n" % 1206 (lineno, test.filename, test.name)) 1207 1208 #///////////////////////////////////////////////////////////////// 1209 # DocTest Running 1210 #///////////////////////////////////////////////////////////////// 1211 1212 # A regular expression for handling `want` strings that contain 1213 # expected exceptions. It divides `want` into two pieces: the 1214 # pre-exception output (`out`) and the exception message (`exc`), 1215 # as generated by traceback.format_exception_only(). (I assume 1216 # that the exception_only message is the first non-indented line 1217 # starting with word characters after the "Traceback ...".) 1218 _EXCEPTION_RE = re.compile(('^(?P<out>.*)' 1219 '^(?P<hdr>Traceback \((?:%s|%s)\):)\s*$.*?' 1220 '^(?P<exc>\w+.*)') % 1221 ('most recent call last', 'innermost last'), 1222 re.MULTILINE | re.DOTALL) 1223 1224 _OPTION_DIRECTIVE_RE = re.compile('\s*doctest:\s*(?P<flags>[^#\n]*)') 1225 1226 def __handle_directive(self, example): 1227 """ 1228 Check if the given example is actually a directive to doctest 1229 (to turn an optionflag on or off); and if it is, then handle 1230 the directive. 1231 1232 Return true iff the example is actually a directive (and so 1233 should not be executed). 1234 1235 """ 1236 m = self._OPTION_DIRECTIVE_RE.match(example.source) 1237 if m is None: 1238 return False 1239 1240 for flag in m.group('flags').upper().split(): 1241 if (flag[:1] not in '+-' or 1242 flag[1:] not in OPTIONFLAGS_BY_NAME): 1243 raise ValueError('Bad doctest option directive: '+flag) 1244 if flag[0] == '+': 1245 self.optionflags |= OPTIONFLAGS_BY_NAME[flag[1:]] 1246 else: 1247 self.optionflags &= ~OPTIONFLAGS_BY_NAME[flag[1:]] 1248 return True 1249 1250 def __run(self, test, compileflags, out): 1251 """ 1252 Run the examples in `test`. Write the outcome of each example 1253 with one of the `DocTestRunner.report_*` methods, using the 1254 writer function `out`. `compileflags` is the set of compiler 1255 flags that should be used to execute examples. Return a tuple 1256 `(f, t)`, where `t` is the number of examples tried, and `f` 1257 is the number of examples that failed. The examples are run 1258 in the namespace `test.globs`. 1259 """ 1260 # Keep track of the number of failures and tries. 1261 failures = tries = 0 1262 1263 # Save the option flags (since option directives can be used 1264 # to modify them). 1265 original_optionflags = self.optionflags 1266 1267 # Process each example. 1268 for example in test.examples: 1269 # Check if it's an option directive. If it is, then handle 1270 # it, and go on to the next example. 1271 if self.__handle_directive(example): 1272 continue 1273 1274 # Record that we started this example. 1275 tries += 1 1276 self.report_start(out, test, example) 1277 1278 # Run the example in the given context (globs), and record 1279 # any exception that gets raised. (But don't intercept 1280 # keyboard interrupts.) 1281 try: 1282 # If the example is a compound statement on one line, 1283 # like "if 1: print 2", then compile() requires a 1284 # trailing newline. Rather than analyze that, always 1285 # append one (it never hurts). 1286 exec compile(example.source + '\n', "<string>", "single", 1287 compileflags, 1) in test.globs 1288 exception = None 1289 except KeyboardInterrupt: 1290 raise 1291 except: 1292 exception = sys.exc_info() 1293 1294 # Extract the example's actual output from fakeout, and 1295 # write it to `got`. Add a terminating newline if it 1296 # doesn't have already one. 1297 got = self._fakeout.getvalue() 1298 self._fakeout.truncate(0) 1299 1300 # If the example executed without raising any exceptions, 1301 # then verify its output and report its outcome. 1302 if exception is None: 1303 if self._checker.check_output(example.want, got, 1304 self.optionflags): 1305 self.report_success(out, test, example, got) 1306 else: 1307 self.report_failure(out, test, example, got) 1308 failures += 1 1309 1310 # If the example raised an exception, then check if it was 1311 # expected. 1312 else: 1313 exc_info = sys.exc_info() 1314 exc_msg = traceback.format_exception_only(*exc_info[:2])[-1] 1315 1316 # Search the `want` string for an exception. If we don't 1317 # find one, then report an unexpected exception. 1318 m = self._EXCEPTION_RE.match(example.want) 1319 if m is None: 1320 self.report_unexpected_exception(out, test, example, 1321 exc_info) 1322 failures += 1 1323 else: 1324 exc_hdr = m.group('hdr')+'\n' # Exception header 1325 # The test passes iff the pre-exception output and 1326 # the exception description match the values given 1327 # in `want`. 1328 if (self._checker.check_output(m.group('out'), got, 1329 self.optionflags) and 1330 self._checker.check_output(m.group('exc'), exc_msg, 1331 self.optionflags)): 1332 # Is +exc_msg the right thing here?? 1333 self.report_success(out, test, example, 1334 got+exc_hdr+exc_msg) 1335 else: 1336 self.report_failure(out, test, example, 1337 got+exc_hdr+exc_msg) 1338 failures += 1 1339 1340 # Restore the option flags (in case they were modified) 1341 self.optionflags = original_optionflags 1342 1343 # Record and return the number of failures and tries. 1344 self.__record_outcome(test, failures, tries) 1345 return failures, tries 1346 1347 def __record_outcome(self, test, f, t): 1348 """ 1349 Record the fact that the given DocTest (`test`) generated `f` 1350 failures out of `t` tried examples. 1351 """ 1352 f2, t2 = self._name2ft.get(test.name, (0,0)) 1353 self._name2ft[test.name] = (f+f2, t+t2) 1354 self.failures += f 1355 self.tries += t 1356 1357 def run(self, test, compileflags=None, out=None, clear_globs=True): 1358 """ 1359 Run the examples in `test`, and display the results using the 1360 writer function `out`. 1361 1362 The examples are run in the namespace `test.globs`. If 1363 `clear_globs` is true (the default), then this namespace will 1364 be cleared after the test runs, to help with garbage 1365 collection. If you would like to examine the namespace after 1366 the test completes, then use `clear_globs=False`. 1367 1368 `compileflags` gives the set of flags that should be used by 1369 the Python compiler when running the examples. If not 1370 specified, then it will default to the set of future-import 1371 flags that apply to `globs`. 1372 1373 The output of each example is checked using 1374 `DocTestRunner.check_output`, and the results are formatted by 1375 the `DocTestRunner.report_*` methods. 1376 """ 1377 if compileflags is None: 1378 compileflags = _extract_future_flags(test.globs) 1379 if out is None: 1380 out = sys.stdout.write 1381 saveout = sys.stdout 1382 1383 try: 1384 sys.stdout = self._fakeout 1385 return self.__run(test, compileflags, out) 1386 finally: 1387 sys.stdout = saveout 1388 if clear_globs: 1389 test.globs.clear() 1390 1391 #///////////////////////////////////////////////////////////////// 1392 # Summarization 1393 #///////////////////////////////////////////////////////////////// 1394 def summarize(self, verbose=None): 1395 """ 1396 Print a summary of all the test cases that have been run by 1397 this DocTestRunner, and return a tuple `(f, t)`, where `f` is 1398 the total number of failed examples, and `t` is the total 1399 number of tried examples. 1400 1401 The optional `verbose` argument controls how detailed the 1402 summary is. If the verbosity is not specified, then the 1403 DocTestRunner's verbosity is used. 1404 """ 1405 if verbose is None: 1406 verbose = self._verbose 1407 notests = [] 1408 passed = [] 1409 failed = [] 1410 totalt = totalf = 0 1411 for x in self._name2ft.items(): 1412 name, (f, t) = x 1413 assert f <= t 1414 totalt += t 1415 totalf += f 1416 if t == 0: 1417 notests.append(name) 1418 elif f == 0: 1419 passed.append( (name, t) ) 1420 else: 1421 failed.append(x) 1422 if verbose: 1423 if notests: 1424 print len(notests), "items had no tests:" 1425 notests.sort() 1426 for thing in notests: 1427 print " ", thing 1428 if passed: 1429 print len(passed), "items passed all tests:" 1430 passed.sort() 1431 for thing, count in passed: 1432 print " %3d tests in %s" % (count, thing) 1433 if failed: 1434 print self.DIVIDER 1435 print len(failed), "items had failures:" 1436 failed.sort() 1437 for thing, (f, t) in failed: 1438 print " %3d of %3d in %s" % (f, t, thing) 1439 if verbose: 1440 print totalt, "tests in", len(self._name2ft), "items." 1441 print totalt - totalf, "passed and", totalf, "failed." 1442 if totalf: 1443 print "***Test Failed***", totalf, "failures." 1444 elif verbose: 1445 print "Test passed." 1446 return totalf, totalt 1447 1448class OutputChecker: 1449 """ 1450 A class used to check the whether the actual output from a doctest 1451 example matches the expected output. `OutputChecker` defines two 1452 methods: `check_output`, which compares a given pair of outputs, 1453 and returns true if they match; and `output_difference`, which 1454 returns a string describing the differences between two outputs. 1455 """ 1456 def check_output(self, want, got, optionflags): 1457 """ 1458 Return True iff the actual output (`got`) matches the expected 1459 output (`want`). These strings are always considered to match 1460 if they are identical; but depending on what option flags the 1461 test runner is using, several non-exact match types are also 1462 possible. See the documentation for `TestRunner` for more 1463 information about option flags. 1464 """ 1465 # Handle the common case first, for efficiency: 1466 # if they're string-identical, always return true. 1467 if got == want: 1468 return True 1469 1470 # The values True and False replaced 1 and 0 as the return 1471 # value for boolean comparisons in Python 2.3. 1472 if not (optionflags & DONT_ACCEPT_TRUE_FOR_1): 1473 if (got,want) == ("True\n", "1\n"): 1474 return True 1475 if (got,want) == ("False\n", "0\n"): 1476 return True 1477 1478 # <BLANKLINE> can be used as a special sequence to signify a 1479 # blank line, unless the DONT_ACCEPT_BLANKLINE flag is used. 1480 if not (optionflags & DONT_ACCEPT_BLANKLINE): 1481 # Replace <BLANKLINE> in want with a blank line. 1482 want = re.sub('(?m)^%s\s*?$' % re.escape(BLANKLINE_MARKER), 1483 '', want) 1484 # If a line in got contains only spaces, then remove the 1485 # spaces. 1486 got = re.sub('(?m)^\s*?$', '', got) 1487 if got == want: 1488 return True 1489 1490 # This flag causes doctest to ignore any differences in the 1491 # contents of whitespace strings. Note that this can be used 1492 # in conjunction with the ELLISPIS flag. 1493 if (optionflags & NORMALIZE_WHITESPACE): 1494 got = ' '.join(got.split()) 1495 want = ' '.join(want.split()) 1496 if got == want: 1497 return True 1498 1499 # The ELLIPSIS flag says to let the sequence "..." in `want` 1500 # match any substring in `got`. We implement this by 1501 # transforming `want` into a regular expression. 1502 if (optionflags & ELLIPSIS): 1503 # Escape any special regexp characters 1504 want_re = re.escape(want) 1505 # Replace ellipsis markers ('...') with .* 1506 want_re = want_re.replace(re.escape(ELLIPSIS_MARKER), '.*') 1507 # Require that it matches the entire string; and set the 1508 # re.DOTALL flag (with '(?s)'). 1509 want_re = '(?s)^%s$' % want_re 1510 # Check if the `want_re` regexp matches got. 1511 if re.match(want_re, got): 1512 return True 1513 1514 # We didn't find any match; return false. 1515 return False 1516 1517 def output_difference(self, want, got, optionflags): 1518 """ 1519 Return a string describing the differences between the 1520 expected output (`want`) and the actual output (`got`). 1521 """ 1522 # If <BLANKLINE>s are being used, then replace <BLANKLINE> 1523 # with blank lines in the expected output string. 1524 if not (optionflags & DONT_ACCEPT_BLANKLINE): 1525 want = re.sub('(?m)^%s$' % re.escape(BLANKLINE_MARKER), '', want) 1526 1527 # Check if we should use diff. Don't use diff if the actual 1528 # or expected outputs are too short, or if the expected output 1529 # contains an ellipsis marker. 1530 if ((optionflags & (UNIFIED_DIFF | CONTEXT_DIFF)) and 1531 want.count('\n') > 2 and got.count('\n') > 2 and 1532 not (optionflags & ELLIPSIS and '...' in want)): 1533 # Split want & got into lines. 1534 want_lines = [l+'\n' for l in want.split('\n')] 1535 got_lines = [l+'\n' for l in got.split('\n')] 1536 # Use difflib to find their differences. 1537 if optionflags & UNIFIED_DIFF: 1538 diff = difflib.unified_diff(want_lines, got_lines, n=2, 1539 fromfile='Expected', tofile='Got') 1540 kind = 'unified' 1541 elif optionflags & CONTEXT_DIFF: 1542 diff = difflib.context_diff(want_lines, got_lines, n=2, 1543 fromfile='Expected', tofile='Got') 1544 kind = 'context' 1545 else: 1546 assert 0, 'Bad diff option' 1547 # Remove trailing whitespace on diff output. 1548 diff = [line.rstrip() + '\n' for line in diff] 1549 return _tag_msg("Differences (" + kind + " diff)", 1550 ''.join(diff)) 1551 1552 # If we're not using diff, then simply list the expected 1553 # output followed by the actual output. 1554 return (_tag_msg("Expected", want or "Nothing") + 1555 _tag_msg("Got", got)) 1556 1557class DocTestFailure(Exception): 1558 """A DocTest example has failed in debugging mode. 1559 1560 The exception instance has variables: 1561 1562 - test: the DocTest object being run 1563 1564 - excample: the Example object that failed 1565 1566 - got: the actual output 1567 """ 1568 def __init__(self, test, example, got): 1569 self.test = test 1570 self.example = example 1571 self.got = got 1572 1573 def __str__(self): 1574 return str(self.test) 1575 1576class UnexpectedException(Exception): 1577 """A DocTest example has encountered an unexpected exception 1578 1579 The exception instance has variables: 1580 1581 - test: the DocTest object being run 1582 1583 - excample: the Example object that failed 1584 1585 - exc_info: the exception info 1586 """ 1587 def __init__(self, test, example, exc_info): 1588 self.test = test 1589 self.example = example 1590 self.exc_info = exc_info 1591 1592 def __str__(self): 1593 return str(self.test) 1594 1595class DebugRunner(DocTestRunner): 1596 r"""Run doc tests but raise an exception as soon as there is a failure. 1597 1598 If an unexpected exception occurs, an UnexpectedException is raised. 1599 It contains the test, the example, and the original exception: 1600 1601 >>> runner = DebugRunner(verbose=False) 1602 >>> test = DocTest('>>> raise KeyError\n42', {}, 'foo', 'foo.py', 0) 1603 >>> try: 1604 ... runner.run(test) 1605 ... except UnexpectedException, failure: 1606 ... pass 1607 1608 >>> failure.test is test 1609 True 1610 1611 >>> failure.example.want 1612 '42\n' 1613 1614 >>> exc_info = failure.exc_info 1615 >>> raise exc_info[0], exc_info[1], exc_info[2] 1616 Traceback (most recent call last): 1617 ... 1618 KeyError 1619 1620 We wrap the original exception to give the calling application 1621 access to the test and example information. 1622 1623 If the output doesn't match, then a DocTestFailure is raised: 1624 1625 >>> test = DocTest(''' 1626 ... >>> x = 1 1627 ... >>> x 1628 ... 2 1629 ... ''', {}, 'foo', 'foo.py', 0) 1630 1631 >>> try: 1632 ... runner.run(test) 1633 ... except DocTestFailure, failure: 1634 ... pass 1635 1636 DocTestFailure objects provide access to the test: 1637 1638 >>> failure.test is test 1639 True 1640 1641 As well as to the example: 1642 1643 >>> failure.example.want 1644 '2\n' 1645 1646 and the actual output: 1647 1648 >>> failure.got 1649 '1\n' 1650 1651 If a failure or error occurs, the globals are left intact: 1652 1653 >>> del test.globs['__builtins__'] 1654 >>> test.globs 1655 {'x': 1} 1656 1657 >>> test = DocTest(''' 1658 ... >>> x = 2 1659 ... >>> raise KeyError 1660 ... ''', {}, 'foo', 'foo.py', 0) 1661 1662 >>> runner.run(test) 1663 Traceback (most recent call last): 1664 ... 1665 UnexpectedException: <DocTest foo from foo.py:0 (2 examples)> 1666 1667 >>> del test.globs['__builtins__'] 1668 >>> test.globs 1669 {'x': 2} 1670 1671 But the globals are cleared if there is no error: 1672 1673 >>> test = DocTest(''' 1674 ... >>> x = 2 1675 ... ''', {}, 'foo', 'foo.py', 0) 1676 1677 >>> runner.run(test) 1678 (0, 1) 1679 1680 >>> test.globs 1681 {} 1682 1683 """ 1684 1685 def run(self, test, compileflags=None, out=None, clear_globs=True): 1686 r = DocTestRunner.run(self, test, compileflags, out, False) 1687 if clear_globs: 1688 test.globs.clear() 1689 return r 1690 1691 def report_unexpected_exception(self, out, test, example, exc_info): 1692 raise UnexpectedException(test, example, exc_info) 1693 1694 def report_failure(self, out, test, example, got): 1695 raise DocTestFailure(test, example, got) 1696 1697###################################################################### 1698## 6. Test Functions 1699###################################################################### 1700# These should be backwards compatible. 1701 1702def testmod(m=None, name=None, globs=None, verbose=None, isprivate=None, 1703 report=True, optionflags=0, extraglobs=None, 1704 raise_on_error=False): 1705 """m=None, name=None, globs=None, verbose=None, isprivate=None, 1706 report=True, optionflags=0, extraglobs=None 1707 1708 Test examples in docstrings in functions and classes reachable 1709 from module m (or the current module if m is not supplied), starting 1710 with m.__doc__. Unless isprivate is specified, private names 1711 are not skipped. 1712 1713 Also test examples reachable from dict m.__test__ if it exists and is 1714 not None. m.__dict__ maps names to functions, classes and strings; 1715 function and class docstrings are tested even if the name is private; 1716 strings are tested directly, as if they were docstrings. 1717 1718 Return (#failures, #tests). 1719 1720 See doctest.__doc__ for an overview. 1721 1722 Optional keyword arg "name" gives the name of the module; by default 1723 use m.__name__. 1724 1725 Optional keyword arg "globs" gives a dict to be used as the globals 1726 when executing examples; by default, use m.__dict__. A copy of this 1727 dict is actually used for each docstring, so that each docstring's 1728 examples start with a clean slate. 1729 1730 Optional keyword arg "extraglobs" gives a dictionary that should be 1731 merged into the globals that are used to execute examples. By 1732 default, no extra globals are used. This is new in 2.4. 1733 1734 Optional keyword arg "verbose" prints lots of stuff if true, prints 1735 only failures if false; by default, it's true iff "-v" is in sys.argv. 1736 1737 Optional keyword arg "report" prints a summary at the end when true, 1738 else prints nothing at the end. In verbose mode, the summary is 1739 detailed, else very brief (in fact, empty if all tests passed). 1740 1741 Optional keyword arg "optionflags" or's together module constants, 1742 and defaults to 0. This is new in 2.3. Possible values: 1743 1744 DONT_ACCEPT_TRUE_FOR_1 1745 By default, if an expected output block contains just "1", 1746 an actual output block containing just "True" is considered 1747 to be a match, and similarly for "0" versus "False". When 1748 DONT_ACCEPT_TRUE_FOR_1 is specified, neither substitution 1749 is allowed. 1750 1751 DONT_ACCEPT_BLANKLINE 1752 By default, if an expected output block contains a line 1753 containing only the string "<BLANKLINE>", then that line 1754 will match a blank line in the actual output. When 1755 DONT_ACCEPT_BLANKLINE is specified, this substitution is 1756 not allowed. 1757 1758 NORMALIZE_WHITESPACE 1759 When NORMALIZE_WHITESPACE is specified, all sequences of 1760 whitespace are treated as equal. I.e., any sequence of 1761 whitespace within the expected output will match any 1762 sequence of whitespace within the actual output. 1763 1764 ELLIPSIS 1765 When ELLIPSIS is specified, then an ellipsis marker 1766 ("...") in the expected output can match any substring in 1767 the actual output. 1768 1769 UNIFIED_DIFF 1770 When UNIFIED_DIFF is specified, failures that involve 1771 multi-line expected and actual outputs will be displayed 1772 using a unified diff. 1773 1774 CONTEXT_DIFF 1775 When CONTEXT_DIFF is specified, failures that involve 1776 multi-line expected and actual outputs will be displayed 1777 using a context diff. 1778 1779 Optional keyword arg "raise_on_error" raises an exception on the 1780 first unexpected exception or failure. This allows failures to be 1781 post-mortem debugged. 1782 1783 Deprecated in Python 2.4: 1784 Optional keyword arg "isprivate" specifies a function used to 1785 determine whether a name is private. The default function is 1786 treat all functions as public. Optionally, "isprivate" can be 1787 set to doctest.is_private to skip over functions marked as private 1788 using the underscore naming convention; see its docs for details. 1789 """ 1790 1791 """ [XX] This is no longer true: 1792 Advanced tomfoolery: testmod runs methods of a local instance of 1793 class doctest.Tester, then merges the results into (or creates) 1794 global Tester instance doctest.master. Methods of doctest.master 1795 can be called directly too, if you want to do something unusual. 1796 Passing report=0 to testmod is especially useful then, to delay 1797 displaying a summary. Invoke doctest.master.summarize(verbose) 1798 when you're done fiddling. 1799 """ 1800 if isprivate is not None: 1801 warnings.warn("the isprivate argument is deprecated; " 1802 "examine DocTestFinder.find() lists instead", 1803 DeprecationWarning) 1804 1805 # If no module was given, then use __main__. 1806 if m is None: 1807 # DWA - m will still be None if this wasn't invoked from the command 1808 # line, in which case the following TypeError is about as good an error 1809 # as we should expect 1810 m = sys.modules.get('__main__') 1811 1812 # Check that we were actually given a module. 1813 if not inspect.ismodule(m): 1814 raise TypeError("testmod: module required; %r" % (m,)) 1815 1816 # If no name was given, then use the module's name. 1817 if name is None: 1818 name = m.__name__ 1819 1820 # Find, parse, and run all tests in the given module. 1821 finder = DocTestFinder(_namefilter=isprivate) 1822 1823 if raise_on_error: 1824 runner = DebugRunner(verbose=verbose, optionflags=optionflags) 1825 else: 1826 runner = DocTestRunner(verbose=verbose, optionflags=optionflags) 1827 1828 for test in finder.find(m, name, globs=globs, extraglobs=extraglobs): 1829 runner.run(test) 1830 1831 if report: 1832 runner.summarize() 1833 1834 return runner.failures, runner.tries 1835 1836def run_docstring_examples(f, globs, verbose=False, name="NoName", 1837 compileflags=None, optionflags=0): 1838 """ 1839 Test examples in the given object's docstring (`f`), using `globs` 1840 as globals. Optional argument `name` is used in failure messages. 1841 If the optional argument `verbose` is true, then generate output 1842 even if there are no failures. 1843 1844 `compileflags` gives the set of flags that should be used by the 1845 Python compiler when running the examples. If not specified, then 1846 it will default to the set of future-import flags that apply to 1847 `globs`. 1848 1849 Optional keyword arg `optionflags` specifies options for the 1850 testing and output. See the documentation for `testmod` for more 1851 information. 1852 """ 1853 # Find, parse, and run all tests in the given module. 1854 finder = DocTestFinder(verbose=verbose, recurse=False) 1855 runner = DocTestRunner(verbose=verbose, optionflags=optionflags) 1856 for test in finder.find(f, name, globs=globs): 1857 runner.run(test, compileflags=compileflags) 1858 1859###################################################################### 1860## 7. Tester 1861###################################################################### 1862# This is provided only for backwards compatibility. It's not 1863# actually used in any way. 1864 1865class Tester: 1866 def __init__(self, mod=None, globs=None, verbose=None, 1867 isprivate=None, optionflags=0): 1868 1869 warnings.warn("class Tester is deprecated; " 1870 "use class doctest.DocTestRunner instead", 1871 DeprecationWarning, stacklevel=2) 1872 if mod is None and globs is None: 1873 raise TypeError("Tester.__init__: must specify mod or globs") 1874 if mod is not None and not _ismodule(mod): 1875 raise TypeError("Tester.__init__: mod must be a module; %r" % 1876 (mod,)) 1877 if globs is None: 1878 globs = mod.__dict__ 1879 self.globs = globs 1880 1881 self.verbose = verbose 1882 self.isprivate = isprivate 1883 self.optionflags = optionflags 1884 self.testfinder = DocTestFinder(_namefilter=isprivate) 1885 self.testrunner = DocTestRunner(verbose=verbose, 1886 optionflags=optionflags) 1887 1888 def runstring(self, s, name): 1889 test = DocTest(s, self.globs, name, None, None) 1890 if self.verbose: 1891 print "Running string", name 1892 (f,t) = self.testrunner.run(test) 1893 if self.verbose: 1894 print f, "of", t, "examples failed in string", name 1895 return (f,t) 1896 1897 def rundoc(self, object, name=None, module=None): 1898 f = t = 0 1899 tests = self.testfinder.find(object, name, module=module, 1900 globs=self.globs) 1901 for test in tests: 1902 (f2, t2) = self.testrunner.run(test) 1903 (f,t) = (f+f2, t+t2) 1904 return (f,t) 1905 1906 def rundict(self, d, name, module=None): 1907 import new 1908 m = new.module(name) 1909 m.__dict__.update(d) 1910 if module is None: 1911 module = False 1912 return self.rundoc(m, name, module) 1913 1914 def run__test__(self, d, name): 1915 import new 1916 m = new.module(name) 1917 m.__test__ = d 1918 return self.rundoc(m, name, module) 1919 1920 def summarize(self, verbose=None): 1921 return self.testrunner.summarize(verbose) 1922 1923 def merge(self, other): 1924 d = self.testrunner._name2ft 1925 for name, (f, t) in other.testrunner._name2ft.items(): 1926 if name in d: 1927 print "*** Tester.merge: '" + name + "' in both" \ 1928 " testers; summing outcomes." 1929 f2, t2 = d[name] 1930 f = f + f2 1931 t = t + t2 1932 d[name] = f, t 1933 1934###################################################################### 1935## 8. Unittest Support 1936###################################################################### 1937 1938class DocTestCase(unittest.TestCase): 1939 1940 def __init__(self, test, optionflags=0, setUp=None, tearDown=None, 1941 checker=None): 1942 unittest.TestCase.__init__(self) 1943 self._dt_optionflags = optionflags 1944 self._dt_checker = checker 1945 self._dt_test = test 1946 self._dt_setUp = setUp 1947 self._dt_tearDown = tearDown 1948 1949 def setUp(self): 1950 if self._dt_setUp is not None: 1951 self._dt_setUp() 1952 1953 def tearDown(self): 1954 if self._dt_tearDown is not None: 1955 self._dt_tearDown() 1956 1957 def runTest(self): 1958 test = self._dt_test 1959 old = sys.stdout 1960 new = StringIO() 1961 runner = DocTestRunner(optionflags=self._dt_optionflags, 1962 checker=self._dt_checker, verbose=False) 1963 1964 try: 1965 runner.DIVIDER = "-"*70 1966 failures, tries = runner.run(test, out=new.write) 1967 finally: 1968 sys.stdout = old 1969 1970 if failures: 1971 raise self.failureException(self.format_failure(new.getvalue())) 1972 1973 def format_failure(self, err): 1974 test = self._dt_test 1975 if test.lineno is None: 1976 lineno = 'unknown line number' 1977 else: 1978 lineno = 'line %s' % test.lineno 1979 lname = '.'.join(test.name.split('.')[-1:]) 1980 return ('Failed doctest test for %s\n' 1981 ' File "%s", line %s, in %s\n\n%s' 1982 % (test.name, test.filename, lineno, lname, err) 1983 ) 1984 1985 def debug(self): 1986 r"""Run the test case without results and without catching exceptions 1987 1988 The unit test framework includes a debug method on test cases 1989 and test suites to support post-mortem debugging. The test code 1990 is run in such a way that errors are not caught. This way a 1991 caller can catch the errors and initiate post-mortem debugging. 1992 1993 The DocTestCase provides a debug method that raises 1994 UnexpectedException errors if there is an unexepcted 1995 exception: 1996 1997 >>> test = DocTest('>>> raise KeyError\n42', 1998 ... {}, 'foo', 'foo.py', 0) 1999 >>> case = DocTestCase(test) 2000 >>> try: 2001 ... case.debug() 2002 ... except UnexpectedException, failure: 2003 ... pass 2004 2005 The UnexpectedException contains the test, the example, and 2006 the original exception: 2007 2008 >>> failure.test is test 2009 True 2010 2011 >>> failure.example.want 2012 '42\n' 2013 2014 >>> exc_info = failure.exc_info 2015 >>> raise exc_info[0], exc_info[1], exc_info[2] 2016 Traceback (most recent call last): 2017 ... 2018 KeyError 2019 2020 If the output doesn't match, then a DocTestFailure is raised: 2021 2022 >>> test = DocTest(''' 2023 ... >>> x = 1 2024 ... >>> x 2025 ... 2 2026 ... ''', {}, 'foo', 'foo.py', 0) 2027 >>> case = DocTestCase(test) 2028 2029 >>> try: 2030 ... case.debug() 2031 ... except DocTestFailure, failure: 2032 ... pass 2033 2034 DocTestFailure objects provide access to the test: 2035 2036 >>> failure.test is test 2037 True 2038 2039 As well as to the example: 2040 2041 >>> failure.example.want 2042 '2\n' 2043 2044 and the actual output: 2045 2046 >>> failure.got 2047 '1\n' 2048 2049 """ 2050 2051 runner = DebugRunner(optionflags=self._dt_optionflags, 2052 checker=self._dt_checker, verbose=False) 2053 runner.run(self._dt_test, out=nooutput) 2054 2055 def id(self): 2056 return self._dt_test.name 2057 2058 def __repr__(self): 2059 name = self._dt_test.name.split('.') 2060 return "%s (%s)" % (name[-1], '.'.join(name[:-1])) 2061 2062 __str__ = __repr__ 2063 2064 def shortDescription(self): 2065 return "Doctest: " + self._dt_test.name 2066 2067def nooutput(*args): 2068 pass 2069 2070def DocTestSuite(module=None, globs=None, extraglobs=None, 2071 optionflags=0, test_finder=None, 2072 setUp=lambda: None, tearDown=lambda: None, 2073 checker=None): 2074 """ 2075 Convert doctest tests for a mudule to a unittest test suite. 2076 2077 This converts each documentation string in a module that 2078 contains doctest tests to a unittest test case. If any of the 2079 tests in a doc string fail, then the test case fails. An exception 2080 is raised showing the name of the file containing the test and a 2081 (sometimes approximate) line number. 2082 2083 The `module` argument provides the module to be tested. The argument 2084 can be either a module or a module name. 2085 2086 If no argument is given, the calling module is used. 2087 """ 2088 2089 if test_finder is None: 2090 test_finder = DocTestFinder() 2091 2092 module = _normalize_module(module) 2093 tests = test_finder.find(module, globs=globs, extraglobs=extraglobs) 2094 if globs is None: 2095 globs = module.__dict__ 2096 if not tests: # [XX] why do we want to do this? 2097 raise ValueError(module, "has no tests") 2098 2099 tests.sort() 2100 suite = unittest.TestSuite() 2101 for test in tests: 2102 if len(test.examples) == 0: 2103 continue 2104 if not test.filename: 2105 filename = module.__file__ 2106 if filename.endswith(".pyc"): 2107 filename = filename[:-1] 2108 elif filename.endswith(".pyo"): 2109 filename = filename[:-1] 2110 test.filename = filename 2111 suite.addTest(DocTestCase(test, optionflags, setUp, tearDown, 2112 checker)) 2113 2114 return suite 2115 2116class DocFileCase(DocTestCase): 2117 2118 def id(self): 2119 return '_'.join(self._dt_test.name.split('.')) 2120 2121 def __repr__(self): 2122 return self._dt_test.filename 2123 __str__ = __repr__ 2124 2125 def format_failure(self, err): 2126 return ('Failed doctest test for %s\n File "%s", line 0\n\n%s' 2127 % (self._dt_test.name, self._dt_test.filename, err) 2128 ) 2129 2130def DocFileTest(path, package=None, globs=None, 2131 setUp=None, tearDown=None, 2132 optionflags=0): 2133 package = _normalize_module(package) 2134 name = path.split('/')[-1] 2135 dir = os.path.split(package.__file__)[0] 2136 path = os.path.join(dir, *(path.split('/'))) 2137 doc = open(path).read() 2138 2139 if globs is None: 2140 globs = {} 2141 2142 test = DocTest(doc, globs, name, path, 0) 2143 2144 return DocFileCase(test, optionflags, setUp, tearDown) 2145 2146def DocFileSuite(*paths, **kw): 2147 """Creates a suite of doctest files. 2148 2149 One or more text file paths are given as strings. These should 2150 use "/" characters to separate path segments. Paths are relative 2151 to the directory of the calling module, or relative to the package 2152 passed as a keyword argument. 2153 2154 A number of options may be provided as keyword arguments: 2155 2156 package 2157 The name of a Python package. Text-file paths will be 2158 interpreted relative to the directory containing this package. 2159 The package may be supplied as a package object or as a dotted 2160 package name. 2161 2162 setUp 2163 The name of a set-up function. This is called before running the 2164 tests in each file. 2165 2166 tearDown 2167 The name of a tear-down function. This is called after running the 2168 tests in each file. 2169 2170 globs 2171 A dictionary containing initial global variables for the tests. 2172 """ 2173 suite = unittest.TestSuite() 2174 2175 # We do this here so that _normalize_module is called at the right 2176 # level. If it were called in DocFileTest, then this function 2177 # would be the caller and we might guess the package incorrectly. 2178 kw['package'] = _normalize_module(kw.get('package')) 2179 2180 for path in paths: 2181 suite.addTest(DocFileTest(path, **kw)) 2182 2183 return suite 2184 2185###################################################################### 2186## 9. Debugging Support 2187###################################################################### 2188 2189def script_from_examples(s): 2190 r"""Extract script from text with examples. 2191 2192 Converts text with examples to a Python script. Example input is 2193 converted to regular code. Example output and all other words 2194 are converted to comments: 2195 2196 >>> text = ''' 2197 ... Here are examples of simple math. 2198 ... 2199 ... Python has super accurate integer addition 2200 ... 2201 ... >>> 2 + 2 2202 ... 5 2203 ... 2204 ... And very friendly error messages: 2205 ... 2206 ... >>> 1/0 2207 ... To Infinity 2208 ... And 2209 ... Beyond 2210 ... 2211 ... You can use logic if you want: 2212 ... 2213 ... >>> if 0: 2214 ... ... blah 2215 ... ... blah 2216 ... ... 2217 ... 2218 ... Ho hum 2219 ... ''' 2220 2221 >>> print script_from_examples(text) 2222 # Here are examples of simple math. 2223 # 2224 # Python has super accurate integer addition 2225 # 2226 2 + 2 2227 # Expected: 2228 # 5 2229 # 2230 # And very friendly error messages: 2231 # 2232 1/0 2233 # Expected: 2234 # To Infinity 2235 # And 2236 # Beyond 2237 # 2238 # You can use logic if you want: 2239 # 2240 if 0: 2241 blah 2242 blah 2243 <BLANKLINE> 2244 # 2245 # Ho hum 2246 """ 2247 2248 return Parser('<string>', s).get_program() 2249 2250def _want_comment(example): 2251 """ 2252 Return a comment containing the expected output for the given example. 2253 """ 2254 # Return the expected output, if any 2255 want = example.want 2256 if want: 2257 if want[-1] == '\n': 2258 want = want[:-1] 2259 want = "\n# ".join(want.split("\n")) 2260 want = "\n# Expected:\n# %s" % want 2261 return want 2262 2263def testsource(module, name): 2264 """Extract the test sources from a doctest docstring as a script. 2265 2266 Provide the module (or dotted name of the module) containing the 2267 test to be debugged and the name (within the module) of the object 2268 with the doc string with tests to be debugged. 2269 """ 2270 module = _normalize_module(module) 2271 tests = DocTestFinder().find(module) 2272 test = [t for t in tests if t.name == name] 2273 if not test: 2274 raise ValueError(name, "not found in tests") 2275 test = test[0] 2276 testsrc = script_from_examples(test.docstring) 2277 return testsrc 2278 2279def debug_src(src, pm=False, globs=None): 2280 """Debug a single doctest docstring, in argument `src`'""" 2281 testsrc = script_from_examples(src) 2282 debug_script(testsrc, pm, globs) 2283 2284def debug_script(src, pm=False, globs=None): 2285 "Debug a test script. `src` is the script, as a string." 2286 import pdb 2287 2288 srcfilename = tempfile.mktemp("doctestdebug.py") 2289 f = open(srcfilename, 'w') 2290 f.write(src) 2291 f.close() 2292 2293 if globs: 2294 globs = globs.copy() 2295 else: 2296 globs = {} 2297 2298 if pm: 2299 try: 2300 execfile(srcfilename, globs, globs) 2301 except: 2302 print sys.exc_info()[1] 2303 pdb.post_mortem(sys.exc_info()[2]) 2304 else: 2305 # Note that %r is vital here. '%s' instead can, e.g., cause 2306 # backslashes to get treated as metacharacters on Windows. 2307 pdb.run("execfile(%r)" % srcfilename, globs, globs) 2308 2309def debug(module, name, pm=False): 2310 """Debug a single doctest docstring. 2311 2312 Provide the module (or dotted name of the module) containing the 2313 test to be debugged and the name (within the module) of the object 2314 with the docstring with tests to be debugged. 2315 """ 2316 module = _normalize_module(module) 2317 testsrc = testsource(module, name) 2318 debug_script(testsrc, pm, module.__dict__) 2319 2320###################################################################### 2321## 10. Example Usage 2322###################################################################### 2323class _TestClass: 2324 """ 2325 A pointless class, for sanity-checking of docstring testing. 2326 2327 Methods: 2328 square() 2329 get() 2330 2331 >>> _TestClass(13).get() + _TestClass(-12).get() 2332 1 2333 >>> hex(_TestClass(13).square().get()) 2334 '0xa9' 2335 """ 2336 2337 def __init__(self, val): 2338 """val -> _TestClass object with associated value val. 2339 2340 >>> t = _TestClass(123) 2341 >>> print t.get() 2342 123 2343 """ 2344 2345 self.val = val 2346 2347 def square(self): 2348 """square() -> square TestClass's associated value 2349 2350 >>> _TestClass(13).square().get() 2351 169 2352 """ 2353 2354 self.val = self.val ** 2 2355 return self 2356 2357 def get(self): 2358 """get() -> return TestClass's associated value. 2359 2360 >>> x = _TestClass(-42) 2361 >>> print x.get() 2362 -42 2363 """ 2364 2365 return self.val 2366 2367__test__ = {"_TestClass": _TestClass, 2368 "string": r""" 2369 Example of a string object, searched as-is. 2370 >>> x = 1; y = 2 2371 >>> x + y, x * y 2372 (3, 2) 2373 """, 2374 "bool-int equivalence": r""" 2375 In 2.2, boolean expressions displayed 2376 0 or 1. By default, we still accept 2377 them. This can be disabled by passing 2378 DONT_ACCEPT_TRUE_FOR_1 to the new 2379 optionflags argument. 2380 >>> 4 == 4 2381 1 2382 >>> 4 == 4 2383 True 2384 >>> 4 > 4 2385 0 2386 >>> 4 > 4 2387 False 2388 """, 2389 "blank lines": r""" 2390 Blank lines can be marked with <BLANKLINE>: 2391 >>> print 'foo\n\nbar\n' 2392 foo 2393 <BLANKLINE> 2394 bar 2395 <BLANKLINE> 2396 """, 2397 } 2398# "ellipsis": r""" 2399# If the ellipsis flag is used, then '...' can be used to 2400# elide substrings in the desired output: 2401# >>> print range(1000) 2402# [0, 1, 2, ..., 999] 2403# """, 2404# "whitespace normalization": r""" 2405# If the whitespace normalization flag is used, then 2406# differences in whitespace are ignored. 2407# >>> print range(30) 2408# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 2409# 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 2410# 27, 28, 29] 2411# """, 2412# } 2413 2414def test1(): r""" 2415>>> warnings.filterwarnings("ignore", "class Tester", DeprecationWarning, 2416... "doctest", 0) 2417>>> from doctest import Tester 2418>>> t = Tester(globs={'x': 42}, verbose=0) 2419>>> t.runstring(r''' 2420... >>> x = x * 2 2421... >>> print x 2422... 42 2423... ''', 'XYZ') 2424********************************************************************** 2425Failure in example: print x 2426from line #2 of XYZ 2427Expected: 42 2428Got: 84 2429(1, 2) 2430>>> t.runstring(">>> x = x * 2\n>>> print x\n84\n", 'example2') 2431(0, 2) 2432>>> t.summarize() 2433********************************************************************** 24341 items had failures: 2435 1 of 2 in XYZ 2436***Test Failed*** 1 failures. 2437(1, 4) 2438>>> t.summarize(verbose=1) 24391 items passed all tests: 2440 2 tests in example2 2441********************************************************************** 24421 items had failures: 2443 1 of 2 in XYZ 24444 tests in 2 items. 24453 passed and 1 failed. 2446***Test Failed*** 1 failures. 2447(1, 4) 2448""" 2449 2450def test2(): r""" 2451 >>> warnings.filterwarnings("ignore", "class Tester", 2452 ... DeprecationWarning, "doctest", 0) 2453 >>> t = Tester(globs={}, verbose=1) 2454 >>> test = r''' 2455 ... # just an example 2456 ... >>> x = 1 + 2 2457 ... >>> x 2458 ... 3 2459 ... ''' 2460 >>> t.runstring(test, "Example") 2461 Running string Example 2462 Trying: x = 1 + 2 2463 Expecting: nothing 2464 ok 2465 Trying: x 2466 Expecting: 3 2467 ok 2468 0 of 2 examples failed in string Example 2469 (0, 2) 2470""" 2471def test3(): r""" 2472 >>> warnings.filterwarnings("ignore", "class Tester", 2473 ... DeprecationWarning, "doctest", 0) 2474 >>> t = Tester(globs={}, verbose=0) 2475 >>> def _f(): 2476 ... '''Trivial docstring example. 2477 ... >>> assert 2 == 2 2478 ... ''' 2479 ... return 32 2480 ... 2481 >>> t.rundoc(_f) # expect 0 failures in 1 example 2482 (0, 1) 2483""" 2484def test4(): """ 2485 >>> import new 2486 >>> m1 = new.module('_m1') 2487 >>> m2 = new.module('_m2') 2488 >>> test_data = \""" 2489 ... def _f(): 2490 ... '''>>> assert 1 == 1 2491 ... ''' 2492 ... def g(): 2493 ... '''>>> assert 2 != 1 2494 ... ''' 2495 ... class H: 2496 ... '''>>> assert 2 > 1 2497 ... ''' 2498 ... def bar(self): 2499 ... '''>>> assert 1 < 2 2500 ... ''' 2501 ... \""" 2502 >>> exec test_data in m1.__dict__ 2503 >>> exec test_data in m2.__dict__ 2504 >>> m1.__dict__.update({"f2": m2._f, "g2": m2.g, "h2": m2.H}) 2505 2506 Tests that objects outside m1 are excluded: 2507 2508 >>> warnings.filterwarnings("ignore", "class Tester", 2509 ... DeprecationWarning, "doctest", 0) 2510 >>> t = Tester(globs={}, verbose=0) 2511 >>> t.rundict(m1.__dict__, "rundict_test", m1) # f2 and g2 and h2 skipped 2512 (0, 4) 2513 2514 Once more, not excluding stuff outside m1: 2515 2516 >>> t = Tester(globs={}, verbose=0) 2517 >>> t.rundict(m1.__dict__, "rundict_test_pvt") # None are skipped. 2518 (0, 8) 2519 2520 The exclusion of objects from outside the designated module is 2521 meant to be invoked automagically by testmod. 2522 2523 >>> testmod(m1, verbose=False) 2524 (0, 4) 2525""" 2526 2527def _test(): 2528 #import doctest 2529 #doctest.testmod(doctest, verbose=False, 2530 # optionflags=ELLIPSIS | NORMALIZE_WHITESPACE | 2531 # UNIFIED_DIFF) 2532 #print '~'*70 2533 r = unittest.TextTestRunner() 2534 r.run(DocTestSuite()) 2535 2536if __name__ == "__main__": 2537 _test() 2538