doctest.py revision f3f57473ed97f7d0c4862639bbf29baef77de3a4
1# Module doctest. 2# Released to the public domain 16-Jan-2001, by Tim Peters (tim@python.org). 3# Major enhancements and refactoring by: 4# Jim Fulton 5# Edward Loper 6 7# Provided as-is; use at your own risk; no warranty; no promises; enjoy! 8 9# [XX] This docstring is out-of-date: 10r"""Module doctest -- a framework for running examples in docstrings. 11 12NORMAL USAGE 13 14In normal use, end each module M with: 15 16def _test(): 17 import doctest, M # replace M with your module's name 18 return doctest.testmod(M) # ditto 19 20if __name__ == "__main__": 21 _test() 22 23Then running the module as a script will cause the examples in the 24docstrings to get executed and verified: 25 26python M.py 27 28This won't display anything unless an example fails, in which case the 29failing example(s) and the cause(s) of the failure(s) are printed to stdout 30(why not stderr? because stderr is a lame hack <0.2 wink>), and the final 31line of output is "Test failed.". 32 33Run it with the -v switch instead: 34 35python M.py -v 36 37and a detailed report of all examples tried is printed to stdout, along 38with assorted summaries at the end. 39 40You can force verbose mode by passing "verbose=1" to testmod, or prohibit 41it by passing "verbose=0". In either of those cases, sys.argv is not 42examined by testmod. 43 44In any case, testmod returns a 2-tuple of ints (f, t), where f is the 45number of docstring examples that failed and t is the total number of 46docstring examples attempted. 47 48 49WHICH DOCSTRINGS ARE EXAMINED? 50 51+ M.__doc__. 52 53+ f.__doc__ for all functions f in M.__dict__.values(), except those 54 defined in other modules. 55 56+ C.__doc__ for all classes C in M.__dict__.values(), except those 57 defined in other modules. 58 59+ If M.__test__ exists and "is true", it must be a dict, and 60 each entry maps a (string) name to a function object, class object, or 61 string. Function and class object docstrings found from M.__test__ 62 are searched even if the name is private, and strings are searched 63 directly as if they were docstrings. In output, a key K in M.__test__ 64 appears with name 65 <name of M>.__test__.K 66 67Any classes found are recursively searched similarly, to test docstrings in 68their contained methods and nested classes. All names reached from 69M.__test__ are searched. 70 71Optionally, functions with private names can be skipped (unless listed in 72M.__test__) by supplying a function to the "isprivate" argument that will 73identify private functions. For convenience, one such function is 74supplied. docttest.is_private considers a name to be private if it begins 75with an underscore (like "_my_func") but doesn't both begin and end with 76(at least) two underscores (like "__init__"). By supplying this function 77or your own "isprivate" function to testmod, the behavior can be customized. 78 79If you want to test docstrings in objects with private names too, stuff 80them into an M.__test__ dict, or see ADVANCED USAGE below (e.g., pass your 81own isprivate function to Tester's constructor, or call the rundoc method 82of a Tester instance). 83 84WHAT'S THE EXECUTION CONTEXT? 85 86By default, each time testmod finds a docstring to test, it uses a *copy* 87of M's globals (so that running tests on a module doesn't change the 88module's real globals, and so that one test in M can't leave behind crumbs 89that accidentally allow another test to work). This means examples can 90freely use any names defined at top-level in M. It also means that sloppy 91imports (see above) can cause examples in external docstrings to use 92globals inappropriate for them. 93 94You can force use of your own dict as the execution context by passing 95"globs=your_dict" to testmod instead. Presumably this would be a copy of 96M.__dict__ merged with the globals from other imported modules. 97 98 99WHAT IF I WANT TO TEST A WHOLE PACKAGE? 100 101Piece o' cake, provided the modules do their testing from docstrings. 102Here's the test.py I use for the world's most elaborate Rational/ 103floating-base-conversion pkg (which I'll distribute some day): 104 105from Rational import Cvt 106from Rational import Format 107from Rational import machprec 108from Rational import Rat 109from Rational import Round 110from Rational import utils 111 112modules = (Cvt, 113 Format, 114 machprec, 115 Rat, 116 Round, 117 utils) 118 119def _test(): 120 import doctest 121 import sys 122 verbose = "-v" in sys.argv 123 for mod in modules: 124 doctest.testmod(mod, verbose=verbose, report=0) 125 doctest.master.summarize() 126 127if __name__ == "__main__": 128 _test() 129 130IOW, it just runs testmod on all the pkg modules. testmod remembers the 131names and outcomes (# of failures, # of tries) for each item it's seen, and 132passing "report=0" prevents it from printing a summary in verbose mode. 133Instead, the summary is delayed until all modules have been tested, and 134then "doctest.master.summarize()" forces the summary at the end. 135 136So this is very nice in practice: each module can be tested individually 137with almost no work beyond writing up docstring examples, and collections 138of modules can be tested too as a unit with no more work than the above. 139 140 141WHAT ABOUT EXCEPTIONS? 142 143No problem, as long as the only output generated by the example is the 144traceback itself. For example: 145 146 >>> [1, 2, 3].remove(42) 147 Traceback (most recent call last): 148 File "<stdin>", line 1, in ? 149 ValueError: list.remove(x): x not in list 150 >>> 151 152Note that only the exception type and value are compared (specifically, 153only the last line in the traceback). 154 155 156ADVANCED USAGE 157 158doctest.testmod() captures the testing policy I find most useful most 159often. You may want other policies. 160 161testmod() actually creates a local instance of class doctest.Tester, runs 162appropriate methods of that class, and merges the results into global 163Tester instance doctest.master. 164 165You can create your own instances of doctest.Tester, and so build your own 166policies, or even run methods of doctest.master directly. See 167doctest.Tester.__doc__ for details. 168 169 170SO WHAT DOES A DOCSTRING EXAMPLE LOOK LIKE ALREADY!? 171 172Oh ya. It's easy! In most cases a copy-and-paste of an interactive 173console session works fine -- just make sure the leading whitespace is 174rigidly consistent (you can mix tabs and spaces if you're too lazy to do it 175right, but doctest is not in the business of guessing what you think a tab 176means). 177 178 >>> # comments are ignored 179 >>> x = 12 180 >>> x 181 12 182 >>> if x == 13: 183 ... print "yes" 184 ... else: 185 ... print "no" 186 ... print "NO" 187 ... print "NO!!!" 188 ... 189 no 190 NO 191 NO!!! 192 >>> 193 194Any expected output must immediately follow the final ">>>" or "..." line 195containing the code, and the expected output (if any) extends to the next 196">>>" or all-whitespace line. That's it. 197 198Bummers: 199 200+ Expected output cannot contain an all-whitespace line, since such a line 201 is taken to signal the end of expected output. 202 203+ Output to stdout is captured, but not output to stderr (exception 204 tracebacks are captured via a different means). 205 206+ If you continue a line via backslashing in an interactive session, 207 or for any other reason use a backslash, you should use a raw 208 docstring, which will preserve your backslahses exactly as you type 209 them: 210 211 >>> def f(x): 212 ... r'''Backslashes in a raw docstring: m\n''' 213 >>> print f.__doc__ 214 Backslashes in a raw docstring: m\n 215 216 Otherwise, the backslash will be interpreted as part of the string. 217 E.g., the "\n" above would be interpreted as a newline character. 218 Alternatively, you can double each backslash in the doctest version 219 (and not use a raw string): 220 221 >>> def f(x): 222 ... '''Backslashes in a raw docstring: m\\n''' 223 >>> print f.__doc__ 224 Backslashes in a raw docstring: m\n 225 226The starting column doesn't matter: 227 228>>> assert "Easy!" 229 >>> import math 230 >>> math.floor(1.9) 231 1.0 232 233and as many leading whitespace characters are stripped from the expected 234output as appeared in the initial ">>>" line that triggered it. 235 236If you execute this very file, the examples above will be found and 237executed, leading to this output in verbose mode: 238 239Running doctest.__doc__ 240Trying: [1, 2, 3].remove(42) 241Expecting: 242Traceback (most recent call last): 243 File "<stdin>", line 1, in ? 244ValueError: list.remove(x): x not in list 245ok 246Trying: x = 12 247Expecting: nothing 248ok 249Trying: x 250Expecting: 12 251ok 252Trying: 253if x == 13: 254 print "yes" 255else: 256 print "no" 257 print "NO" 258 print "NO!!!" 259Expecting: 260no 261NO 262NO!!! 263ok 264... and a bunch more like that, with this summary at the end: 265 2665 items had no tests: 267 doctest.Tester.__init__ 268 doctest.Tester.run__test__ 269 doctest.Tester.summarize 270 doctest.run_docstring_examples 271 doctest.testmod 27212 items passed all tests: 273 8 tests in doctest 274 6 tests in doctest.Tester 275 10 tests in doctest.Tester.merge 276 14 tests in doctest.Tester.rundict 277 3 tests in doctest.Tester.rundoc 278 3 tests in doctest.Tester.runstring 279 2 tests in doctest.__test__._TestClass 280 2 tests in doctest.__test__._TestClass.__init__ 281 2 tests in doctest.__test__._TestClass.get 282 1 tests in doctest.__test__._TestClass.square 283 2 tests in doctest.__test__.string 284 7 tests in doctest.is_private 28560 tests in 17 items. 28660 passed and 0 failed. 287Test passed. 288""" 289 290__all__ = [ 291 'is_private', 292 'Example', 293 'DocTest', 294 'DocTestFinder', 295 'DocTestRunner', 296 'testmod', 297 'run_docstring_examples', 298 'Tester', 299 'DocTestCase', 300 'DocTestSuite', 301 'testsource', 302 'debug', 303# 'master', 304] 305 306import __future__ 307 308import sys, traceback, inspect, linecache, os, re, types 309import unittest, difflib, tempfile 310import warnings 311from StringIO import StringIO 312 313# Option constants. 314DONT_ACCEPT_TRUE_FOR_1 = 1 << 0 315DONT_ACCEPT_BLANKLINE = 1 << 1 316NORMALIZE_WHITESPACE = 1 << 2 317ELLIPSIS = 1 << 3 318UNIFIED_DIFF = 1 << 4 319CONTEXT_DIFF = 1 << 5 320 321OPTIONFLAGS_BY_NAME = { 322 'DONT_ACCEPT_TRUE_FOR_1': DONT_ACCEPT_TRUE_FOR_1, 323 'DONT_ACCEPT_BLANKLINE': DONT_ACCEPT_BLANKLINE, 324 'NORMALIZE_WHITESPACE': NORMALIZE_WHITESPACE, 325 'ELLIPSIS': ELLIPSIS, 326 'UNIFIED_DIFF': UNIFIED_DIFF, 327 'CONTEXT_DIFF': CONTEXT_DIFF, 328 } 329 330# Special string markers for use in `want` strings: 331BLANKLINE_MARKER = '<BLANKLINE>' 332ELLIPSIS_MARKER = '...' 333 334 335# There are 4 basic classes: 336# - Example: a <source, want> pair, plus an intra-docstring line number. 337# - DocTest: a collection of examples, parsed from a docstring, plus 338# info about where the docstring came from (name, filename, lineno). 339# - DocTestFinder: extracts DocTests from a given object's docstring and 340# its contained objects' docstrings. 341# - DocTestRunner: runs DocTest cases, and accumulates statistics. 342# 343# So the basic picture is: 344# 345# list of: 346# +------+ +---------+ +-------+ 347# |object| --DocTestFinder-> | DocTest | --DocTestRunner-> |results| 348# +------+ +---------+ +-------+ 349# | Example | 350# | ... | 351# | Example | 352# +---------+ 353 354###################################################################### 355## Table of Contents 356###################################################################### 357# 1. Utility Functions 358# 2. Example & DocTest -- store test cases 359# 3. DocTest Finder -- extracts test cases from objects 360# 4. DocTest Runner -- runs test cases 361# 5. Test Functions -- convenient wrappers for testing 362# 6. Tester Class -- for backwards compatibility 363# 7. Unittest Support 364# 8. Debugging Support 365# 9. Example Usage 366 367###################################################################### 368## 1. Utility Functions 369###################################################################### 370 371def is_private(prefix, base): 372 """prefix, base -> true iff name prefix + "." + base is "private". 373 374 Prefix may be an empty string, and base does not contain a period. 375 Prefix is ignored (although functions you write conforming to this 376 protocol may make use of it). 377 Return true iff base begins with an (at least one) underscore, but 378 does not both begin and end with (at least) two underscores. 379 380 >>> warnings.filterwarnings("ignore", "is_private", DeprecationWarning, 381 ... "doctest", 0) 382 >>> is_private("a.b", "my_func") 383 False 384 >>> is_private("____", "_my_func") 385 True 386 >>> is_private("someclass", "__init__") 387 False 388 >>> is_private("sometypo", "__init_") 389 True 390 >>> is_private("x.y.z", "_") 391 True 392 >>> is_private("_x.y.z", "__") 393 False 394 >>> is_private("", "") # senseless but consistent 395 False 396 """ 397 warnings.warn("is_private is deprecated; it wasn't useful; " 398 "examine DocTestFinder.find() lists instead", 399 DeprecationWarning, stacklevel=2) 400 return base[:1] == "_" and not base[:2] == "__" == base[-2:] 401 402def _extract_future_flags(globs): 403 """ 404 Return the compiler-flags associated with the future features that 405 have been imported into the given namespace (globs). 406 """ 407 flags = 0 408 for fname in __future__.all_feature_names: 409 feature = globs.get(fname, None) 410 if feature is getattr(__future__, fname): 411 flags |= feature.compiler_flag 412 return flags 413 414def _normalize_module(module, depth=2): 415 """ 416 Return the module specified by `module`. In particular: 417 - If `module` is a module, then return module. 418 - If `module` is a string, then import and return the 419 module with that name. 420 - If `module` is None, then return the calling module. 421 The calling module is assumed to be the module of 422 the stack frame at the given depth in the call stack. 423 """ 424 if inspect.ismodule(module): 425 return module 426 elif isinstance(module, (str, unicode)): 427 return __import__(module, globals(), locals(), ["*"]) 428 elif module is None: 429 return sys.modules[sys._getframe(depth).f_globals['__name__']] 430 else: 431 raise TypeError("Expected a module, string, or None") 432 433def _tag_msg(tag, msg, indent_msg=True): 434 """ 435 Return a string that displays a tag-and-message pair nicely, 436 keeping the tag and its message on the same line when that 437 makes sense. If `indent_msg` is true, then messages that are 438 put on separate lines will be indented. 439 """ 440 # What string should we use to indent contents? 441 INDENT = ' ' 442 443 # If the message doesn't end in a newline, then add one. 444 if msg[-1:] != '\n': 445 msg += '\n' 446 # If the message is short enough, and contains no internal 447 # newlines, then display it on the same line as the tag. 448 # Otherwise, display the tag on its own line. 449 if (len(tag) + len(msg) < 75 and 450 msg.find('\n', 0, len(msg)-1) == -1): 451 return '%s: %s' % (tag, msg) 452 else: 453 if indent_msg: 454 msg = '\n'.join([INDENT+l for l in msg.split('\n')]) 455 msg = msg[:-len(INDENT)] 456 return '%s:\n%s' % (tag, msg) 457 458# Override some StringIO methods. 459class _SpoofOut(StringIO): 460 def getvalue(self): 461 result = StringIO.getvalue(self) 462 # If anything at all was written, make sure there's a trailing 463 # newline. There's no way for the expected output to indicate 464 # that a trailing newline is missing. 465 if result and not result.endswith("\n"): 466 result += "\n" 467 # Prevent softspace from screwing up the next test case, in 468 # case they used print with a trailing comma in an example. 469 if hasattr(self, "softspace"): 470 del self.softspace 471 return result 472 473 def truncate(self, size=None): 474 StringIO.truncate(self, size) 475 if hasattr(self, "softspace"): 476 del self.softspace 477 478class Parser: 479 """ 480 Extract doctests from a string. 481 """ 482 483 _PS1 = ">>>" 484 _PS2 = "..." 485 _isPS1 = re.compile(r"(\s*)" + re.escape(_PS1)).match 486 _isPS2 = re.compile(r"(\s*)" + re.escape(_PS2)).match 487 _isEmpty = re.compile(r"\s*$").match 488 _isComment = re.compile(r"\s*#").match 489 490 def __init__(self, name, string): 491 """ 492 Prepare to extract doctests from string `string`. 493 494 `name` is an arbitrary (string) name associated with the string, 495 and is used only in error messages. 496 """ 497 self.name = name 498 self.source = string 499 500 def get_examples(self): 501 """ 502 Return the doctest examples from the string. 503 504 This is a list of (source, want, lineno) triples, one per example 505 in the string. "source" is a single Python statement; it ends 506 with a newline iff the statement contains more than one 507 physical line. "want" is the expected output from running the 508 example (either from stdout, or a traceback in case of exception). 509 "want" always ends with a newline, unless no output is expected, 510 in which case "want" is an empty string. "lineno" is the 0-based 511 line number of the first line of "source" within the string. It's 512 0-based because it's most common in doctests that nothing 513 interesting appears on the same line as opening triple-quote, 514 and so the first interesting line is called "line 1" then. 515 516 >>> text = ''' 517 ... >>> x, y = 2, 3 # no output expected 518 ... >>> if 1: 519 ... ... print x 520 ... ... print y 521 ... 2 522 ... 3 523 ... 524 ... Some text. 525 ... >>> x+y 526 ... 5 527 ... ''' 528 >>> for x in Parser('<string>', text).get_examples(): 529 ... print x 530 ('x, y = 2, 3 # no output expected', '', 1) 531 ('if 1:\\n print x\\n print y\\n', '2\\n3\\n', 2) 532 ('x+y', '5\\n', 9) 533 """ 534 return self._parse(kind='examples') 535 536 def get_program(self): 537 """ 538 Return an executable program from the string, as a string. 539 540 The format of this isn't rigidly defined. In general, doctest 541 examples become the executable statements in the result, and 542 their expected outputs become comments, preceded by an "#Expected:" 543 comment. Everything else (text, comments, everything not part of 544 a doctest test) is also placed in comments. 545 546 >>> text = ''' 547 ... >>> x, y = 2, 3 # no output expected 548 ... >>> if 1: 549 ... ... print x 550 ... ... print y 551 ... 2 552 ... 3 553 ... 554 ... Some text. 555 ... >>> x+y 556 ... 5 557 ... ''' 558 >>> print Parser('<string>', text).get_program() 559 x, y = 2, 3 # no output expected 560 if 1: 561 print x 562 print y 563 # Expected: 564 # 2 565 # 3 566 # 567 # Some text. 568 x+y 569 # Expected: 570 # 5 571 """ 572 return self._parse(kind='program') 573 574 def _parse(self, kind): 575 assert kind in ('examples', 'program') 576 do_program = kind == 'program' 577 output = [] 578 push = output.append 579 580 string = self.source 581 if not string.endswith('\n'): 582 string += '\n' 583 584 isPS1, isPS2 = self._isPS1, self._isPS2 585 isEmpty, isComment = self._isEmpty, self._isComment 586 lines = string.split("\n") 587 i, n = 0, len(lines) 588 while i < n: 589 # Search for an example (a PS1 line). 590 line = lines[i] 591 i += 1 592 m = isPS1(line) 593 if m is None: 594 if do_program: 595 line = line.rstrip() 596 if line: 597 line = ' ' + line 598 push('#' + line) 599 continue 600 # line is a PS1 line. 601 j = m.end(0) # beyond the prompt 602 if isEmpty(line, j) or isComment(line, j): 603 # a bare prompt or comment -- not interesting 604 if do_program: 605 push("# " + line[j:]) 606 continue 607 # line is a non-trivial PS1 line. 608 lineno = i - 1 609 if line[j] != " ": 610 raise ValueError('line %r of the docstring for %s lacks ' 611 'blank after %s: %r' % 612 (lineno, self.name, self._PS1, line)) 613 614 j += 1 615 blanks = m.group(1) 616 nblanks = len(blanks) 617 # suck up this and following PS2 lines 618 source = [] 619 while 1: 620 source.append(line[j:]) 621 line = lines[i] 622 m = isPS2(line) 623 if m: 624 if m.group(1) != blanks: 625 raise ValueError('line %r of the docstring for %s ' 626 'has inconsistent leading whitespace: %r' % 627 (i, self.name, line)) 628 i += 1 629 else: 630 break 631 632 if do_program: 633 output.extend(source) 634 else: 635 # get rid of useless null line from trailing empty "..." 636 if source[-1] == "": 637 assert len(source) > 1 638 del source[-1] 639 if len(source) == 1: 640 source = source[0] 641 else: 642 source = "\n".join(source) + "\n" 643 644 # suck up response 645 if isPS1(line) or isEmpty(line): 646 if not do_program: 647 push((source, "", lineno)) 648 continue 649 650 # There is a response. 651 want = [] 652 if do_program: 653 push("# Expected:") 654 while 1: 655 if line[:nblanks] != blanks: 656 raise ValueError('line %r of the docstring for %s ' 657 'has inconsistent leading whitespace: %r' % 658 (i, self.name, line)) 659 want.append(line[nblanks:]) 660 i += 1 661 line = lines[i] 662 if isPS1(line) or isEmpty(line): 663 break 664 665 if do_program: 666 output.extend(['# ' + x for x in want]) 667 else: 668 want = "\n".join(want) + "\n" 669 push((source, want, lineno)) 670 671 if do_program: 672 # Trim junk on both ends. 673 while output and output[-1] == '#': 674 output.pop() 675 while output and output[0] == '#': 676 output.pop(0) 677 output = '\n'.join(output) 678 679 return output 680 681###################################################################### 682## 2. Example & DocTest 683###################################################################### 684## - An "example" is a <source, want> pair, where "source" is a 685## fragment of source code, and "want" is the expected output for 686## "source." The Example class also includes information about 687## where the example was extracted from. 688## 689## - A "doctest" is a collection of examples extracted from a string 690## (such as an object's docstring). The DocTest class also includes 691## information about where the string was extracted from. 692 693class Example: 694 """ 695 A single doctest example, consisting of source code and expected 696 output. Example defines the following attributes: 697 698 - source: The source code that should be run. It ends with a 699 newline iff the source spans more than one line. 700 701 - want: The expected output from running the source code. If 702 not empty, then this string ends with a newline. 703 704 - lineno: The line number within the DocTest string containing 705 this Example where the Example begins. This line number is 706 zero-based, with respect to the beginning of the DocTest. 707 """ 708 def __init__(self, source, want, lineno): 709 # Check invariants. 710 if (source[-1:] == '\n') != ('\n' in source[:-1]): 711 raise AssertionError("source must end with newline iff " 712 "source contains more than one line") 713 if want and want[-1] != '\n': 714 raise AssertionError("non-empty want must end with newline") 715 # Store properties. 716 self.source = source 717 self.want = want 718 self.lineno = lineno 719 720class DocTest: 721 """ 722 A collection of doctest examples that should be run in a single 723 namespace. Each DocTest defines the following attributes: 724 725 - examples: the list of examples. 726 727 - globs: The namespace (aka globals) that the examples should 728 be run in. 729 730 - name: A name identifying the DocTest (typically, the name of 731 the object whose docstring this DocTest was extracted from). 732 733 - docstring: The docstring being tested 734 735 - filename: The name of the file that this DocTest was extracted 736 from. 737 738 - lineno: The line number within filename where this DocTest 739 begins. This line number is zero-based, with respect to the 740 beginning of the file. 741 """ 742 def __init__(self, docstring, globs, name, filename, lineno): 743 """ 744 Create a new DocTest, by extracting examples from `docstring`. 745 The DocTest's globals are initialized with a copy of `globs`. 746 """ 747 # Store a copy of the globals 748 self.globs = globs.copy() 749 # Store identifying information 750 self.name = name 751 self.filename = filename 752 self.lineno = lineno 753 # Parse the docstring. 754 self.docstring = docstring 755 examples = Parser(name, docstring).get_examples() 756 self.examples = [Example(*example) for example in examples] 757 758 def __repr__(self): 759 if len(self.examples) == 0: 760 examples = 'no examples' 761 elif len(self.examples) == 1: 762 examples = '1 example' 763 else: 764 examples = '%d examples' % len(self.examples) 765 return ('<DocTest %s from %s:%s (%s)>' % 766 (self.name, self.filename, self.lineno, examples)) 767 768 769 # This lets us sort tests by name: 770 def __cmp__(self, other): 771 if not isinstance(other, DocTest): 772 return -1 773 return cmp((self.name, self.filename, self.lineno, id(self)), 774 (other.name, other.filename, other.lineno, id(other))) 775 776###################################################################### 777## 3. DocTest Finder 778###################################################################### 779 780class DocTestFinder: 781 """ 782 A class used to extract the DocTests that are relevant to a given 783 object, from its docstring and the docstrings of its contained 784 objects. Doctests can currently be extracted from the following 785 object types: modules, functions, classes, methods, staticmethods, 786 classmethods, and properties. 787 """ 788 789 def __init__(self, verbose=False, doctest_factory=DocTest, 790 recurse=True, _namefilter=None): 791 """ 792 Create a new doctest finder. 793 794 The optional argument `doctest_factory` specifies a class or 795 function that should be used to create new DocTest objects (or 796 objects that implement the same interface as DocTest). The 797 signature for this factory function should match the signature 798 of the DocTest constructor. 799 800 If the optional argument `recurse` is false, then `find` will 801 only examine the given object, and not any contained objects. 802 """ 803 self._doctest_factory = doctest_factory 804 self._verbose = verbose 805 self._recurse = recurse 806 # _namefilter is undocumented, and exists only for temporary backward- 807 # compatibility support of testmod's deprecated isprivate mess. 808 self._namefilter = _namefilter 809 810 def find(self, obj, name=None, module=None, globs=None, 811 extraglobs=None): 812 """ 813 Return a list of the DocTests that are defined by the given 814 object's docstring, or by any of its contained objects' 815 docstrings. 816 817 The optional parameter `module` is the module that contains 818 the given object. If the module is not specified or is None, then 819 the test finder will attempt to automatically determine the 820 correct module. The object's module is used: 821 822 - As a default namespace, if `globs` is not specified. 823 - To prevent the DocTestFinder from extracting DocTests 824 from objects that are imported from other modules. 825 - To find the name of the file containing the object. 826 - To help find the line number of the object within its 827 file. 828 829 Contained objects whose module does not match `module` are ignored. 830 831 If `module` is False, no attempt to find the module will be made. 832 This is obscure, of use mostly in tests: if `module` is False, or 833 is None but cannot be found automatically, then all objects are 834 considered to belong to the (non-existent) module, so all contained 835 objects will (recursively) be searched for doctests. 836 837 The globals for each DocTest is formed by combining `globs` 838 and `extraglobs` (bindings in `extraglobs` override bindings 839 in `globs`). A new copy of the globals dictionary is created 840 for each DocTest. If `globs` is not specified, then it 841 defaults to the module's `__dict__`, if specified, or {} 842 otherwise. If `extraglobs` is not specified, then it defaults 843 to {}. 844 845 """ 846 # If name was not specified, then extract it from the object. 847 if name is None: 848 name = getattr(obj, '__name__', None) 849 if name is None: 850 raise ValueError("DocTestFinder.find: name must be given " 851 "when obj.__name__ doesn't exist: %r" % 852 (type(obj),)) 853 854 # Find the module that contains the given object (if obj is 855 # a module, then module=obj.). Note: this may fail, in which 856 # case module will be None. 857 if module is False: 858 module = None 859 elif module is None: 860 module = inspect.getmodule(obj) 861 862 # Read the module's source code. This is used by 863 # DocTestFinder._find_lineno to find the line number for a 864 # given object's docstring. 865 try: 866 file = inspect.getsourcefile(obj) or inspect.getfile(obj) 867 source_lines = linecache.getlines(file) 868 if not source_lines: 869 source_lines = None 870 except TypeError: 871 source_lines = None 872 873 # Initialize globals, and merge in extraglobs. 874 if globs is None: 875 if module is None: 876 globs = {} 877 else: 878 globs = module.__dict__.copy() 879 else: 880 globs = globs.copy() 881 if extraglobs is not None: 882 globs.update(extraglobs) 883 884 # Recursively expore `obj`, extracting DocTests. 885 tests = [] 886 self._find(tests, obj, name, module, source_lines, globs, {}) 887 return tests 888 889 def _filter(self, obj, prefix, base): 890 """ 891 Return true if the given object should not be examined. 892 """ 893 return (self._namefilter is not None and 894 self._namefilter(prefix, base)) 895 896 def _from_module(self, module, object): 897 """ 898 Return true if the given object is defined in the given 899 module. 900 """ 901 if module is None: 902 return True 903 elif inspect.isfunction(object): 904 return module.__dict__ is object.func_globals 905 elif inspect.isclass(object): 906 return module.__name__ == object.__module__ 907 elif inspect.getmodule(object) is not None: 908 return module is inspect.getmodule(object) 909 elif hasattr(object, '__module__'): 910 return module.__name__ == object.__module__ 911 elif isinstance(object, property): 912 return True # [XX] no way not be sure. 913 else: 914 raise ValueError("object must be a class or function") 915 916 def _find(self, tests, obj, name, module, source_lines, globs, seen): 917 """ 918 Find tests for the given object and any contained objects, and 919 add them to `tests`. 920 """ 921 if self._verbose: 922 print 'Finding tests in %s' % name 923 924 # If we've already processed this object, then ignore it. 925 if id(obj) in seen: 926 return 927 seen[id(obj)] = 1 928 929 # Find a test for this object, and add it to the list of tests. 930 test = self._get_test(obj, name, module, globs, source_lines) 931 if test is not None: 932 tests.append(test) 933 934 # Look for tests in a module's contained objects. 935 if inspect.ismodule(obj) and self._recurse: 936 for valname, val in obj.__dict__.items(): 937 # Check if this contained object should be ignored. 938 if self._filter(val, name, valname): 939 continue 940 valname = '%s.%s' % (name, valname) 941 # Recurse to functions & classes. 942 if ((inspect.isfunction(val) or inspect.isclass(val)) and 943 self._from_module(module, val)): 944 self._find(tests, val, valname, module, source_lines, 945 globs, seen) 946 947 # Look for tests in a module's __test__ dictionary. 948 if inspect.ismodule(obj) and self._recurse: 949 for valname, val in getattr(obj, '__test__', {}).items(): 950 if not isinstance(valname, basestring): 951 raise ValueError("DocTestFinder.find: __test__ keys " 952 "must be strings: %r" % 953 (type(valname),)) 954 if not (inspect.isfunction(val) or inspect.isclass(val) or 955 inspect.ismethod(val) or inspect.ismodule(val) or 956 isinstance(val, basestring)): 957 raise ValueError("DocTestFinder.find: __test__ values " 958 "must be strings, functions, methods, " 959 "classes, or modules: %r" % 960 (type(val),)) 961 valname = '%s.%s' % (name, valname) 962 self._find(tests, val, valname, module, source_lines, 963 globs, seen) 964 965 # Look for tests in a class's contained objects. 966 if inspect.isclass(obj) and self._recurse: 967 for valname, val in obj.__dict__.items(): 968 # Check if this contained object should be ignored. 969 if self._filter(val, name, valname): 970 continue 971 # Special handling for staticmethod/classmethod. 972 if isinstance(val, staticmethod): 973 val = getattr(obj, valname) 974 if isinstance(val, classmethod): 975 val = getattr(obj, valname).im_func 976 977 # Recurse to methods, properties, and nested classes. 978 if ((inspect.isfunction(val) or inspect.isclass(val) or 979 isinstance(val, property)) and 980 self._from_module(module, val)): 981 valname = '%s.%s' % (name, valname) 982 self._find(tests, val, valname, module, source_lines, 983 globs, seen) 984 985 def _get_test(self, obj, name, module, globs, source_lines): 986 """ 987 Return a DocTest for the given object, if it defines a docstring; 988 otherwise, return None. 989 """ 990 # Extract the object's docstring. If it doesn't have one, 991 # then return None (no test for this object). 992 if isinstance(obj, basestring): 993 docstring = obj 994 else: 995 try: 996 if obj.__doc__ is None: 997 return None 998 docstring = str(obj.__doc__) 999 except (TypeError, AttributeError): 1000 return None 1001 1002 # Don't bother if the docstring is empty. 1003 if not docstring: 1004 return None 1005 1006 # Find the docstring's location in the file. 1007 lineno = self._find_lineno(obj, source_lines) 1008 1009 # Return a DocTest for this object. 1010 if module is None: 1011 filename = None 1012 else: 1013 filename = getattr(module, '__file__', module.__name__) 1014 return self._doctest_factory(docstring, globs, name, filename, lineno) 1015 1016 def _find_lineno(self, obj, source_lines): 1017 """ 1018 Return a line number of the given object's docstring. Note: 1019 this method assumes that the object has a docstring. 1020 """ 1021 lineno = None 1022 1023 # Find the line number for modules. 1024 if inspect.ismodule(obj): 1025 lineno = 0 1026 1027 # Find the line number for classes. 1028 # Note: this could be fooled if a class is defined multiple 1029 # times in a single file. 1030 if inspect.isclass(obj): 1031 if source_lines is None: 1032 return None 1033 pat = re.compile(r'^\s*class\s*%s\b' % 1034 getattr(obj, '__name__', '-')) 1035 for i, line in enumerate(source_lines): 1036 if pat.match(line): 1037 lineno = i 1038 break 1039 1040 # Find the line number for functions & methods. 1041 if inspect.ismethod(obj): obj = obj.im_func 1042 if inspect.isfunction(obj): obj = obj.func_code 1043 if inspect.istraceback(obj): obj = obj.tb_frame 1044 if inspect.isframe(obj): obj = obj.f_code 1045 if inspect.iscode(obj): 1046 lineno = getattr(obj, 'co_firstlineno', None)-1 1047 1048 # Find the line number where the docstring starts. Assume 1049 # that it's the first line that begins with a quote mark. 1050 # Note: this could be fooled by a multiline function 1051 # signature, where a continuation line begins with a quote 1052 # mark. 1053 if lineno is not None: 1054 if source_lines is None: 1055 return lineno+1 1056 pat = re.compile('(^|.*:)\s*\w*("|\')') 1057 for lineno in range(lineno, len(source_lines)): 1058 if pat.match(source_lines[lineno]): 1059 return lineno 1060 1061 # We couldn't find the line number. 1062 return None 1063 1064###################################################################### 1065## 4. DocTest Runner 1066###################################################################### 1067 1068# [XX] Should overridable methods (eg DocTestRunner.check_output) be 1069# named with a leading underscore? 1070 1071class DocTestRunner: 1072 """ 1073 A class used to run DocTest test cases, and accumulate statistics. 1074 The `run` method is used to process a single DocTest case. It 1075 returns a tuple `(f, t)`, where `t` is the number of test cases 1076 tried, and `f` is the number of test cases that failed. 1077 1078 >>> tests = DocTestFinder().find(_TestClass) 1079 >>> runner = DocTestRunner(verbose=False) 1080 >>> for test in tests: 1081 ... print runner.run(test) 1082 (0, 2) 1083 (0, 1) 1084 (0, 2) 1085 (0, 2) 1086 1087 The `summarize` method prints a summary of all the test cases that 1088 have been run by the runner, and returns an aggregated `(f, t)` 1089 tuple: 1090 1091 >>> runner.summarize(verbose=1) 1092 4 items passed all tests: 1093 2 tests in _TestClass 1094 2 tests in _TestClass.__init__ 1095 2 tests in _TestClass.get 1096 1 tests in _TestClass.square 1097 7 tests in 4 items. 1098 7 passed and 0 failed. 1099 Test passed. 1100 (0, 7) 1101 1102 The aggregated number of tried examples and failed examples is 1103 also available via the `tries` and `failures` attributes: 1104 1105 >>> runner.tries 1106 7 1107 >>> runner.failures 1108 0 1109 1110 The comparison between expected outputs and actual outputs is done 1111 by the `check_output` method. This comparison may be customized 1112 with a number of option flags; see the documentation for `testmod` 1113 for more information. If the option flags are insufficient, then 1114 the comparison may also be customized by subclassing 1115 DocTestRunner, and overriding the methods `check_output` and 1116 `output_difference`. 1117 1118 The test runner's display output can be controlled in two ways. 1119 First, an output function (`out) can be passed to 1120 `TestRunner.run`; this function will be called with strings that 1121 should be displayed. It defaults to `sys.stdout.write`. If 1122 capturing the output is not sufficient, then the display output 1123 can be also customized by subclassing DocTestRunner, and 1124 overriding the methods `report_start`, `report_success`, 1125 `report_unexpected_exception`, and `report_failure`. 1126 """ 1127 # This divider string is used to separate failure messages, and to 1128 # separate sections of the summary. 1129 DIVIDER = "*" * 70 1130 1131 def __init__(self, verbose=None, optionflags=0): 1132 """ 1133 Create a new test runner. 1134 1135 Optional keyword arg 'verbose' prints lots of stuff if true, 1136 only failures if false; by default, it's true iff '-v' is in 1137 sys.argv. 1138 1139 Optional argument `optionflags` can be used to control how the 1140 test runner compares expected output to actual output, and how 1141 it displays failures. See the documentation for `testmod` for 1142 more information. 1143 """ 1144 if verbose is None: 1145 verbose = '-v' in sys.argv 1146 self._verbose = verbose 1147 self.optionflags = optionflags 1148 1149 # Keep track of the examples we've run. 1150 self.tries = 0 1151 self.failures = 0 1152 self._name2ft = {} 1153 1154 # Create a fake output target for capturing doctest output. 1155 self._fakeout = _SpoofOut() 1156 1157 #///////////////////////////////////////////////////////////////// 1158 # Output verification methods 1159 #///////////////////////////////////////////////////////////////// 1160 # These two methods should be updated together, since the 1161 # output_difference method needs to know what should be considered 1162 # to match by check_output. 1163 1164 def check_output(self, want, got): 1165 """ 1166 Return True iff the actual output (`got`) matches the expected 1167 output (`want`). These strings are always considered to match 1168 if they are identical; but depending on what option flags the 1169 test runner is using, several non-exact match types are also 1170 possible. See the documentation for `TestRunner` for more 1171 information about option flags. 1172 """ 1173 # Handle the common case first, for efficiency: 1174 # if they're string-identical, always return true. 1175 if got == want: 1176 return True 1177 1178 # The values True and False replaced 1 and 0 as the return 1179 # value for boolean comparisons in Python 2.3. 1180 if not (self.optionflags & DONT_ACCEPT_TRUE_FOR_1): 1181 if (got,want) == ("True\n", "1\n"): 1182 return True 1183 if (got,want) == ("False\n", "0\n"): 1184 return True 1185 1186 # <BLANKLINE> can be used as a special sequence to signify a 1187 # blank line, unless the DONT_ACCEPT_BLANKLINE flag is used. 1188 if not (self.optionflags & DONT_ACCEPT_BLANKLINE): 1189 # Replace <BLANKLINE> in want with a blank line. 1190 want = re.sub('(?m)^%s\s*?$' % re.escape(BLANKLINE_MARKER), 1191 '', want) 1192 # If a line in got contains only spaces, then remove the 1193 # spaces. 1194 got = re.sub('(?m)^\s*?$', '', got) 1195 if got == want: 1196 return True 1197 1198 # This flag causes doctest to ignore any differences in the 1199 # contents of whitespace strings. Note that this can be used 1200 # in conjunction with the ELLISPIS flag. 1201 if (self.optionflags & NORMALIZE_WHITESPACE): 1202 got = ' '.join(got.split()) 1203 want = ' '.join(want.split()) 1204 if got == want: 1205 return True 1206 1207 # The ELLIPSIS flag says to let the sequence "..." in `want` 1208 # match any substring in `got`. We implement this by 1209 # transforming `want` into a regular expression. 1210 if (self.optionflags & ELLIPSIS): 1211 # Escape any special regexp characters 1212 want_re = re.escape(want) 1213 # Replace ellipsis markers ('...') with .* 1214 want_re = want_re.replace(re.escape(ELLIPSIS_MARKER), '.*') 1215 # Require that it matches the entire string; and set the 1216 # re.DOTALL flag (with '(?s)'). 1217 want_re = '(?s)^%s$' % want_re 1218 # Check if the `want_re` regexp matches got. 1219 if re.match(want_re, got): 1220 return True 1221 1222 # We didn't find any match; return false. 1223 return False 1224 1225 def output_difference(self, want, got): 1226 """ 1227 Return a string describing the differences between the 1228 expected output (`want`) and the actual output (`got`). 1229 """ 1230 # If <BLANKLINE>s are being used, then replace <BLANKLINE> 1231 # with blank lines in the expected output string. 1232 if not (self.optionflags & DONT_ACCEPT_BLANKLINE): 1233 want = re.sub('(?m)^%s$' % re.escape(BLANKLINE_MARKER), '', want) 1234 1235 # Check if we should use diff. Don't use diff if the actual 1236 # or expected outputs are too short, or if the expected output 1237 # contains an ellipsis marker. 1238 if ((self.optionflags & (UNIFIED_DIFF | CONTEXT_DIFF)) and 1239 want.count('\n') > 2 and got.count('\n') > 2 and 1240 not (self.optionflags & ELLIPSIS and '...' in want)): 1241 # Split want & got into lines. 1242 want_lines = [l+'\n' for l in want.split('\n')] 1243 got_lines = [l+'\n' for l in got.split('\n')] 1244 # Use difflib to find their differences. 1245 if self.optionflags & UNIFIED_DIFF: 1246 diff = difflib.unified_diff(want_lines, got_lines, n=2, 1247 fromfile='Expected', tofile='Got') 1248 kind = 'unified' 1249 elif self.optionflags & CONTEXT_DIFF: 1250 diff = difflib.context_diff(want_lines, got_lines, n=2, 1251 fromfile='Expected', tofile='Got') 1252 kind = 'context' 1253 else: 1254 assert 0, 'Bad diff option' 1255 # Remove trailing whitespace on diff output. 1256 diff = [line.rstrip() + '\n' for line in diff] 1257 return _tag_msg("Differences (" + kind + " diff)", 1258 ''.join(diff)) 1259 1260 # If we're not using diff, then simply list the expected 1261 # output followed by the actual output. 1262 return (_tag_msg("Expected", want or "Nothing") + 1263 _tag_msg("Got", got)) 1264 1265 #///////////////////////////////////////////////////////////////// 1266 # Reporting methods 1267 #///////////////////////////////////////////////////////////////// 1268 1269 def report_start(self, out, test, example): 1270 """ 1271 Report that the test runner is about to process the given 1272 example. (Only displays a message if verbose=True) 1273 """ 1274 if self._verbose: 1275 out(_tag_msg("Trying", example.source) + 1276 _tag_msg("Expecting", example.want or "nothing")) 1277 1278 def report_success(self, out, test, example, got): 1279 """ 1280 Report that the given example ran successfully. (Only 1281 displays a message if verbose=True) 1282 """ 1283 if self._verbose: 1284 out("ok\n") 1285 1286 def report_failure(self, out, test, example, got): 1287 """ 1288 Report that the given example failed. 1289 """ 1290 # Print an error message. 1291 out(self.__failure_header(test, example) + 1292 self.output_difference(example.want, got)) 1293 1294 def report_unexpected_exception(self, out, test, example, exc_info): 1295 """ 1296 Report that the given example raised an unexpected exception. 1297 """ 1298 # Get a traceback message. 1299 excout = StringIO() 1300 exc_type, exc_val, exc_tb = exc_info 1301 traceback.print_exception(exc_type, exc_val, exc_tb, file=excout) 1302 exception_tb = excout.getvalue() 1303 # Print an error message. 1304 out(self.__failure_header(test, example) + 1305 _tag_msg("Exception raised", exception_tb)) 1306 1307 def __failure_header(self, test, example): 1308 s = (self.DIVIDER + "\n" + 1309 _tag_msg("Failure in example", example.source)) 1310 if test.filename is None: 1311 # [XX] I'm not putting +1 here, to give the same output 1312 # as the old version. But I think it *should* go here. 1313 return s + ("from line #%s of %s\n" % 1314 (example.lineno, test.name)) 1315 elif test.lineno is None: 1316 return s + ("from line #%s of %s in %s\n" % 1317 (example.lineno+1, test.name, test.filename)) 1318 else: 1319 lineno = test.lineno+example.lineno+1 1320 return s + ("from line #%s of %s (%s)\n" % 1321 (lineno, test.filename, test.name)) 1322 1323 #///////////////////////////////////////////////////////////////// 1324 # DocTest Running 1325 #///////////////////////////////////////////////////////////////// 1326 1327 # A regular expression for handling `want` strings that contain 1328 # expected exceptions. It divides `want` into two pieces: the 1329 # pre-exception output (`out`) and the exception message (`exc`), 1330 # as generated by traceback.format_exception_only(). (I assume 1331 # that the exception_only message is the first non-indented line 1332 # starting with word characters after the "Traceback ...".) 1333 _EXCEPTION_RE = re.compile(('^(?P<out>.*)' 1334 '^(?P<hdr>Traceback \((?:%s|%s)\):)\s*$.*?' 1335 '^(?P<exc>\w+.*)') % 1336 ('most recent call last', 'innermost last'), 1337 re.MULTILINE | re.DOTALL) 1338 1339 _OPTION_DIRECTIVE_RE = re.compile('\s*doctest:\s*(?P<flags>[^#\n]*)') 1340 1341 def __handle_directive(self, example): 1342 """ 1343 Check if the given example is actually a directive to doctest 1344 (to turn an optionflag on or off); and if it is, then handle 1345 the directive. 1346 1347 Return true iff the example is actually a directive (and so 1348 should not be executed). 1349 1350 """ 1351 m = self._OPTION_DIRECTIVE_RE.match(example.source) 1352 if m is None: 1353 return False 1354 1355 for flag in m.group('flags').upper().split(): 1356 if (flag[:1] not in '+-' or 1357 flag[1:] not in OPTIONFLAGS_BY_NAME): 1358 raise ValueError('Bad doctest option directive: '+flag) 1359 if flag[0] == '+': 1360 self.optionflags |= OPTIONFLAGS_BY_NAME[flag[1:]] 1361 else: 1362 self.optionflags &= ~OPTIONFLAGS_BY_NAME[flag[1:]] 1363 return True 1364 1365 def __run(self, test, compileflags, out): 1366 """ 1367 Run the examples in `test`. Write the outcome of each example 1368 with one of the `DocTestRunner.report_*` methods, using the 1369 writer function `out`. `compileflags` is the set of compiler 1370 flags that should be used to execute examples. Return a tuple 1371 `(f, t)`, where `t` is the number of examples tried, and `f` 1372 is the number of examples that failed. The examples are run 1373 in the namespace `test.globs`. 1374 """ 1375 # Keep track of the number of failures and tries. 1376 failures = tries = 0 1377 1378 # Save the option flags (since option directives can be used 1379 # to modify them). 1380 original_optionflags = self.optionflags 1381 1382 # Process each example. 1383 for example in test.examples: 1384 # Check if it's an option directive. If it is, then handle 1385 # it, and go on to the next example. 1386 if self.__handle_directive(example): 1387 continue 1388 1389 # Record that we started this example. 1390 tries += 1 1391 self.report_start(out, test, example) 1392 1393 # Run the example in the given context (globs), and record 1394 # any exception that gets raised. (But don't intercept 1395 # keyboard interrupts.) 1396 try: 1397 # If the example is a compound statement on one line, 1398 # like "if 1: print 2", then compile() requires a 1399 # trailing newline. Rather than analyze that, always 1400 # append one (it never hurts). 1401 exec compile(example.source + '\n', "<string>", "single", 1402 compileflags, 1) in test.globs 1403 exception = None 1404 except KeyboardInterrupt: 1405 raise 1406 except: 1407 exception = sys.exc_info() 1408 1409 # Extract the example's actual output from fakeout, and 1410 # write it to `got`. Add a terminating newline if it 1411 # doesn't have already one. 1412 got = self._fakeout.getvalue() 1413 self._fakeout.truncate(0) 1414 1415 # If the example executed without raising any exceptions, 1416 # then verify its output and report its outcome. 1417 if exception is None: 1418 if self.check_output(example.want, got): 1419 self.report_success(out, test, example, got) 1420 else: 1421 self.report_failure(out, test, example, got) 1422 failures += 1 1423 1424 # If the example raised an exception, then check if it was 1425 # expected. 1426 else: 1427 exc_info = sys.exc_info() 1428 exc_msg = traceback.format_exception_only(*exc_info[:2])[-1] 1429 1430 # Search the `want` string for an exception. If we don't 1431 # find one, then report an unexpected exception. 1432 m = self._EXCEPTION_RE.match(example.want) 1433 if m is None: 1434 self.report_unexpected_exception(out, test, example, 1435 exc_info) 1436 failures += 1 1437 else: 1438 exc_hdr = m.group('hdr')+'\n' # Exception header 1439 # The test passes iff the pre-exception output and 1440 # the exception description match the values given 1441 # in `want`. 1442 if (self.check_output(m.group('out'), got) and 1443 self.check_output(m.group('exc'), exc_msg)): 1444 # Is +exc_msg the right thing here?? 1445 self.report_success(out, test, example, 1446 got+exc_hdr+exc_msg) 1447 else: 1448 self.report_failure(out, test, example, 1449 got+exc_hdr+exc_msg) 1450 failures += 1 1451 1452 # Restore the option flags (in case they were modified) 1453 self.optionflags = original_optionflags 1454 1455 # Record and return the number of failures and tries. 1456 self.__record_outcome(test, failures, tries) 1457 return failures, tries 1458 1459 def __record_outcome(self, test, f, t): 1460 """ 1461 Record the fact that the given DocTest (`test`) generated `f` 1462 failures out of `t` tried examples. 1463 """ 1464 f2, t2 = self._name2ft.get(test.name, (0,0)) 1465 self._name2ft[test.name] = (f+f2, t+t2) 1466 self.failures += f 1467 self.tries += t 1468 1469 def run(self, test, compileflags=None, out=None, clear_globs=True): 1470 """ 1471 Run the examples in `test`, and display the results using the 1472 writer function `out`. 1473 1474 The examples are run in the namespace `test.globs`. If 1475 `clear_globs` is true (the default), then this namespace will 1476 be cleared after the test runs, to help with garbage 1477 collection. If you would like to examine the namespace after 1478 the test completes, then use `clear_globs=False`. 1479 1480 `compileflags` gives the set of flags that should be used by 1481 the Python compiler when running the examples. If not 1482 specified, then it will default to the set of future-import 1483 flags that apply to `globs`. 1484 1485 The output of each example is checked using 1486 `DocTestRunner.check_output`, and the results are formatted by 1487 the `DocTestRunner.report_*` methods. 1488 """ 1489 if compileflags is None: 1490 compileflags = _extract_future_flags(test.globs) 1491 if out is None: 1492 out = sys.stdout.write 1493 saveout = sys.stdout 1494 1495 try: 1496 sys.stdout = self._fakeout 1497 return self.__run(test, compileflags, out) 1498 finally: 1499 sys.stdout = saveout 1500 if clear_globs: 1501 test.globs.clear() 1502 1503 #///////////////////////////////////////////////////////////////// 1504 # Summarization 1505 #///////////////////////////////////////////////////////////////// 1506 def summarize(self, verbose=None): 1507 """ 1508 Print a summary of all the test cases that have been run by 1509 this DocTestRunner, and return a tuple `(f, t)`, where `f` is 1510 the total number of failed examples, and `t` is the total 1511 number of tried examples. 1512 1513 The optional `verbose` argument controls how detailed the 1514 summary is. If the verbosity is not specified, then the 1515 DocTestRunner's verbosity is used. 1516 """ 1517 if verbose is None: 1518 verbose = self._verbose 1519 notests = [] 1520 passed = [] 1521 failed = [] 1522 totalt = totalf = 0 1523 for x in self._name2ft.items(): 1524 name, (f, t) = x 1525 assert f <= t 1526 totalt += t 1527 totalf += f 1528 if t == 0: 1529 notests.append(name) 1530 elif f == 0: 1531 passed.append( (name, t) ) 1532 else: 1533 failed.append(x) 1534 if verbose: 1535 if notests: 1536 print len(notests), "items had no tests:" 1537 notests.sort() 1538 for thing in notests: 1539 print " ", thing 1540 if passed: 1541 print len(passed), "items passed all tests:" 1542 passed.sort() 1543 for thing, count in passed: 1544 print " %3d tests in %s" % (count, thing) 1545 if failed: 1546 print self.DIVIDER 1547 print len(failed), "items had failures:" 1548 failed.sort() 1549 for thing, (f, t) in failed: 1550 print " %3d of %3d in %s" % (f, t, thing) 1551 if verbose: 1552 print totalt, "tests in", len(self._name2ft), "items." 1553 print totalt - totalf, "passed and", totalf, "failed." 1554 if totalf: 1555 print "***Test Failed***", totalf, "failures." 1556 elif verbose: 1557 print "Test passed." 1558 return totalf, totalt 1559 1560class DocTestFailure(Exception): 1561 """A DocTest example has failed in debugging mode. 1562 1563 The exception instance has variables: 1564 1565 - test: the DocTest object being run 1566 1567 - excample: the Example object that failed 1568 1569 - got: the actual output 1570 """ 1571 def __init__(self, test, example, got): 1572 self.test = test 1573 self.example = example 1574 self.got = got 1575 1576 def __str__(self): 1577 return str(self.test) 1578 1579class UnexpectedException(Exception): 1580 """A DocTest example has encountered an unexpected exception 1581 1582 The exception instance has variables: 1583 1584 - test: the DocTest object being run 1585 1586 - excample: the Example object that failed 1587 1588 - exc_info: the exception info 1589 """ 1590 def __init__(self, test, example, exc_info): 1591 self.test = test 1592 self.example = example 1593 self.exc_info = exc_info 1594 1595 def __str__(self): 1596 return str(self.test) 1597 1598class DebugRunner(DocTestRunner): 1599 r"""Run doc tests but raise an exception as soon as there is a failure. 1600 1601 If an unexpected exception occurs, an UnexpectedException is raised. 1602 It contains the test, the example, and the original exception: 1603 1604 >>> runner = DebugRunner(verbose=False) 1605 >>> test = DocTest('>>> raise KeyError\n42', {}, 'foo', 'foo.py', 0) 1606 >>> try: 1607 ... runner.run(test) 1608 ... except UnexpectedException, failure: 1609 ... pass 1610 1611 >>> failure.test is test 1612 True 1613 1614 >>> failure.example.want 1615 '42\n' 1616 1617 >>> exc_info = failure.exc_info 1618 >>> raise exc_info[0], exc_info[1], exc_info[2] 1619 Traceback (most recent call last): 1620 ... 1621 KeyError 1622 1623 We wrap the original exception to give the calling application 1624 access to the test and example information. 1625 1626 If the output doesn't match, then a DocTestFailure is raised: 1627 1628 >>> test = DocTest(''' 1629 ... >>> x = 1 1630 ... >>> x 1631 ... 2 1632 ... ''', {}, 'foo', 'foo.py', 0) 1633 1634 >>> try: 1635 ... runner.run(test) 1636 ... except DocTestFailure, failure: 1637 ... pass 1638 1639 DocTestFailure objects provide access to the test: 1640 1641 >>> failure.test is test 1642 True 1643 1644 As well as to the example: 1645 1646 >>> failure.example.want 1647 '2\n' 1648 1649 and the actual output: 1650 1651 >>> failure.got 1652 '1\n' 1653 1654 If a failure or error occurs, the globals are left intact: 1655 1656 >>> del test.globs['__builtins__'] 1657 >>> test.globs 1658 {'x': 1} 1659 1660 >>> test = DocTest(''' 1661 ... >>> x = 2 1662 ... >>> raise KeyError 1663 ... ''', {}, 'foo', 'foo.py', 0) 1664 1665 >>> runner.run(test) 1666 Traceback (most recent call last): 1667 ... 1668 UnexpectedException: <DocTest foo from foo.py:0 (2 examples)> 1669 1670 >>> del test.globs['__builtins__'] 1671 >>> test.globs 1672 {'x': 2} 1673 1674 But the globals are cleared if there is no error: 1675 1676 >>> test = DocTest(''' 1677 ... >>> x = 2 1678 ... ''', {}, 'foo', 'foo.py', 0) 1679 1680 >>> runner.run(test) 1681 (0, 1) 1682 1683 >>> test.globs 1684 {} 1685 1686 """ 1687 1688 def run(self, test, compileflags=None, out=None, clear_globs=True): 1689 r = DocTestRunner.run(self, test, compileflags, out, False) 1690 if clear_globs: 1691 test.globs.clear() 1692 return r 1693 1694 def report_unexpected_exception(self, out, test, example, exc_info): 1695 raise UnexpectedException(test, example, exc_info) 1696 1697 def report_failure(self, out, test, example, got): 1698 raise DocTestFailure(test, example, got) 1699 1700###################################################################### 1701## 5. Test Functions 1702###################################################################### 1703# These should be backwards compatible. 1704 1705def testmod(m=None, name=None, globs=None, verbose=None, isprivate=None, 1706 report=True, optionflags=0, extraglobs=None, 1707 raise_on_error=False): 1708 """m=None, name=None, globs=None, verbose=None, isprivate=None, 1709 report=True, optionflags=0, extraglobs=None 1710 1711 Test examples in docstrings in functions and classes reachable 1712 from module m (or the current module if m is not supplied), starting 1713 with m.__doc__. Unless isprivate is specified, private names 1714 are not skipped. 1715 1716 Also test examples reachable from dict m.__test__ if it exists and is 1717 not None. m.__dict__ maps names to functions, classes and strings; 1718 function and class docstrings are tested even if the name is private; 1719 strings are tested directly, as if they were docstrings. 1720 1721 Return (#failures, #tests). 1722 1723 See doctest.__doc__ for an overview. 1724 1725 Optional keyword arg "name" gives the name of the module; by default 1726 use m.__name__. 1727 1728 Optional keyword arg "globs" gives a dict to be used as the globals 1729 when executing examples; by default, use m.__dict__. A copy of this 1730 dict is actually used for each docstring, so that each docstring's 1731 examples start with a clean slate. 1732 1733 Optional keyword arg "extraglobs" gives a dictionary that should be 1734 merged into the globals that are used to execute examples. By 1735 default, no extra globals are used. This is new in 2.4. 1736 1737 Optional keyword arg "verbose" prints lots of stuff if true, prints 1738 only failures if false; by default, it's true iff "-v" is in sys.argv. 1739 1740 Optional keyword arg "report" prints a summary at the end when true, 1741 else prints nothing at the end. In verbose mode, the summary is 1742 detailed, else very brief (in fact, empty if all tests passed). 1743 1744 Optional keyword arg "optionflags" or's together module constants, 1745 and defaults to 0. This is new in 2.3. Possible values: 1746 1747 DONT_ACCEPT_TRUE_FOR_1 1748 By default, if an expected output block contains just "1", 1749 an actual output block containing just "True" is considered 1750 to be a match, and similarly for "0" versus "False". When 1751 DONT_ACCEPT_TRUE_FOR_1 is specified, neither substitution 1752 is allowed. 1753 1754 DONT_ACCEPT_BLANKLINE 1755 By default, if an expected output block contains a line 1756 containing only the string "<BLANKLINE>", then that line 1757 will match a blank line in the actual output. When 1758 DONT_ACCEPT_BLANKLINE is specified, this substitution is 1759 not allowed. 1760 1761 NORMALIZE_WHITESPACE 1762 When NORMALIZE_WHITESPACE is specified, all sequences of 1763 whitespace are treated as equal. I.e., any sequence of 1764 whitespace within the expected output will match any 1765 sequence of whitespace within the actual output. 1766 1767 ELLIPSIS 1768 When ELLIPSIS is specified, then an ellipsis marker 1769 ("...") in the expected output can match any substring in 1770 the actual output. 1771 1772 UNIFIED_DIFF 1773 When UNIFIED_DIFF is specified, failures that involve 1774 multi-line expected and actual outputs will be displayed 1775 using a unified diff. 1776 1777 CONTEXT_DIFF 1778 When CONTEXT_DIFF is specified, failures that involve 1779 multi-line expected and actual outputs will be displayed 1780 using a context diff. 1781 1782 Optional keyword arg "raise_on_error" raises an exception on the 1783 first unexpected exception or failure. This allows failures to be 1784 post-mortem debugged. 1785 1786 Deprecated in Python 2.4: 1787 Optional keyword arg "isprivate" specifies a function used to 1788 determine whether a name is private. The default function is 1789 treat all functions as public. Optionally, "isprivate" can be 1790 set to doctest.is_private to skip over functions marked as private 1791 using the underscore naming convention; see its docs for details. 1792 """ 1793 1794 """ [XX] This is no longer true: 1795 Advanced tomfoolery: testmod runs methods of a local instance of 1796 class doctest.Tester, then merges the results into (or creates) 1797 global Tester instance doctest.master. Methods of doctest.master 1798 can be called directly too, if you want to do something unusual. 1799 Passing report=0 to testmod is especially useful then, to delay 1800 displaying a summary. Invoke doctest.master.summarize(verbose) 1801 when you're done fiddling. 1802 """ 1803 if isprivate is not None: 1804 warnings.warn("the isprivate argument is deprecated; " 1805 "examine DocTestFinder.find() lists instead", 1806 DeprecationWarning) 1807 1808 # If no module was given, then use __main__. 1809 if m is None: 1810 # DWA - m will still be None if this wasn't invoked from the command 1811 # line, in which case the following TypeError is about as good an error 1812 # as we should expect 1813 m = sys.modules.get('__main__') 1814 1815 # Check that we were actually given a module. 1816 if not inspect.ismodule(m): 1817 raise TypeError("testmod: module required; %r" % (m,)) 1818 1819 # If no name was given, then use the module's name. 1820 if name is None: 1821 name = m.__name__ 1822 1823 # Find, parse, and run all tests in the given module. 1824 finder = DocTestFinder(_namefilter=isprivate) 1825 1826 if raise_on_error: 1827 runner = DebugRunner(verbose=verbose, optionflags=optionflags) 1828 else: 1829 runner = DocTestRunner(verbose=verbose, optionflags=optionflags) 1830 1831 for test in finder.find(m, name, globs=globs, extraglobs=extraglobs): 1832 runner.run(test) 1833 1834 if report: 1835 runner.summarize() 1836 1837 return runner.failures, runner.tries 1838 1839def run_docstring_examples(f, globs, verbose=False, name="NoName", 1840 compileflags=None, optionflags=0): 1841 """ 1842 Test examples in the given object's docstring (`f`), using `globs` 1843 as globals. Optional argument `name` is used in failure messages. 1844 If the optional argument `verbose` is true, then generate output 1845 even if there are no failures. 1846 1847 `compileflags` gives the set of flags that should be used by the 1848 Python compiler when running the examples. If not specified, then 1849 it will default to the set of future-import flags that apply to 1850 `globs`. 1851 1852 Optional keyword arg `optionflags` specifies options for the 1853 testing and output. See the documentation for `testmod` for more 1854 information. 1855 """ 1856 # Find, parse, and run all tests in the given module. 1857 finder = DocTestFinder(verbose=verbose, recurse=False) 1858 runner = DocTestRunner(verbose=verbose, optionflags=optionflags) 1859 for test in finder.find(f, name, globs=globs): 1860 runner.run(test, compileflags=compileflags) 1861 1862###################################################################### 1863## 6. Tester 1864###################################################################### 1865# This is provided only for backwards compatibility. It's not 1866# actually used in any way. 1867 1868class Tester: 1869 def __init__(self, mod=None, globs=None, verbose=None, 1870 isprivate=None, optionflags=0): 1871 1872 warnings.warn("class Tester is deprecated; " 1873 "use class doctest.DocTestRunner instead", 1874 DeprecationWarning, stacklevel=2) 1875 if mod is None and globs is None: 1876 raise TypeError("Tester.__init__: must specify mod or globs") 1877 if mod is not None and not _ismodule(mod): 1878 raise TypeError("Tester.__init__: mod must be a module; %r" % 1879 (mod,)) 1880 if globs is None: 1881 globs = mod.__dict__ 1882 self.globs = globs 1883 1884 self.verbose = verbose 1885 self.isprivate = isprivate 1886 self.optionflags = optionflags 1887 self.testfinder = DocTestFinder(_namefilter=isprivate) 1888 self.testrunner = DocTestRunner(verbose=verbose, 1889 optionflags=optionflags) 1890 1891 def runstring(self, s, name): 1892 test = DocTest(s, self.globs, name, None, None) 1893 if self.verbose: 1894 print "Running string", name 1895 (f,t) = self.testrunner.run(test) 1896 if self.verbose: 1897 print f, "of", t, "examples failed in string", name 1898 return (f,t) 1899 1900 def rundoc(self, object, name=None, module=None): 1901 f = t = 0 1902 tests = self.testfinder.find(object, name, module=module, 1903 globs=self.globs) 1904 for test in tests: 1905 (f2, t2) = self.testrunner.run(test) 1906 (f,t) = (f+f2, t+t2) 1907 return (f,t) 1908 1909 def rundict(self, d, name, module=None): 1910 import new 1911 m = new.module(name) 1912 m.__dict__.update(d) 1913 if module is None: 1914 module = False 1915 return self.rundoc(m, name, module) 1916 1917 def run__test__(self, d, name): 1918 import new 1919 m = new.module(name) 1920 m.__test__ = d 1921 return self.rundoc(m, name, module) 1922 1923 def summarize(self, verbose=None): 1924 return self.testrunner.summarize(verbose) 1925 1926 def merge(self, other): 1927 d = self.testrunner._name2ft 1928 for name, (f, t) in other.testrunner._name2ft.items(): 1929 if name in d: 1930 print "*** Tester.merge: '" + name + "' in both" \ 1931 " testers; summing outcomes." 1932 f2, t2 = d[name] 1933 f = f + f2 1934 t = t + t2 1935 d[name] = f, t 1936 1937###################################################################### 1938## 7. Unittest Support 1939###################################################################### 1940 1941class DocTestCase(unittest.TestCase): 1942 1943 def __init__(self, test, optionflags=0, setUp=None, tearDown=None): 1944 unittest.TestCase.__init__(self) 1945 self._dt_optionflags = optionflags 1946 self._dt_test = test 1947 self._dt_setUp = setUp 1948 self._dt_tearDown = tearDown 1949 1950 def setUp(self): 1951 if self._dt_setUp is not None: 1952 self._dt_setUp() 1953 1954 def tearDown(self): 1955 if self._dt_tearDown is not None: 1956 self._dt_tearDown() 1957 1958 def runTest(self): 1959 test = self._dt_test 1960 old = sys.stdout 1961 new = StringIO() 1962 runner = DocTestRunner(optionflags=self._dt_optionflags, verbose=False) 1963 1964 try: 1965 runner.DIVIDER = "-"*70 1966 failures, tries = runner.run(test, out=new.write) 1967 finally: 1968 sys.stdout = old 1969 1970 if failures: 1971 raise self.failureException(self.format_failure(new.getvalue())) 1972 1973 def format_failure(self, err): 1974 test = self._dt_test 1975 if test.lineno is None: 1976 lineno = 'unknown line number' 1977 else: 1978 lineno = 'line %s' % test.lineno 1979 lname = '.'.join(test.name.split('.')[-1:]) 1980 return ('Failed doctest test for %s\n' 1981 ' File "%s", line %s, in %s\n\n%s' 1982 % (test.name, test.filename, lineno, lname, err) 1983 ) 1984 1985 def debug(self): 1986 r"""Run the test case without results and without catching exceptions 1987 1988 The unit test framework includes a debug method on test cases 1989 and test suites to support post-mortem debugging. The test code 1990 is run in such a way that errors are not caught. This way a 1991 caller can catch the errors and initiate post-mortem debugging. 1992 1993 The DocTestCase provides a debug method that raises 1994 UnexpectedException errors if there is an unexepcted 1995 exception: 1996 1997 >>> test = DocTest('>>> raise KeyError\n42', 1998 ... {}, 'foo', 'foo.py', 0) 1999 >>> case = DocTestCase(test) 2000 >>> try: 2001 ... case.debug() 2002 ... except UnexpectedException, failure: 2003 ... pass 2004 2005 The UnexpectedException contains the test, the example, and 2006 the original exception: 2007 2008 >>> failure.test is test 2009 True 2010 2011 >>> failure.example.want 2012 '42\n' 2013 2014 >>> exc_info = failure.exc_info 2015 >>> raise exc_info[0], exc_info[1], exc_info[2] 2016 Traceback (most recent call last): 2017 ... 2018 KeyError 2019 2020 If the output doesn't match, then a DocTestFailure is raised: 2021 2022 >>> test = DocTest(''' 2023 ... >>> x = 1 2024 ... >>> x 2025 ... 2 2026 ... ''', {}, 'foo', 'foo.py', 0) 2027 >>> case = DocTestCase(test) 2028 2029 >>> try: 2030 ... case.debug() 2031 ... except DocTestFailure, failure: 2032 ... pass 2033 2034 DocTestFailure objects provide access to the test: 2035 2036 >>> failure.test is test 2037 True 2038 2039 As well as to the example: 2040 2041 >>> failure.example.want 2042 '2\n' 2043 2044 and the actual output: 2045 2046 >>> failure.got 2047 '1\n' 2048 2049 """ 2050 2051 runner = DebugRunner(verbose = False, optionflags=self._dt_optionflags) 2052 runner.run(self._dt_test, out=nooutput) 2053 2054 def id(self): 2055 return self._dt_test.name 2056 2057 def __repr__(self): 2058 name = self._dt_test.name.split('.') 2059 return "%s (%s)" % (name[-1], '.'.join(name[:-1])) 2060 2061 __str__ = __repr__ 2062 2063 def shortDescription(self): 2064 return "Doctest: " + self._dt_test.name 2065 2066def nooutput(*args): 2067 pass 2068 2069def DocTestSuite(module=None, globs=None, extraglobs=None, 2070 optionflags=0, test_finder=None, 2071 setUp=lambda: None, tearDown=lambda: None): 2072 """ 2073 Convert doctest tests for a mudule to a unittest test suite. 2074 2075 This converts each documentation string in a module that 2076 contains doctest tests to a unittest test case. If any of the 2077 tests in a doc string fail, then the test case fails. An exception 2078 is raised showing the name of the file containing the test and a 2079 (sometimes approximate) line number. 2080 2081 The `module` argument provides the module to be tested. The argument 2082 can be either a module or a module name. 2083 2084 If no argument is given, the calling module is used. 2085 """ 2086 2087 if test_finder is None: 2088 test_finder = DocTestFinder() 2089 2090 module = _normalize_module(module) 2091 tests = test_finder.find(module, globs=globs, extraglobs=extraglobs) 2092 if globs is None: 2093 globs = module.__dict__ 2094 if not tests: # [XX] why do we want to do this? 2095 raise ValueError(module, "has no tests") 2096 2097 tests.sort() 2098 suite = unittest.TestSuite() 2099 for test in tests: 2100 if len(test.examples) == 0: 2101 continue 2102 if not test.filename: 2103 filename = module.__file__ 2104 if filename.endswith(".pyc"): 2105 filename = filename[:-1] 2106 elif filename.endswith(".pyo"): 2107 filename = filename[:-1] 2108 test.filename = filename 2109 suite.addTest(DocTestCase(test, optionflags, setUp, tearDown)) 2110 2111 return suite 2112 2113class DocFileCase(DocTestCase): 2114 2115 def id(self): 2116 return '_'.join(self._dt_test.name.split('.')) 2117 2118 def __repr__(self): 2119 return self._dt_test.filename 2120 __str__ = __repr__ 2121 2122 def format_failure(self, err): 2123 return ('Failed doctest test for %s\n File "%s", line 0\n\n%s' 2124 % (self._dt_test.name, self._dt_test.filename, err) 2125 ) 2126 2127def DocFileTest(path, package=None, globs=None, 2128 setUp=None, tearDown=None, 2129 optionflags=0): 2130 package = _normalize_module(package) 2131 name = path.split('/')[-1] 2132 dir = os.path.split(package.__file__)[0] 2133 path = os.path.join(dir, *(path.split('/'))) 2134 doc = open(path).read() 2135 2136 if globs is None: 2137 globs = {} 2138 2139 test = DocTest(doc, globs, name, path, 0) 2140 2141 return DocFileCase(test, optionflags, setUp, tearDown) 2142 2143def DocFileSuite(*paths, **kw): 2144 """Creates a suite of doctest files. 2145 2146 One or more text file paths are given as strings. These should 2147 use "/" characters to separate path segments. Paths are relative 2148 to the directory of the calling module, or relative to the package 2149 passed as a keyword argument. 2150 2151 A number of options may be provided as keyword arguments: 2152 2153 package 2154 The name of a Python package. Text-file paths will be 2155 interpreted relative to the directory containing this package. 2156 The package may be supplied as a package object or as a dotted 2157 package name. 2158 2159 setUp 2160 The name of a set-up function. This is called before running the 2161 tests in each file. 2162 2163 tearDown 2164 The name of a tear-down function. This is called after running the 2165 tests in each file. 2166 2167 globs 2168 A dictionary containing initial global variables for the tests. 2169 """ 2170 suite = unittest.TestSuite() 2171 2172 # We do this here so that _normalize_module is called at the right 2173 # level. If it were called in DocFileTest, then this function 2174 # would be the caller and we might guess the package incorrectly. 2175 kw['package'] = _normalize_module(kw.get('package')) 2176 2177 for path in paths: 2178 suite.addTest(DocFileTest(path, **kw)) 2179 2180 return suite 2181 2182###################################################################### 2183## 8. Debugging Support 2184###################################################################### 2185 2186def script_from_examples(s): 2187 r"""Extract script from text with examples. 2188 2189 Converts text with examples to a Python script. Example input is 2190 converted to regular code. Example output and all other words 2191 are converted to comments: 2192 2193 >>> text = ''' 2194 ... Here are examples of simple math. 2195 ... 2196 ... Python has super accurate integer addition 2197 ... 2198 ... >>> 2 + 2 2199 ... 5 2200 ... 2201 ... And very friendly error messages: 2202 ... 2203 ... >>> 1/0 2204 ... To Infinity 2205 ... And 2206 ... Beyond 2207 ... 2208 ... You can use logic if you want: 2209 ... 2210 ... >>> if 0: 2211 ... ... blah 2212 ... ... blah 2213 ... ... 2214 ... 2215 ... Ho hum 2216 ... ''' 2217 2218 >>> print script_from_examples(text) 2219 # Here are examples of simple math. 2220 # 2221 # Python has super accurate integer addition 2222 # 2223 2 + 2 2224 # Expected: 2225 # 5 2226 # 2227 # And very friendly error messages: 2228 # 2229 1/0 2230 # Expected: 2231 # To Infinity 2232 # And 2233 # Beyond 2234 # 2235 # You can use logic if you want: 2236 # 2237 if 0: 2238 blah 2239 blah 2240 <BLANKLINE> 2241 # 2242 # Ho hum 2243 """ 2244 2245 return Parser('<string>', s).get_program() 2246 2247def _want_comment(example): 2248 """ 2249 Return a comment containing the expected output for the given example. 2250 """ 2251 # Return the expected output, if any 2252 want = example.want 2253 if want: 2254 if want[-1] == '\n': 2255 want = want[:-1] 2256 want = "\n# ".join(want.split("\n")) 2257 want = "\n# Expected:\n# %s" % want 2258 return want 2259 2260def testsource(module, name): 2261 """Extract the test sources from a doctest docstring as a script. 2262 2263 Provide the module (or dotted name of the module) containing the 2264 test to be debugged and the name (within the module) of the object 2265 with the doc string with tests to be debugged. 2266 """ 2267 module = _normalize_module(module) 2268 tests = DocTestFinder().find(module) 2269 test = [t for t in tests if t.name == name] 2270 if not test: 2271 raise ValueError(name, "not found in tests") 2272 test = test[0] 2273 testsrc = script_from_examples(test.docstring) 2274 return testsrc 2275 2276def debug_src(src, pm=False, globs=None): 2277 """Debug a single doctest docstring, in argument `src`'""" 2278 testsrc = script_from_examples(src) 2279 debug_script(testsrc, pm, globs) 2280 2281def debug_script(src, pm=False, globs=None): 2282 "Debug a test script. `src` is the script, as a string." 2283 import pdb 2284 2285 srcfilename = tempfile.mktemp("doctestdebug.py") 2286 f = open(srcfilename, 'w') 2287 f.write(src) 2288 f.close() 2289 2290 if globs: 2291 globs = globs.copy() 2292 else: 2293 globs = {} 2294 2295 if pm: 2296 try: 2297 execfile(srcfilename, globs, globs) 2298 except: 2299 print sys.exc_info()[1] 2300 pdb.post_mortem(sys.exc_info()[2]) 2301 else: 2302 # Note that %r is vital here. '%s' instead can, e.g., cause 2303 # backslashes to get treated as metacharacters on Windows. 2304 pdb.run("execfile(%r)" % srcfilename, globs, globs) 2305 2306def debug(module, name, pm=False): 2307 """Debug a single doctest docstring. 2308 2309 Provide the module (or dotted name of the module) containing the 2310 test to be debugged and the name (within the module) of the object 2311 with the docstring with tests to be debugged. 2312 """ 2313 module = _normalize_module(module) 2314 testsrc = testsource(module, name) 2315 debug_script(testsrc, pm, module.__dict__) 2316 2317###################################################################### 2318## 9. Example Usage 2319###################################################################### 2320class _TestClass: 2321 """ 2322 A pointless class, for sanity-checking of docstring testing. 2323 2324 Methods: 2325 square() 2326 get() 2327 2328 >>> _TestClass(13).get() + _TestClass(-12).get() 2329 1 2330 >>> hex(_TestClass(13).square().get()) 2331 '0xa9' 2332 """ 2333 2334 def __init__(self, val): 2335 """val -> _TestClass object with associated value val. 2336 2337 >>> t = _TestClass(123) 2338 >>> print t.get() 2339 123 2340 """ 2341 2342 self.val = val 2343 2344 def square(self): 2345 """square() -> square TestClass's associated value 2346 2347 >>> _TestClass(13).square().get() 2348 169 2349 """ 2350 2351 self.val = self.val ** 2 2352 return self 2353 2354 def get(self): 2355 """get() -> return TestClass's associated value. 2356 2357 >>> x = _TestClass(-42) 2358 >>> print x.get() 2359 -42 2360 """ 2361 2362 return self.val 2363 2364__test__ = {"_TestClass": _TestClass, 2365 "string": r""" 2366 Example of a string object, searched as-is. 2367 >>> x = 1; y = 2 2368 >>> x + y, x * y 2369 (3, 2) 2370 """, 2371 "bool-int equivalence": r""" 2372 In 2.2, boolean expressions displayed 2373 0 or 1. By default, we still accept 2374 them. This can be disabled by passing 2375 DONT_ACCEPT_TRUE_FOR_1 to the new 2376 optionflags argument. 2377 >>> 4 == 4 2378 1 2379 >>> 4 == 4 2380 True 2381 >>> 4 > 4 2382 0 2383 >>> 4 > 4 2384 False 2385 """, 2386 "blank lines": r""" 2387 Blank lines can be marked with <BLANKLINE>: 2388 >>> print 'foo\n\nbar\n' 2389 foo 2390 <BLANKLINE> 2391 bar 2392 <BLANKLINE> 2393 """, 2394 } 2395# "ellipsis": r""" 2396# If the ellipsis flag is used, then '...' can be used to 2397# elide substrings in the desired output: 2398# >>> print range(1000) 2399# [0, 1, 2, ..., 999] 2400# """, 2401# "whitespace normalization": r""" 2402# If the whitespace normalization flag is used, then 2403# differences in whitespace are ignored. 2404# >>> print range(30) 2405# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 2406# 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 2407# 27, 28, 29] 2408# """, 2409# } 2410 2411def test1(): r""" 2412>>> warnings.filterwarnings("ignore", "class Tester", DeprecationWarning, 2413... "doctest", 0) 2414>>> from doctest import Tester 2415>>> t = Tester(globs={'x': 42}, verbose=0) 2416>>> t.runstring(r''' 2417... >>> x = x * 2 2418... >>> print x 2419... 42 2420... ''', 'XYZ') 2421********************************************************************** 2422Failure in example: print x 2423from line #2 of XYZ 2424Expected: 42 2425Got: 84 2426(1, 2) 2427>>> t.runstring(">>> x = x * 2\n>>> print x\n84\n", 'example2') 2428(0, 2) 2429>>> t.summarize() 2430********************************************************************** 24311 items had failures: 2432 1 of 2 in XYZ 2433***Test Failed*** 1 failures. 2434(1, 4) 2435>>> t.summarize(verbose=1) 24361 items passed all tests: 2437 2 tests in example2 2438********************************************************************** 24391 items had failures: 2440 1 of 2 in XYZ 24414 tests in 2 items. 24423 passed and 1 failed. 2443***Test Failed*** 1 failures. 2444(1, 4) 2445""" 2446 2447def test2(): r""" 2448 >>> warnings.filterwarnings("ignore", "class Tester", 2449 ... DeprecationWarning, "doctest", 0) 2450 >>> t = Tester(globs={}, verbose=1) 2451 >>> test = r''' 2452 ... # just an example 2453 ... >>> x = 1 + 2 2454 ... >>> x 2455 ... 3 2456 ... ''' 2457 >>> t.runstring(test, "Example") 2458 Running string Example 2459 Trying: x = 1 + 2 2460 Expecting: nothing 2461 ok 2462 Trying: x 2463 Expecting: 3 2464 ok 2465 0 of 2 examples failed in string Example 2466 (0, 2) 2467""" 2468def test3(): r""" 2469 >>> warnings.filterwarnings("ignore", "class Tester", 2470 ... DeprecationWarning, "doctest", 0) 2471 >>> t = Tester(globs={}, verbose=0) 2472 >>> def _f(): 2473 ... '''Trivial docstring example. 2474 ... >>> assert 2 == 2 2475 ... ''' 2476 ... return 32 2477 ... 2478 >>> t.rundoc(_f) # expect 0 failures in 1 example 2479 (0, 1) 2480""" 2481def test4(): """ 2482 >>> import new 2483 >>> m1 = new.module('_m1') 2484 >>> m2 = new.module('_m2') 2485 >>> test_data = \""" 2486 ... def _f(): 2487 ... '''>>> assert 1 == 1 2488 ... ''' 2489 ... def g(): 2490 ... '''>>> assert 2 != 1 2491 ... ''' 2492 ... class H: 2493 ... '''>>> assert 2 > 1 2494 ... ''' 2495 ... def bar(self): 2496 ... '''>>> assert 1 < 2 2497 ... ''' 2498 ... \""" 2499 >>> exec test_data in m1.__dict__ 2500 >>> exec test_data in m2.__dict__ 2501 >>> m1.__dict__.update({"f2": m2._f, "g2": m2.g, "h2": m2.H}) 2502 2503 Tests that objects outside m1 are excluded: 2504 2505 >>> warnings.filterwarnings("ignore", "class Tester", 2506 ... DeprecationWarning, "doctest", 0) 2507 >>> t = Tester(globs={}, verbose=0) 2508 >>> t.rundict(m1.__dict__, "rundict_test", m1) # f2 and g2 and h2 skipped 2509 (0, 4) 2510 2511 Once more, not excluding stuff outside m1: 2512 2513 >>> t = Tester(globs={}, verbose=0) 2514 >>> t.rundict(m1.__dict__, "rundict_test_pvt") # None are skipped. 2515 (0, 8) 2516 2517 The exclusion of objects from outside the designated module is 2518 meant to be invoked automagically by testmod. 2519 2520 >>> testmod(m1, verbose=False) 2521 (0, 4) 2522""" 2523 2524def _test(): 2525 #import doctest 2526 #doctest.testmod(doctest, verbose=False, 2527 # optionflags=ELLIPSIS | NORMALIZE_WHITESPACE | 2528 # UNIFIED_DIFF) 2529 #print '~'*70 2530 r = unittest.TextTestRunner() 2531 r.run(DocTestSuite()) 2532 2533if __name__ == "__main__": 2534 _test() 2535