doctest.py revision b7503ff631648db6b54641db3154f4a70193c997
1# Module doctest. 2# Released to the public domain 16-Jan-2001, by Tim Peters (tim@python.org). 3# Major enhancements and refactoring by: 4# Jim Fulton 5# Edward Loper 6 7# Provided as-is; use at your own risk; no warranty; no promises; enjoy! 8 9r"""Module doctest -- a framework for running examples in docstrings. 10 11NORMAL USAGE 12 13In simplest use, end each module M to be tested with: 14 15def _test(): 16 import doctest 17 return doctest.testmod() 18 19if __name__ == "__main__": 20 _test() 21 22Then running the module as a script will cause the examples in the 23docstrings to get executed and verified: 24 25python M.py 26 27This won't display anything unless an example fails, in which case the 28failing example(s) and the cause(s) of the failure(s) are printed to stdout 29(why not stderr? because stderr is a lame hack <0.2 wink>), and the final 30line of output is "Test failed.". 31 32Run it with the -v switch instead: 33 34python M.py -v 35 36and a detailed report of all examples tried is printed to stdout, along 37with assorted summaries at the end. 38 39You can force verbose mode by passing "verbose=True" to testmod, or prohibit 40it by passing "verbose=False". In either of those cases, sys.argv is not 41examined by testmod. 42 43In any case, testmod returns a 2-tuple of ints (f, t), where f is the 44number of docstring examples that failed and t is the total number of 45docstring examples attempted. 46 47There are a variety of other ways to run doctests, including integration 48with the unittest framework, and support for running non-Python text 49files containing doctests. There are also many ways to override parts 50of doctest's default behaviors. See the Library Reference Manual for 51details. 52 53 54WHICH DOCSTRINGS ARE EXAMINED? 55 56+ M.__doc__. 57 58+ f.__doc__ for all functions f in M.__dict__.values(), except those 59 defined in other modules. 60 61+ C.__doc__ for all classes C in M.__dict__.values(), except those 62 defined in other modules. 63 64+ If M.__test__ exists and "is true", it must be a dict, and 65 each entry maps a (string) name to a function object, class object, or 66 string. Function and class object docstrings found from M.__test__ 67 are searched, and strings are searched directly as if they were docstrings. 68 In output, a key K in M.__test__ appears with name 69 <name of M>.__test__.K 70 71Any classes found are recursively searched similarly, to test docstrings in 72their contained methods and nested classes. 73 74 75WHAT'S THE EXECUTION CONTEXT? 76 77By default, each time testmod finds a docstring to test, it uses a *copy* 78of M's globals (so that running tests on a module doesn't change the 79module's real globals, and so that one test in M can't leave behind crumbs 80that accidentally allow another test to work). This means examples can 81freely use any names defined at top-level in M. It also means that sloppy 82imports (see above) can cause examples in external docstrings to use 83globals inappropriate for them. 84 85You can force use of your own dict as the execution context by passing 86"globs=your_dict" to testmod instead. Presumably this would be a copy of 87M.__dict__ merged with the globals from other imported modules. 88 89 90WHAT ABOUT EXCEPTIONS? 91 92No problem, as long as the only output generated by the example is the 93traceback itself. For example: 94 95 >>> [1, 2, 3].remove(42) 96 Traceback (most recent call last): 97 File "<stdin>", line 1, in ? 98 ValueError: list.remove(x): x not in list 99 >>> 100 101Note that only the exception type and value are compared. 102 103 104SO WHAT DOES A DOCTEST EXAMPLE LOOK LIKE ALREADY!? 105 106Oh ya. It's easy! In most cases a copy-and-paste of an interactive 107console session works fine -- just make sure the leading whitespace is 108rigidly consistent (you can mix tabs and spaces if you're too lazy to do it 109right, but doctest is not in the business of guessing what you think a tab 110means). 111 112 >>> # comments are ignored 113 >>> x = 12 114 >>> x 115 12 116 >>> if x == 13: 117 ... print "yes" 118 ... else: 119 ... print "no" 120 ... print "NO" 121 ... print "NO!!!" 122 ... 123 no 124 NO 125 NO!!! 126 >>> 127 128Any expected output must immediately follow the final ">>>" or "..." line 129containing the code, and the expected output (if any) extends to the next 130">>>" or all-whitespace line. That's it. 131 132Bummers: 133 134+ Output to stdout is captured, but not output to stderr (exception 135 tracebacks are captured via a different means). 136 137+ If you continue a line via backslashing in an interactive session, 138 or for any other reason use a backslash, you should use a raw 139 docstring, which will preserve your backslahses exactly as you type 140 them: 141 142 >>> def f(x): 143 ... r'''Backslashes in a raw docstring: m\n''' 144 >>> print f.__doc__ 145 Backslashes in a raw docstring: m\n 146 147 Otherwise, the backslash will be interpreted as part of the string. 148 E.g., the "\n" above would be interpreted as a newline character. 149 Alternatively, you can double each backslash in the doctest version 150 (and not use a raw string): 151 152 >>> def f(x): 153 ... '''Backslashes in a raw docstring: m\\n''' 154 >>> print f.__doc__ 155 Backslashes in a raw docstring: m\n 156 157The starting column doesn't matter: 158 159>>> assert "Easy!" 160 >>> import math 161 >>> math.floor(1.9) 162 1.0 163 164and as many leading whitespace characters are stripped from the expected 165output as appeared in the initial ">>>" line that triggered it. 166 167If you execute this very file, the examples above will be found and 168executed. 169""" 170__docformat__ = 'reStructuredText en' 171 172__all__ = [ 173 # 0, Option Flags 174 'register_optionflag', 175 'DONT_ACCEPT_TRUE_FOR_1', 176 'DONT_ACCEPT_BLANKLINE', 177 'NORMALIZE_WHITESPACE', 178 'ELLIPSIS', 179 'UNIFIED_DIFF', 180 'CONTEXT_DIFF', 181 # 1. Utility Functions 182 'is_private', 183 # 2. Example & DocTest 184 'Example', 185 'DocTest', 186 # 3. Doctest Parser 187 'DocTestParser', 188 # 4. Doctest Finder 189 'DocTestFinder', 190 # 5. Doctest Runner 191 'DocTestRunner', 192 'OutputChecker', 193 'DocTestFailure', 194 'UnexpectedException', 195 'DebugRunner', 196 # 6. Test Functions 197 'testmod', 198 'run_docstring_examples', 199 # 7. Tester 200 'Tester', 201 # 8. Unittest Support 202 'DocTestCase', 203 'nooutput', 204 'DocTestSuite', 205 'DocFileCase', 206 'DocFileTest', 207 'DocFileSuite', 208 # 9. Debugging Support 209 'script_from_examples', 210 'testsource', 211 'debug_src', 212 'debug_script', 213 'debug', 214] 215 216import __future__ 217 218import sys, traceback, inspect, linecache, os, re, types 219import unittest, difflib, pdb, tempfile 220import warnings 221from StringIO import StringIO 222 223real_pdb_set_trace = pdb.set_trace 224 225# There are 4 basic classes: 226# - Example: a <source, want> pair, plus an intra-docstring line number. 227# - DocTest: a collection of examples, parsed from a docstring, plus 228# info about where the docstring came from (name, filename, lineno). 229# - DocTestFinder: extracts DocTests from a given object's docstring and 230# its contained objects' docstrings. 231# - DocTestRunner: runs DocTest cases, and accumulates statistics. 232# 233# So the basic picture is: 234# 235# list of: 236# +------+ +---------+ +-------+ 237# |object| --DocTestFinder-> | DocTest | --DocTestRunner-> |results| 238# +------+ +---------+ +-------+ 239# | Example | 240# | ... | 241# | Example | 242# +---------+ 243 244# Option constants. 245OPTIONFLAGS_BY_NAME = {} 246def register_optionflag(name): 247 flag = 1 << len(OPTIONFLAGS_BY_NAME) 248 OPTIONFLAGS_BY_NAME[name] = flag 249 return flag 250 251DONT_ACCEPT_TRUE_FOR_1 = register_optionflag('DONT_ACCEPT_TRUE_FOR_1') 252DONT_ACCEPT_BLANKLINE = register_optionflag('DONT_ACCEPT_BLANKLINE') 253NORMALIZE_WHITESPACE = register_optionflag('NORMALIZE_WHITESPACE') 254ELLIPSIS = register_optionflag('ELLIPSIS') 255UNIFIED_DIFF = register_optionflag('UNIFIED_DIFF') 256CONTEXT_DIFF = register_optionflag('CONTEXT_DIFF') 257 258# Special string markers for use in `want` strings: 259BLANKLINE_MARKER = '<BLANKLINE>' 260ELLIPSIS_MARKER = '...' 261 262###################################################################### 263## Table of Contents 264###################################################################### 265# 1. Utility Functions 266# 2. Example & DocTest -- store test cases 267# 3. DocTest Parser -- extracts examples from strings 268# 4. DocTest Finder -- extracts test cases from objects 269# 5. DocTest Runner -- runs test cases 270# 6. Test Functions -- convenient wrappers for testing 271# 7. Tester Class -- for backwards compatibility 272# 8. Unittest Support 273# 9. Debugging Support 274# 10. Example Usage 275 276###################################################################### 277## 1. Utility Functions 278###################################################################### 279 280def is_private(prefix, base): 281 """prefix, base -> true iff name prefix + "." + base is "private". 282 283 Prefix may be an empty string, and base does not contain a period. 284 Prefix is ignored (although functions you write conforming to this 285 protocol may make use of it). 286 Return true iff base begins with an (at least one) underscore, but 287 does not both begin and end with (at least) two underscores. 288 289 >>> warnings.filterwarnings("ignore", "is_private", DeprecationWarning, 290 ... "doctest", 0) 291 >>> is_private("a.b", "my_func") 292 False 293 >>> is_private("____", "_my_func") 294 True 295 >>> is_private("someclass", "__init__") 296 False 297 >>> is_private("sometypo", "__init_") 298 True 299 >>> is_private("x.y.z", "_") 300 True 301 >>> is_private("_x.y.z", "__") 302 False 303 >>> is_private("", "") # senseless but consistent 304 False 305 """ 306 warnings.warn("is_private is deprecated; it wasn't useful; " 307 "examine DocTestFinder.find() lists instead", 308 DeprecationWarning, stacklevel=2) 309 return base[:1] == "_" and not base[:2] == "__" == base[-2:] 310 311def _extract_future_flags(globs): 312 """ 313 Return the compiler-flags associated with the future features that 314 have been imported into the given namespace (globs). 315 """ 316 flags = 0 317 for fname in __future__.all_feature_names: 318 feature = globs.get(fname, None) 319 if feature is getattr(__future__, fname): 320 flags |= feature.compiler_flag 321 return flags 322 323def _normalize_module(module, depth=2): 324 """ 325 Return the module specified by `module`. In particular: 326 - If `module` is a module, then return module. 327 - If `module` is a string, then import and return the 328 module with that name. 329 - If `module` is None, then return the calling module. 330 The calling module is assumed to be the module of 331 the stack frame at the given depth in the call stack. 332 """ 333 if inspect.ismodule(module): 334 return module 335 elif isinstance(module, (str, unicode)): 336 return __import__(module, globals(), locals(), ["*"]) 337 elif module is None: 338 return sys.modules[sys._getframe(depth).f_globals['__name__']] 339 else: 340 raise TypeError("Expected a module, string, or None") 341 342def _tag_msg(tag, msg, indent=' '): 343 """ 344 Return a string that displays a tag-and-message pair nicely, 345 keeping the tag and its message on the same line when that 346 makes sense. If the message is displayed on separate lines, 347 then `indent` is added to the beginning of each line. 348 """ 349 # If the message doesn't end in a newline, then add one. 350 if msg[-1:] != '\n': 351 msg += '\n' 352 # If the message is short enough, and contains no internal 353 # newlines, then display it on the same line as the tag. 354 # Otherwise, display the tag on its own line. 355 if (len(tag) + len(msg) < 75 and 356 msg.find('\n', 0, len(msg)-1) == -1): 357 return '%s: %s' % (tag, msg) 358 else: 359 msg = '\n'.join([indent+l for l in msg[:-1].split('\n')]) 360 return '%s:\n%s\n' % (tag, msg) 361 362def _exception_traceback(exc_info): 363 """ 364 Return a string containing a traceback message for the given 365 exc_info tuple (as returned by sys.exc_info()). 366 """ 367 # Get a traceback message. 368 excout = StringIO() 369 exc_type, exc_val, exc_tb = exc_info 370 traceback.print_exception(exc_type, exc_val, exc_tb, file=excout) 371 return excout.getvalue() 372 373# Override some StringIO methods. 374class _SpoofOut(StringIO): 375 def getvalue(self): 376 result = StringIO.getvalue(self) 377 # If anything at all was written, make sure there's a trailing 378 # newline. There's no way for the expected output to indicate 379 # that a trailing newline is missing. 380 if result and not result.endswith("\n"): 381 result += "\n" 382 # Prevent softspace from screwing up the next test case, in 383 # case they used print with a trailing comma in an example. 384 if hasattr(self, "softspace"): 385 del self.softspace 386 return result 387 388 def truncate(self, size=None): 389 StringIO.truncate(self, size) 390 if hasattr(self, "softspace"): 391 del self.softspace 392 393# Worst-case linear-time ellipsis matching. 394def ellipsis_match(want, got): 395 """ 396 Essentially the only subtle case: 397 >>> ellipsis_match('aa...aa', 'aaa') 398 False 399 """ 400 if ELLIPSIS_MARKER not in want: 401 return want == got 402 # Remove \n from ...\n, else the newline will be required, 403 # and (for example) ... on a line by itself can't match 404 # nothing gracefully. 405 want = want.replace(ELLIPSIS_MARKER + '\n', ELLIPSIS_MARKER) 406 407 # Find "the real" strings. 408 ws = want.split(ELLIPSIS_MARKER) 409 assert len(ws) >= 2 410 411 # Deal with exact matches possibly needed at one or both ends. 412 startpos, endpos = 0, len(got) 413 w = ws[0] 414 if w: # starts with exact match 415 if got.startswith(w): 416 startpos = len(w) 417 del ws[0] 418 else: 419 return False 420 w = ws[-1] 421 if w: # ends with exact match 422 if got.endswith(w): 423 endpos -= len(w) 424 del ws[-1] 425 else: 426 return False 427 428 if startpos > endpos: 429 # Exact end matches required more characters than we have, as in 430 # ellipsis_match('aa...aa', 'aaa') 431 return False 432 433 # For the rest, we only need to find the leftmost non-overlapping 434 # match for each piece. If there's no overall match that way alone, 435 # there's no overall match period. 436 for w in ws: 437 # w may be '' at times, if there are consecutive ellipses, or 438 # due to an ellipsis at the start or end of `want`. That's OK. 439 # Search for an empty string succeeds, and doesn't change startpos. 440 startpos = got.find(w, startpos, endpos) 441 if startpos < 0: 442 return False 443 startpos += len(w) 444 445 return True 446 447###################################################################### 448## 2. Example & DocTest 449###################################################################### 450## - An "example" is a <source, want> pair, where "source" is a 451## fragment of source code, and "want" is the expected output for 452## "source." The Example class also includes information about 453## where the example was extracted from. 454## 455## - A "doctest" is a collection of examples, typically extracted from 456## a string (such as an object's docstring). The DocTest class also 457## includes information about where the string was extracted from. 458 459class Example: 460 """ 461 A single doctest example, consisting of source code and expected 462 output. `Example` defines the following attributes: 463 464 - source: A single Python statement, always ending with a newline. 465 The constructor adds a newline if needed. 466 467 - want: The expected output from running the source code (either 468 from stdout, or a traceback in case of exception). `want` ends 469 with a newline unless it's empty, in which case it's an empty 470 string. The constructor adds a newline if needed. 471 472 - lineno: The line number within the DocTest string containing 473 this Example where the Example begins. This line number is 474 zero-based, with respect to the beginning of the DocTest. 475 476 - indent: The example's indentation in the DocTest string. 477 I.e., the number of space characters that preceed the 478 example's first prompt. 479 480 - options: A dictionary mapping from option flags to True or 481 False, which is used to override default options for this 482 example. Any option flags not contained in this dictionary 483 are left at their default value (as specified by the 484 DocTestRunner's optionflags). By default, no options are set. 485 """ 486 def __init__(self, source, want, lineno, indent=0, options=None): 487 # Normalize inputs. 488 if not source.endswith('\n'): 489 source += '\n' 490 if want and not want.endswith('\n'): 491 want += '\n' 492 # Store properties. 493 self.source = source 494 self.want = want 495 self.lineno = lineno 496 self.indent = indent 497 if options is None: options = {} 498 self.options = options 499 500class DocTest: 501 """ 502 A collection of doctest examples that should be run in a single 503 namespace. Each `DocTest` defines the following attributes: 504 505 - examples: the list of examples. 506 507 - globs: The namespace (aka globals) that the examples should 508 be run in. 509 510 - name: A name identifying the DocTest (typically, the name of 511 the object whose docstring this DocTest was extracted from). 512 513 - filename: The name of the file that this DocTest was extracted 514 from, or `None` if the filename is unknown. 515 516 - lineno: The line number within filename where this DocTest 517 begins, or `None` if the line number is unavailable. This 518 line number is zero-based, with respect to the beginning of 519 the file. 520 521 - docstring: The string that the examples were extracted from, 522 or `None` if the string is unavailable. 523 """ 524 def __init__(self, examples, globs, name, filename, lineno, docstring): 525 """ 526 Create a new DocTest containing the given examples. The 527 DocTest's globals are initialized with a copy of `globs`. 528 """ 529 assert not isinstance(examples, basestring), \ 530 "DocTest no longer accepts str; use DocTestParser instead" 531 self.examples = examples 532 self.docstring = docstring 533 self.globs = globs.copy() 534 self.name = name 535 self.filename = filename 536 self.lineno = lineno 537 538 def __repr__(self): 539 if len(self.examples) == 0: 540 examples = 'no examples' 541 elif len(self.examples) == 1: 542 examples = '1 example' 543 else: 544 examples = '%d examples' % len(self.examples) 545 return ('<DocTest %s from %s:%s (%s)>' % 546 (self.name, self.filename, self.lineno, examples)) 547 548 549 # This lets us sort tests by name: 550 def __cmp__(self, other): 551 if not isinstance(other, DocTest): 552 return -1 553 return cmp((self.name, self.filename, self.lineno, id(self)), 554 (other.name, other.filename, other.lineno, id(other))) 555 556###################################################################### 557## 3. DocTestParser 558###################################################################### 559 560class DocTestParser: 561 """ 562 A class used to parse strings containing doctest examples. 563 """ 564 # This regular expression is used to find doctest examples in a 565 # string. It defines three groups: `source` is the source code 566 # (including leading indentation and prompts); `indent` is the 567 # indentation of the first (PS1) line of the source code; and 568 # `want` is the expected output (including leading indentation). 569 _EXAMPLE_RE = re.compile(r''' 570 # Source consists of a PS1 line followed by zero or more PS2 lines. 571 (?P<source> 572 (?:^(?P<indent> [ ]*) >>> .*) # PS1 line 573 (?:\n [ ]* \.\.\. .*)*) # PS2 lines 574 \n? 575 # Want consists of any non-blank lines that do not start with PS1. 576 (?P<want> (?:(?![ ]*$) # Not a blank line 577 (?![ ]*>>>) # Not a line starting with PS1 578 .*$\n? # But any other line 579 )*) 580 ''', re.MULTILINE | re.VERBOSE) 581 582 # A callable returning a true value iff its argument is a blank line 583 # or contains a single comment. 584 _IS_BLANK_OR_COMMENT = re.compile(r'^[ ]*(#.*)?$').match 585 586 def get_doctest(self, string, globs, name, filename, lineno): 587 """ 588 Extract all doctest examples from the given string, and 589 collect them into a `DocTest` object. 590 591 `globs`, `name`, `filename`, and `lineno` are attributes for 592 the new `DocTest` object. See the documentation for `DocTest` 593 for more information. 594 """ 595 return DocTest(self.get_examples(string, name), globs, 596 name, filename, lineno, string) 597 598 def get_examples(self, string, name='<string>'): 599 """ 600 Extract all doctest examples from the given string, and return 601 them as a list of `Example` objects. Line numbers are 602 0-based, because it's most common in doctests that nothing 603 interesting appears on the same line as opening triple-quote, 604 and so the first interesting line is called \"line 1\" then. 605 606 The optional argument `name` is a name identifying this 607 string, and is only used for error messages. 608 609 >>> text = ''' 610 ... >>> x, y = 2, 3 # no output expected 611 ... >>> if 1: 612 ... ... print x 613 ... ... print y 614 ... 2 615 ... 3 616 ... 617 ... Some text. 618 ... >>> x+y 619 ... 5 620 ... ''' 621 >>> for x in DocTestParser().get_examples(text): 622 ... print (x.source, x.want, x.lineno) 623 ('x, y = 2, 3 # no output expected\\n', '', 1) 624 ('if 1:\\n print x\\n print y\\n', '2\\n3\\n', 2) 625 ('x+y\\n', '5\\n', 9) 626 """ 627 examples = [] 628 charno, lineno = 0, 0 629 # Find all doctest examples in the string: 630 for m in self._EXAMPLE_RE.finditer(string.expandtabs()): 631 # Update lineno (lines before this example) 632 lineno += string.count('\n', charno, m.start()) 633 # Extract source/want from the regexp match. 634 (source, want) = self._parse_example(m, name, lineno) 635 # Extract extra options from the source. 636 options = self._find_options(source, name, lineno) 637 # Create an Example, and add it to the list. 638 if not self._IS_BLANK_OR_COMMENT(source): 639 examples.append( Example(source, want, lineno, 640 len(m.group('indent')), options) ) 641 # Update lineno (lines inside this example) 642 lineno += string.count('\n', m.start(), m.end()) 643 # Update charno. 644 charno = m.end() 645 return examples 646 647 def get_program(self, string, name="<string>"): 648 """ 649 Return an executable program from the given string, as a string. 650 651 The format of this isn't rigidly defined. In general, doctest 652 examples become the executable statements in the result, and 653 their expected outputs become comments, preceded by an \"#Expected:\" 654 comment. Everything else (text, comments, everything not part of 655 a doctest test) is also placed in comments. 656 657 The optional argument `name` is a name identifying this 658 string, and is only used for error messages. 659 660 >>> text = ''' 661 ... >>> x, y = 2, 3 # no output expected 662 ... >>> if 1: 663 ... ... print x 664 ... ... print y 665 ... 2 666 ... 3 667 ... 668 ... Some text. 669 ... >>> x+y 670 ... 5 671 ... ''' 672 >>> print DocTestParser().get_program(text) 673 x, y = 2, 3 # no output expected 674 if 1: 675 print x 676 print y 677 # Expected: 678 ## 2 679 ## 3 680 # 681 # Some text. 682 x+y 683 # Expected: 684 ## 5 685 """ 686 string = string.expandtabs() 687 # If all lines begin with the same indentation, then strip it. 688 min_indent = self._min_indent(string) 689 if min_indent > 0: 690 string = '\n'.join([l[min_indent:] for l in string.split('\n')]) 691 692 output = [] 693 charnum, lineno = 0, 0 694 # Find all doctest examples in the string: 695 for m in self._EXAMPLE_RE.finditer(string.expandtabs()): 696 # Add any text before this example, as a comment. 697 if m.start() > charnum: 698 lines = string[charnum:m.start()-1].split('\n') 699 output.extend([self._comment_line(l) for l in lines]) 700 lineno += len(lines) 701 702 # Extract source/want from the regexp match. 703 (source, want) = self._parse_example(m, name, lineno) 704 # Display the source 705 output.append(source) 706 # Display the expected output, if any 707 if want: 708 output.append('# Expected:') 709 output.extend(['## '+l for l in want.split('\n')]) 710 711 # Update the line number & char number. 712 lineno += string.count('\n', m.start(), m.end()) 713 charnum = m.end() 714 # Add any remaining text, as comments. 715 output.extend([self._comment_line(l) 716 for l in string[charnum:].split('\n')]) 717 # Trim junk on both ends. 718 while output and output[-1] == '#': 719 output.pop() 720 while output and output[0] == '#': 721 output.pop(0) 722 # Combine the output, and return it. 723 return '\n'.join(output) 724 725 def _parse_example(self, m, name, lineno): 726 """ 727 Given a regular expression match from `_EXAMPLE_RE` (`m`), 728 return a pair `(source, want)`, where `source` is the matched 729 example's source code (with prompts and indentation stripped); 730 and `want` is the example's expected output (with indentation 731 stripped). 732 733 `name` is the string's name, and `lineno` is the line number 734 where the example starts; both are used for error messages. 735 """ 736 # Get the example's indentation level. 737 indent = len(m.group('indent')) 738 739 # Divide source into lines; check that they're properly 740 # indented; and then strip their indentation & prompts. 741 source_lines = m.group('source').split('\n') 742 self._check_prompt_blank(source_lines, indent, name, lineno) 743 self._check_prefix(source_lines[1:], ' '*indent+'.', name, lineno) 744 source = '\n'.join([sl[indent+4:] for sl in source_lines]) 745 746 # Divide want into lines; check that it's properly 747 # indented; and then strip the indentation. 748 want_lines = m.group('want').rstrip().split('\n') 749 self._check_prefix(want_lines, ' '*indent, name, 750 lineno+len(source_lines)) 751 want = '\n'.join([wl[indent:] for wl in want_lines]) 752 753 return source, want 754 755 # This regular expression looks for option directives in the 756 # source code of an example. Option directives are comments 757 # starting with "doctest:". Warning: this may give false 758 # positives for string-literals that contain the string 759 # "#doctest:". Eliminating these false positives would require 760 # actually parsing the string; but we limit them by ignoring any 761 # line containing "#doctest:" that is *followed* by a quote mark. 762 _OPTION_DIRECTIVE_RE = re.compile(r'#\s*doctest:\s*([^\n\'"]*)$', 763 re.MULTILINE) 764 765 def _find_options(self, source, name, lineno): 766 """ 767 Return a dictionary containing option overrides extracted from 768 option directives in the given source string. 769 770 `name` is the string's name, and `lineno` is the line number 771 where the example starts; both are used for error messages. 772 """ 773 options = {} 774 # (note: with the current regexp, this will match at most once:) 775 for m in self._OPTION_DIRECTIVE_RE.finditer(source): 776 option_strings = m.group(1).replace(',', ' ').split() 777 for option in option_strings: 778 if (option[0] not in '+-' or 779 option[1:] not in OPTIONFLAGS_BY_NAME): 780 raise ValueError('line %r of the doctest for %s ' 781 'has an invalid option: %r' % 782 (lineno+1, name, option)) 783 flag = OPTIONFLAGS_BY_NAME[option[1:]] 784 options[flag] = (option[0] == '+') 785 if options and self._IS_BLANK_OR_COMMENT(source): 786 raise ValueError('line %r of the doctest for %s has an option ' 787 'directive on a line with no example: %r' % 788 (lineno, name, source)) 789 return options 790 791 # This regular expression finds the indentation of every non-blank 792 # line in a string. 793 _INDENT_RE = re.compile('^([ ]+)(?=\S)', re.MULTILINE) 794 795 def _min_indent(self, s): 796 "Return the minimum indentation of any non-blank line in `s`" 797 return min([len(indent) for indent in self._INDENT_RE.findall(s)]) 798 799 def _comment_line(self, line): 800 "Return a commented form of the given line" 801 line = line.rstrip() 802 if line: 803 return '# '+line 804 else: 805 return '#' 806 807 def _check_prompt_blank(self, lines, indent, name, lineno): 808 """ 809 Given the lines of a source string (including prompts and 810 leading indentation), check to make sure that every prompt is 811 followed by a space character. If any line is not followed by 812 a space character, then raise ValueError. 813 """ 814 for i, line in enumerate(lines): 815 if len(line) >= indent+4 and line[indent+3] != ' ': 816 raise ValueError('line %r of the docstring for %s ' 817 'lacks blank after %s: %r' % 818 (lineno+i+1, name, 819 line[indent:indent+3], line)) 820 821 def _check_prefix(self, lines, prefix, name, lineno): 822 """ 823 Check that every line in the given list starts with the given 824 prefix; if any line does not, then raise a ValueError. 825 """ 826 for i, line in enumerate(lines): 827 if line and not line.startswith(prefix): 828 raise ValueError('line %r of the docstring for %s has ' 829 'inconsistent leading whitespace: %r' % 830 (lineno+i+1, name, line)) 831 832 833###################################################################### 834## 4. DocTest Finder 835###################################################################### 836 837class DocTestFinder: 838 """ 839 A class used to extract the DocTests that are relevant to a given 840 object, from its docstring and the docstrings of its contained 841 objects. Doctests can currently be extracted from the following 842 object types: modules, functions, classes, methods, staticmethods, 843 classmethods, and properties. 844 """ 845 846 def __init__(self, verbose=False, parser=DocTestParser(), 847 recurse=True, _namefilter=None): 848 """ 849 Create a new doctest finder. 850 851 The optional argument `parser` specifies a class or 852 function that should be used to create new DocTest objects (or 853 objects that implement the same interface as DocTest). The 854 signature for this factory function should match the signature 855 of the DocTest constructor. 856 857 If the optional argument `recurse` is false, then `find` will 858 only examine the given object, and not any contained objects. 859 """ 860 self._parser = parser 861 self._verbose = verbose 862 self._recurse = recurse 863 # _namefilter is undocumented, and exists only for temporary backward- 864 # compatibility support of testmod's deprecated isprivate mess. 865 self._namefilter = _namefilter 866 867 def find(self, obj, name=None, module=None, globs=None, 868 extraglobs=None): 869 """ 870 Return a list of the DocTests that are defined by the given 871 object's docstring, or by any of its contained objects' 872 docstrings. 873 874 The optional parameter `module` is the module that contains 875 the given object. If the module is not specified or is None, then 876 the test finder will attempt to automatically determine the 877 correct module. The object's module is used: 878 879 - As a default namespace, if `globs` is not specified. 880 - To prevent the DocTestFinder from extracting DocTests 881 from objects that are imported from other modules. 882 - To find the name of the file containing the object. 883 - To help find the line number of the object within its 884 file. 885 886 Contained objects whose module does not match `module` are ignored. 887 888 If `module` is False, no attempt to find the module will be made. 889 This is obscure, of use mostly in tests: if `module` is False, or 890 is None but cannot be found automatically, then all objects are 891 considered to belong to the (non-existent) module, so all contained 892 objects will (recursively) be searched for doctests. 893 894 The globals for each DocTest is formed by combining `globs` 895 and `extraglobs` (bindings in `extraglobs` override bindings 896 in `globs`). A new copy of the globals dictionary is created 897 for each DocTest. If `globs` is not specified, then it 898 defaults to the module's `__dict__`, if specified, or {} 899 otherwise. If `extraglobs` is not specified, then it defaults 900 to {}. 901 902 """ 903 # If name was not specified, then extract it from the object. 904 if name is None: 905 name = getattr(obj, '__name__', None) 906 if name is None: 907 raise ValueError("DocTestFinder.find: name must be given " 908 "when obj.__name__ doesn't exist: %r" % 909 (type(obj),)) 910 911 # Find the module that contains the given object (if obj is 912 # a module, then module=obj.). Note: this may fail, in which 913 # case module will be None. 914 if module is False: 915 module = None 916 elif module is None: 917 module = inspect.getmodule(obj) 918 919 # Read the module's source code. This is used by 920 # DocTestFinder._find_lineno to find the line number for a 921 # given object's docstring. 922 try: 923 file = inspect.getsourcefile(obj) or inspect.getfile(obj) 924 source_lines = linecache.getlines(file) 925 if not source_lines: 926 source_lines = None 927 except TypeError: 928 source_lines = None 929 930 # Initialize globals, and merge in extraglobs. 931 if globs is None: 932 if module is None: 933 globs = {} 934 else: 935 globs = module.__dict__.copy() 936 else: 937 globs = globs.copy() 938 if extraglobs is not None: 939 globs.update(extraglobs) 940 941 # Recursively expore `obj`, extracting DocTests. 942 tests = [] 943 self._find(tests, obj, name, module, source_lines, globs, {}) 944 return tests 945 946 def _filter(self, obj, prefix, base): 947 """ 948 Return true if the given object should not be examined. 949 """ 950 return (self._namefilter is not None and 951 self._namefilter(prefix, base)) 952 953 def _from_module(self, module, object): 954 """ 955 Return true if the given object is defined in the given 956 module. 957 """ 958 if module is None: 959 return True 960 elif inspect.isfunction(object): 961 return module.__dict__ is object.func_globals 962 elif inspect.isclass(object): 963 return module.__name__ == object.__module__ 964 elif inspect.getmodule(object) is not None: 965 return module is inspect.getmodule(object) 966 elif hasattr(object, '__module__'): 967 return module.__name__ == object.__module__ 968 elif isinstance(object, property): 969 return True # [XX] no way not be sure. 970 else: 971 raise ValueError("object must be a class or function") 972 973 def _find(self, tests, obj, name, module, source_lines, globs, seen): 974 """ 975 Find tests for the given object and any contained objects, and 976 add them to `tests`. 977 """ 978 if self._verbose: 979 print 'Finding tests in %s' % name 980 981 # If we've already processed this object, then ignore it. 982 if id(obj) in seen: 983 return 984 seen[id(obj)] = 1 985 986 # Find a test for this object, and add it to the list of tests. 987 test = self._get_test(obj, name, module, globs, source_lines) 988 if test is not None: 989 tests.append(test) 990 991 # Look for tests in a module's contained objects. 992 if inspect.ismodule(obj) and self._recurse: 993 for valname, val in obj.__dict__.items(): 994 # Check if this contained object should be ignored. 995 if self._filter(val, name, valname): 996 continue 997 valname = '%s.%s' % (name, valname) 998 # Recurse to functions & classes. 999 if ((inspect.isfunction(val) or inspect.isclass(val)) and 1000 self._from_module(module, val)): 1001 self._find(tests, val, valname, module, source_lines, 1002 globs, seen) 1003 1004 # Look for tests in a module's __test__ dictionary. 1005 if inspect.ismodule(obj) and self._recurse: 1006 for valname, val in getattr(obj, '__test__', {}).items(): 1007 if not isinstance(valname, basestring): 1008 raise ValueError("DocTestFinder.find: __test__ keys " 1009 "must be strings: %r" % 1010 (type(valname),)) 1011 if not (inspect.isfunction(val) or inspect.isclass(val) or 1012 inspect.ismethod(val) or inspect.ismodule(val) or 1013 isinstance(val, basestring)): 1014 raise ValueError("DocTestFinder.find: __test__ values " 1015 "must be strings, functions, methods, " 1016 "classes, or modules: %r" % 1017 (type(val),)) 1018 valname = '%s.%s' % (name, valname) 1019 self._find(tests, val, valname, module, source_lines, 1020 globs, seen) 1021 1022 # Look for tests in a class's contained objects. 1023 if inspect.isclass(obj) and self._recurse: 1024 for valname, val in obj.__dict__.items(): 1025 # Check if this contained object should be ignored. 1026 if self._filter(val, name, valname): 1027 continue 1028 # Special handling for staticmethod/classmethod. 1029 if isinstance(val, staticmethod): 1030 val = getattr(obj, valname) 1031 if isinstance(val, classmethod): 1032 val = getattr(obj, valname).im_func 1033 1034 # Recurse to methods, properties, and nested classes. 1035 if ((inspect.isfunction(val) or inspect.isclass(val) or 1036 isinstance(val, property)) and 1037 self._from_module(module, val)): 1038 valname = '%s.%s' % (name, valname) 1039 self._find(tests, val, valname, module, source_lines, 1040 globs, seen) 1041 1042 def _get_test(self, obj, name, module, globs, source_lines): 1043 """ 1044 Return a DocTest for the given object, if it defines a docstring; 1045 otherwise, return None. 1046 """ 1047 # Extract the object's docstring. If it doesn't have one, 1048 # then return None (no test for this object). 1049 if isinstance(obj, basestring): 1050 docstring = obj 1051 else: 1052 try: 1053 if obj.__doc__ is None: 1054 return None 1055 docstring = str(obj.__doc__) 1056 except (TypeError, AttributeError): 1057 return None 1058 1059 # Don't bother if the docstring is empty. 1060 if not docstring: 1061 return None 1062 1063 # Find the docstring's location in the file. 1064 lineno = self._find_lineno(obj, source_lines) 1065 1066 # Return a DocTest for this object. 1067 if module is None: 1068 filename = None 1069 else: 1070 filename = getattr(module, '__file__', module.__name__) 1071 return self._parser.get_doctest(docstring, globs, name, 1072 filename, lineno) 1073 1074 def _find_lineno(self, obj, source_lines): 1075 """ 1076 Return a line number of the given object's docstring. Note: 1077 this method assumes that the object has a docstring. 1078 """ 1079 lineno = None 1080 1081 # Find the line number for modules. 1082 if inspect.ismodule(obj): 1083 lineno = 0 1084 1085 # Find the line number for classes. 1086 # Note: this could be fooled if a class is defined multiple 1087 # times in a single file. 1088 if inspect.isclass(obj): 1089 if source_lines is None: 1090 return None 1091 pat = re.compile(r'^\s*class\s*%s\b' % 1092 getattr(obj, '__name__', '-')) 1093 for i, line in enumerate(source_lines): 1094 if pat.match(line): 1095 lineno = i 1096 break 1097 1098 # Find the line number for functions & methods. 1099 if inspect.ismethod(obj): obj = obj.im_func 1100 if inspect.isfunction(obj): obj = obj.func_code 1101 if inspect.istraceback(obj): obj = obj.tb_frame 1102 if inspect.isframe(obj): obj = obj.f_code 1103 if inspect.iscode(obj): 1104 lineno = getattr(obj, 'co_firstlineno', None)-1 1105 1106 # Find the line number where the docstring starts. Assume 1107 # that it's the first line that begins with a quote mark. 1108 # Note: this could be fooled by a multiline function 1109 # signature, where a continuation line begins with a quote 1110 # mark. 1111 if lineno is not None: 1112 if source_lines is None: 1113 return lineno+1 1114 pat = re.compile('(^|.*:)\s*\w*("|\')') 1115 for lineno in range(lineno, len(source_lines)): 1116 if pat.match(source_lines[lineno]): 1117 return lineno 1118 1119 # We couldn't find the line number. 1120 return None 1121 1122###################################################################### 1123## 5. DocTest Runner 1124###################################################################### 1125 1126class DocTestRunner: 1127 """ 1128 A class used to run DocTest test cases, and accumulate statistics. 1129 The `run` method is used to process a single DocTest case. It 1130 returns a tuple `(f, t)`, where `t` is the number of test cases 1131 tried, and `f` is the number of test cases that failed. 1132 1133 >>> tests = DocTestFinder().find(_TestClass) 1134 >>> runner = DocTestRunner(verbose=False) 1135 >>> for test in tests: 1136 ... print runner.run(test) 1137 (0, 2) 1138 (0, 1) 1139 (0, 2) 1140 (0, 2) 1141 1142 The `summarize` method prints a summary of all the test cases that 1143 have been run by the runner, and returns an aggregated `(f, t)` 1144 tuple: 1145 1146 >>> runner.summarize(verbose=1) 1147 4 items passed all tests: 1148 2 tests in _TestClass 1149 2 tests in _TestClass.__init__ 1150 2 tests in _TestClass.get 1151 1 tests in _TestClass.square 1152 7 tests in 4 items. 1153 7 passed and 0 failed. 1154 Test passed. 1155 (0, 7) 1156 1157 The aggregated number of tried examples and failed examples is 1158 also available via the `tries` and `failures` attributes: 1159 1160 >>> runner.tries 1161 7 1162 >>> runner.failures 1163 0 1164 1165 The comparison between expected outputs and actual outputs is done 1166 by an `OutputChecker`. This comparison may be customized with a 1167 number of option flags; see the documentation for `testmod` for 1168 more information. If the option flags are insufficient, then the 1169 comparison may also be customized by passing a subclass of 1170 `OutputChecker` to the constructor. 1171 1172 The test runner's display output can be controlled in two ways. 1173 First, an output function (`out) can be passed to 1174 `TestRunner.run`; this function will be called with strings that 1175 should be displayed. It defaults to `sys.stdout.write`. If 1176 capturing the output is not sufficient, then the display output 1177 can be also customized by subclassing DocTestRunner, and 1178 overriding the methods `report_start`, `report_success`, 1179 `report_unexpected_exception`, and `report_failure`. 1180 """ 1181 # This divider string is used to separate failure messages, and to 1182 # separate sections of the summary. 1183 DIVIDER = "*" * 70 1184 1185 def __init__(self, checker=None, verbose=None, optionflags=0): 1186 """ 1187 Create a new test runner. 1188 1189 Optional keyword arg `checker` is the `OutputChecker` that 1190 should be used to compare the expected outputs and actual 1191 outputs of doctest examples. 1192 1193 Optional keyword arg 'verbose' prints lots of stuff if true, 1194 only failures if false; by default, it's true iff '-v' is in 1195 sys.argv. 1196 1197 Optional argument `optionflags` can be used to control how the 1198 test runner compares expected output to actual output, and how 1199 it displays failures. See the documentation for `testmod` for 1200 more information. 1201 """ 1202 self._checker = checker or OutputChecker() 1203 if verbose is None: 1204 verbose = '-v' in sys.argv 1205 self._verbose = verbose 1206 self.optionflags = optionflags 1207 1208 # Keep track of the examples we've run. 1209 self.tries = 0 1210 self.failures = 0 1211 self._name2ft = {} 1212 1213 # Create a fake output target for capturing doctest output. 1214 self._fakeout = _SpoofOut() 1215 1216 #///////////////////////////////////////////////////////////////// 1217 # Reporting methods 1218 #///////////////////////////////////////////////////////////////// 1219 1220 def report_start(self, out, test, example): 1221 """ 1222 Report that the test runner is about to process the given 1223 example. (Only displays a message if verbose=True) 1224 """ 1225 if self._verbose: 1226 out(_tag_msg("Trying", example.source) + 1227 _tag_msg("Expecting", example.want or "nothing")) 1228 1229 def report_success(self, out, test, example, got): 1230 """ 1231 Report that the given example ran successfully. (Only 1232 displays a message if verbose=True) 1233 """ 1234 if self._verbose: 1235 out("ok\n") 1236 1237 def report_failure(self, out, test, example, got): 1238 """ 1239 Report that the given example failed. 1240 """ 1241 # Print an error message. 1242 out(self._failure_header(test, example) + 1243 self._checker.output_difference(example.want, got, 1244 self.optionflags)) 1245 1246 def report_unexpected_exception(self, out, test, example, exc_info): 1247 """ 1248 Report that the given example raised an unexpected exception. 1249 """ 1250 out(self._failure_header(test, example) + 1251 _tag_msg("Exception raised", _exception_traceback(exc_info))) 1252 1253 def _failure_header(self, test, example): 1254 s = (self.DIVIDER + "\n" + 1255 _tag_msg("Failure in example", example.source)) 1256 if test.filename is None: 1257 # [XX] I'm not putting +1 here, to give the same output 1258 # as the old version. But I think it *should* go here. 1259 return s + ("from line #%s of %s\n" % 1260 (example.lineno, test.name)) 1261 elif test.lineno is None: 1262 return s + ("from line #%s of %s in %s\n" % 1263 (example.lineno+1, test.name, test.filename)) 1264 else: 1265 lineno = test.lineno+example.lineno+1 1266 return s + ("from line #%s of %s (%s)\n" % 1267 (lineno, test.filename, test.name)) 1268 1269 #///////////////////////////////////////////////////////////////// 1270 # DocTest Running 1271 #///////////////////////////////////////////////////////////////// 1272 1273 # A regular expression for handling `want` strings that contain 1274 # expected exceptions. It divides `want` into three pieces: 1275 # - the pre-exception output (`want`) 1276 # - the traceback header line (`hdr`) 1277 # - the exception message (`msg`), as generated by 1278 # traceback.format_exception_only() 1279 # `msg` may have multiple lines. We assume/require that the 1280 # exception message is the first non-indented line starting with a word 1281 # character following the traceback header line. 1282 _EXCEPTION_RE = re.compile(r""" 1283 (?P<want> .*?) # suck up everything until traceback header 1284 # Grab the traceback header. Different versions of Python have 1285 # said different things on the first traceback line. 1286 ^(?P<hdr> Traceback\ \( 1287 (?: most\ recent\ call\ last 1288 | innermost\ last 1289 ) \) : 1290 ) 1291 \s* $ # toss trailing whitespace on traceback header 1292 .*? # don't blink: absorb stuff until a line *starts* with \w 1293 ^ (?P<msg> \w+ .*) 1294 """, re.VERBOSE | re.MULTILINE | re.DOTALL) 1295 1296 def __run(self, test, compileflags, out): 1297 """ 1298 Run the examples in `test`. Write the outcome of each example 1299 with one of the `DocTestRunner.report_*` methods, using the 1300 writer function `out`. `compileflags` is the set of compiler 1301 flags that should be used to execute examples. Return a tuple 1302 `(f, t)`, where `t` is the number of examples tried, and `f` 1303 is the number of examples that failed. The examples are run 1304 in the namespace `test.globs`. 1305 """ 1306 # Keep track of the number of failures and tries. 1307 failures = tries = 0 1308 1309 # Save the option flags (since option directives can be used 1310 # to modify them). 1311 original_optionflags = self.optionflags 1312 1313 # Process each example. 1314 for example in test.examples: 1315 # Merge in the example's options. 1316 self.optionflags = original_optionflags 1317 if example.options: 1318 for (optionflag, val) in example.options.items(): 1319 if val: 1320 self.optionflags |= optionflag 1321 else: 1322 self.optionflags &= ~optionflag 1323 1324 # Record that we started this example. 1325 tries += 1 1326 self.report_start(out, test, example) 1327 1328 # Run the example in the given context (globs), and record 1329 # any exception that gets raised. (But don't intercept 1330 # keyboard interrupts.) 1331 try: 1332 # Don't blink! This is where the user's code gets run. 1333 exec compile(example.source, "<string>", "single", 1334 compileflags, 1) in test.globs 1335 exception = None 1336 except KeyboardInterrupt: 1337 raise 1338 except: 1339 exception = sys.exc_info() 1340 1341 got = self._fakeout.getvalue() # the actual output 1342 self._fakeout.truncate(0) 1343 1344 # If the example executed without raising any exceptions, 1345 # then verify its output and report its outcome. 1346 if exception is None: 1347 if self._checker.check_output(example.want, got, 1348 self.optionflags): 1349 self.report_success(out, test, example, got) 1350 else: 1351 self.report_failure(out, test, example, got) 1352 failures += 1 1353 1354 # If the example raised an exception, then check if it was 1355 # expected. 1356 else: 1357 exc_info = sys.exc_info() 1358 exc_msg = traceback.format_exception_only(*exc_info[:2])[-1] 1359 1360 # Search the `want` string for an exception. If we don't 1361 # find one, then report an unexpected exception. 1362 m = self._EXCEPTION_RE.match(example.want) 1363 if m is None: 1364 self.report_unexpected_exception(out, test, example, 1365 exc_info) 1366 failures += 1 1367 else: 1368 e_want, e_msg = m.group('want', 'msg') 1369 # The test passes iff the pre-exception output and 1370 # the exception description match the values given 1371 # in `want`. 1372 if (self._checker.check_output(e_want, got, 1373 self.optionflags) and 1374 self._checker.check_output(e_msg, exc_msg, 1375 self.optionflags)): 1376 self.report_success(out, test, example, 1377 got + _exception_traceback(exc_info)) 1378 else: 1379 self.report_failure(out, test, example, 1380 got + _exception_traceback(exc_info)) 1381 failures += 1 1382 1383 # Restore the option flags (in case they were modified) 1384 self.optionflags = original_optionflags 1385 1386 # Record and return the number of failures and tries. 1387 self.__record_outcome(test, failures, tries) 1388 return failures, tries 1389 1390 def __record_outcome(self, test, f, t): 1391 """ 1392 Record the fact that the given DocTest (`test`) generated `f` 1393 failures out of `t` tried examples. 1394 """ 1395 f2, t2 = self._name2ft.get(test.name, (0,0)) 1396 self._name2ft[test.name] = (f+f2, t+t2) 1397 self.failures += f 1398 self.tries += t 1399 1400 def run(self, test, compileflags=None, out=None, clear_globs=True): 1401 """ 1402 Run the examples in `test`, and display the results using the 1403 writer function `out`. 1404 1405 The examples are run in the namespace `test.globs`. If 1406 `clear_globs` is true (the default), then this namespace will 1407 be cleared after the test runs, to help with garbage 1408 collection. If you would like to examine the namespace after 1409 the test completes, then use `clear_globs=False`. 1410 1411 `compileflags` gives the set of flags that should be used by 1412 the Python compiler when running the examples. If not 1413 specified, then it will default to the set of future-import 1414 flags that apply to `globs`. 1415 1416 The output of each example is checked using 1417 `DocTestRunner.check_output`, and the results are formatted by 1418 the `DocTestRunner.report_*` methods. 1419 """ 1420 if compileflags is None: 1421 compileflags = _extract_future_flags(test.globs) 1422 1423 save_stdout = sys.stdout 1424 if out is None: 1425 out = save_stdout.write 1426 sys.stdout = self._fakeout 1427 1428 # Patch pdb.set_trace to restore sys.stdout, so that interactive 1429 # debugging output is visible (not still redirected to self._fakeout). 1430 # Note that we run "the real" pdb.set_trace (captured at doctest 1431 # import time) in our replacement. Because the current run() may 1432 # run another doctest (and so on), the current pdb.set_trace may be 1433 # our set_trace function, which changes sys.stdout. If we called 1434 # a chain of those, we wouldn't be left with the save_stdout 1435 # *this* run() invocation wants. 1436 def set_trace(): 1437 sys.stdout = save_stdout 1438 real_pdb_set_trace() 1439 1440 save_set_trace = pdb.set_trace 1441 pdb.set_trace = set_trace 1442 try: 1443 return self.__run(test, compileflags, out) 1444 finally: 1445 sys.stdout = save_stdout 1446 pdb.set_trace = save_set_trace 1447 if clear_globs: 1448 test.globs.clear() 1449 1450 #///////////////////////////////////////////////////////////////// 1451 # Summarization 1452 #///////////////////////////////////////////////////////////////// 1453 def summarize(self, verbose=None): 1454 """ 1455 Print a summary of all the test cases that have been run by 1456 this DocTestRunner, and return a tuple `(f, t)`, where `f` is 1457 the total number of failed examples, and `t` is the total 1458 number of tried examples. 1459 1460 The optional `verbose` argument controls how detailed the 1461 summary is. If the verbosity is not specified, then the 1462 DocTestRunner's verbosity is used. 1463 """ 1464 if verbose is None: 1465 verbose = self._verbose 1466 notests = [] 1467 passed = [] 1468 failed = [] 1469 totalt = totalf = 0 1470 for x in self._name2ft.items(): 1471 name, (f, t) = x 1472 assert f <= t 1473 totalt += t 1474 totalf += f 1475 if t == 0: 1476 notests.append(name) 1477 elif f == 0: 1478 passed.append( (name, t) ) 1479 else: 1480 failed.append(x) 1481 if verbose: 1482 if notests: 1483 print len(notests), "items had no tests:" 1484 notests.sort() 1485 for thing in notests: 1486 print " ", thing 1487 if passed: 1488 print len(passed), "items passed all tests:" 1489 passed.sort() 1490 for thing, count in passed: 1491 print " %3d tests in %s" % (count, thing) 1492 if failed: 1493 print self.DIVIDER 1494 print len(failed), "items had failures:" 1495 failed.sort() 1496 for thing, (f, t) in failed: 1497 print " %3d of %3d in %s" % (f, t, thing) 1498 if verbose: 1499 print totalt, "tests in", len(self._name2ft), "items." 1500 print totalt - totalf, "passed and", totalf, "failed." 1501 if totalf: 1502 print "***Test Failed***", totalf, "failures." 1503 elif verbose: 1504 print "Test passed." 1505 return totalf, totalt 1506 1507class OutputChecker: 1508 """ 1509 A class used to check the whether the actual output from a doctest 1510 example matches the expected output. `OutputChecker` defines two 1511 methods: `check_output`, which compares a given pair of outputs, 1512 and returns true if they match; and `output_difference`, which 1513 returns a string describing the differences between two outputs. 1514 """ 1515 def check_output(self, want, got, optionflags): 1516 """ 1517 Return True iff the actual output from an example (`got`) 1518 matches the expected output (`want`). These strings are 1519 always considered to match if they are identical; but 1520 depending on what option flags the test runner is using, 1521 several non-exact match types are also possible. See the 1522 documentation for `TestRunner` for more information about 1523 option flags. 1524 """ 1525 # Handle the common case first, for efficiency: 1526 # if they're string-identical, always return true. 1527 if got == want: 1528 return True 1529 1530 # The values True and False replaced 1 and 0 as the return 1531 # value for boolean comparisons in Python 2.3. 1532 if not (optionflags & DONT_ACCEPT_TRUE_FOR_1): 1533 if (got,want) == ("True\n", "1\n"): 1534 return True 1535 if (got,want) == ("False\n", "0\n"): 1536 return True 1537 1538 # <BLANKLINE> can be used as a special sequence to signify a 1539 # blank line, unless the DONT_ACCEPT_BLANKLINE flag is used. 1540 if not (optionflags & DONT_ACCEPT_BLANKLINE): 1541 # Replace <BLANKLINE> in want with a blank line. 1542 want = re.sub('(?m)^%s\s*?$' % re.escape(BLANKLINE_MARKER), 1543 '', want) 1544 # If a line in got contains only spaces, then remove the 1545 # spaces. 1546 got = re.sub('(?m)^\s*?$', '', got) 1547 if got == want: 1548 return True 1549 1550 # This flag causes doctest to ignore any differences in the 1551 # contents of whitespace strings. Note that this can be used 1552 # in conjunction with the ELLISPIS flag. 1553 if optionflags & NORMALIZE_WHITESPACE: 1554 got = ' '.join(got.split()) 1555 want = ' '.join(want.split()) 1556 if got == want: 1557 return True 1558 1559 # The ELLIPSIS flag says to let the sequence "..." in `want` 1560 # match any substring in `got`. 1561 if optionflags & ELLIPSIS: 1562 if ellipsis_match(want, got): 1563 return True 1564 1565 # We didn't find any match; return false. 1566 return False 1567 1568 def output_difference(self, want, got, optionflags): 1569 """ 1570 Return a string describing the differences between the 1571 expected output for an example (`want`) and the actual output 1572 (`got`). `optionflags` is the set of option flags used to 1573 compare `want` and `got`. `indent` is the indentation of the 1574 original example. 1575 """ 1576 # If <BLANKLINE>s are being used, then replace blank lines 1577 # with <BLANKLINE> in the actual output string. 1578 if not (optionflags & DONT_ACCEPT_BLANKLINE): 1579 got = re.sub('(?m)^[ ]*(?=\n)', BLANKLINE_MARKER, got) 1580 1581 # Check if we should use diff. Don't use diff if the actual 1582 # or expected outputs are too short, or if the expected output 1583 # contains an ellipsis marker. 1584 if ((optionflags & (UNIFIED_DIFF | CONTEXT_DIFF)) and 1585 want.count('\n') > 2 and got.count('\n') > 2 and 1586 not (optionflags & ELLIPSIS and '...' in want)): 1587 # Split want & got into lines. 1588 want_lines = [l+'\n' for l in want.split('\n')] 1589 got_lines = [l+'\n' for l in got.split('\n')] 1590 # Use difflib to find their differences. 1591 if optionflags & UNIFIED_DIFF: 1592 diff = difflib.unified_diff(want_lines, got_lines, n=2, 1593 fromfile='Expected', tofile='Got') 1594 kind = 'unified' 1595 elif optionflags & CONTEXT_DIFF: 1596 diff = difflib.context_diff(want_lines, got_lines, n=2, 1597 fromfile='Expected', tofile='Got') 1598 kind = 'context' 1599 else: 1600 assert 0, 'Bad diff option' 1601 # Remove trailing whitespace on diff output. 1602 diff = [line.rstrip() + '\n' for line in diff] 1603 return _tag_msg("Differences (" + kind + " diff)", 1604 ''.join(diff)) 1605 1606 # If we're not using diff, then simply list the expected 1607 # output followed by the actual output. 1608 return (_tag_msg("Expected", want or "Nothing") + 1609 _tag_msg("Got", got)) 1610 1611class DocTestFailure(Exception): 1612 """A DocTest example has failed in debugging mode. 1613 1614 The exception instance has variables: 1615 1616 - test: the DocTest object being run 1617 1618 - excample: the Example object that failed 1619 1620 - got: the actual output 1621 """ 1622 def __init__(self, test, example, got): 1623 self.test = test 1624 self.example = example 1625 self.got = got 1626 1627 def __str__(self): 1628 return str(self.test) 1629 1630class UnexpectedException(Exception): 1631 """A DocTest example has encountered an unexpected exception 1632 1633 The exception instance has variables: 1634 1635 - test: the DocTest object being run 1636 1637 - excample: the Example object that failed 1638 1639 - exc_info: the exception info 1640 """ 1641 def __init__(self, test, example, exc_info): 1642 self.test = test 1643 self.example = example 1644 self.exc_info = exc_info 1645 1646 def __str__(self): 1647 return str(self.test) 1648 1649class DebugRunner(DocTestRunner): 1650 r"""Run doc tests but raise an exception as soon as there is a failure. 1651 1652 If an unexpected exception occurs, an UnexpectedException is raised. 1653 It contains the test, the example, and the original exception: 1654 1655 >>> runner = DebugRunner(verbose=False) 1656 >>> test = DocTestParser().get_doctest('>>> raise KeyError\n42', 1657 ... {}, 'foo', 'foo.py', 0) 1658 >>> try: 1659 ... runner.run(test) 1660 ... except UnexpectedException, failure: 1661 ... pass 1662 1663 >>> failure.test is test 1664 True 1665 1666 >>> failure.example.want 1667 '42\n' 1668 1669 >>> exc_info = failure.exc_info 1670 >>> raise exc_info[0], exc_info[1], exc_info[2] 1671 Traceback (most recent call last): 1672 ... 1673 KeyError 1674 1675 We wrap the original exception to give the calling application 1676 access to the test and example information. 1677 1678 If the output doesn't match, then a DocTestFailure is raised: 1679 1680 >>> test = DocTestParser().get_doctest(''' 1681 ... >>> x = 1 1682 ... >>> x 1683 ... 2 1684 ... ''', {}, 'foo', 'foo.py', 0) 1685 1686 >>> try: 1687 ... runner.run(test) 1688 ... except DocTestFailure, failure: 1689 ... pass 1690 1691 DocTestFailure objects provide access to the test: 1692 1693 >>> failure.test is test 1694 True 1695 1696 As well as to the example: 1697 1698 >>> failure.example.want 1699 '2\n' 1700 1701 and the actual output: 1702 1703 >>> failure.got 1704 '1\n' 1705 1706 If a failure or error occurs, the globals are left intact: 1707 1708 >>> del test.globs['__builtins__'] 1709 >>> test.globs 1710 {'x': 1} 1711 1712 >>> test = DocTestParser().get_doctest(''' 1713 ... >>> x = 2 1714 ... >>> raise KeyError 1715 ... ''', {}, 'foo', 'foo.py', 0) 1716 1717 >>> runner.run(test) 1718 Traceback (most recent call last): 1719 ... 1720 UnexpectedException: <DocTest foo from foo.py:0 (2 examples)> 1721 1722 >>> del test.globs['__builtins__'] 1723 >>> test.globs 1724 {'x': 2} 1725 1726 But the globals are cleared if there is no error: 1727 1728 >>> test = DocTestParser().get_doctest(''' 1729 ... >>> x = 2 1730 ... ''', {}, 'foo', 'foo.py', 0) 1731 1732 >>> runner.run(test) 1733 (0, 1) 1734 1735 >>> test.globs 1736 {} 1737 1738 """ 1739 1740 def run(self, test, compileflags=None, out=None, clear_globs=True): 1741 r = DocTestRunner.run(self, test, compileflags, out, False) 1742 if clear_globs: 1743 test.globs.clear() 1744 return r 1745 1746 def report_unexpected_exception(self, out, test, example, exc_info): 1747 raise UnexpectedException(test, example, exc_info) 1748 1749 def report_failure(self, out, test, example, got): 1750 raise DocTestFailure(test, example, got) 1751 1752###################################################################### 1753## 6. Test Functions 1754###################################################################### 1755# These should be backwards compatible. 1756 1757def testmod(m=None, name=None, globs=None, verbose=None, isprivate=None, 1758 report=True, optionflags=0, extraglobs=None, 1759 raise_on_error=False): 1760 """m=None, name=None, globs=None, verbose=None, isprivate=None, 1761 report=True, optionflags=0, extraglobs=None 1762 1763 Test examples in docstrings in functions and classes reachable 1764 from module m (or the current module if m is not supplied), starting 1765 with m.__doc__. Unless isprivate is specified, private names 1766 are not skipped. 1767 1768 Also test examples reachable from dict m.__test__ if it exists and is 1769 not None. m.__test__ maps names to functions, classes and strings; 1770 function and class docstrings are tested even if the name is private; 1771 strings are tested directly, as if they were docstrings. 1772 1773 Return (#failures, #tests). 1774 1775 See doctest.__doc__ for an overview. 1776 1777 Optional keyword arg "name" gives the name of the module; by default 1778 use m.__name__. 1779 1780 Optional keyword arg "globs" gives a dict to be used as the globals 1781 when executing examples; by default, use m.__dict__. A copy of this 1782 dict is actually used for each docstring, so that each docstring's 1783 examples start with a clean slate. 1784 1785 Optional keyword arg "extraglobs" gives a dictionary that should be 1786 merged into the globals that are used to execute examples. By 1787 default, no extra globals are used. This is new in 2.4. 1788 1789 Optional keyword arg "verbose" prints lots of stuff if true, prints 1790 only failures if false; by default, it's true iff "-v" is in sys.argv. 1791 1792 Optional keyword arg "report" prints a summary at the end when true, 1793 else prints nothing at the end. In verbose mode, the summary is 1794 detailed, else very brief (in fact, empty if all tests passed). 1795 1796 Optional keyword arg "optionflags" or's together module constants, 1797 and defaults to 0. This is new in 2.3. Possible values: 1798 1799 DONT_ACCEPT_TRUE_FOR_1 1800 By default, if an expected output block contains just "1", 1801 an actual output block containing just "True" is considered 1802 to be a match, and similarly for "0" versus "False". When 1803 DONT_ACCEPT_TRUE_FOR_1 is specified, neither substitution 1804 is allowed. 1805 1806 DONT_ACCEPT_BLANKLINE 1807 By default, if an expected output block contains a line 1808 containing only the string "<BLANKLINE>", then that line 1809 will match a blank line in the actual output. When 1810 DONT_ACCEPT_BLANKLINE is specified, this substitution is 1811 not allowed. 1812 1813 NORMALIZE_WHITESPACE 1814 When NORMALIZE_WHITESPACE is specified, all sequences of 1815 whitespace are treated as equal. I.e., any sequence of 1816 whitespace within the expected output will match any 1817 sequence of whitespace within the actual output. 1818 1819 ELLIPSIS 1820 When ELLIPSIS is specified, then an ellipsis marker 1821 ("...") in the expected output can match any substring in 1822 the actual output. 1823 1824 UNIFIED_DIFF 1825 When UNIFIED_DIFF is specified, failures that involve 1826 multi-line expected and actual outputs will be displayed 1827 using a unified diff. 1828 1829 CONTEXT_DIFF 1830 When CONTEXT_DIFF is specified, failures that involve 1831 multi-line expected and actual outputs will be displayed 1832 using a context diff. 1833 1834 Optional keyword arg "raise_on_error" raises an exception on the 1835 first unexpected exception or failure. This allows failures to be 1836 post-mortem debugged. 1837 1838 Deprecated in Python 2.4: 1839 Optional keyword arg "isprivate" specifies a function used to 1840 determine whether a name is private. The default function is 1841 treat all functions as public. Optionally, "isprivate" can be 1842 set to doctest.is_private to skip over functions marked as private 1843 using the underscore naming convention; see its docs for details. 1844 """ 1845 1846 """ [XX] This is no longer true: 1847 Advanced tomfoolery: testmod runs methods of a local instance of 1848 class doctest.Tester, then merges the results into (or creates) 1849 global Tester instance doctest.master. Methods of doctest.master 1850 can be called directly too, if you want to do something unusual. 1851 Passing report=0 to testmod is especially useful then, to delay 1852 displaying a summary. Invoke doctest.master.summarize(verbose) 1853 when you're done fiddling. 1854 """ 1855 if isprivate is not None: 1856 warnings.warn("the isprivate argument is deprecated; " 1857 "examine DocTestFinder.find() lists instead", 1858 DeprecationWarning) 1859 1860 # If no module was given, then use __main__. 1861 if m is None: 1862 # DWA - m will still be None if this wasn't invoked from the command 1863 # line, in which case the following TypeError is about as good an error 1864 # as we should expect 1865 m = sys.modules.get('__main__') 1866 1867 # Check that we were actually given a module. 1868 if not inspect.ismodule(m): 1869 raise TypeError("testmod: module required; %r" % (m,)) 1870 1871 # If no name was given, then use the module's name. 1872 if name is None: 1873 name = m.__name__ 1874 1875 # Find, parse, and run all tests in the given module. 1876 finder = DocTestFinder(_namefilter=isprivate) 1877 1878 if raise_on_error: 1879 runner = DebugRunner(verbose=verbose, optionflags=optionflags) 1880 else: 1881 runner = DocTestRunner(verbose=verbose, optionflags=optionflags) 1882 1883 for test in finder.find(m, name, globs=globs, extraglobs=extraglobs): 1884 runner.run(test) 1885 1886 if report: 1887 runner.summarize() 1888 1889 return runner.failures, runner.tries 1890 1891def run_docstring_examples(f, globs, verbose=False, name="NoName", 1892 compileflags=None, optionflags=0): 1893 """ 1894 Test examples in the given object's docstring (`f`), using `globs` 1895 as globals. Optional argument `name` is used in failure messages. 1896 If the optional argument `verbose` is true, then generate output 1897 even if there are no failures. 1898 1899 `compileflags` gives the set of flags that should be used by the 1900 Python compiler when running the examples. If not specified, then 1901 it will default to the set of future-import flags that apply to 1902 `globs`. 1903 1904 Optional keyword arg `optionflags` specifies options for the 1905 testing and output. See the documentation for `testmod` for more 1906 information. 1907 """ 1908 # Find, parse, and run all tests in the given module. 1909 finder = DocTestFinder(verbose=verbose, recurse=False) 1910 runner = DocTestRunner(verbose=verbose, optionflags=optionflags) 1911 for test in finder.find(f, name, globs=globs): 1912 runner.run(test, compileflags=compileflags) 1913 1914###################################################################### 1915## 7. Tester 1916###################################################################### 1917# This is provided only for backwards compatibility. It's not 1918# actually used in any way. 1919 1920class Tester: 1921 def __init__(self, mod=None, globs=None, verbose=None, 1922 isprivate=None, optionflags=0): 1923 1924 warnings.warn("class Tester is deprecated; " 1925 "use class doctest.DocTestRunner instead", 1926 DeprecationWarning, stacklevel=2) 1927 if mod is None and globs is None: 1928 raise TypeError("Tester.__init__: must specify mod or globs") 1929 if mod is not None and not _ismodule(mod): 1930 raise TypeError("Tester.__init__: mod must be a module; %r" % 1931 (mod,)) 1932 if globs is None: 1933 globs = mod.__dict__ 1934 self.globs = globs 1935 1936 self.verbose = verbose 1937 self.isprivate = isprivate 1938 self.optionflags = optionflags 1939 self.testfinder = DocTestFinder(_namefilter=isprivate) 1940 self.testrunner = DocTestRunner(verbose=verbose, 1941 optionflags=optionflags) 1942 1943 def runstring(self, s, name): 1944 test = DocTestParser().get_doctest(s, self.globs, name, None, None) 1945 if self.verbose: 1946 print "Running string", name 1947 (f,t) = self.testrunner.run(test) 1948 if self.verbose: 1949 print f, "of", t, "examples failed in string", name 1950 return (f,t) 1951 1952 def rundoc(self, object, name=None, module=None): 1953 f = t = 0 1954 tests = self.testfinder.find(object, name, module=module, 1955 globs=self.globs) 1956 for test in tests: 1957 (f2, t2) = self.testrunner.run(test) 1958 (f,t) = (f+f2, t+t2) 1959 return (f,t) 1960 1961 def rundict(self, d, name, module=None): 1962 import new 1963 m = new.module(name) 1964 m.__dict__.update(d) 1965 if module is None: 1966 module = False 1967 return self.rundoc(m, name, module) 1968 1969 def run__test__(self, d, name): 1970 import new 1971 m = new.module(name) 1972 m.__test__ = d 1973 return self.rundoc(m, name, module) 1974 1975 def summarize(self, verbose=None): 1976 return self.testrunner.summarize(verbose) 1977 1978 def merge(self, other): 1979 d = self.testrunner._name2ft 1980 for name, (f, t) in other.testrunner._name2ft.items(): 1981 if name in d: 1982 print "*** Tester.merge: '" + name + "' in both" \ 1983 " testers; summing outcomes." 1984 f2, t2 = d[name] 1985 f = f + f2 1986 t = t + t2 1987 d[name] = f, t 1988 1989###################################################################### 1990## 8. Unittest Support 1991###################################################################### 1992 1993class DocTestCase(unittest.TestCase): 1994 1995 def __init__(self, test, optionflags=0, setUp=None, tearDown=None, 1996 checker=None): 1997 unittest.TestCase.__init__(self) 1998 self._dt_optionflags = optionflags 1999 self._dt_checker = checker 2000 self._dt_test = test 2001 self._dt_setUp = setUp 2002 self._dt_tearDown = tearDown 2003 2004 def setUp(self): 2005 if self._dt_setUp is not None: 2006 self._dt_setUp() 2007 2008 def tearDown(self): 2009 if self._dt_tearDown is not None: 2010 self._dt_tearDown() 2011 2012 def runTest(self): 2013 test = self._dt_test 2014 old = sys.stdout 2015 new = StringIO() 2016 runner = DocTestRunner(optionflags=self._dt_optionflags, 2017 checker=self._dt_checker, verbose=False) 2018 2019 try: 2020 runner.DIVIDER = "-"*70 2021 failures, tries = runner.run(test, out=new.write) 2022 finally: 2023 sys.stdout = old 2024 2025 if failures: 2026 raise self.failureException(self.format_failure(new.getvalue())) 2027 2028 def format_failure(self, err): 2029 test = self._dt_test 2030 if test.lineno is None: 2031 lineno = 'unknown line number' 2032 else: 2033 lineno = 'line %s' % test.lineno 2034 lname = '.'.join(test.name.split('.')[-1:]) 2035 return ('Failed doctest test for %s\n' 2036 ' File "%s", line %s, in %s\n\n%s' 2037 % (test.name, test.filename, lineno, lname, err) 2038 ) 2039 2040 def debug(self): 2041 r"""Run the test case without results and without catching exceptions 2042 2043 The unit test framework includes a debug method on test cases 2044 and test suites to support post-mortem debugging. The test code 2045 is run in such a way that errors are not caught. This way a 2046 caller can catch the errors and initiate post-mortem debugging. 2047 2048 The DocTestCase provides a debug method that raises 2049 UnexpectedException errors if there is an unexepcted 2050 exception: 2051 2052 >>> test = DocTestParser().get_doctest('>>> raise KeyError\n42', 2053 ... {}, 'foo', 'foo.py', 0) 2054 >>> case = DocTestCase(test) 2055 >>> try: 2056 ... case.debug() 2057 ... except UnexpectedException, failure: 2058 ... pass 2059 2060 The UnexpectedException contains the test, the example, and 2061 the original exception: 2062 2063 >>> failure.test is test 2064 True 2065 2066 >>> failure.example.want 2067 '42\n' 2068 2069 >>> exc_info = failure.exc_info 2070 >>> raise exc_info[0], exc_info[1], exc_info[2] 2071 Traceback (most recent call last): 2072 ... 2073 KeyError 2074 2075 If the output doesn't match, then a DocTestFailure is raised: 2076 2077 >>> test = DocTestParser().get_doctest(''' 2078 ... >>> x = 1 2079 ... >>> x 2080 ... 2 2081 ... ''', {}, 'foo', 'foo.py', 0) 2082 >>> case = DocTestCase(test) 2083 2084 >>> try: 2085 ... case.debug() 2086 ... except DocTestFailure, failure: 2087 ... pass 2088 2089 DocTestFailure objects provide access to the test: 2090 2091 >>> failure.test is test 2092 True 2093 2094 As well as to the example: 2095 2096 >>> failure.example.want 2097 '2\n' 2098 2099 and the actual output: 2100 2101 >>> failure.got 2102 '1\n' 2103 2104 """ 2105 2106 runner = DebugRunner(optionflags=self._dt_optionflags, 2107 checker=self._dt_checker, verbose=False) 2108 runner.run(self._dt_test, out=nooutput) 2109 2110 def id(self): 2111 return self._dt_test.name 2112 2113 def __repr__(self): 2114 name = self._dt_test.name.split('.') 2115 return "%s (%s)" % (name[-1], '.'.join(name[:-1])) 2116 2117 __str__ = __repr__ 2118 2119 def shortDescription(self): 2120 return "Doctest: " + self._dt_test.name 2121 2122def nooutput(*args): 2123 pass 2124 2125def DocTestSuite(module=None, globs=None, extraglobs=None, 2126 optionflags=0, test_finder=None, 2127 setUp=lambda: None, tearDown=lambda: None, 2128 checker=None): 2129 """ 2130 Convert doctest tests for a mudule to a unittest test suite. 2131 2132 This converts each documentation string in a module that 2133 contains doctest tests to a unittest test case. If any of the 2134 tests in a doc string fail, then the test case fails. An exception 2135 is raised showing the name of the file containing the test and a 2136 (sometimes approximate) line number. 2137 2138 The `module` argument provides the module to be tested. The argument 2139 can be either a module or a module name. 2140 2141 If no argument is given, the calling module is used. 2142 """ 2143 2144 if test_finder is None: 2145 test_finder = DocTestFinder() 2146 2147 module = _normalize_module(module) 2148 tests = test_finder.find(module, globs=globs, extraglobs=extraglobs) 2149 if globs is None: 2150 globs = module.__dict__ 2151 if not tests: # [XX] why do we want to do this? 2152 raise ValueError(module, "has no tests") 2153 2154 tests.sort() 2155 suite = unittest.TestSuite() 2156 for test in tests: 2157 if len(test.examples) == 0: 2158 continue 2159 if not test.filename: 2160 filename = module.__file__ 2161 if filename.endswith(".pyc"): 2162 filename = filename[:-1] 2163 elif filename.endswith(".pyo"): 2164 filename = filename[:-1] 2165 test.filename = filename 2166 suite.addTest(DocTestCase(test, optionflags, setUp, tearDown, 2167 checker)) 2168 2169 return suite 2170 2171class DocFileCase(DocTestCase): 2172 2173 def id(self): 2174 return '_'.join(self._dt_test.name.split('.')) 2175 2176 def __repr__(self): 2177 return self._dt_test.filename 2178 __str__ = __repr__ 2179 2180 def format_failure(self, err): 2181 return ('Failed doctest test for %s\n File "%s", line 0\n\n%s' 2182 % (self._dt_test.name, self._dt_test.filename, err) 2183 ) 2184 2185def DocFileTest(path, package=None, globs=None, 2186 setUp=None, tearDown=None, 2187 optionflags=0): 2188 package = _normalize_module(package) 2189 name = path.split('/')[-1] 2190 dir = os.path.split(package.__file__)[0] 2191 path = os.path.join(dir, *(path.split('/'))) 2192 doc = open(path).read() 2193 2194 if globs is None: 2195 globs = {} 2196 2197 test = DocTestParser().get_doctest(doc, globs, name, path, 0) 2198 2199 return DocFileCase(test, optionflags, setUp, tearDown) 2200 2201def DocFileSuite(*paths, **kw): 2202 """Creates a suite of doctest files. 2203 2204 One or more text file paths are given as strings. These should 2205 use "/" characters to separate path segments. Paths are relative 2206 to the directory of the calling module, or relative to the package 2207 passed as a keyword argument. 2208 2209 A number of options may be provided as keyword arguments: 2210 2211 package 2212 The name of a Python package. Text-file paths will be 2213 interpreted relative to the directory containing this package. 2214 The package may be supplied as a package object or as a dotted 2215 package name. 2216 2217 setUp 2218 The name of a set-up function. This is called before running the 2219 tests in each file. 2220 2221 tearDown 2222 The name of a tear-down function. This is called after running the 2223 tests in each file. 2224 2225 globs 2226 A dictionary containing initial global variables for the tests. 2227 """ 2228 suite = unittest.TestSuite() 2229 2230 # We do this here so that _normalize_module is called at the right 2231 # level. If it were called in DocFileTest, then this function 2232 # would be the caller and we might guess the package incorrectly. 2233 kw['package'] = _normalize_module(kw.get('package')) 2234 2235 for path in paths: 2236 suite.addTest(DocFileTest(path, **kw)) 2237 2238 return suite 2239 2240###################################################################### 2241## 9. Debugging Support 2242###################################################################### 2243 2244def script_from_examples(s): 2245 r"""Extract script from text with examples. 2246 2247 Converts text with examples to a Python script. Example input is 2248 converted to regular code. Example output and all other words 2249 are converted to comments: 2250 2251 >>> text = ''' 2252 ... Here are examples of simple math. 2253 ... 2254 ... Python has super accurate integer addition 2255 ... 2256 ... >>> 2 + 2 2257 ... 5 2258 ... 2259 ... And very friendly error messages: 2260 ... 2261 ... >>> 1/0 2262 ... To Infinity 2263 ... And 2264 ... Beyond 2265 ... 2266 ... You can use logic if you want: 2267 ... 2268 ... >>> if 0: 2269 ... ... blah 2270 ... ... blah 2271 ... ... 2272 ... 2273 ... Ho hum 2274 ... ''' 2275 2276 >>> print script_from_examples(text) 2277 # Here are examples of simple math. 2278 # 2279 # Python has super accurate integer addition 2280 # 2281 2 + 2 2282 # Expected: 2283 ## 5 2284 # 2285 # And very friendly error messages: 2286 # 2287 1/0 2288 # Expected: 2289 ## To Infinity 2290 ## And 2291 ## Beyond 2292 # 2293 # You can use logic if you want: 2294 # 2295 if 0: 2296 blah 2297 blah 2298 <BLANKLINE> 2299 # 2300 # Ho hum 2301 """ 2302 2303 return DocTestParser().get_program(s) 2304 2305def _want_comment(example): 2306 """ 2307 Return a comment containing the expected output for the given example. 2308 """ 2309 # Return the expected output, if any 2310 want = example.want 2311 if want: 2312 if want[-1] == '\n': 2313 want = want[:-1] 2314 want = "\n# ".join(want.split("\n")) 2315 want = "\n# Expected:\n# %s" % want 2316 return want 2317 2318def testsource(module, name): 2319 """Extract the test sources from a doctest docstring as a script. 2320 2321 Provide the module (or dotted name of the module) containing the 2322 test to be debugged and the name (within the module) of the object 2323 with the doc string with tests to be debugged. 2324 """ 2325 module = _normalize_module(module) 2326 tests = DocTestFinder().find(module) 2327 test = [t for t in tests if t.name == name] 2328 if not test: 2329 raise ValueError(name, "not found in tests") 2330 test = test[0] 2331 testsrc = script_from_examples(test.docstring) 2332 return testsrc 2333 2334def debug_src(src, pm=False, globs=None): 2335 """Debug a single doctest docstring, in argument `src`'""" 2336 testsrc = script_from_examples(src) 2337 debug_script(testsrc, pm, globs) 2338 2339def debug_script(src, pm=False, globs=None): 2340 "Debug a test script. `src` is the script, as a string." 2341 import pdb 2342 2343 srcfilename = tempfile.mktemp("doctestdebug.py") 2344 f = open(srcfilename, 'w') 2345 f.write(src) 2346 f.close() 2347 2348 if globs: 2349 globs = globs.copy() 2350 else: 2351 globs = {} 2352 2353 if pm: 2354 try: 2355 execfile(srcfilename, globs, globs) 2356 except: 2357 print sys.exc_info()[1] 2358 pdb.post_mortem(sys.exc_info()[2]) 2359 else: 2360 # Note that %r is vital here. '%s' instead can, e.g., cause 2361 # backslashes to get treated as metacharacters on Windows. 2362 pdb.run("execfile(%r)" % srcfilename, globs, globs) 2363 2364def debug(module, name, pm=False): 2365 """Debug a single doctest docstring. 2366 2367 Provide the module (or dotted name of the module) containing the 2368 test to be debugged and the name (within the module) of the object 2369 with the docstring with tests to be debugged. 2370 """ 2371 module = _normalize_module(module) 2372 testsrc = testsource(module, name) 2373 debug_script(testsrc, pm, module.__dict__) 2374 2375###################################################################### 2376## 10. Example Usage 2377###################################################################### 2378class _TestClass: 2379 """ 2380 A pointless class, for sanity-checking of docstring testing. 2381 2382 Methods: 2383 square() 2384 get() 2385 2386 >>> _TestClass(13).get() + _TestClass(-12).get() 2387 1 2388 >>> hex(_TestClass(13).square().get()) 2389 '0xa9' 2390 """ 2391 2392 def __init__(self, val): 2393 """val -> _TestClass object with associated value val. 2394 2395 >>> t = _TestClass(123) 2396 >>> print t.get() 2397 123 2398 """ 2399 2400 self.val = val 2401 2402 def square(self): 2403 """square() -> square TestClass's associated value 2404 2405 >>> _TestClass(13).square().get() 2406 169 2407 """ 2408 2409 self.val = self.val ** 2 2410 return self 2411 2412 def get(self): 2413 """get() -> return TestClass's associated value. 2414 2415 >>> x = _TestClass(-42) 2416 >>> print x.get() 2417 -42 2418 """ 2419 2420 return self.val 2421 2422__test__ = {"_TestClass": _TestClass, 2423 "string": r""" 2424 Example of a string object, searched as-is. 2425 >>> x = 1; y = 2 2426 >>> x + y, x * y 2427 (3, 2) 2428 """, 2429 "bool-int equivalence": r""" 2430 In 2.2, boolean expressions displayed 2431 0 or 1. By default, we still accept 2432 them. This can be disabled by passing 2433 DONT_ACCEPT_TRUE_FOR_1 to the new 2434 optionflags argument. 2435 >>> 4 == 4 2436 1 2437 >>> 4 == 4 2438 True 2439 >>> 4 > 4 2440 0 2441 >>> 4 > 4 2442 False 2443 """, 2444 "blank lines": r""" 2445 Blank lines can be marked with <BLANKLINE>: 2446 >>> print 'foo\n\nbar\n' 2447 foo 2448 <BLANKLINE> 2449 bar 2450 <BLANKLINE> 2451 """, 2452 } 2453# "ellipsis": r""" 2454# If the ellipsis flag is used, then '...' can be used to 2455# elide substrings in the desired output: 2456# >>> print range(1000) 2457# [0, 1, 2, ..., 999] 2458# """, 2459# "whitespace normalization": r""" 2460# If the whitespace normalization flag is used, then 2461# differences in whitespace are ignored. 2462# >>> print range(30) 2463# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 2464# 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 2465# 27, 28, 29] 2466# """, 2467# } 2468 2469def test1(): r""" 2470>>> warnings.filterwarnings("ignore", "class Tester", DeprecationWarning, 2471... "doctest", 0) 2472>>> from doctest import Tester 2473>>> t = Tester(globs={'x': 42}, verbose=0) 2474>>> t.runstring(r''' 2475... >>> x = x * 2 2476... >>> print x 2477... 42 2478... ''', 'XYZ') 2479********************************************************************** 2480Failure in example: print x 2481from line #2 of XYZ 2482Expected: 42 2483Got: 84 2484(1, 2) 2485>>> t.runstring(">>> x = x * 2\n>>> print x\n84\n", 'example2') 2486(0, 2) 2487>>> t.summarize() 2488********************************************************************** 24891 items had failures: 2490 1 of 2 in XYZ 2491***Test Failed*** 1 failures. 2492(1, 4) 2493>>> t.summarize(verbose=1) 24941 items passed all tests: 2495 2 tests in example2 2496********************************************************************** 24971 items had failures: 2498 1 of 2 in XYZ 24994 tests in 2 items. 25003 passed and 1 failed. 2501***Test Failed*** 1 failures. 2502(1, 4) 2503""" 2504 2505def test2(): r""" 2506 >>> warnings.filterwarnings("ignore", "class Tester", 2507 ... DeprecationWarning, "doctest", 0) 2508 >>> t = Tester(globs={}, verbose=1) 2509 >>> test = r''' 2510 ... # just an example 2511 ... >>> x = 1 + 2 2512 ... >>> x 2513 ... 3 2514 ... ''' 2515 >>> t.runstring(test, "Example") 2516 Running string Example 2517 Trying: x = 1 + 2 2518 Expecting: nothing 2519 ok 2520 Trying: x 2521 Expecting: 3 2522 ok 2523 0 of 2 examples failed in string Example 2524 (0, 2) 2525""" 2526def test3(): r""" 2527 >>> warnings.filterwarnings("ignore", "class Tester", 2528 ... DeprecationWarning, "doctest", 0) 2529 >>> t = Tester(globs={}, verbose=0) 2530 >>> def _f(): 2531 ... '''Trivial docstring example. 2532 ... >>> assert 2 == 2 2533 ... ''' 2534 ... return 32 2535 ... 2536 >>> t.rundoc(_f) # expect 0 failures in 1 example 2537 (0, 1) 2538""" 2539def test4(): """ 2540 >>> import new 2541 >>> m1 = new.module('_m1') 2542 >>> m2 = new.module('_m2') 2543 >>> test_data = \""" 2544 ... def _f(): 2545 ... '''>>> assert 1 == 1 2546 ... ''' 2547 ... def g(): 2548 ... '''>>> assert 2 != 1 2549 ... ''' 2550 ... class H: 2551 ... '''>>> assert 2 > 1 2552 ... ''' 2553 ... def bar(self): 2554 ... '''>>> assert 1 < 2 2555 ... ''' 2556 ... \""" 2557 >>> exec test_data in m1.__dict__ 2558 >>> exec test_data in m2.__dict__ 2559 >>> m1.__dict__.update({"f2": m2._f, "g2": m2.g, "h2": m2.H}) 2560 2561 Tests that objects outside m1 are excluded: 2562 2563 >>> warnings.filterwarnings("ignore", "class Tester", 2564 ... DeprecationWarning, "doctest", 0) 2565 >>> t = Tester(globs={}, verbose=0) 2566 >>> t.rundict(m1.__dict__, "rundict_test", m1) # f2 and g2 and h2 skipped 2567 (0, 4) 2568 2569 Once more, not excluding stuff outside m1: 2570 2571 >>> t = Tester(globs={}, verbose=0) 2572 >>> t.rundict(m1.__dict__, "rundict_test_pvt") # None are skipped. 2573 (0, 8) 2574 2575 The exclusion of objects from outside the designated module is 2576 meant to be invoked automagically by testmod. 2577 2578 >>> testmod(m1, verbose=False) 2579 (0, 4) 2580""" 2581 2582def _test(): 2583 #import doctest 2584 #doctest.testmod(doctest, verbose=False, 2585 # optionflags=ELLIPSIS | NORMALIZE_WHITESPACE | 2586 # UNIFIED_DIFF) 2587 #print '~'*70 2588 r = unittest.TextTestRunner() 2589 r.run(DocTestSuite()) 2590 2591if __name__ == "__main__": 2592 _test() 2593