doctest.py revision 8e4a34ba090a4ad9753ea8d9ccf95703c85fdbb4
1# Module doctest. 2# Released to the public domain 16-Jan-2001, by Tim Peters (tim@python.org). 3# Major enhancements and refactoring by: 4# Jim Fulton 5# Edward Loper 6 7# Provided as-is; use at your own risk; no warranty; no promises; enjoy! 8 9r"""Module doctest -- a framework for running examples in docstrings. 10 11NORMAL USAGE 12 13In simplest use, end each module M to be tested with: 14 15def _test(): 16 import doctest 17 return doctest.testmod() 18 19if __name__ == "__main__": 20 _test() 21 22Then running the module as a script will cause the examples in the 23docstrings to get executed and verified: 24 25python M.py 26 27This won't display anything unless an example fails, in which case the 28failing example(s) and the cause(s) of the failure(s) are printed to stdout 29(why not stderr? because stderr is a lame hack <0.2 wink>), and the final 30line of output is "Test failed.". 31 32Run it with the -v switch instead: 33 34python M.py -v 35 36and a detailed report of all examples tried is printed to stdout, along 37with assorted summaries at the end. 38 39You can force verbose mode by passing "verbose=True" to testmod, or prohibit 40it by passing "verbose=False". In either of those cases, sys.argv is not 41examined by testmod. 42 43In any case, testmod returns a 2-tuple of ints (f, t), where f is the 44number of docstring examples that failed and t is the total number of 45docstring examples attempted. 46 47There are a variety of other ways to run doctests, including integration 48with the unittest framework, and support for running non-Python text 49files containing doctests. There are also many ways to override parts 50of doctest's default behaviors. See the Library Reference Manual for 51details. 52 53 54WHICH DOCSTRINGS ARE EXAMINED? 55 56+ M.__doc__. 57 58+ f.__doc__ for all functions f in M.__dict__.values(), except those 59 defined in other modules. 60 61+ C.__doc__ for all classes C in M.__dict__.values(), except those 62 defined in other modules. 63 64+ If M.__test__ exists and "is true", it must be a dict, and 65 each entry maps a (string) name to a function object, class object, or 66 string. Function and class object docstrings found from M.__test__ 67 are searched, and strings are searched directly as if they were docstrings. 68 In output, a key K in M.__test__ appears with name 69 <name of M>.__test__.K 70 71Any classes found are recursively searched similarly, to test docstrings in 72their contained methods and nested classes. 73 74 75WHAT'S THE EXECUTION CONTEXT? 76 77By default, each time testmod finds a docstring to test, it uses a *copy* 78of M's globals (so that running tests on a module doesn't change the 79module's real globals, and so that one test in M can't leave behind crumbs 80that accidentally allow another test to work). This means examples can 81freely use any names defined at top-level in M. It also means that sloppy 82imports (see above) can cause examples in external docstrings to use 83globals inappropriate for them. 84 85You can force use of your own dict as the execution context by passing 86"globs=your_dict" to testmod instead. Presumably this would be a copy of 87M.__dict__ merged with the globals from other imported modules. 88 89 90WHAT ABOUT EXCEPTIONS? 91 92No problem, as long as the only output generated by the example is the 93traceback itself. For example: 94 95 >>> [1, 2, 3].remove(42) 96 Traceback (most recent call last): 97 File "<stdin>", line 1, in ? 98 ValueError: list.remove(x): x not in list 99 >>> 100 101Note that only the exception type and value are compared. 102 103 104SO WHAT DOES A DOCTEST EXAMPLE LOOK LIKE ALREADY!? 105 106Oh ya. It's easy! In most cases a copy-and-paste of an interactive 107console session works fine -- just make sure the leading whitespace is 108rigidly consistent (you can mix tabs and spaces if you're too lazy to do it 109right, but doctest is not in the business of guessing what you think a tab 110means). 111 112 >>> # comments are ignored 113 >>> x = 12 114 >>> x 115 12 116 >>> if x == 13: 117 ... print "yes" 118 ... else: 119 ... print "no" 120 ... print "NO" 121 ... print "NO!!!" 122 ... 123 no 124 NO 125 NO!!! 126 >>> 127 128Any expected output must immediately follow the final ">>>" or "..." line 129containing the code, and the expected output (if any) extends to the next 130">>>" or all-whitespace line. That's it. 131 132Bummers: 133 134+ Output to stdout is captured, but not output to stderr (exception 135 tracebacks are captured via a different means). 136 137+ If you continue a line via backslashing in an interactive session, 138 or for any other reason use a backslash, you should use a raw 139 docstring, which will preserve your backslahses exactly as you type 140 them: 141 142 >>> def f(x): 143 ... r'''Backslashes in a raw docstring: m\n''' 144 >>> print f.__doc__ 145 Backslashes in a raw docstring: m\n 146 147 Otherwise, the backslash will be interpreted as part of the string. 148 E.g., the "\n" above would be interpreted as a newline character. 149 Alternatively, you can double each backslash in the doctest version 150 (and not use a raw string): 151 152 >>> def f(x): 153 ... '''Backslashes in a raw docstring: m\\n''' 154 >>> print f.__doc__ 155 Backslashes in a raw docstring: m\n 156 157The starting column doesn't matter: 158 159>>> assert "Easy!" 160 >>> import math 161 >>> math.floor(1.9) 162 1.0 163 164and as many leading whitespace characters are stripped from the expected 165output as appeared in the initial ">>>" line that triggered it. 166 167If you execute this very file, the examples above will be found and 168executed. 169""" 170__docformat__ = 'reStructuredText en' 171 172__all__ = [ 173 'is_private', 174 'Example', 175 'DocTest', 176 'DocTestFinder', 177 'DocTestRunner', 178 'testmod', 179 'run_docstring_examples', 180 'Tester', 181 'DocTestCase', 182 'DocTestSuite', 183 'testsource', 184 'debug', 185# 'master', 186] 187 188import __future__ 189 190import sys, traceback, inspect, linecache, os, re, types 191import unittest, difflib, pdb, tempfile 192import warnings 193from StringIO import StringIO 194 195real_pdb_set_trace = pdb.set_trace 196 197# There are 4 basic classes: 198# - Example: a <source, want> pair, plus an intra-docstring line number. 199# - DocTest: a collection of examples, parsed from a docstring, plus 200# info about where the docstring came from (name, filename, lineno). 201# - DocTestFinder: extracts DocTests from a given object's docstring and 202# its contained objects' docstrings. 203# - DocTestRunner: runs DocTest cases, and accumulates statistics. 204# 205# So the basic picture is: 206# 207# list of: 208# +------+ +---------+ +-------+ 209# |object| --DocTestFinder-> | DocTest | --DocTestRunner-> |results| 210# +------+ +---------+ +-------+ 211# | Example | 212# | ... | 213# | Example | 214# +---------+ 215 216# Option constants. 217OPTIONFLAGS_BY_NAME = {} 218def register_optionflag(name): 219 flag = 1 << len(OPTIONFLAGS_BY_NAME) 220 OPTIONFLAGS_BY_NAME[name] = flag 221 return flag 222 223DONT_ACCEPT_TRUE_FOR_1 = register_optionflag('DONT_ACCEPT_TRUE_FOR_1') 224DONT_ACCEPT_BLANKLINE = register_optionflag('DONT_ACCEPT_BLANKLINE') 225NORMALIZE_WHITESPACE = register_optionflag('NORMALIZE_WHITESPACE') 226ELLIPSIS = register_optionflag('ELLIPSIS') 227UNIFIED_DIFF = register_optionflag('UNIFIED_DIFF') 228CONTEXT_DIFF = register_optionflag('CONTEXT_DIFF') 229 230# Special string markers for use in `want` strings: 231BLANKLINE_MARKER = '<BLANKLINE>' 232ELLIPSIS_MARKER = '...' 233 234###################################################################### 235## Table of Contents 236###################################################################### 237# 1. Utility Functions 238# 2. Example & DocTest -- store test cases 239# 3. DocTest Parser -- extracts examples from strings 240# 4. DocTest Finder -- extracts test cases from objects 241# 5. DocTest Runner -- runs test cases 242# 6. Test Functions -- convenient wrappers for testing 243# 7. Tester Class -- for backwards compatibility 244# 8. Unittest Support 245# 9. Debugging Support 246# 10. Example Usage 247 248###################################################################### 249## 1. Utility Functions 250###################################################################### 251 252def is_private(prefix, base): 253 """prefix, base -> true iff name prefix + "." + base is "private". 254 255 Prefix may be an empty string, and base does not contain a period. 256 Prefix is ignored (although functions you write conforming to this 257 protocol may make use of it). 258 Return true iff base begins with an (at least one) underscore, but 259 does not both begin and end with (at least) two underscores. 260 261 >>> warnings.filterwarnings("ignore", "is_private", DeprecationWarning, 262 ... "doctest", 0) 263 >>> is_private("a.b", "my_func") 264 False 265 >>> is_private("____", "_my_func") 266 True 267 >>> is_private("someclass", "__init__") 268 False 269 >>> is_private("sometypo", "__init_") 270 True 271 >>> is_private("x.y.z", "_") 272 True 273 >>> is_private("_x.y.z", "__") 274 False 275 >>> is_private("", "") # senseless but consistent 276 False 277 """ 278 warnings.warn("is_private is deprecated; it wasn't useful; " 279 "examine DocTestFinder.find() lists instead", 280 DeprecationWarning, stacklevel=2) 281 return base[:1] == "_" and not base[:2] == "__" == base[-2:] 282 283def _extract_future_flags(globs): 284 """ 285 Return the compiler-flags associated with the future features that 286 have been imported into the given namespace (globs). 287 """ 288 flags = 0 289 for fname in __future__.all_feature_names: 290 feature = globs.get(fname, None) 291 if feature is getattr(__future__, fname): 292 flags |= feature.compiler_flag 293 return flags 294 295def _normalize_module(module, depth=2): 296 """ 297 Return the module specified by `module`. In particular: 298 - If `module` is a module, then return module. 299 - If `module` is a string, then import and return the 300 module with that name. 301 - If `module` is None, then return the calling module. 302 The calling module is assumed to be the module of 303 the stack frame at the given depth in the call stack. 304 """ 305 if inspect.ismodule(module): 306 return module 307 elif isinstance(module, (str, unicode)): 308 return __import__(module, globals(), locals(), ["*"]) 309 elif module is None: 310 return sys.modules[sys._getframe(depth).f_globals['__name__']] 311 else: 312 raise TypeError("Expected a module, string, or None") 313 314def _tag_msg(tag, msg, indent=' '): 315 """ 316 Return a string that displays a tag-and-message pair nicely, 317 keeping the tag and its message on the same line when that 318 makes sense. If the message is displayed on separate lines, 319 then `indent` is added to the beginning of each line. 320 """ 321 # If the message doesn't end in a newline, then add one. 322 if msg[-1:] != '\n': 323 msg += '\n' 324 # If the message is short enough, and contains no internal 325 # newlines, then display it on the same line as the tag. 326 # Otherwise, display the tag on its own line. 327 if (len(tag) + len(msg) < 75 and 328 msg.find('\n', 0, len(msg)-1) == -1): 329 return '%s: %s' % (tag, msg) 330 else: 331 msg = '\n'.join([indent+l for l in msg[:-1].split('\n')]) 332 return '%s:\n%s\n' % (tag, msg) 333 334def _exception_traceback(exc_info): 335 """ 336 Return a string containing a traceback message for the given 337 exc_info tuple (as returned by sys.exc_info()). 338 """ 339 # Get a traceback message. 340 excout = StringIO() 341 exc_type, exc_val, exc_tb = exc_info 342 traceback.print_exception(exc_type, exc_val, exc_tb, file=excout) 343 return excout.getvalue() 344 345# Override some StringIO methods. 346class _SpoofOut(StringIO): 347 def getvalue(self): 348 result = StringIO.getvalue(self) 349 # If anything at all was written, make sure there's a trailing 350 # newline. There's no way for the expected output to indicate 351 # that a trailing newline is missing. 352 if result and not result.endswith("\n"): 353 result += "\n" 354 # Prevent softspace from screwing up the next test case, in 355 # case they used print with a trailing comma in an example. 356 if hasattr(self, "softspace"): 357 del self.softspace 358 return result 359 360 def truncate(self, size=None): 361 StringIO.truncate(self, size) 362 if hasattr(self, "softspace"): 363 del self.softspace 364 365###################################################################### 366## 2. Example & DocTest 367###################################################################### 368## - An "example" is a <source, want> pair, where "source" is a 369## fragment of source code, and "want" is the expected output for 370## "source." The Example class also includes information about 371## where the example was extracted from. 372## 373## - A "doctest" is a collection of examples, typically extracted from 374## a string (such as an object's docstring). The DocTest class also 375## includes information about where the string was extracted from. 376 377class Example: 378 """ 379 A single doctest example, consisting of source code and expected 380 output. `Example` defines the following attributes: 381 382 - source: A single Python statement, always ending with a newline. 383 The constructor adds a newline if needed. 384 385 - want: The expected output from running the source code (either 386 from stdout, or a traceback in case of exception). `want` ends 387 with a newline unless it's empty, in which case it's an empty 388 string. The constructor adds a newline if needed. 389 390 - lineno: The line number within the DocTest string containing 391 this Example where the Example begins. This line number is 392 zero-based, with respect to the beginning of the DocTest. 393 394 - indent: The example's indentation in the DocTest string. 395 I.e., the number of space characters that preceed the 396 example's first prompt. 397 398 - options: A dictionary mapping from option flags to True or 399 False, which is used to override default options for this 400 example. Any option flags not contained in this dictionary 401 are left at their default value (as specified by the 402 DocTestRunner's optionflags). By default, no options are set. 403 """ 404 def __init__(self, source, want, lineno, indent=0, options=None): 405 # Normalize inputs. 406 if not source.endswith('\n'): 407 source += '\n' 408 if want and not want.endswith('\n'): 409 want += '\n' 410 # Store properties. 411 self.source = source 412 self.want = want 413 self.lineno = lineno 414 self.indent = indent 415 if options is None: options = {} 416 self.options = options 417 418class DocTest: 419 """ 420 A collection of doctest examples that should be run in a single 421 namespace. Each `DocTest` defines the following attributes: 422 423 - examples: the list of examples. 424 425 - globs: The namespace (aka globals) that the examples should 426 be run in. 427 428 - name: A name identifying the DocTest (typically, the name of 429 the object whose docstring this DocTest was extracted from). 430 431 - filename: The name of the file that this DocTest was extracted 432 from, or `None` if the filename is unknown. 433 434 - lineno: The line number within filename where this DocTest 435 begins, or `None` if the line number is unavailable. This 436 line number is zero-based, with respect to the beginning of 437 the file. 438 439 - docstring: The string that the examples were extracted from, 440 or `None` if the string is unavailable. 441 """ 442 def __init__(self, examples, globs, name, filename, lineno, docstring): 443 """ 444 Create a new DocTest containing the given examples. The 445 DocTest's globals are initialized with a copy of `globs`. 446 """ 447 assert not isinstance(examples, basestring), \ 448 "DocTest no longer accepts str; use DocTestParser instead" 449 self.examples = examples 450 self.docstring = docstring 451 self.globs = globs.copy() 452 self.name = name 453 self.filename = filename 454 self.lineno = lineno 455 456 def __repr__(self): 457 if len(self.examples) == 0: 458 examples = 'no examples' 459 elif len(self.examples) == 1: 460 examples = '1 example' 461 else: 462 examples = '%d examples' % len(self.examples) 463 return ('<DocTest %s from %s:%s (%s)>' % 464 (self.name, self.filename, self.lineno, examples)) 465 466 467 # This lets us sort tests by name: 468 def __cmp__(self, other): 469 if not isinstance(other, DocTest): 470 return -1 471 return cmp((self.name, self.filename, self.lineno, id(self)), 472 (other.name, other.filename, other.lineno, id(other))) 473 474###################################################################### 475## 2. DocTestParser 476###################################################################### 477 478class DocTestParser: 479 """ 480 A class used to parse strings containing doctest examples. 481 """ 482 # This regular expression is used to find doctest examples in a 483 # string. It defines three groups: `source` is the source code 484 # (including leading indentation and prompts); `indent` is the 485 # indentation of the first (PS1) line of the source code; and 486 # `want` is the expected output (including leading indentation). 487 _EXAMPLE_RE = re.compile(r''' 488 # Source consists of a PS1 line followed by zero or more PS2 lines. 489 (?P<source> 490 (?:^(?P<indent> [ ]*) >>> .*) # PS1 line 491 (?:\n [ ]* \.\.\. .*)*) # PS2 lines 492 \n? 493 # Want consists of any non-blank lines that do not start with PS1. 494 (?P<want> (?:(?![ ]*$) # Not a blank line 495 (?![ ]*>>>) # Not a line starting with PS1 496 .*$\n? # But any other line 497 )*) 498 ''', re.MULTILINE | re.VERBOSE) 499 500 # This regular expression matcher checks if a given string is a 501 # blank line or contains a single comment. 502 _IS_BLANK_OR_COMMENT = re.compile(r'^[ ]*(#.*)?$').match 503 504 def get_doctest(self, string, globs, name, filename, lineno): 505 """ 506 Extract all doctest examples from the given string, and 507 collect them into a `DocTest` object. 508 509 `globs`, `name`, `filename`, and `lineno` are attributes for 510 the new `DocTest` object. See the documentation for `DocTest` 511 for more information. 512 """ 513 return DocTest(self.get_examples(string, name), globs, 514 name, filename, lineno, string) 515 516 def get_examples(self, string, name='<string>'): 517 """ 518 Extract all doctest examples from the given string, and return 519 them as a list of `Example` objects. Line numbers are 520 0-based, because it's most common in doctests that nothing 521 interesting appears on the same line as opening triple-quote, 522 and so the first interesting line is called \"line 1\" then. 523 524 The optional argument `name` is a name identifying this 525 string, and is only used for error messages. 526 527 >>> text = ''' 528 ... >>> x, y = 2, 3 # no output expected 529 ... >>> if 1: 530 ... ... print x 531 ... ... print y 532 ... 2 533 ... 3 534 ... 535 ... Some text. 536 ... >>> x+y 537 ... 5 538 ... ''' 539 >>> for x in DocTestParser().get_examples(text): 540 ... print (x.source, x.want, x.lineno) 541 ('x, y = 2, 3 # no output expected\\n', '', 1) 542 ('if 1:\\n print x\\n print y\\n', '2\\n3\\n', 2) 543 ('x+y\\n', '5\\n', 9) 544 """ 545 examples = [] 546 charno, lineno = 0, 0 547 # Find all doctest examples in the string: 548 for m in self._EXAMPLE_RE.finditer(string.expandtabs()): 549 # Update lineno (lines before this example) 550 lineno += string.count('\n', charno, m.start()) 551 # Extract source/want from the regexp match. 552 (source, want) = self._parse_example(m, name, lineno) 553 # Extract extra options from the source. 554 options = self._find_options(source, name, lineno) 555 # If it contains no real source, then ignore it. 556 if self._IS_BLANK_OR_COMMENT(source): 557 continue 558 # Create an Example, and add it to the list. 559 examples.append( Example(source, want, lineno, 560 len(m.group('indent')), options) ) 561 # Update lineno (lines inside this example) 562 lineno += string.count('\n', m.start(), m.end()) 563 # Update charno. 564 charno = m.end() 565 return examples 566 567 def get_program(self, string, name="<string>"): 568 """ 569 Return an executable program from the given string, as a string. 570 571 The format of this isn't rigidly defined. In general, doctest 572 examples become the executable statements in the result, and 573 their expected outputs become comments, preceded by an \"#Expected:\" 574 comment. Everything else (text, comments, everything not part of 575 a doctest test) is also placed in comments. 576 577 The optional argument `name` is a name identifying this 578 string, and is only used for error messages. 579 580 >>> text = ''' 581 ... >>> x, y = 2, 3 # no output expected 582 ... >>> if 1: 583 ... ... print x 584 ... ... print y 585 ... 2 586 ... 3 587 ... 588 ... Some text. 589 ... >>> x+y 590 ... 5 591 ... ''' 592 >>> print DocTestParser().get_program(text) 593 x, y = 2, 3 # no output expected 594 if 1: 595 print x 596 print y 597 # Expected: 598 # 2 599 # 3 600 # 601 # Some text. 602 x+y 603 # Expected: 604 # 5 605 """ 606 output = [] 607 charnum, lineno = 0, 0 608 # Find all doctest examples in the string: 609 for m in self._EXAMPLE_RE.finditer(string.expandtabs()): 610 # Add any text before this example, as a comment. 611 if m.start() > charnum: 612 lines = string[charnum:m.start()-1].split('\n') 613 output.extend([self._comment_line(l) for l in lines]) 614 lineno += len(lines) 615 616 # Extract source/want from the regexp match. 617 (source, want) = self._parse_example(m, name, lineno) 618 # Display the source 619 output.append(source) 620 # Display the expected output, if any 621 if want: 622 output.append('# Expected:') 623 output.extend(['# '+l for l in want.split('\n')]) 624 625 # Update the line number & char number. 626 lineno += string.count('\n', m.start(), m.end()) 627 charnum = m.end() 628 # Add any remaining text, as comments. 629 output.extend([self._comment_line(l) 630 for l in string[charnum:].split('\n')]) 631 # Trim junk on both ends. 632 while output and output[-1] == '#': 633 output.pop() 634 while output and output[0] == '#': 635 output.pop(0) 636 # Combine the output, and return it. 637 return '\n'.join(output) 638 639 def _parse_example(self, m, name, lineno): 640 """ 641 Given a regular expression match from `_EXAMPLE_RE` (`m`), 642 return a pair `(source, want)`, where `source` is the matched 643 example's source code (with prompts and indentation stripped); 644 and `want` is the example's expected output (with indentation 645 stripped). 646 647 `name` is the string's name, and `lineno` is the line number 648 where the example starts; both are used for error messages. 649 """ 650 # Get the example's indentation level. 651 indent = len(m.group('indent')) 652 653 # Divide source into lines; check that they're properly 654 # indented; and then strip their indentation & prompts. 655 source_lines = m.group('source').split('\n') 656 self._check_prompt_blank(source_lines, indent, name, lineno) 657 self._check_prefix(source_lines[1:], ' '*indent+'.', name, lineno) 658 source = '\n'.join([sl[indent+4:] for sl in source_lines]) 659 660 # Divide want into lines; check that it's properly 661 # indented; and then strip the indentation. 662 want_lines = m.group('want').rstrip().split('\n') 663 self._check_prefix(want_lines, ' '*indent, name, 664 lineno+len(source_lines)) 665 want = '\n'.join([wl[indent:] for wl in want_lines]) 666 667 return source, want 668 669 # This regular expression looks for option directives in the 670 # source code of an example. Option directives are comments 671 # starting with "doctest:". Warning: this may give false 672 # positives for string-literals that contain the string 673 # "#doctest:". Eliminating these false positives would require 674 # actually parsing the string; but we limit them by ignoring any 675 # line containing "#doctest:" that is *followed* by a quote mark. 676 _OPTION_DIRECTIVE_RE = re.compile(r'#\s*doctest:\s*([^\n\'"]*)$', 677 re.MULTILINE) 678 679 def _find_options(self, source, name, lineno): 680 """ 681 Return a dictionary containing option overrides extracted from 682 option directives in the given source string. 683 684 `name` is the string's name, and `lineno` is the line number 685 where the example starts; both are used for error messages. 686 """ 687 options = {} 688 # (note: with the current regexp, this will match at most once:) 689 for m in self._OPTION_DIRECTIVE_RE.finditer(source): 690 option_strings = m.group(1).replace(',', ' ').split() 691 for option in option_strings: 692 if (option[0] not in '+-' or 693 option[1:] not in OPTIONFLAGS_BY_NAME): 694 raise ValueError('line %r of the doctest for %s ' 695 'has an invalid option: %r' % 696 (lineno+1, name, option)) 697 flag = OPTIONFLAGS_BY_NAME[option[1:]] 698 options[flag] = (option[0] == '+') 699 if options and self._IS_BLANK_OR_COMMENT(source): 700 raise ValueError('line %r of the doctest for %s has an option ' 701 'directive on a line with no example: %r' % 702 (lineno, name, source)) 703 return options 704 705 def _comment_line(self, line): 706 "Return a commented form of the given line" 707 line = line.rstrip() 708 if line: 709 return '# '+line 710 else: 711 return '#' 712 713 def _check_prompt_blank(self, lines, indent, name, lineno): 714 """ 715 Given the lines of a source string (including prompts and 716 leading indentation), check to make sure that every prompt is 717 followed by a space character. If any line is not followed by 718 a space character, then raise ValueError. 719 """ 720 for i, line in enumerate(lines): 721 if len(line) >= indent+4 and line[indent+3] != ' ': 722 raise ValueError('line %r of the docstring for %s ' 723 'lacks blank after %s: %r' % 724 (lineno+i+1, name, 725 line[indent:indent+3], line)) 726 727 def _check_prefix(self, lines, prefix, name, lineno): 728 """ 729 Check that every line in the given list starts with the given 730 prefix; if any line does not, then raise a ValueError. 731 """ 732 for i, line in enumerate(lines): 733 if line and not line.startswith(prefix): 734 raise ValueError('line %r of the docstring for %s has ' 735 'inconsistent leading whitespace: %r' % 736 (lineno+i+1, name, line)) 737 738 739###################################################################### 740## 4. DocTest Finder 741###################################################################### 742 743class DocTestFinder: 744 """ 745 A class used to extract the DocTests that are relevant to a given 746 object, from its docstring and the docstrings of its contained 747 objects. Doctests can currently be extracted from the following 748 object types: modules, functions, classes, methods, staticmethods, 749 classmethods, and properties. 750 """ 751 752 def __init__(self, verbose=False, parser=DocTestParser(), 753 recurse=True, _namefilter=None): 754 """ 755 Create a new doctest finder. 756 757 The optional argument `parser` specifies a class or 758 function that should be used to create new DocTest objects (or 759 objects that implement the same interface as DocTest). The 760 signature for this factory function should match the signature 761 of the DocTest constructor. 762 763 If the optional argument `recurse` is false, then `find` will 764 only examine the given object, and not any contained objects. 765 """ 766 self._parser = parser 767 self._verbose = verbose 768 self._recurse = recurse 769 # _namefilter is undocumented, and exists only for temporary backward- 770 # compatibility support of testmod's deprecated isprivate mess. 771 self._namefilter = _namefilter 772 773 def find(self, obj, name=None, module=None, globs=None, 774 extraglobs=None): 775 """ 776 Return a list of the DocTests that are defined by the given 777 object's docstring, or by any of its contained objects' 778 docstrings. 779 780 The optional parameter `module` is the module that contains 781 the given object. If the module is not specified or is None, then 782 the test finder will attempt to automatically determine the 783 correct module. The object's module is used: 784 785 - As a default namespace, if `globs` is not specified. 786 - To prevent the DocTestFinder from extracting DocTests 787 from objects that are imported from other modules. 788 - To find the name of the file containing the object. 789 - To help find the line number of the object within its 790 file. 791 792 Contained objects whose module does not match `module` are ignored. 793 794 If `module` is False, no attempt to find the module will be made. 795 This is obscure, of use mostly in tests: if `module` is False, or 796 is None but cannot be found automatically, then all objects are 797 considered to belong to the (non-existent) module, so all contained 798 objects will (recursively) be searched for doctests. 799 800 The globals for each DocTest is formed by combining `globs` 801 and `extraglobs` (bindings in `extraglobs` override bindings 802 in `globs`). A new copy of the globals dictionary is created 803 for each DocTest. If `globs` is not specified, then it 804 defaults to the module's `__dict__`, if specified, or {} 805 otherwise. If `extraglobs` is not specified, then it defaults 806 to {}. 807 808 """ 809 # If name was not specified, then extract it from the object. 810 if name is None: 811 name = getattr(obj, '__name__', None) 812 if name is None: 813 raise ValueError("DocTestFinder.find: name must be given " 814 "when obj.__name__ doesn't exist: %r" % 815 (type(obj),)) 816 817 # Find the module that contains the given object (if obj is 818 # a module, then module=obj.). Note: this may fail, in which 819 # case module will be None. 820 if module is False: 821 module = None 822 elif module is None: 823 module = inspect.getmodule(obj) 824 825 # Read the module's source code. This is used by 826 # DocTestFinder._find_lineno to find the line number for a 827 # given object's docstring. 828 try: 829 file = inspect.getsourcefile(obj) or inspect.getfile(obj) 830 source_lines = linecache.getlines(file) 831 if not source_lines: 832 source_lines = None 833 except TypeError: 834 source_lines = None 835 836 # Initialize globals, and merge in extraglobs. 837 if globs is None: 838 if module is None: 839 globs = {} 840 else: 841 globs = module.__dict__.copy() 842 else: 843 globs = globs.copy() 844 if extraglobs is not None: 845 globs.update(extraglobs) 846 847 # Recursively expore `obj`, extracting DocTests. 848 tests = [] 849 self._find(tests, obj, name, module, source_lines, globs, {}) 850 return tests 851 852 def _filter(self, obj, prefix, base): 853 """ 854 Return true if the given object should not be examined. 855 """ 856 return (self._namefilter is not None and 857 self._namefilter(prefix, base)) 858 859 def _from_module(self, module, object): 860 """ 861 Return true if the given object is defined in the given 862 module. 863 """ 864 if module is None: 865 return True 866 elif inspect.isfunction(object): 867 return module.__dict__ is object.func_globals 868 elif inspect.isclass(object): 869 return module.__name__ == object.__module__ 870 elif inspect.getmodule(object) is not None: 871 return module is inspect.getmodule(object) 872 elif hasattr(object, '__module__'): 873 return module.__name__ == object.__module__ 874 elif isinstance(object, property): 875 return True # [XX] no way not be sure. 876 else: 877 raise ValueError("object must be a class or function") 878 879 def _find(self, tests, obj, name, module, source_lines, globs, seen): 880 """ 881 Find tests for the given object and any contained objects, and 882 add them to `tests`. 883 """ 884 if self._verbose: 885 print 'Finding tests in %s' % name 886 887 # If we've already processed this object, then ignore it. 888 if id(obj) in seen: 889 return 890 seen[id(obj)] = 1 891 892 # Find a test for this object, and add it to the list of tests. 893 test = self._get_test(obj, name, module, globs, source_lines) 894 if test is not None: 895 tests.append(test) 896 897 # Look for tests in a module's contained objects. 898 if inspect.ismodule(obj) and self._recurse: 899 for valname, val in obj.__dict__.items(): 900 # Check if this contained object should be ignored. 901 if self._filter(val, name, valname): 902 continue 903 valname = '%s.%s' % (name, valname) 904 # Recurse to functions & classes. 905 if ((inspect.isfunction(val) or inspect.isclass(val)) and 906 self._from_module(module, val)): 907 self._find(tests, val, valname, module, source_lines, 908 globs, seen) 909 910 # Look for tests in a module's __test__ dictionary. 911 if inspect.ismodule(obj) and self._recurse: 912 for valname, val in getattr(obj, '__test__', {}).items(): 913 if not isinstance(valname, basestring): 914 raise ValueError("DocTestFinder.find: __test__ keys " 915 "must be strings: %r" % 916 (type(valname),)) 917 if not (inspect.isfunction(val) or inspect.isclass(val) or 918 inspect.ismethod(val) or inspect.ismodule(val) or 919 isinstance(val, basestring)): 920 raise ValueError("DocTestFinder.find: __test__ values " 921 "must be strings, functions, methods, " 922 "classes, or modules: %r" % 923 (type(val),)) 924 valname = '%s.%s' % (name, valname) 925 self._find(tests, val, valname, module, source_lines, 926 globs, seen) 927 928 # Look for tests in a class's contained objects. 929 if inspect.isclass(obj) and self._recurse: 930 for valname, val in obj.__dict__.items(): 931 # Check if this contained object should be ignored. 932 if self._filter(val, name, valname): 933 continue 934 # Special handling for staticmethod/classmethod. 935 if isinstance(val, staticmethod): 936 val = getattr(obj, valname) 937 if isinstance(val, classmethod): 938 val = getattr(obj, valname).im_func 939 940 # Recurse to methods, properties, and nested classes. 941 if ((inspect.isfunction(val) or inspect.isclass(val) or 942 isinstance(val, property)) and 943 self._from_module(module, val)): 944 valname = '%s.%s' % (name, valname) 945 self._find(tests, val, valname, module, source_lines, 946 globs, seen) 947 948 def _get_test(self, obj, name, module, globs, source_lines): 949 """ 950 Return a DocTest for the given object, if it defines a docstring; 951 otherwise, return None. 952 """ 953 # Extract the object's docstring. If it doesn't have one, 954 # then return None (no test for this object). 955 if isinstance(obj, basestring): 956 docstring = obj 957 else: 958 try: 959 if obj.__doc__ is None: 960 return None 961 docstring = str(obj.__doc__) 962 except (TypeError, AttributeError): 963 return None 964 965 # Don't bother if the docstring is empty. 966 if not docstring: 967 return None 968 969 # Find the docstring's location in the file. 970 lineno = self._find_lineno(obj, source_lines) 971 972 # Return a DocTest for this object. 973 if module is None: 974 filename = None 975 else: 976 filename = getattr(module, '__file__', module.__name__) 977 return self._parser.get_doctest(docstring, globs, name, 978 filename, lineno) 979 980 def _find_lineno(self, obj, source_lines): 981 """ 982 Return a line number of the given object's docstring. Note: 983 this method assumes that the object has a docstring. 984 """ 985 lineno = None 986 987 # Find the line number for modules. 988 if inspect.ismodule(obj): 989 lineno = 0 990 991 # Find the line number for classes. 992 # Note: this could be fooled if a class is defined multiple 993 # times in a single file. 994 if inspect.isclass(obj): 995 if source_lines is None: 996 return None 997 pat = re.compile(r'^\s*class\s*%s\b' % 998 getattr(obj, '__name__', '-')) 999 for i, line in enumerate(source_lines): 1000 if pat.match(line): 1001 lineno = i 1002 break 1003 1004 # Find the line number for functions & methods. 1005 if inspect.ismethod(obj): obj = obj.im_func 1006 if inspect.isfunction(obj): obj = obj.func_code 1007 if inspect.istraceback(obj): obj = obj.tb_frame 1008 if inspect.isframe(obj): obj = obj.f_code 1009 if inspect.iscode(obj): 1010 lineno = getattr(obj, 'co_firstlineno', None)-1 1011 1012 # Find the line number where the docstring starts. Assume 1013 # that it's the first line that begins with a quote mark. 1014 # Note: this could be fooled by a multiline function 1015 # signature, where a continuation line begins with a quote 1016 # mark. 1017 if lineno is not None: 1018 if source_lines is None: 1019 return lineno+1 1020 pat = re.compile('(^|.*:)\s*\w*("|\')') 1021 for lineno in range(lineno, len(source_lines)): 1022 if pat.match(source_lines[lineno]): 1023 return lineno 1024 1025 # We couldn't find the line number. 1026 return None 1027 1028###################################################################### 1029## 5. DocTest Runner 1030###################################################################### 1031 1032class DocTestRunner: 1033 """ 1034 A class used to run DocTest test cases, and accumulate statistics. 1035 The `run` method is used to process a single DocTest case. It 1036 returns a tuple `(f, t)`, where `t` is the number of test cases 1037 tried, and `f` is the number of test cases that failed. 1038 1039 >>> tests = DocTestFinder().find(_TestClass) 1040 >>> runner = DocTestRunner(verbose=False) 1041 >>> for test in tests: 1042 ... print runner.run(test) 1043 (0, 2) 1044 (0, 1) 1045 (0, 2) 1046 (0, 2) 1047 1048 The `summarize` method prints a summary of all the test cases that 1049 have been run by the runner, and returns an aggregated `(f, t)` 1050 tuple: 1051 1052 >>> runner.summarize(verbose=1) 1053 4 items passed all tests: 1054 2 tests in _TestClass 1055 2 tests in _TestClass.__init__ 1056 2 tests in _TestClass.get 1057 1 tests in _TestClass.square 1058 7 tests in 4 items. 1059 7 passed and 0 failed. 1060 Test passed. 1061 (0, 7) 1062 1063 The aggregated number of tried examples and failed examples is 1064 also available via the `tries` and `failures` attributes: 1065 1066 >>> runner.tries 1067 7 1068 >>> runner.failures 1069 0 1070 1071 The comparison between expected outputs and actual outputs is done 1072 by an `OutputChecker`. This comparison may be customized with a 1073 number of option flags; see the documentation for `testmod` for 1074 more information. If the option flags are insufficient, then the 1075 comparison may also be customized by passing a subclass of 1076 `OutputChecker` to the constructor. 1077 1078 The test runner's display output can be controlled in two ways. 1079 First, an output function (`out) can be passed to 1080 `TestRunner.run`; this function will be called with strings that 1081 should be displayed. It defaults to `sys.stdout.write`. If 1082 capturing the output is not sufficient, then the display output 1083 can be also customized by subclassing DocTestRunner, and 1084 overriding the methods `report_start`, `report_success`, 1085 `report_unexpected_exception`, and `report_failure`. 1086 """ 1087 # This divider string is used to separate failure messages, and to 1088 # separate sections of the summary. 1089 DIVIDER = "*" * 70 1090 1091 def __init__(self, checker=None, verbose=None, optionflags=0): 1092 """ 1093 Create a new test runner. 1094 1095 Optional keyword arg `checker` is the `OutputChecker` that 1096 should be used to compare the expected outputs and actual 1097 outputs of doctest examples. 1098 1099 Optional keyword arg 'verbose' prints lots of stuff if true, 1100 only failures if false; by default, it's true iff '-v' is in 1101 sys.argv. 1102 1103 Optional argument `optionflags` can be used to control how the 1104 test runner compares expected output to actual output, and how 1105 it displays failures. See the documentation for `testmod` for 1106 more information. 1107 """ 1108 self._checker = checker or OutputChecker() 1109 if verbose is None: 1110 verbose = '-v' in sys.argv 1111 self._verbose = verbose 1112 self.optionflags = optionflags 1113 1114 # Keep track of the examples we've run. 1115 self.tries = 0 1116 self.failures = 0 1117 self._name2ft = {} 1118 1119 # Create a fake output target for capturing doctest output. 1120 self._fakeout = _SpoofOut() 1121 1122 #///////////////////////////////////////////////////////////////// 1123 # Reporting methods 1124 #///////////////////////////////////////////////////////////////// 1125 1126 def report_start(self, out, test, example): 1127 """ 1128 Report that the test runner is about to process the given 1129 example. (Only displays a message if verbose=True) 1130 """ 1131 if self._verbose: 1132 out(_tag_msg("Trying", example.source) + 1133 _tag_msg("Expecting", example.want or "nothing")) 1134 1135 def report_success(self, out, test, example, got): 1136 """ 1137 Report that the given example ran successfully. (Only 1138 displays a message if verbose=True) 1139 """ 1140 if self._verbose: 1141 out("ok\n") 1142 1143 def report_failure(self, out, test, example, got): 1144 """ 1145 Report that the given example failed. 1146 """ 1147 # Print an error message. 1148 out(self._failure_header(test, example) + 1149 self._checker.output_difference(example.want, got, 1150 self.optionflags)) 1151 1152 def report_unexpected_exception(self, out, test, example, exc_info): 1153 """ 1154 Report that the given example raised an unexpected exception. 1155 """ 1156 out(self._failure_header(test, example) + 1157 _tag_msg("Exception raised", _exception_traceback(exc_info))) 1158 1159 def _failure_header(self, test, example): 1160 s = (self.DIVIDER + "\n" + 1161 _tag_msg("Failure in example", example.source)) 1162 if test.filename is None: 1163 # [XX] I'm not putting +1 here, to give the same output 1164 # as the old version. But I think it *should* go here. 1165 return s + ("from line #%s of %s\n" % 1166 (example.lineno, test.name)) 1167 elif test.lineno is None: 1168 return s + ("from line #%s of %s in %s\n" % 1169 (example.lineno+1, test.name, test.filename)) 1170 else: 1171 lineno = test.lineno+example.lineno+1 1172 return s + ("from line #%s of %s (%s)\n" % 1173 (lineno, test.filename, test.name)) 1174 1175 #///////////////////////////////////////////////////////////////// 1176 # DocTest Running 1177 #///////////////////////////////////////////////////////////////// 1178 1179 # A regular expression for handling `want` strings that contain 1180 # expected exceptions. It divides `want` into two pieces: the 1181 # pre-exception output (`out`) and the exception message (`exc`), 1182 # as generated by traceback.format_exception_only(). (I assume 1183 # that the exception_only message is the first non-indented line 1184 # starting with word characters after the "Traceback ...".) 1185 _EXCEPTION_RE = re.compile(('^(?P<out>.*)' 1186 '^(?P<hdr>Traceback \((?:%s|%s)\):)\s*$.*?' 1187 '^(?P<exc>\w+.*)') % 1188 ('most recent call last', 'innermost last'), 1189 re.MULTILINE | re.DOTALL) 1190 1191 def __run(self, test, compileflags, out): 1192 """ 1193 Run the examples in `test`. Write the outcome of each example 1194 with one of the `DocTestRunner.report_*` methods, using the 1195 writer function `out`. `compileflags` is the set of compiler 1196 flags that should be used to execute examples. Return a tuple 1197 `(f, t)`, where `t` is the number of examples tried, and `f` 1198 is the number of examples that failed. The examples are run 1199 in the namespace `test.globs`. 1200 """ 1201 # Keep track of the number of failures and tries. 1202 failures = tries = 0 1203 1204 # Save the option flags (since option directives can be used 1205 # to modify them). 1206 original_optionflags = self.optionflags 1207 1208 # Process each example. 1209 for example in test.examples: 1210 # Merge in the example's options. 1211 self.optionflags = original_optionflags 1212 if example.options: 1213 for (optionflag, val) in example.options.items(): 1214 if val: 1215 self.optionflags |= optionflag 1216 else: 1217 self.optionflags &= ~optionflag 1218 1219 # Record that we started this example. 1220 tries += 1 1221 self.report_start(out, test, example) 1222 1223 # Run the example in the given context (globs), and record 1224 # any exception that gets raised. (But don't intercept 1225 # keyboard interrupts.) 1226 try: 1227 # Don't blink! This is where the user's code gets run. 1228 exec compile(example.source, "<string>", "single", 1229 compileflags, 1) in test.globs 1230 exception = None 1231 except KeyboardInterrupt: 1232 raise 1233 except: 1234 exception = sys.exc_info() 1235 1236 got = self._fakeout.getvalue() # the actual output 1237 self._fakeout.truncate(0) 1238 1239 # If the example executed without raising any exceptions, 1240 # then verify its output and report its outcome. 1241 if exception is None: 1242 if self._checker.check_output(example.want, got, 1243 self.optionflags): 1244 self.report_success(out, test, example, got) 1245 else: 1246 self.report_failure(out, test, example, got) 1247 failures += 1 1248 1249 # If the example raised an exception, then check if it was 1250 # expected. 1251 else: 1252 exc_info = sys.exc_info() 1253 exc_msg = traceback.format_exception_only(*exc_info[:2])[-1] 1254 1255 # Search the `want` string for an exception. If we don't 1256 # find one, then report an unexpected exception. 1257 m = self._EXCEPTION_RE.match(example.want) 1258 if m is None: 1259 self.report_unexpected_exception(out, test, example, 1260 exc_info) 1261 failures += 1 1262 else: 1263 exc_hdr = m.group('hdr')+'\n' # Exception header 1264 # The test passes iff the pre-exception output and 1265 # the exception description match the values given 1266 # in `want`. 1267 if (self._checker.check_output(m.group('out'), got, 1268 self.optionflags) and 1269 self._checker.check_output(m.group('exc'), exc_msg, 1270 self.optionflags)): 1271 # Is +exc_msg the right thing here?? 1272 self.report_success(out, test, example, 1273 got+_exception_traceback(exc_info)) 1274 else: 1275 self.report_failure(out, test, example, 1276 got+_exception_traceback(exc_info)) 1277 failures += 1 1278 1279 # Restore the option flags (in case they were modified) 1280 self.optionflags = original_optionflags 1281 1282 # Record and return the number of failures and tries. 1283 self.__record_outcome(test, failures, tries) 1284 return failures, tries 1285 1286 def __record_outcome(self, test, f, t): 1287 """ 1288 Record the fact that the given DocTest (`test`) generated `f` 1289 failures out of `t` tried examples. 1290 """ 1291 f2, t2 = self._name2ft.get(test.name, (0,0)) 1292 self._name2ft[test.name] = (f+f2, t+t2) 1293 self.failures += f 1294 self.tries += t 1295 1296 def run(self, test, compileflags=None, out=None, clear_globs=True): 1297 """ 1298 Run the examples in `test`, and display the results using the 1299 writer function `out`. 1300 1301 The examples are run in the namespace `test.globs`. If 1302 `clear_globs` is true (the default), then this namespace will 1303 be cleared after the test runs, to help with garbage 1304 collection. If you would like to examine the namespace after 1305 the test completes, then use `clear_globs=False`. 1306 1307 `compileflags` gives the set of flags that should be used by 1308 the Python compiler when running the examples. If not 1309 specified, then it will default to the set of future-import 1310 flags that apply to `globs`. 1311 1312 The output of each example is checked using 1313 `DocTestRunner.check_output`, and the results are formatted by 1314 the `DocTestRunner.report_*` methods. 1315 """ 1316 if compileflags is None: 1317 compileflags = _extract_future_flags(test.globs) 1318 1319 save_stdout = sys.stdout 1320 if out is None: 1321 out = save_stdout.write 1322 sys.stdout = self._fakeout 1323 1324 # Patch pdb.set_trace to restore sys.stdout, so that interactive 1325 # debugging output is visible (not still redirected to self._fakeout). 1326 # Note that we run "the real" pdb.set_trace (captured at doctest 1327 # import time) in our replacement. Because the current run() may 1328 # run another doctest (and so on), the current pdb.set_trace may be 1329 # our set_trace function, which changes sys.stdout. If we called 1330 # a chain of those, we wouldn't be left with the save_stdout 1331 # *this* run() invocation wants. 1332 def set_trace(): 1333 sys.stdout = save_stdout 1334 real_pdb_set_trace() 1335 1336 save_set_trace = pdb.set_trace 1337 pdb.set_trace = set_trace 1338 try: 1339 return self.__run(test, compileflags, out) 1340 finally: 1341 sys.stdout = save_stdout 1342 pdb.set_trace = save_set_trace 1343 if clear_globs: 1344 test.globs.clear() 1345 1346 #///////////////////////////////////////////////////////////////// 1347 # Summarization 1348 #///////////////////////////////////////////////////////////////// 1349 def summarize(self, verbose=None): 1350 """ 1351 Print a summary of all the test cases that have been run by 1352 this DocTestRunner, and return a tuple `(f, t)`, where `f` is 1353 the total number of failed examples, and `t` is the total 1354 number of tried examples. 1355 1356 The optional `verbose` argument controls how detailed the 1357 summary is. If the verbosity is not specified, then the 1358 DocTestRunner's verbosity is used. 1359 """ 1360 if verbose is None: 1361 verbose = self._verbose 1362 notests = [] 1363 passed = [] 1364 failed = [] 1365 totalt = totalf = 0 1366 for x in self._name2ft.items(): 1367 name, (f, t) = x 1368 assert f <= t 1369 totalt += t 1370 totalf += f 1371 if t == 0: 1372 notests.append(name) 1373 elif f == 0: 1374 passed.append( (name, t) ) 1375 else: 1376 failed.append(x) 1377 if verbose: 1378 if notests: 1379 print len(notests), "items had no tests:" 1380 notests.sort() 1381 for thing in notests: 1382 print " ", thing 1383 if passed: 1384 print len(passed), "items passed all tests:" 1385 passed.sort() 1386 for thing, count in passed: 1387 print " %3d tests in %s" % (count, thing) 1388 if failed: 1389 print self.DIVIDER 1390 print len(failed), "items had failures:" 1391 failed.sort() 1392 for thing, (f, t) in failed: 1393 print " %3d of %3d in %s" % (f, t, thing) 1394 if verbose: 1395 print totalt, "tests in", len(self._name2ft), "items." 1396 print totalt - totalf, "passed and", totalf, "failed." 1397 if totalf: 1398 print "***Test Failed***", totalf, "failures." 1399 elif verbose: 1400 print "Test passed." 1401 return totalf, totalt 1402 1403class OutputChecker: 1404 """ 1405 A class used to check the whether the actual output from a doctest 1406 example matches the expected output. `OutputChecker` defines two 1407 methods: `check_output`, which compares a given pair of outputs, 1408 and returns true if they match; and `output_difference`, which 1409 returns a string describing the differences between two outputs. 1410 """ 1411 def check_output(self, want, got, optionflags): 1412 """ 1413 Return True iff the actual output from an example (`got`) 1414 matches the expected output (`want`). These strings are 1415 always considered to match if they are identical; but 1416 depending on what option flags the test runner is using, 1417 several non-exact match types are also possible. See the 1418 documentation for `TestRunner` for more information about 1419 option flags. 1420 """ 1421 # Handle the common case first, for efficiency: 1422 # if they're string-identical, always return true. 1423 if got == want: 1424 return True 1425 1426 # The values True and False replaced 1 and 0 as the return 1427 # value for boolean comparisons in Python 2.3. 1428 if not (optionflags & DONT_ACCEPT_TRUE_FOR_1): 1429 if (got,want) == ("True\n", "1\n"): 1430 return True 1431 if (got,want) == ("False\n", "0\n"): 1432 return True 1433 1434 # <BLANKLINE> can be used as a special sequence to signify a 1435 # blank line, unless the DONT_ACCEPT_BLANKLINE flag is used. 1436 if not (optionflags & DONT_ACCEPT_BLANKLINE): 1437 # Replace <BLANKLINE> in want with a blank line. 1438 want = re.sub('(?m)^%s\s*?$' % re.escape(BLANKLINE_MARKER), 1439 '', want) 1440 # If a line in got contains only spaces, then remove the 1441 # spaces. 1442 got = re.sub('(?m)^\s*?$', '', got) 1443 if got == want: 1444 return True 1445 1446 # This flag causes doctest to ignore any differences in the 1447 # contents of whitespace strings. Note that this can be used 1448 # in conjunction with the ELLISPIS flag. 1449 if (optionflags & NORMALIZE_WHITESPACE): 1450 got = ' '.join(got.split()) 1451 want = ' '.join(want.split()) 1452 if got == want: 1453 return True 1454 1455 # The ELLIPSIS flag says to let the sequence "..." in `want` 1456 # match any substring in `got`. We implement this by 1457 # transforming `want` into a regular expression. 1458 if (optionflags & ELLIPSIS): 1459 # Escape any special regexp characters 1460 want_re = re.escape(want) 1461 # Replace ellipsis markers ('...') with .* 1462 want_re = want_re.replace(re.escape(ELLIPSIS_MARKER), '.*') 1463 # Require that it matches the entire string; and set the 1464 # re.DOTALL flag (with '(?s)'). 1465 want_re = '(?s)^%s$' % want_re 1466 # Check if the `want_re` regexp matches got. 1467 if re.match(want_re, got): 1468 return True 1469 1470 # We didn't find any match; return false. 1471 return False 1472 1473 def output_difference(self, want, got, optionflags): 1474 """ 1475 Return a string describing the differences between the 1476 expected output for an example (`want`) and the actual output 1477 (`got`). `optionflags` is the set of option flags used to 1478 compare `want` and `got`. `indent` is the indentation of the 1479 original example. 1480 """ 1481 # If <BLANKLINE>s are being used, then replace <BLANKLINE> 1482 # with blank lines in the expected output string. 1483 if not (optionflags & DONT_ACCEPT_BLANKLINE): 1484 want = re.sub('(?m)^%s$' % re.escape(BLANKLINE_MARKER), '', want) 1485 1486 # Check if we should use diff. Don't use diff if the actual 1487 # or expected outputs are too short, or if the expected output 1488 # contains an ellipsis marker. 1489 if ((optionflags & (UNIFIED_DIFF | CONTEXT_DIFF)) and 1490 want.count('\n') > 2 and got.count('\n') > 2 and 1491 not (optionflags & ELLIPSIS and '...' in want)): 1492 # Split want & got into lines. 1493 want_lines = [l+'\n' for l in want.split('\n')] 1494 got_lines = [l+'\n' for l in got.split('\n')] 1495 # Use difflib to find their differences. 1496 if optionflags & UNIFIED_DIFF: 1497 diff = difflib.unified_diff(want_lines, got_lines, n=2, 1498 fromfile='Expected', tofile='Got') 1499 kind = 'unified' 1500 elif optionflags & CONTEXT_DIFF: 1501 diff = difflib.context_diff(want_lines, got_lines, n=2, 1502 fromfile='Expected', tofile='Got') 1503 kind = 'context' 1504 else: 1505 assert 0, 'Bad diff option' 1506 # Remove trailing whitespace on diff output. 1507 diff = [line.rstrip() + '\n' for line in diff] 1508 return _tag_msg("Differences (" + kind + " diff)", 1509 ''.join(diff)) 1510 1511 # If we're not using diff, then simply list the expected 1512 # output followed by the actual output. 1513 return (_tag_msg("Expected", want or "Nothing") + 1514 _tag_msg("Got", got)) 1515 1516class DocTestFailure(Exception): 1517 """A DocTest example has failed in debugging mode. 1518 1519 The exception instance has variables: 1520 1521 - test: the DocTest object being run 1522 1523 - excample: the Example object that failed 1524 1525 - got: the actual output 1526 """ 1527 def __init__(self, test, example, got): 1528 self.test = test 1529 self.example = example 1530 self.got = got 1531 1532 def __str__(self): 1533 return str(self.test) 1534 1535class UnexpectedException(Exception): 1536 """A DocTest example has encountered an unexpected exception 1537 1538 The exception instance has variables: 1539 1540 - test: the DocTest object being run 1541 1542 - excample: the Example object that failed 1543 1544 - exc_info: the exception info 1545 """ 1546 def __init__(self, test, example, exc_info): 1547 self.test = test 1548 self.example = example 1549 self.exc_info = exc_info 1550 1551 def __str__(self): 1552 return str(self.test) 1553 1554class DebugRunner(DocTestRunner): 1555 r"""Run doc tests but raise an exception as soon as there is a failure. 1556 1557 If an unexpected exception occurs, an UnexpectedException is raised. 1558 It contains the test, the example, and the original exception: 1559 1560 >>> runner = DebugRunner(verbose=False) 1561 >>> test = DocTestParser().get_doctest('>>> raise KeyError\n42', 1562 ... {}, 'foo', 'foo.py', 0) 1563 >>> try: 1564 ... runner.run(test) 1565 ... except UnexpectedException, failure: 1566 ... pass 1567 1568 >>> failure.test is test 1569 True 1570 1571 >>> failure.example.want 1572 '42\n' 1573 1574 >>> exc_info = failure.exc_info 1575 >>> raise exc_info[0], exc_info[1], exc_info[2] 1576 Traceback (most recent call last): 1577 ... 1578 KeyError 1579 1580 We wrap the original exception to give the calling application 1581 access to the test and example information. 1582 1583 If the output doesn't match, then a DocTestFailure is raised: 1584 1585 >>> test = DocTestParser().get_doctest(''' 1586 ... >>> x = 1 1587 ... >>> x 1588 ... 2 1589 ... ''', {}, 'foo', 'foo.py', 0) 1590 1591 >>> try: 1592 ... runner.run(test) 1593 ... except DocTestFailure, failure: 1594 ... pass 1595 1596 DocTestFailure objects provide access to the test: 1597 1598 >>> failure.test is test 1599 True 1600 1601 As well as to the example: 1602 1603 >>> failure.example.want 1604 '2\n' 1605 1606 and the actual output: 1607 1608 >>> failure.got 1609 '1\n' 1610 1611 If a failure or error occurs, the globals are left intact: 1612 1613 >>> del test.globs['__builtins__'] 1614 >>> test.globs 1615 {'x': 1} 1616 1617 >>> test = DocTestParser().get_doctest(''' 1618 ... >>> x = 2 1619 ... >>> raise KeyError 1620 ... ''', {}, 'foo', 'foo.py', 0) 1621 1622 >>> runner.run(test) 1623 Traceback (most recent call last): 1624 ... 1625 UnexpectedException: <DocTest foo from foo.py:0 (2 examples)> 1626 1627 >>> del test.globs['__builtins__'] 1628 >>> test.globs 1629 {'x': 2} 1630 1631 But the globals are cleared if there is no error: 1632 1633 >>> test = DocTestParser().get_doctest(''' 1634 ... >>> x = 2 1635 ... ''', {}, 'foo', 'foo.py', 0) 1636 1637 >>> runner.run(test) 1638 (0, 1) 1639 1640 >>> test.globs 1641 {} 1642 1643 """ 1644 1645 def run(self, test, compileflags=None, out=None, clear_globs=True): 1646 r = DocTestRunner.run(self, test, compileflags, out, False) 1647 if clear_globs: 1648 test.globs.clear() 1649 return r 1650 1651 def report_unexpected_exception(self, out, test, example, exc_info): 1652 raise UnexpectedException(test, example, exc_info) 1653 1654 def report_failure(self, out, test, example, got): 1655 raise DocTestFailure(test, example, got) 1656 1657###################################################################### 1658## 6. Test Functions 1659###################################################################### 1660# These should be backwards compatible. 1661 1662def testmod(m=None, name=None, globs=None, verbose=None, isprivate=None, 1663 report=True, optionflags=0, extraglobs=None, 1664 raise_on_error=False): 1665 """m=None, name=None, globs=None, verbose=None, isprivate=None, 1666 report=True, optionflags=0, extraglobs=None 1667 1668 Test examples in docstrings in functions and classes reachable 1669 from module m (or the current module if m is not supplied), starting 1670 with m.__doc__. Unless isprivate is specified, private names 1671 are not skipped. 1672 1673 Also test examples reachable from dict m.__test__ if it exists and is 1674 not None. m.__test__ maps names to functions, classes and strings; 1675 function and class docstrings are tested even if the name is private; 1676 strings are tested directly, as if they were docstrings. 1677 1678 Return (#failures, #tests). 1679 1680 See doctest.__doc__ for an overview. 1681 1682 Optional keyword arg "name" gives the name of the module; by default 1683 use m.__name__. 1684 1685 Optional keyword arg "globs" gives a dict to be used as the globals 1686 when executing examples; by default, use m.__dict__. A copy of this 1687 dict is actually used for each docstring, so that each docstring's 1688 examples start with a clean slate. 1689 1690 Optional keyword arg "extraglobs" gives a dictionary that should be 1691 merged into the globals that are used to execute examples. By 1692 default, no extra globals are used. This is new in 2.4. 1693 1694 Optional keyword arg "verbose" prints lots of stuff if true, prints 1695 only failures if false; by default, it's true iff "-v" is in sys.argv. 1696 1697 Optional keyword arg "report" prints a summary at the end when true, 1698 else prints nothing at the end. In verbose mode, the summary is 1699 detailed, else very brief (in fact, empty if all tests passed). 1700 1701 Optional keyword arg "optionflags" or's together module constants, 1702 and defaults to 0. This is new in 2.3. Possible values: 1703 1704 DONT_ACCEPT_TRUE_FOR_1 1705 By default, if an expected output block contains just "1", 1706 an actual output block containing just "True" is considered 1707 to be a match, and similarly for "0" versus "False". When 1708 DONT_ACCEPT_TRUE_FOR_1 is specified, neither substitution 1709 is allowed. 1710 1711 DONT_ACCEPT_BLANKLINE 1712 By default, if an expected output block contains a line 1713 containing only the string "<BLANKLINE>", then that line 1714 will match a blank line in the actual output. When 1715 DONT_ACCEPT_BLANKLINE is specified, this substitution is 1716 not allowed. 1717 1718 NORMALIZE_WHITESPACE 1719 When NORMALIZE_WHITESPACE is specified, all sequences of 1720 whitespace are treated as equal. I.e., any sequence of 1721 whitespace within the expected output will match any 1722 sequence of whitespace within the actual output. 1723 1724 ELLIPSIS 1725 When ELLIPSIS is specified, then an ellipsis marker 1726 ("...") in the expected output can match any substring in 1727 the actual output. 1728 1729 UNIFIED_DIFF 1730 When UNIFIED_DIFF is specified, failures that involve 1731 multi-line expected and actual outputs will be displayed 1732 using a unified diff. 1733 1734 CONTEXT_DIFF 1735 When CONTEXT_DIFF is specified, failures that involve 1736 multi-line expected and actual outputs will be displayed 1737 using a context diff. 1738 1739 Optional keyword arg "raise_on_error" raises an exception on the 1740 first unexpected exception or failure. This allows failures to be 1741 post-mortem debugged. 1742 1743 Deprecated in Python 2.4: 1744 Optional keyword arg "isprivate" specifies a function used to 1745 determine whether a name is private. The default function is 1746 treat all functions as public. Optionally, "isprivate" can be 1747 set to doctest.is_private to skip over functions marked as private 1748 using the underscore naming convention; see its docs for details. 1749 """ 1750 1751 """ [XX] This is no longer true: 1752 Advanced tomfoolery: testmod runs methods of a local instance of 1753 class doctest.Tester, then merges the results into (or creates) 1754 global Tester instance doctest.master. Methods of doctest.master 1755 can be called directly too, if you want to do something unusual. 1756 Passing report=0 to testmod is especially useful then, to delay 1757 displaying a summary. Invoke doctest.master.summarize(verbose) 1758 when you're done fiddling. 1759 """ 1760 if isprivate is not None: 1761 warnings.warn("the isprivate argument is deprecated; " 1762 "examine DocTestFinder.find() lists instead", 1763 DeprecationWarning) 1764 1765 # If no module was given, then use __main__. 1766 if m is None: 1767 # DWA - m will still be None if this wasn't invoked from the command 1768 # line, in which case the following TypeError is about as good an error 1769 # as we should expect 1770 m = sys.modules.get('__main__') 1771 1772 # Check that we were actually given a module. 1773 if not inspect.ismodule(m): 1774 raise TypeError("testmod: module required; %r" % (m,)) 1775 1776 # If no name was given, then use the module's name. 1777 if name is None: 1778 name = m.__name__ 1779 1780 # Find, parse, and run all tests in the given module. 1781 finder = DocTestFinder(_namefilter=isprivate) 1782 1783 if raise_on_error: 1784 runner = DebugRunner(verbose=verbose, optionflags=optionflags) 1785 else: 1786 runner = DocTestRunner(verbose=verbose, optionflags=optionflags) 1787 1788 for test in finder.find(m, name, globs=globs, extraglobs=extraglobs): 1789 runner.run(test) 1790 1791 if report: 1792 runner.summarize() 1793 1794 return runner.failures, runner.tries 1795 1796def run_docstring_examples(f, globs, verbose=False, name="NoName", 1797 compileflags=None, optionflags=0): 1798 """ 1799 Test examples in the given object's docstring (`f`), using `globs` 1800 as globals. Optional argument `name` is used in failure messages. 1801 If the optional argument `verbose` is true, then generate output 1802 even if there are no failures. 1803 1804 `compileflags` gives the set of flags that should be used by the 1805 Python compiler when running the examples. If not specified, then 1806 it will default to the set of future-import flags that apply to 1807 `globs`. 1808 1809 Optional keyword arg `optionflags` specifies options for the 1810 testing and output. See the documentation for `testmod` for more 1811 information. 1812 """ 1813 # Find, parse, and run all tests in the given module. 1814 finder = DocTestFinder(verbose=verbose, recurse=False) 1815 runner = DocTestRunner(verbose=verbose, optionflags=optionflags) 1816 for test in finder.find(f, name, globs=globs): 1817 runner.run(test, compileflags=compileflags) 1818 1819###################################################################### 1820## 7. Tester 1821###################################################################### 1822# This is provided only for backwards compatibility. It's not 1823# actually used in any way. 1824 1825class Tester: 1826 def __init__(self, mod=None, globs=None, verbose=None, 1827 isprivate=None, optionflags=0): 1828 1829 warnings.warn("class Tester is deprecated; " 1830 "use class doctest.DocTestRunner instead", 1831 DeprecationWarning, stacklevel=2) 1832 if mod is None and globs is None: 1833 raise TypeError("Tester.__init__: must specify mod or globs") 1834 if mod is not None and not _ismodule(mod): 1835 raise TypeError("Tester.__init__: mod must be a module; %r" % 1836 (mod,)) 1837 if globs is None: 1838 globs = mod.__dict__ 1839 self.globs = globs 1840 1841 self.verbose = verbose 1842 self.isprivate = isprivate 1843 self.optionflags = optionflags 1844 self.testfinder = DocTestFinder(_namefilter=isprivate) 1845 self.testrunner = DocTestRunner(verbose=verbose, 1846 optionflags=optionflags) 1847 1848 def runstring(self, s, name): 1849 test = DocTestParser().get_doctest(s, self.globs, name, None, None) 1850 if self.verbose: 1851 print "Running string", name 1852 (f,t) = self.testrunner.run(test) 1853 if self.verbose: 1854 print f, "of", t, "examples failed in string", name 1855 return (f,t) 1856 1857 def rundoc(self, object, name=None, module=None): 1858 f = t = 0 1859 tests = self.testfinder.find(object, name, module=module, 1860 globs=self.globs) 1861 for test in tests: 1862 (f2, t2) = self.testrunner.run(test) 1863 (f,t) = (f+f2, t+t2) 1864 return (f,t) 1865 1866 def rundict(self, d, name, module=None): 1867 import new 1868 m = new.module(name) 1869 m.__dict__.update(d) 1870 if module is None: 1871 module = False 1872 return self.rundoc(m, name, module) 1873 1874 def run__test__(self, d, name): 1875 import new 1876 m = new.module(name) 1877 m.__test__ = d 1878 return self.rundoc(m, name, module) 1879 1880 def summarize(self, verbose=None): 1881 return self.testrunner.summarize(verbose) 1882 1883 def merge(self, other): 1884 d = self.testrunner._name2ft 1885 for name, (f, t) in other.testrunner._name2ft.items(): 1886 if name in d: 1887 print "*** Tester.merge: '" + name + "' in both" \ 1888 " testers; summing outcomes." 1889 f2, t2 = d[name] 1890 f = f + f2 1891 t = t + t2 1892 d[name] = f, t 1893 1894###################################################################### 1895## 8. Unittest Support 1896###################################################################### 1897 1898class DocTestCase(unittest.TestCase): 1899 1900 def __init__(self, test, optionflags=0, setUp=None, tearDown=None, 1901 checker=None): 1902 unittest.TestCase.__init__(self) 1903 self._dt_optionflags = optionflags 1904 self._dt_checker = checker 1905 self._dt_test = test 1906 self._dt_setUp = setUp 1907 self._dt_tearDown = tearDown 1908 1909 def setUp(self): 1910 if self._dt_setUp is not None: 1911 self._dt_setUp() 1912 1913 def tearDown(self): 1914 if self._dt_tearDown is not None: 1915 self._dt_tearDown() 1916 1917 def runTest(self): 1918 test = self._dt_test 1919 old = sys.stdout 1920 new = StringIO() 1921 runner = DocTestRunner(optionflags=self._dt_optionflags, 1922 checker=self._dt_checker, verbose=False) 1923 1924 try: 1925 runner.DIVIDER = "-"*70 1926 failures, tries = runner.run(test, out=new.write) 1927 finally: 1928 sys.stdout = old 1929 1930 if failures: 1931 raise self.failureException(self.format_failure(new.getvalue())) 1932 1933 def format_failure(self, err): 1934 test = self._dt_test 1935 if test.lineno is None: 1936 lineno = 'unknown line number' 1937 else: 1938 lineno = 'line %s' % test.lineno 1939 lname = '.'.join(test.name.split('.')[-1:]) 1940 return ('Failed doctest test for %s\n' 1941 ' File "%s", line %s, in %s\n\n%s' 1942 % (test.name, test.filename, lineno, lname, err) 1943 ) 1944 1945 def debug(self): 1946 r"""Run the test case without results and without catching exceptions 1947 1948 The unit test framework includes a debug method on test cases 1949 and test suites to support post-mortem debugging. The test code 1950 is run in such a way that errors are not caught. This way a 1951 caller can catch the errors and initiate post-mortem debugging. 1952 1953 The DocTestCase provides a debug method that raises 1954 UnexpectedException errors if there is an unexepcted 1955 exception: 1956 1957 >>> test = DocTestParser().get_doctest('>>> raise KeyError\n42', 1958 ... {}, 'foo', 'foo.py', 0) 1959 >>> case = DocTestCase(test) 1960 >>> try: 1961 ... case.debug() 1962 ... except UnexpectedException, failure: 1963 ... pass 1964 1965 The UnexpectedException contains the test, the example, and 1966 the original exception: 1967 1968 >>> failure.test is test 1969 True 1970 1971 >>> failure.example.want 1972 '42\n' 1973 1974 >>> exc_info = failure.exc_info 1975 >>> raise exc_info[0], exc_info[1], exc_info[2] 1976 Traceback (most recent call last): 1977 ... 1978 KeyError 1979 1980 If the output doesn't match, then a DocTestFailure is raised: 1981 1982 >>> test = DocTestParser().get_doctest(''' 1983 ... >>> x = 1 1984 ... >>> x 1985 ... 2 1986 ... ''', {}, 'foo', 'foo.py', 0) 1987 >>> case = DocTestCase(test) 1988 1989 >>> try: 1990 ... case.debug() 1991 ... except DocTestFailure, failure: 1992 ... pass 1993 1994 DocTestFailure objects provide access to the test: 1995 1996 >>> failure.test is test 1997 True 1998 1999 As well as to the example: 2000 2001 >>> failure.example.want 2002 '2\n' 2003 2004 and the actual output: 2005 2006 >>> failure.got 2007 '1\n' 2008 2009 """ 2010 2011 runner = DebugRunner(optionflags=self._dt_optionflags, 2012 checker=self._dt_checker, verbose=False) 2013 runner.run(self._dt_test, out=nooutput) 2014 2015 def id(self): 2016 return self._dt_test.name 2017 2018 def __repr__(self): 2019 name = self._dt_test.name.split('.') 2020 return "%s (%s)" % (name[-1], '.'.join(name[:-1])) 2021 2022 __str__ = __repr__ 2023 2024 def shortDescription(self): 2025 return "Doctest: " + self._dt_test.name 2026 2027def nooutput(*args): 2028 pass 2029 2030def DocTestSuite(module=None, globs=None, extraglobs=None, 2031 optionflags=0, test_finder=None, 2032 setUp=lambda: None, tearDown=lambda: None, 2033 checker=None): 2034 """ 2035 Convert doctest tests for a mudule to a unittest test suite. 2036 2037 This converts each documentation string in a module that 2038 contains doctest tests to a unittest test case. If any of the 2039 tests in a doc string fail, then the test case fails. An exception 2040 is raised showing the name of the file containing the test and a 2041 (sometimes approximate) line number. 2042 2043 The `module` argument provides the module to be tested. The argument 2044 can be either a module or a module name. 2045 2046 If no argument is given, the calling module is used. 2047 """ 2048 2049 if test_finder is None: 2050 test_finder = DocTestFinder() 2051 2052 module = _normalize_module(module) 2053 tests = test_finder.find(module, globs=globs, extraglobs=extraglobs) 2054 if globs is None: 2055 globs = module.__dict__ 2056 if not tests: # [XX] why do we want to do this? 2057 raise ValueError(module, "has no tests") 2058 2059 tests.sort() 2060 suite = unittest.TestSuite() 2061 for test in tests: 2062 if len(test.examples) == 0: 2063 continue 2064 if not test.filename: 2065 filename = module.__file__ 2066 if filename.endswith(".pyc"): 2067 filename = filename[:-1] 2068 elif filename.endswith(".pyo"): 2069 filename = filename[:-1] 2070 test.filename = filename 2071 suite.addTest(DocTestCase(test, optionflags, setUp, tearDown, 2072 checker)) 2073 2074 return suite 2075 2076class DocFileCase(DocTestCase): 2077 2078 def id(self): 2079 return '_'.join(self._dt_test.name.split('.')) 2080 2081 def __repr__(self): 2082 return self._dt_test.filename 2083 __str__ = __repr__ 2084 2085 def format_failure(self, err): 2086 return ('Failed doctest test for %s\n File "%s", line 0\n\n%s' 2087 % (self._dt_test.name, self._dt_test.filename, err) 2088 ) 2089 2090def DocFileTest(path, package=None, globs=None, 2091 setUp=None, tearDown=None, 2092 optionflags=0): 2093 package = _normalize_module(package) 2094 name = path.split('/')[-1] 2095 dir = os.path.split(package.__file__)[0] 2096 path = os.path.join(dir, *(path.split('/'))) 2097 doc = open(path).read() 2098 2099 if globs is None: 2100 globs = {} 2101 2102 test = DocTestParser().get_doctest(doc, globs, name, path, 0) 2103 2104 return DocFileCase(test, optionflags, setUp, tearDown) 2105 2106def DocFileSuite(*paths, **kw): 2107 """Creates a suite of doctest files. 2108 2109 One or more text file paths are given as strings. These should 2110 use "/" characters to separate path segments. Paths are relative 2111 to the directory of the calling module, or relative to the package 2112 passed as a keyword argument. 2113 2114 A number of options may be provided as keyword arguments: 2115 2116 package 2117 The name of a Python package. Text-file paths will be 2118 interpreted relative to the directory containing this package. 2119 The package may be supplied as a package object or as a dotted 2120 package name. 2121 2122 setUp 2123 The name of a set-up function. This is called before running the 2124 tests in each file. 2125 2126 tearDown 2127 The name of a tear-down function. This is called after running the 2128 tests in each file. 2129 2130 globs 2131 A dictionary containing initial global variables for the tests. 2132 """ 2133 suite = unittest.TestSuite() 2134 2135 # We do this here so that _normalize_module is called at the right 2136 # level. If it were called in DocFileTest, then this function 2137 # would be the caller and we might guess the package incorrectly. 2138 kw['package'] = _normalize_module(kw.get('package')) 2139 2140 for path in paths: 2141 suite.addTest(DocFileTest(path, **kw)) 2142 2143 return suite 2144 2145###################################################################### 2146## 9. Debugging Support 2147###################################################################### 2148 2149def script_from_examples(s): 2150 r"""Extract script from text with examples. 2151 2152 Converts text with examples to a Python script. Example input is 2153 converted to regular code. Example output and all other words 2154 are converted to comments: 2155 2156 >>> text = ''' 2157 ... Here are examples of simple math. 2158 ... 2159 ... Python has super accurate integer addition 2160 ... 2161 ... >>> 2 + 2 2162 ... 5 2163 ... 2164 ... And very friendly error messages: 2165 ... 2166 ... >>> 1/0 2167 ... To Infinity 2168 ... And 2169 ... Beyond 2170 ... 2171 ... You can use logic if you want: 2172 ... 2173 ... >>> if 0: 2174 ... ... blah 2175 ... ... blah 2176 ... ... 2177 ... 2178 ... Ho hum 2179 ... ''' 2180 2181 >>> print script_from_examples(text) 2182 # Here are examples of simple math. 2183 # 2184 # Python has super accurate integer addition 2185 # 2186 2 + 2 2187 # Expected: 2188 # 5 2189 # 2190 # And very friendly error messages: 2191 # 2192 1/0 2193 # Expected: 2194 # To Infinity 2195 # And 2196 # Beyond 2197 # 2198 # You can use logic if you want: 2199 # 2200 if 0: 2201 blah 2202 blah 2203 <BLANKLINE> 2204 # 2205 # Ho hum 2206 """ 2207 2208 return DocTestParser().get_program(s) 2209 2210def _want_comment(example): 2211 """ 2212 Return a comment containing the expected output for the given example. 2213 """ 2214 # Return the expected output, if any 2215 want = example.want 2216 if want: 2217 if want[-1] == '\n': 2218 want = want[:-1] 2219 want = "\n# ".join(want.split("\n")) 2220 want = "\n# Expected:\n# %s" % want 2221 return want 2222 2223def testsource(module, name): 2224 """Extract the test sources from a doctest docstring as a script. 2225 2226 Provide the module (or dotted name of the module) containing the 2227 test to be debugged and the name (within the module) of the object 2228 with the doc string with tests to be debugged. 2229 """ 2230 module = _normalize_module(module) 2231 tests = DocTestFinder().find(module) 2232 test = [t for t in tests if t.name == name] 2233 if not test: 2234 raise ValueError(name, "not found in tests") 2235 test = test[0] 2236 testsrc = script_from_examples(test.docstring) 2237 return testsrc 2238 2239def debug_src(src, pm=False, globs=None): 2240 """Debug a single doctest docstring, in argument `src`'""" 2241 testsrc = script_from_examples(src) 2242 debug_script(testsrc, pm, globs) 2243 2244def debug_script(src, pm=False, globs=None): 2245 "Debug a test script. `src` is the script, as a string." 2246 import pdb 2247 2248 srcfilename = tempfile.mktemp("doctestdebug.py") 2249 f = open(srcfilename, 'w') 2250 f.write(src) 2251 f.close() 2252 2253 if globs: 2254 globs = globs.copy() 2255 else: 2256 globs = {} 2257 2258 if pm: 2259 try: 2260 execfile(srcfilename, globs, globs) 2261 except: 2262 print sys.exc_info()[1] 2263 pdb.post_mortem(sys.exc_info()[2]) 2264 else: 2265 # Note that %r is vital here. '%s' instead can, e.g., cause 2266 # backslashes to get treated as metacharacters on Windows. 2267 pdb.run("execfile(%r)" % srcfilename, globs, globs) 2268 2269def debug(module, name, pm=False): 2270 """Debug a single doctest docstring. 2271 2272 Provide the module (or dotted name of the module) containing the 2273 test to be debugged and the name (within the module) of the object 2274 with the docstring with tests to be debugged. 2275 """ 2276 module = _normalize_module(module) 2277 testsrc = testsource(module, name) 2278 debug_script(testsrc, pm, module.__dict__) 2279 2280###################################################################### 2281## 10. Example Usage 2282###################################################################### 2283class _TestClass: 2284 """ 2285 A pointless class, for sanity-checking of docstring testing. 2286 2287 Methods: 2288 square() 2289 get() 2290 2291 >>> _TestClass(13).get() + _TestClass(-12).get() 2292 1 2293 >>> hex(_TestClass(13).square().get()) 2294 '0xa9' 2295 """ 2296 2297 def __init__(self, val): 2298 """val -> _TestClass object with associated value val. 2299 2300 >>> t = _TestClass(123) 2301 >>> print t.get() 2302 123 2303 """ 2304 2305 self.val = val 2306 2307 def square(self): 2308 """square() -> square TestClass's associated value 2309 2310 >>> _TestClass(13).square().get() 2311 169 2312 """ 2313 2314 self.val = self.val ** 2 2315 return self 2316 2317 def get(self): 2318 """get() -> return TestClass's associated value. 2319 2320 >>> x = _TestClass(-42) 2321 >>> print x.get() 2322 -42 2323 """ 2324 2325 return self.val 2326 2327__test__ = {"_TestClass": _TestClass, 2328 "string": r""" 2329 Example of a string object, searched as-is. 2330 >>> x = 1; y = 2 2331 >>> x + y, x * y 2332 (3, 2) 2333 """, 2334 "bool-int equivalence": r""" 2335 In 2.2, boolean expressions displayed 2336 0 or 1. By default, we still accept 2337 them. This can be disabled by passing 2338 DONT_ACCEPT_TRUE_FOR_1 to the new 2339 optionflags argument. 2340 >>> 4 == 4 2341 1 2342 >>> 4 == 4 2343 True 2344 >>> 4 > 4 2345 0 2346 >>> 4 > 4 2347 False 2348 """, 2349 "blank lines": r""" 2350 Blank lines can be marked with <BLANKLINE>: 2351 >>> print 'foo\n\nbar\n' 2352 foo 2353 <BLANKLINE> 2354 bar 2355 <BLANKLINE> 2356 """, 2357 } 2358# "ellipsis": r""" 2359# If the ellipsis flag is used, then '...' can be used to 2360# elide substrings in the desired output: 2361# >>> print range(1000) 2362# [0, 1, 2, ..., 999] 2363# """, 2364# "whitespace normalization": r""" 2365# If the whitespace normalization flag is used, then 2366# differences in whitespace are ignored. 2367# >>> print range(30) 2368# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 2369# 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 2370# 27, 28, 29] 2371# """, 2372# } 2373 2374def test1(): r""" 2375>>> warnings.filterwarnings("ignore", "class Tester", DeprecationWarning, 2376... "doctest", 0) 2377>>> from doctest import Tester 2378>>> t = Tester(globs={'x': 42}, verbose=0) 2379>>> t.runstring(r''' 2380... >>> x = x * 2 2381... >>> print x 2382... 42 2383... ''', 'XYZ') 2384********************************************************************** 2385Failure in example: print x 2386from line #2 of XYZ 2387Expected: 42 2388Got: 84 2389(1, 2) 2390>>> t.runstring(">>> x = x * 2\n>>> print x\n84\n", 'example2') 2391(0, 2) 2392>>> t.summarize() 2393********************************************************************** 23941 items had failures: 2395 1 of 2 in XYZ 2396***Test Failed*** 1 failures. 2397(1, 4) 2398>>> t.summarize(verbose=1) 23991 items passed all tests: 2400 2 tests in example2 2401********************************************************************** 24021 items had failures: 2403 1 of 2 in XYZ 24044 tests in 2 items. 24053 passed and 1 failed. 2406***Test Failed*** 1 failures. 2407(1, 4) 2408""" 2409 2410def test2(): r""" 2411 >>> warnings.filterwarnings("ignore", "class Tester", 2412 ... DeprecationWarning, "doctest", 0) 2413 >>> t = Tester(globs={}, verbose=1) 2414 >>> test = r''' 2415 ... # just an example 2416 ... >>> x = 1 + 2 2417 ... >>> x 2418 ... 3 2419 ... ''' 2420 >>> t.runstring(test, "Example") 2421 Running string Example 2422 Trying: x = 1 + 2 2423 Expecting: nothing 2424 ok 2425 Trying: x 2426 Expecting: 3 2427 ok 2428 0 of 2 examples failed in string Example 2429 (0, 2) 2430""" 2431def test3(): r""" 2432 >>> warnings.filterwarnings("ignore", "class Tester", 2433 ... DeprecationWarning, "doctest", 0) 2434 >>> t = Tester(globs={}, verbose=0) 2435 >>> def _f(): 2436 ... '''Trivial docstring example. 2437 ... >>> assert 2 == 2 2438 ... ''' 2439 ... return 32 2440 ... 2441 >>> t.rundoc(_f) # expect 0 failures in 1 example 2442 (0, 1) 2443""" 2444def test4(): """ 2445 >>> import new 2446 >>> m1 = new.module('_m1') 2447 >>> m2 = new.module('_m2') 2448 >>> test_data = \""" 2449 ... def _f(): 2450 ... '''>>> assert 1 == 1 2451 ... ''' 2452 ... def g(): 2453 ... '''>>> assert 2 != 1 2454 ... ''' 2455 ... class H: 2456 ... '''>>> assert 2 > 1 2457 ... ''' 2458 ... def bar(self): 2459 ... '''>>> assert 1 < 2 2460 ... ''' 2461 ... \""" 2462 >>> exec test_data in m1.__dict__ 2463 >>> exec test_data in m2.__dict__ 2464 >>> m1.__dict__.update({"f2": m2._f, "g2": m2.g, "h2": m2.H}) 2465 2466 Tests that objects outside m1 are excluded: 2467 2468 >>> warnings.filterwarnings("ignore", "class Tester", 2469 ... DeprecationWarning, "doctest", 0) 2470 >>> t = Tester(globs={}, verbose=0) 2471 >>> t.rundict(m1.__dict__, "rundict_test", m1) # f2 and g2 and h2 skipped 2472 (0, 4) 2473 2474 Once more, not excluding stuff outside m1: 2475 2476 >>> t = Tester(globs={}, verbose=0) 2477 >>> t.rundict(m1.__dict__, "rundict_test_pvt") # None are skipped. 2478 (0, 8) 2479 2480 The exclusion of objects from outside the designated module is 2481 meant to be invoked automagically by testmod. 2482 2483 >>> testmod(m1, verbose=False) 2484 (0, 4) 2485""" 2486 2487def _test(): 2488 #import doctest 2489 #doctest.testmod(doctest, verbose=False, 2490 # optionflags=ELLIPSIS | NORMALIZE_WHITESPACE | 2491 # UNIFIED_DIFF) 2492 #print '~'*70 2493 r = unittest.TextTestRunner() 2494 r.run(DocTestSuite()) 2495 2496if __name__ == "__main__": 2497 _test() 2498