pickletools.py revision 5445594e20baf1ac0c0c1aef30d55d789c4b9694
15445594e20baf1ac0c0c1aef30d55d789c4b9694Skip Montanaro'''"Executable documentation" for the pickle module. 28ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 38ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersExtensive comments about the pickle protocols and pickle-machine opcodes 48ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterscan be found here. Some functions meant for external use: 58ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 68ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersgenops(pickle) 78ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Generate all the opcodes in a pickle, as (opcode, arg, position) triples. 88ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 98ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdis(pickle, out=None, indentlevel=4) 108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Print a symbolic disassembly of a pickle. 115445594e20baf1ac0c0c1aef30d55d789c4b9694Skip Montanaro''' 128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# Other ideas: 148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# 158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# - A pickle verifier: read a pickle and check it exhaustively for 168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# well-formedness. 178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# 188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# - A protocol identifier: examine a pickle and return its protocol number 198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# (== the highest .proto attr value among all the opcodes in the pickle). 208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# 218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# - A pickle optimizer: for example, tuple-building code is sometimes more 228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# elaborate than necessary, catering for the possibility that the tuple 238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# is recursive. Or lots of times a PUT is generated that's never accessed 248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# by a later GET. 258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters""" 288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters"A pickle" is a program for a virtual pickle machine (PM, but more accurately 298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterscalled an unpickling machine). It's a sequence of opcodes, interpreted by the 308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersPM, building an arbitrarily complex Python object. 318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersFor the most part, the PM is very simple: there are no looping, testing, or 338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersconditional instructions, no arithmetic and no function calls. Opcodes are 348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersexecuted once each, from first to last, until a STOP opcode is reached. 358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersThe PM has two data areas, "the stack" and "the memo". 378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersMany opcodes push Python objects onto the stack; e.g., INT pushes a Python 398ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersinteger object on the stack, whose value is gotten from a decimal string 408ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersliteral immediately following the INT opcode in the pickle bytestream. Other 418ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersopcodes take Python objects off the stack. The result of unpickling is 428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterswhatever object is left on the stack when the final STOP opcode is executed. 438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersThe memo is simply an array of objects, or it can be implemented as a dict 458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersmapping little integers to objects. The memo serves as the PM's "long term 468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersmemory", and the little integers indexing the memo are akin to variable 478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersnames. Some opcodes pop a stack object into the memo at a given index, 488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersand others push a memo object at a given index onto the stack again. 498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersAt heart, that's all the PM has. Subtleties arise for these reasons: 518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters+ Object identity. Objects can be arbitrarily complex, and subobjects 538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters may be shared (for example, the list [a, a] refers to the same object a 548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters twice). It can be vital that unpickling recreate an isomorphic object 558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters graph, faithfully reproducing sharing. 568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters+ Recursive objects. For example, after "L = []; L.append(L)", L is a 588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters list, and L[0] is the same list. This is related to the object identity 598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters point, and some sequences of pickle opcodes are subtle in order to 608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters get the right result in all cases. 618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters+ Things pickle doesn't know everything about. Examples of things pickle 638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters does know everything about are Python's builtin scalar and container 648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters types, like ints and tuples. They generally have opcodes dedicated to 658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters them. For things like module references and instances of user-defined 668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters classes, pickle's knowledge is limited. Historically, many enhancements 678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters have been made to the pickle protocol in order to do a better (faster, 688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters and/or more compact) job on those. 698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters+ Backward compatibility and micro-optimization. As explained below, 718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters pickle opcodes never go away, not even when better ways to do a thing 728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters get invented. The repertoire of the PM just keeps growing over time. 73fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters For example, protocol 0 had two opcodes for building Python integers (INT 74fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters and LONG), protocol 1 added three more for more-efficient pickling of short 75fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters integers, and protocol 2 added two more for more-efficient pickling of 76fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters long integers (before protocol 2, the only ways to pickle a Python long 77fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters took time quadratic in the number of digits, for both pickling and 78fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters unpickling). "Opcode bloat" isn't so much a subtlety as a source of 798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters wearying complication. 808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersPickle protocols: 838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersFor compatibility, the meaning of a pickle opcode never changes. Instead new 858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspickle opcodes get added, and each version's unpickler can handle all the 868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspickle opcodes in all protocol versions to date. So old pickles continue to 878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersbe readable forever. The pickler can generally be told to restrict itself to 888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersthe subset of opcodes available under previous protocol versions too, so that 898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersusers can create pickles under the current version readable by older 908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersversions. However, a pickle does not contain its version number embedded 918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterswithin it. If an older unpickler tries to read a pickle using a later 928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersprotocol, the result is most likely an exception due to seeing an unknown (in 938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersthe older unpickler) opcode. 948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersThe original pickle used what's now called "protocol 0", and what was called 968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters"text mode" before Python 2.3. The entire pickle bytestream is made up of 978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersprintable 7-bit ASCII characters, plus the newline character, in protocol 0. 98fdc03462b3e0796ae6474da6f0f9844773d1da8fTim PetersThat's why it was called text mode. Protocol 0 is small and elegant, but 99fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peterssometimes painfully inefficient. 1008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersThe second major set of additions is now called "protocol 1", and was called 1028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters"binary mode" before Python 2.3. This added many opcodes with arguments 1038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersconsisting of arbitrary bytes, including NUL bytes and unprintable "high bit" 1048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersbytes. Binary mode pickles can be substantially smaller than equivalent 1058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterstext mode pickles, and sometimes faster too; e.g., BININT represents a 4-byte 1068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersint as 4 bytes following the opcode, which is cheaper to unpickle than the 107fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters(perhaps) 11-character decimal string attached to INT. Protocol 1 also added 108fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Petersa number of opcodes that operate on many stack elements at once (like APPENDS 10981098ac1c8c88f9f1dcf7511555d2ad1f7124575Tim Petersand SETITEMS), and "shortcut" opcodes (like EMPTY_DICT and EMPTY_TUPLE). 1108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersThe third major set of additions came in Python 2.3, and is called "protocol 112fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters2". This added: 113fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 114fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters- A better way to pickle instances of new-style classes (NEWOBJ). 115fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 116fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters- A way for a pickle to identify its protocol (PROTO). 117fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 118fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters- Time- and space- efficient pickling of long ints (LONG{1,4}). 119fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 120fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters- Shortcuts for small tuples (TUPLE{1,2,3}}. 121fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 122fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters- Dedicated opcodes for bools (NEWTRUE, NEWFALSE). 123fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 124fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters- The "extension registry", a vector of popular objects that can be pushed 125fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters efficiently by index (EXT{1,2,4}). This is akin to the memo and GET, but 126fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters the registry contents are predefined (there's nothing akin to the memo's 127fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters PUT). 128ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van Rossum 1295445594e20baf1ac0c0c1aef30d55d789c4b9694Skip MontanaroAnother independent change with Python 2.3 is the abandonment of any 1305445594e20baf1ac0c0c1aef30d55d789c4b9694Skip Montanaropretense that it might be safe to load pickles received from untrusted 131ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van Rossumparties -- no sufficient security analysis has been done to guarantee 1325445594e20baf1ac0c0c1aef30d55d789c4b9694Skip Montanarothis and there isn't a use case that warrants the expense of such an 133ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van Rossumanalysis. 134ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van Rossum 135ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van RossumTo this end, all tests for __safe_for_unpickling__ or for 136ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van Rossumcopy_reg.safe_constructors are removed from the unpickling code. 137ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van RossumReferences to these variables in the descriptions below are to be seen 138ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van Rossumas describing unpickling in Python 2.2 and before. 1398ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters""" 1408ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1418ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# Meta-rule: Descriptions are stored in instances of descriptor objects, 1428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# with plain constructors. No meta-language is defined from which 1438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# descriptors could be constructed. If you want, e.g., XML, write a little 1448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# program to generate XML from the objects. 1458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters############################################################################## 1478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# Some pickle opcodes have an argument, following the opcode in the 1488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# bytestream. An argument is of a specific type, described by an instance 1498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# of ArgumentDescriptor. These are not to be confused with arguments taken 1508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# off the stack -- ArgumentDescriptor applies only to arguments embedded in 1518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# the opcode stream, immediately following an opcode. 1528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# Represents the number of bytes consumed by an argument delimited by the 1548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# next newline character. 1558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersUP_TO_NEWLINE = -1 1568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# Represents the number of bytes consumed by a two-argument opcode where 1588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# the first argument gives the number of bytes in the second argument. 159fdb8cfab085d0f412411b090796e9f856ee0cab5Tim PetersTAKEN_FROM_ARGUMENT1 = -2 # num bytes is 1-byte unsigned int 160fdb8cfab085d0f412411b090796e9f856ee0cab5Tim PetersTAKEN_FROM_ARGUMENT4 = -3 # num bytes is 4-byte signed little-endian int 1618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersclass ArgumentDescriptor(object): 1638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters __slots__ = ( 1648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # name of descriptor record, also a module global name; a string 1658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'name', 1668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # length of argument, in bytes; an int; UP_TO_NEWLINE and 168fdb8cfab085d0f412411b090796e9f856ee0cab5Tim Peters # TAKEN_FROM_ARGUMENT{1,4} are negative values for variable-length 169fdb8cfab085d0f412411b090796e9f856ee0cab5Tim Peters # cases 1708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'n', 1718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # a function taking a file-like object, reading this kind of argument 1738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # from the object at the current position, advancing the current 1748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # position by n bytes, and returning the value of the argument 1758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'reader', 1768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # human-readable docs for this arg descriptor; a string 1788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'doc', 1798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ) 1808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters def __init__(self, name, n, reader, doc): 1828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(name, str) 1838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.name = name 1848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(n, int) and (n >= 0 or 186fdb8cfab085d0f412411b090796e9f856ee0cab5Tim Peters n in (UP_TO_NEWLINE, 187fdb8cfab085d0f412411b090796e9f856ee0cab5Tim Peters TAKEN_FROM_ARGUMENT1, 188fdb8cfab085d0f412411b090796e9f856ee0cab5Tim Peters TAKEN_FROM_ARGUMENT4)) 1898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.n = n 1908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.reader = reader 1928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(doc, str) 1948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.doc = doc 1958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersfrom struct import unpack as _unpack 1978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_uint1(f): 19955762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 2008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO 20155762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_uint1(StringIO.StringIO('\xff')) 2028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 255 2038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 2048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = f.read(1) 2068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if data: 2078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return ord(data) 2088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("not enough data in stream to read uint1") 2098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersuint1 = ArgumentDescriptor( 2118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='uint1', 2128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n=1, 2138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_uint1, 2148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="One-byte unsigned integer.") 2158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_uint2(f): 21855762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 2198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO 22055762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_uint2(StringIO.StringIO('\xff\x00')) 2218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 255 22255762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_uint2(StringIO.StringIO('\xff\xff')) 2238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 65535 2248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 2258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = f.read(2) 2278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if len(data) == 2: 2288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return _unpack("<H", data)[0] 2298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("not enough data in stream to read uint2") 2308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersuint2 = ArgumentDescriptor( 2328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='uint2', 2338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n=2, 2348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_uint2, 2358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="Two-byte unsigned integer, little-endian.") 2368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_int4(f): 23955762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 2408ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO 24155762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_int4(StringIO.StringIO('\xff\x00\x00\x00')) 2428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 255 24355762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_int4(StringIO.StringIO('\x00\x00\x00\x80')) == -(2**31) 2448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters True 2458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 2468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = f.read(4) 2488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if len(data) == 4: 2498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return _unpack("<i", data)[0] 2508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("not enough data in stream to read int4") 2518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersint4 = ArgumentDescriptor( 2538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='int4', 2548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n=4, 2558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_int4, 2568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="Four-byte signed integer, little-endian, 2's complement.") 2578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_stringnl(f, decode=True, stripquotes=True): 26055762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 2618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO 26255762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_stringnl(StringIO.StringIO("'abcd'\nefg\n")) 2638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'abcd' 2648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 26555762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_stringnl(StringIO.StringIO("\n")) 2668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Traceback (most recent call last): 2678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ... 2688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ValueError: no string quotes around '' 2698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 27055762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_stringnl(StringIO.StringIO("\n"), stripquotes=False) 2718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters '' 2728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 27355762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_stringnl(StringIO.StringIO("''\n")) 2748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters '' 2758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> read_stringnl(StringIO.StringIO('"abcd"')) 2778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Traceback (most recent call last): 2788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ... 2798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ValueError: no newline found when trying to read stringnl 2808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Embedded escapes are undone in the result. 28255762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_stringnl(StringIO.StringIO(r"'a\n\\b\x00c\td'" + "\n'e'")) 28355762f5f804c4848bbce323b085101d450f89ff6Tim Peters 'a\n\\b\x00c\td' 2848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 2858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = f.readline() 2878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if not data.endswith('\n'): 2888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("no newline found when trying to read stringnl") 2898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = data[:-1] # lose the newline 2908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if stripquotes: 2928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters for q in "'\"": 2938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if data.startswith(q): 2948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if not data.endswith(q): 2958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("strinq quote %r not found at both " 2968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters "ends of %r" % (q, data)) 2978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = data[1:-1] 2988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters break 2998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters else: 3008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("no string quotes around %r" % data) 3018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # I'm not sure when 'string_escape' was added to the std codecs; it's 3038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # crazy not to use it if it's there. 3048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if decode: 3058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = data.decode('string_escape') 3068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return data 3078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersstringnl = ArgumentDescriptor( 3098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='stringnl', 3108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n=UP_TO_NEWLINE, 3118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_stringnl, 3128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""A newline-terminated string. 3138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This is a repr-style string, with embedded escapes, and 3158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters bracketing quotes. 3168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 3178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_stringnl_noescape(f): 3198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return read_stringnl(f, decode=False, stripquotes=False) 3208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersstringnl_noescape = ArgumentDescriptor( 3228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='stringnl_noescape', 3238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n=UP_TO_NEWLINE, 3248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_stringnl_noescape, 3258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""A newline-terminated string. 3268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This is a str-style string, without embedded escapes, 3288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters or bracketing quotes. It should consist solely of 3298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters printable ASCII characters. 3308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 3318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_stringnl_noescape_pair(f): 33355762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 3348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO 33555762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_stringnl_noescape_pair(StringIO.StringIO("Queue\nEmpty\njunk")) 336d916cf4ec7014e9f6b25b8d63728bda01a17d3f9Tim Peters 'Queue Empty' 3378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 3388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 339d916cf4ec7014e9f6b25b8d63728bda01a17d3f9Tim Peters return "%s %s" % (read_stringnl_noescape(f), read_stringnl_noescape(f)) 3408ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3418ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersstringnl_noescape_pair = ArgumentDescriptor( 3428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='stringnl_noescape_pair', 3438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n=UP_TO_NEWLINE, 3448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_stringnl_noescape_pair, 3458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""A pair of newline-terminated strings. 3468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters These are str-style strings, without embedded 3488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters escapes, or bracketing quotes. They should 3498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters consist solely of printable ASCII characters. 3508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The pair is returned as a single string, with 351d916cf4ec7014e9f6b25b8d63728bda01a17d3f9Tim Peters a single blank separating the two strings. 3528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 3538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_string4(f): 35555762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 3568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO 35755762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_string4(StringIO.StringIO("\x00\x00\x00\x00abc")) 3588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters '' 35955762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_string4(StringIO.StringIO("\x03\x00\x00\x00abcdef")) 3608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'abc' 36155762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_string4(StringIO.StringIO("\x00\x00\x00\x03abcdef")) 3628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Traceback (most recent call last): 3638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ... 3648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ValueError: expected 50331648 bytes in a string4, but only 6 remain 3658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 3668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n = read_int4(f) 3688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if n < 0: 3698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("string4 byte count < 0: %d" % n) 3708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = f.read(n) 3718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if len(data) == n: 3728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return data 3738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("expected %d bytes in a string4, but only %d remain" % 3748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters (n, len(data))) 3758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersstring4 = ArgumentDescriptor( 3778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name="string4", 378fdb8cfab085d0f412411b090796e9f856ee0cab5Tim Peters n=TAKEN_FROM_ARGUMENT4, 3798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_string4, 3808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""A counted string. 3818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The first argument is a 4-byte little-endian signed int giving 3838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters the number of bytes in the string, and the second argument is 3848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters that many bytes. 3858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 3868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_string1(f): 38955762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 3908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO 39155762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_string1(StringIO.StringIO("\x00")) 3928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters '' 39355762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_string1(StringIO.StringIO("\x03abcdef")) 3948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'abc' 3958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 3968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 3978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n = read_uint1(f) 3988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert n >= 0 3998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = f.read(n) 4008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if len(data) == n: 4018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return data 4028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("expected %d bytes in a string1, but only %d remain" % 4038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters (n, len(data))) 4048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersstring1 = ArgumentDescriptor( 4068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name="string1", 407fdb8cfab085d0f412411b090796e9f856ee0cab5Tim Peters n=TAKEN_FROM_ARGUMENT1, 4088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_string1, 4098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""A counted string. 4108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The first argument is a 1-byte unsigned int giving the number 4128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters of bytes in the string, and the second argument is that many 4138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters bytes. 4148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 4158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_unicodestringnl(f): 41855762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 4198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO 42055762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_unicodestringnl(StringIO.StringIO("abc\uabcd\njunk")) 42155762f5f804c4848bbce323b085101d450f89ff6Tim Peters u'abc\uabcd' 4228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 4238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = f.readline() 4258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if not data.endswith('\n'): 4268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("no newline found when trying to read " 4278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters "unicodestringnl") 4288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = data[:-1] # lose the newline 4298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return unicode(data, 'raw-unicode-escape') 4308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersunicodestringnl = ArgumentDescriptor( 4328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='unicodestringnl', 4338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n=UP_TO_NEWLINE, 4348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_unicodestringnl, 4358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""A newline-terminated Unicode string. 4368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This is raw-unicode-escape encoded, so consists of 4388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters printable ASCII characters, and may contain embedded 4398ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters escape sequences. 4408ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 4418ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_unicodestring4(f): 44355762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 4448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO 44555762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> s = u'abcd\uabcd' 4468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> enc = s.encode('utf-8') 4478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> enc 44855762f5f804c4848bbce323b085101d450f89ff6Tim Peters 'abcd\xea\xaf\x8d' 4498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> n = chr(len(enc)) + chr(0) * 3 # little-endian 4-byte length 4508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> t = read_unicodestring4(StringIO.StringIO(n + enc + 'junk')) 4518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> s == t 4528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters True 4538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> read_unicodestring4(StringIO.StringIO(n + enc[:-1])) 4558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Traceback (most recent call last): 4568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ... 4578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ValueError: expected 7 bytes in a unicodestring4, but only 6 remain 4588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 4598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n = read_int4(f) 4618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if n < 0: 4628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("unicodestring4 byte count < 0: %d" % n) 4638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = f.read(n) 4648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if len(data) == n: 4658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return unicode(data, 'utf-8') 4668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("expected %d bytes in a unicodestring4, but only %d " 4678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters "remain" % (n, len(data))) 4688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersunicodestring4 = ArgumentDescriptor( 4708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name="unicodestring4", 471fdb8cfab085d0f412411b090796e9f856ee0cab5Tim Peters n=TAKEN_FROM_ARGUMENT4, 4728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_unicodestring4, 4738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""A counted Unicode string. 4748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The first argument is a 4-byte little-endian signed int 4768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters giving the number of bytes in the string, and the second 4778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters argument-- the UTF-8 encoding of the Unicode string -- 4788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters contains that many bytes. 4798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 4808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_decimalnl_short(f): 48355762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 4848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO 48555762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_decimalnl_short(StringIO.StringIO("1234\n56")) 4868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1234 4878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 48855762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_decimalnl_short(StringIO.StringIO("1234L\n56")) 4898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Traceback (most recent call last): 4908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ... 4918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ValueError: trailing 'L' not allowed in '1234L' 4928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 4938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters s = read_stringnl(f, decode=False, stripquotes=False) 4958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if s.endswith("L"): 4968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("trailing 'L' not allowed in %r" % s) 4978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 4988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # It's not necessarily true that the result fits in a Python short int: 4998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # the pickle may have been written on a 64-bit box. There's also a hack 5008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # for True and False here. 5018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if s == "00": 5028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return False 5038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters elif s == "01": 5048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return True 5058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters try: 5078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return int(s) 5088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters except OverflowError: 5098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return long(s) 5108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_decimalnl_long(f): 51255762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 5138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO 5148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 51555762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_decimalnl_long(StringIO.StringIO("1234\n56")) 5168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Traceback (most recent call last): 5178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ... 5188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ValueError: trailing 'L' required in '1234' 5198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Someday the trailing 'L' will probably go away from this output. 5218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 52255762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_decimalnl_long(StringIO.StringIO("1234L\n56")) 5238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1234L 5248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 52555762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_decimalnl_long(StringIO.StringIO("123456789012345678901234L\n6")) 5268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 123456789012345678901234L 5278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 5288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters s = read_stringnl(f, decode=False, stripquotes=False) 5308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if not s.endswith("L"): 5318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("trailing 'L' required in %r" % s) 5328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return long(s) 5338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdecimalnl_short = ArgumentDescriptor( 5368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='decimalnl_short', 5378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n=UP_TO_NEWLINE, 5388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_decimalnl_short, 5398ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""A newline-terminated decimal integer literal. 5408ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5418ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This never has a trailing 'L', and the integer fit 5428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters in a short Python int on the box where the pickle 5438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters was written -- but there's no guarantee it will fit 5448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters in a short Python int on the box where the pickle 5458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters is read. 5468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 5478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdecimalnl_long = ArgumentDescriptor( 5498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='decimalnl_long', 5508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n=UP_TO_NEWLINE, 5518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_decimalnl_long, 5528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""A newline-terminated decimal integer literal. 5538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This has a trailing 'L', and can represent integers 5558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters of any size. 5568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 5578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_floatnl(f): 56055762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 5618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO 56255762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_floatnl(StringIO.StringIO("-1.25\n6")) 5638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters -1.25 5648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 5658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters s = read_stringnl(f, decode=False, stripquotes=False) 5668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return float(s) 5678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersfloatnl = ArgumentDescriptor( 5698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='floatnl', 5708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n=UP_TO_NEWLINE, 5718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_floatnl, 5728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""A newline-terminated decimal floating literal. 5738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters In general this requires 17 significant digits for roundtrip 5758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters identity, and pickling then unpickling infinities, NaNs, and 5768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters minus zero doesn't work across boxes, or on some boxes even 5778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters on itself (e.g., Windows can't read the strings it produces 5788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters for infinities or NaNs). 5798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 5808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef read_float8(f): 58255762f5f804c4848bbce323b085101d450f89ff6Tim Peters r""" 5838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> import StringIO, struct 5848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> raw = struct.pack(">d", -1.25) 5858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters >>> raw 58655762f5f804c4848bbce323b085101d450f89ff6Tim Peters '\xbf\xf4\x00\x00\x00\x00\x00\x00' 58755762f5f804c4848bbce323b085101d450f89ff6Tim Peters >>> read_float8(StringIO.StringIO(raw + "\n")) 5888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters -1.25 5898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 5908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters data = f.read(8) 5928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if len(data) == 8: 5938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return _unpack(">d", data)[0] 5948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("not enough data in stream to read float8") 5958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 5978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersfloat8 = ArgumentDescriptor( 5988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='float8', 5998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters n=8, 6008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters reader=read_float8, 6018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""An 8-byte binary representation of a float, big-endian. 6028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 6038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The format is unique to Python, and shared with the struct 6048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters module (format string '>d') "in theory" (the struct and cPickle 6058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters implementations don't share the code -- they should). It's 6068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters strongly related to the IEEE-754 double format, and, in normal 6078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters cases, is in fact identical to the big-endian 754 double format. 6088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters On other boxes the dynamic range is limited to that of a 754 6098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters double, and "add a half and chop" rounding is used to reduce 6108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters the precision to 53 bits. However, even on a 754 box, 6118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters infinities, NaNs, and minus zero may not be handled correctly 6128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters (may not survive roundtrip pickling intact). 6138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 6148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 6155a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum# Protocol 2 formats 6165a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 617c0c12b57070a5b494662bebc418e3958bf5bdbeeTim Petersfrom pickle import decode_long 6185a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 6195a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossumdef read_long1(f): 6205a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum r""" 6215a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum >>> import StringIO 6225a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum >>> read_long1(StringIO.StringIO("\x02\xff\x00")) 6235a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 255L 6245a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum >>> read_long1(StringIO.StringIO("\x02\xff\x7f")) 6255a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 32767L 6265a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum >>> read_long1(StringIO.StringIO("\x02\x00\xff")) 6275a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum -256L 6285a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum >>> read_long1(StringIO.StringIO("\x02\x00\x80")) 6295a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum -32768L 6305eed340267868c54775b49613b81dff7eb2461bbTim Peters >>> 6315a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum """ 6325a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 6335a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum n = read_uint1(f) 6345a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum data = f.read(n) 6355a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum if len(data) != n: 6365a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum raise ValueError("not enough data in stream to read long1") 6375a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum return decode_long(data) 6385a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 6395a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossumlong1 = ArgumentDescriptor( 6405a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum name="long1", 641fdb8cfab085d0f412411b090796e9f856ee0cab5Tim Peters n=TAKEN_FROM_ARGUMENT1, 6425a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum reader=read_long1, 6435a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum doc="""A binary long, little-endian, using 1-byte size. 6445a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 6455a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum This first reads one byte as an unsigned size, then reads that 646bdbe74183ceb7aedd71dd1afdf8755ee93e8c3caTim Peters many bytes and interprets them as a little-endian 2's-complement long. 6475a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum """) 6485a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 6495a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossumdef read_long4(f): 6505a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum r""" 6515a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum >>> import StringIO 6525a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum >>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\xff\x00")) 6535a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 255L 6545a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum >>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\xff\x7f")) 6555a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 32767L 6565a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum >>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\x00\xff")) 6575a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum -256L 6585a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum >>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\x00\x80")) 6595a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum -32768L 6605eed340267868c54775b49613b81dff7eb2461bbTim Peters >>> 6615a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum """ 6625a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 6635a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum n = read_int4(f) 6645a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum if n < 0: 665784a3f53a8f78995d4c8ca22f612a68828bc4838Neal Norwitz raise ValueError("long4 byte count < 0: %d" % n) 6665a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum data = f.read(n) 6675a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum if len(data) != n: 668784a3f53a8f78995d4c8ca22f612a68828bc4838Neal Norwitz raise ValueError("not enough data in stream to read long4") 6695a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum return decode_long(data) 6705a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 6715a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossumlong4 = ArgumentDescriptor( 6725a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum name="long4", 673fdb8cfab085d0f412411b090796e9f856ee0cab5Tim Peters n=TAKEN_FROM_ARGUMENT4, 6745a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum reader=read_long4, 6755a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum doc="""A binary representation of a long, little-endian. 6765a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 6775a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum This first reads four bytes as a signed size (but requires the 6785a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum size to be >= 0), then reads that many bytes and interprets them 679bdbe74183ceb7aedd71dd1afdf8755ee93e8c3caTim Peters as a little-endian 2's-complement long. 6805a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum """) 6815a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 6825a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 6838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters############################################################################## 6848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# Object descriptors. The stack used by the pickle machine holds objects, 6858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# and in the stack_before and stack_after attributes of OpcodeInfo 6868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# descriptors we need names to describe the various types of objects that can 6878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# appear on the stack. 6888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 6898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersclass StackObject(object): 6908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters __slots__ = ( 6918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # name of descriptor record, for info only 6928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'name', 6938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 6948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # type of object, or tuple of type objects (meaning the object can 6958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # be of any type in the tuple) 6968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'obtype', 6978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 6988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # human-readable docs for this kind of stack object; a string 6998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'doc', 7008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ) 7018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters def __init__(self, name, obtype, doc): 7038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(name, str) 7048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.name = name 7058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(obtype, type) or isinstance(obtype, tuple) 7078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if isinstance(obtype, tuple): 7088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters for contained in obtype: 7098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(contained, type) 7108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.obtype = obtype 7118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(doc, str) 7138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.doc = doc 7148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspyint = StackObject( 7178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='int', 7188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=int, 7198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="A short (as opposed to long) Python integer object.") 7208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspylong = StackObject( 7228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='long', 7238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=long, 7248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="A long (as opposed to short) Python integer object.") 7258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspyinteger_or_bool = StackObject( 7278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='int_or_bool', 7288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=(int, long, bool), 7298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="A Python integer object (short or long), or " 7308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters "a Python bool.") 7318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7325a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossumpybool = StackObject( 7335a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum name='bool', 7345a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum obtype=(bool,), 7355a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum doc="A Python bool object.") 7365a2d8f5e9af0cbd513f02eb5576ff497e3693ffeGuido van Rossum 7378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspyfloat = StackObject( 7388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='float', 7398ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=float, 7408ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="A Python float object.") 7418ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspystring = StackObject( 7438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='str', 7448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=str, 7458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="A Python string object.") 7468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspyunicode = StackObject( 7488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='unicode', 7498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=unicode, 7508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="A Python Unicode string object.") 7518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspynone = StackObject( 7538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name="None", 7548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=type(None), 7558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="The Python None object.") 7568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspytuple = StackObject( 7588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name="tuple", 7598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=tuple, 7608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="A Python tuple object.") 7618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspylist = StackObject( 7638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name="list", 7648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=list, 7658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="A Python list object.") 7668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspydict = StackObject( 7688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name="dict", 7698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=dict, 7708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="A Python dict object.") 7718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersanyobject = StackObject( 7738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name='any', 7748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=object, 7758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="Any kind of object whatsoever.") 7768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersmarkobject = StackObject( 7788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name="mark", 7798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=StackObject, 7808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""'The mark' is a unique object. 7818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Opcodes that operate on a variable number of objects 7838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters generally don't embed the count of objects in the opcode, 7848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters or pull it off the stack. Instead the MARK opcode is used 7858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters to push a special marker object on the stack, and then 7868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters some other opcodes grab all the objects from the top of 7878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters the stack down to (but not including) the topmost marker 7888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters object. 7898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 7908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersstackslice = StackObject( 7928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name="stackslice", 7938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters obtype=StackObject, 7948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""An object representing a contiguous slice of the stack. 7958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 7968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This is used in conjuction with markobject, to represent all 7978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters of the stack following the topmost markobject. For example, 7988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters the POP_MARK opcode changes the stack from 7998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters [..., markobject, stackslice] 8018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters to 8028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters [...] 8038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters No matter how many object are on the stack after the topmost 8058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters markobject, POP_MARK gets rid of all of them (including the 8068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters topmost markobject too). 8078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """) 8088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters############################################################################## 8108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# Descriptors for pickle opcodes. 8118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersclass OpcodeInfo(object): 8138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters __slots__ = ( 8158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # symbolic name of opcode; a string 8168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'name', 8178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # the code used in a bytestream to represent the opcode; a 8198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # one-character string 8208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'code', 8218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # If the opcode has an argument embedded in the byte string, an 8238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # instance of ArgumentDescriptor specifying its type. Note that 8248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # arg.reader(s) can be used to read and decode the argument from 8258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # the bytestream s, and arg.doc documents the format of the raw 8268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # argument bytes. If the opcode doesn't have an argument embedded 8278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # in the bytestream, arg should be None. 8288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'arg', 8298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # what the stack looks like before this opcode runs; a list 8318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'stack_before', 8328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # what the stack looks like after this opcode runs; a list 8348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'stack_after', 8358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # the protocol number in which this opcode was introduced; an int 8378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'proto', 8388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8398ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # human-readable docs for this opcode; a string 8408ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'doc', 8418ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ) 8428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters def __init__(self, name, code, arg, 8448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before, stack_after, proto, doc): 8458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(name, str) 8468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.name = name 8478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(code, str) 8498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert len(code) == 1 8508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.code = code 8518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert arg is None or isinstance(arg, ArgumentDescriptor) 8538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.arg = arg 8548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(stack_before, list) 8568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters for x in stack_before: 8578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(x, StackObject) 8588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.stack_before = stack_before 8598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(stack_after, list) 8618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters for x in stack_after: 8628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(x, StackObject) 8638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.stack_after = stack_after 8648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(proto, int) and 0 <= proto <= 2 8668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.proto = proto 8678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert isinstance(doc, str) 8698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters self.doc = doc 8708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersI = OpcodeInfo 8728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersopcodes = [ 8738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Ways to spell integers. 8758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='INT', 8778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='I', 8788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=decimalnl_short, 8798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 8808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pyinteger_or_bool], 8818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 8828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push an integer or bool. 8838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The argument is a newline-terminated decimal literal string. 8858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The intent may have been that this always fit in a short Python int, 8878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters but INT can be generated in pickles written on a 64-bit box that 8888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters require a Python long on a 32-bit box. The difference between this 8898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters and LONG then is that INT skips a trailing 'L', and produces a short 8908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters int whenever possible. 8918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 8928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Another difference is due to that, when bool was introduced as a 8938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters distinct type in 2.3, builtin names True and False were also added to 8948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2.2.2, mapping to ints 1 and 0. For compatibility in both directions, 8958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters True gets pickled as INT + "I01\\n", and False as INT + "I00\\n". 8968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Leading zeroes are never produced for a genuine integer. The 2.3 8978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters (and later) unpicklers special-case these and return bool instead; 8988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters earlier unpicklers ignore the leading "0" and return the int. 8998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 9008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 9018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='BININT', 9028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='J', 9038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=int4, 9048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 9058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pyint], 9068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 9078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push a four-byte signed integer. 9088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 9098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This handles the full range of Python (short) integers on a 32-bit 9108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters box, directly as binary bytes (1 for the opcode and 4 for the integer). 9118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters If the integer is non-negative and fits in 1 or 2 bytes, pickling via 9128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters BININT1 or BININT2 saves space. 9138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 9148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 9158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='BININT1', 9168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='K', 9178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=uint1, 9188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 9198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pyint], 9208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 9218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push a one-byte unsigned integer. 9228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 9238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This is a space optimization for pickling very small non-negative ints, 9248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters in range(256). 9258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 9268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 9278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='BININT2', 9288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='M', 9298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=uint2, 9308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 9318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pyint], 9328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 9338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push a two-byte unsigned integer. 9348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 9358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This is a space optimization for pickling small positive ints, in 9368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters range(256, 2**16). Integers in range(256) can also be pickled via 9378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters BININT2, but BININT1 instead saves a byte. 9388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 9398ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 940fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name='LONG', 941fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='L', 942fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=decimalnl_long, 943fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[], 944fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[pylong], 945fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=0, 946fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""Push a long integer. 947fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 948fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters The same as INT, except that the literal ends with 'L', and always 949fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters unpickles to a Python long. There doesn't seem a real purpose to the 950fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters trailing 'L'. 951fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 952fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters Note that LONG takes time quadratic in the number of digits when 953fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters unpickling (this is simply due to the nature of decimal->binary 954fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters conversion). Proto 2 added linear-time (in C; still quadratic-time 955fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters in Python) LONG1 and LONG4 opcodes. 956fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters """), 957fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 958fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name="LONG1", 959fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='\x8a', 960fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=long1, 961fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[], 962fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[pylong], 963fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=2, 964fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""Long integer using one-byte length. 965fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 966fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters A more efficient encoding of a Python long; the long1 encoding 967fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters says it all."""), 968fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 969fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name="LONG4", 970fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='\x8b', 971fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=long4, 972fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[], 973fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[pylong], 974fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=2, 975fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""Long integer using found-byte length. 976fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 977fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters A more efficient encoding of a Python long; the long4 encoding 978fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters says it all."""), 979fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 9808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Ways to spell strings (8-bit, not Unicode). 9818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 9828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='STRING', 9838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='S', 9848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=stringnl, 9858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 9868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pystring], 9878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 9888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push a Python string object. 9898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 9908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The argument is a repr-style string, with bracketing quote characters, 9918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters and perhaps embedded escapes. The argument extends until the next 9928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters newline character. 9938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 9948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 9958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='BINSTRING', 9968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='T', 9978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=string4, 9988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 9998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pystring], 10008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 10018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push a Python string object. 10028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters There are two arguments: the first is a 4-byte little-endian signed int 10048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters giving the number of bytes in the string, and the second is that many 10058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters bytes, which are taken literally as the string content. 10068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 10078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='SHORT_BINSTRING', 10098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='U', 10108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=string1, 10118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 10128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pystring], 10138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 10148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push a Python string object. 10158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters There are two arguments: the first is a 1-byte unsigned int giving 10178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters the number of bytes in the string, and the second is that many bytes, 10188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters which are taken literally as the string content. 10198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 10208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Ways to spell None. 10228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='NONE', 10248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='N', 10258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 10268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 10278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pynone], 10288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 10298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="Push None on the stack."), 10308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1031fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters # Ways to spell bools, starting with proto 2. See INT for how this was 1032fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters # done before proto 2. 1033fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1034fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name='NEWTRUE', 1035fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='\x88', 1036fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=None, 1037fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[], 1038fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[pybool], 1039fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=2, 1040fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""True. 1041fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1042fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters Push True onto the stack."""), 1043fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1044fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name='NEWFALSE', 1045fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='\x89', 1046fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=None, 1047fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[], 1048fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[pybool], 1049fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=2, 1050fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""True. 1051fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1052fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters Push False onto the stack."""), 1053fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 10548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Ways to spell Unicode strings. 10558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='UNICODE', 10578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='V', 10588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=unicodestringnl, 10598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 10608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pyunicode], 10618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, # this may be pure-text, but it's a later addition 10628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push a Python Unicode string object. 10638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The argument is a raw-unicode-escape encoding of a Unicode string, 10658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters and so may contain embedded escape sequences. The argument extends 10668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters until the next newline character. 10678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 10688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='BINUNICODE', 10708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='X', 10718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=unicodestring4, 10728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 10738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pyunicode], 10748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 10758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push a Python Unicode string object. 10768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters There are two arguments: the first is a 4-byte little-endian signed int 10788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters giving the number of bytes in the string. The second is that many 10798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters bytes, and is the UTF-8 encoding of the Unicode string. 10808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 10818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Ways to spell floats. 10838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='FLOAT', 10858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='F', 10868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=floatnl, 10878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 10888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pyfloat], 10898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 10908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Newline-terminated decimal float literal. 10918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The argument is repr(a_float), and in general requires 17 significant 10938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters digits for roundtrip conversion to be an identity (this is so for 10948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters IEEE-754 double precision values, which is what Python float maps to 10958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters on most boxes). 10968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 10978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters In general, FLOAT cannot be used to transport infinities, NaNs, or 10988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters minus zero across boxes (or even on a single box, if the platform C 10998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters library can't read the strings it produces for such things -- Windows 11008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters is like that), but may do less damage than BINFLOAT on boxes with 11018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters greater precision or dynamic range than IEEE-754 double. 11028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 11038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='BINFLOAT', 11058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='G', 11068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=float8, 11078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 11088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pyfloat], 11098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 11108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Float stored in binary form, with 8 bytes of data. 11118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This generally requires less than half the space of FLOAT encoding. 11138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters In general, BINFLOAT cannot be used to transport infinities, NaNs, or 11148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters minus zero, raises an exception if the exponent exceeds the range of 11158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters an IEEE-754 double, and retains no more than 53 bits of precision (if 11168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters there are more than that, "add a half and chop" rounding is used to 11178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters cut it back to 53 significant bits). 11188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 11198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Ways to build lists. 11218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='EMPTY_LIST', 11238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code=']', 11248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 11258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 11268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pylist], 11278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 11288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="Push an empty list."), 11298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='APPEND', 11318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='a', 11328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 11338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[pylist, anyobject], 11348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pylist], 11358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 11368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Append an object to a list. 11378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack before: ... pylist anyobject 11398ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack after: ... pylist+[anyobject] 114081098ac1c8c88f9f1dcf7511555d2ad1f7124575Tim Peters 114181098ac1c8c88f9f1dcf7511555d2ad1f7124575Tim Peters although pylist is really extended in-place. 11428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 11438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='APPENDS', 11458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='e', 11468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 11478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[pylist, markobject, stackslice], 11488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pylist], 11498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 11508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Extend a list by a slice of stack objects. 11518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack before: ... pylist markobject stackslice 11538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack after: ... pylist+stackslice 115481098ac1c8c88f9f1dcf7511555d2ad1f7124575Tim Peters 115581098ac1c8c88f9f1dcf7511555d2ad1f7124575Tim Peters although pylist is really extended in-place. 11568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 11578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='LIST', 11598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='l', 11608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 11618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[markobject, stackslice], 11628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pylist], 11638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 11648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Build a list out of the topmost stack slice, after markobject. 11658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters All the stack entries following the topmost markobject are placed into 11678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters a single Python list, which single list object replaces all of the 11688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack from the topmost markobject onward. For example, 11698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack before: ... markobject 1 2 3 'abc' 11718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack after: ... [1, 2, 3, 'abc'] 11728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 11738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Ways to build tuples. 11758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='EMPTY_TUPLE', 11778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code=')', 11788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 11798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 11808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pytuple], 11818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 11828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="Push an empty tuple."), 11838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='TUPLE', 11858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='t', 11868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 11878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[markobject, stackslice], 11888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pytuple], 11898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 11908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Build a tuple out of the topmost stack slice, after markobject. 11918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters All the stack entries following the topmost markobject are placed into 11938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters a single Python tuple, which single tuple object replaces all of the 11948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack from the topmost markobject onward. For example, 11958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 11968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack before: ... markobject 1 2 3 'abc' 11978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack after: ... (1, 2, 3, 'abc') 11988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 11998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1200fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name='TUPLE1', 1201fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='\x85', 1202fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=None, 1203fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[anyobject], 1204fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[pytuple], 1205fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=2, 1206fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""One-tuple. 1207fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1208fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters This code pops one value off the stack and pushes a tuple of 1209fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters length 1 whose one item is that value back onto it. IOW: 1210fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1211fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack[-1] = tuple(stack[-1:]) 1212fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters """), 1213fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1214fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name='TUPLE2', 1215fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='\x86', 1216fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=None, 1217fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[anyobject, anyobject], 1218fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[pytuple], 1219fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=2, 1220fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""One-tuple. 1221fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1222fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters This code pops two values off the stack and pushes a tuple 1223fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters of length 2 whose items are those values back onto it. IOW: 1224fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1225fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack[-2:] = [tuple(stack[-2:])] 1226fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters """), 1227fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1228fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name='TUPLE3', 1229fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='\x87', 1230fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=None, 1231fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[anyobject, anyobject, anyobject], 1232fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[pytuple], 1233fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=2, 1234fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""One-tuple. 1235fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1236fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters This code pops three values off the stack and pushes a tuple 1237fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters of length 3 whose items are those values back onto it. IOW: 1238fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1239fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack[-3:] = [tuple(stack[-3:])] 1240fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters """), 1241fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 12428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Ways to build dicts. 12438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 12448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='EMPTY_DICT', 12458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='}', 12468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 12478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 12488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pydict], 12498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 12508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="Push an empty dict."), 12518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 12528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='DICT', 12538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='d', 12548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 12558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[markobject, stackslice], 12568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pydict], 12578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 12588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Build a dict out of the topmost stack slice, after markobject. 12598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 12608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters All the stack entries following the topmost markobject are placed into 12618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters a single Python dict, which single dict object replaces all of the 12628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack from the topmost markobject onward. The stack slice alternates 12638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters key, value, key, value, .... For example, 12648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 12658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack before: ... markobject 1 2 3 'abc' 12668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack after: ... {1: 2, 3: 'abc'} 12678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 12688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 12698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='SETITEM', 12708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='s', 12718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 12728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[pydict, anyobject, anyobject], 12738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pydict], 12748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 12758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Add a key+value pair to an existing dict. 12768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 12778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack before: ... pydict key value 12788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack after: ... pydict 12798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 12808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters where pydict has been modified via pydict[key] = value. 12818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 12828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 12838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='SETITEMS', 12848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='u', 12858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 12868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[pydict, markobject, stackslice], 12878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[pydict], 12888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 12898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Add an arbitrary number of key+value pairs to an existing dict. 12908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 12918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The slice of the stack following the topmost markobject is taken as 12928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters an alternating sequence of keys and values, added to the dict 12938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters immediately under the topmost markobject. Everything at and after the 12948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters topmost markobject is popped, leaving the mutated dict at the top 12958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters of the stack. 12968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 12978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack before: ... pydict markobject key_1 value_1 ... key_n value_n 12988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack after: ... pydict 12998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters where pydict has been modified via pydict[key_i] = value_i for i in 13018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1, 2, ..., n, and in that order. 13028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 13038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Stack manipulation. 13058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='POP', 13078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='0', 13088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 13098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[anyobject], 13108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[], 13118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 13128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="Discard the top stack item, shrinking the stack by one item."), 13138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='DUP', 13158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='2', 13168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 13178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[anyobject], 13188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[anyobject, anyobject], 13198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 13208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="Push the top stack item onto the stack again, duplicating it."), 13218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='MARK', 13238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='(', 13248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 13258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 13268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[markobject], 13278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 13288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push markobject onto the stack. 13298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters markobject is a unique object, used by other opcodes to identify a 13318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters region of the stack containing a variable number of objects for them 13328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters to work on. See markobject.doc for more detail. 13338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 13348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='POP_MARK', 13368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='1', 13378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 13388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[markobject, stackslice], 13398ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[], 13408ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 13418ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Pop all the stack objects at and above the topmost markobject. 13428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters When an opcode using a variable number of stack objects is done, 13448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters POP_MARK is used to remove those objects, and to remove the markobject 13458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters that delimited their starting position on the stack. 13468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 13478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Memo manipulation. There are really only two operations (get and put), 13498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # each in all-text, "short binary", and "long binary" flavors. 13508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='GET', 13528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='g', 13538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=decimalnl_short, 13548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 13558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[anyobject], 13568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 13578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Read an object from the memo and push it on the stack. 13588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The index of the memo object to push is given by the newline-teriminated 13608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters decimal string following. BINGET and LONG_BINGET are space-optimized 13618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters versions. 13628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 13638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='BINGET', 13658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='h', 13668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=uint1, 13678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 13688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[anyobject], 13698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 13708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Read an object from the memo and push it on the stack. 13718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The index of the memo object to push is given by the 1-byte unsigned 13738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters integer following. 13748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 13758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='LONG_BINGET', 13778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='j', 13788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=int4, 13798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 13808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[anyobject], 13818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 13828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Read an object from the memo and push it on the stack. 13838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The index of the memo object to push is given by the 4-byte signed 13858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters little-endian integer following. 13868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 13878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='PUT', 13898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='p', 13908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=decimalnl_short, 13918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 13928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[], 13938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 13948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Store the stack top into the memo. The stack is not popped. 13958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 13968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The index of the memo location to write into is given by the newline- 13978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters terminated decimal string following. BINPUT and LONG_BINPUT are 13988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters space-optimized versions. 13998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 14008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 14018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='BINPUT', 14028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='q', 14038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=uint1, 14048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 14058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[], 14068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 14078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Store the stack top into the memo. The stack is not popped. 14088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 14098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The index of the memo location to write into is given by the 1-byte 14108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters unsigned integer following. 14118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 14128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 14138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='LONG_BINPUT', 14148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='r', 14158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=int4, 14168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 14178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[], 14188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 14198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Store the stack top into the memo. The stack is not popped. 14208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 14218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The index of the memo location to write into is given by the 4-byte 14228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters signed little-endian integer following. 14238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 14248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1425fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters # Access the extension registry (predefined objects). Akin to the GET 1426fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters # family. 1427fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1428fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name='EXT1', 1429fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='\x82', 1430fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=uint1, 1431fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[], 1432fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[anyobject], 1433fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=2, 1434fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""Extension code. 1435fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1436fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters This code and the similar EXT2 and EXT4 allow using a registry 1437fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters of popular objects that are pickled by name, typically classes. 1438fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters It is envisioned that through a global negotiation and 1439fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters registration process, third parties can set up a mapping between 1440fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters ints and object names. 1441fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1442fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters In order to guarantee pickle interchangeability, the extension 1443fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code registry ought to be global, although a range of codes may 1444fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters be reserved for private use. 1445fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1446fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters EXT1 has a 1-byte integer argument. This is used to index into the 1447fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters extension registry, and the object at that index is pushed on the stack. 1448fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters """), 1449fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1450fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name='EXT2', 1451fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='\x83', 1452fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=uint2, 1453fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[], 1454fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[anyobject], 1455fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=2, 1456fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""Extension code. 1457fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1458fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters See EXT1. EXT2 has a two-byte integer argument. 1459fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters """), 1460fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1461fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name='EXT4', 1462fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='\x84', 1463fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=int4, 1464fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[], 1465fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[anyobject], 1466fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=2, 1467fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""Extension code. 1468fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1469fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters See EXT1. EXT4 has a four-byte integer argument. 1470fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters """), 1471fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 14728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Push a class object, or module function, on the stack, via its module 14738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # and name. 14748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 14758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='GLOBAL', 14768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='c', 14778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=stringnl_noescape_pair, 14788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 14798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[anyobject], 14808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 14818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push a global object (module.attr) on the stack. 14828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 14838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Two newline-terminated strings follow the GLOBAL opcode. The first is 14848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters taken as a module name, and the second as a class name. The class 14858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters object module.class is pushed on the stack. More accurately, the 14868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters object returned by self.find_class(module, class) is pushed on the 14878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack, so unpickling subclasses can override this form of lookup. 14888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 14898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 14908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Ways to build objects of classes pickle doesn't know about directly 14918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # (user-defined classes). I despair of documenting this accurately 14928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # and comprehensibly -- you really have to read the pickle code to 14938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # find all the special cases. 14948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 14958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='REDUCE', 14968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='R', 14978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 14988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[anyobject, anyobject], 14998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[anyobject], 15008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 15018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push an object built from a callable and an argument tuple. 15028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The opcode is named to remind of the __reduce__() method. 15048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack before: ... callable pytuple 15068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack after: ... callable(*pytuple) 15078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The callable and the argument tuple are the first two items returned 15098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters by a __reduce__ method. Applying the callable to the argtuple is 15108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters supposed to reproduce the original object, or at least get it started. 15118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters If the __reduce__ method returns a 3-tuple, the last component is an 15128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters argument to be passed to the object's __setstate__, and then the REDUCE 15138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters opcode is followed by code to create setstate's argument, and then a 15148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters BUILD opcode to apply __setstate__ to that argument. 15158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters There are lots of special cases here. The argtuple can be None, in 15178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters which case callable.__basicnew__() is called instead to produce the 15188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters object to be pushed on the stack. This appears to be a trick unique 15198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters to ExtensionClasses, and is deprecated regardless. 15208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters If type(callable) is not ClassType, REDUCE complains unless the 15228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters callable has been registered with the copy_reg module's 15238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters safe_constructors dict, or the callable has a magic 15248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters '__safe_for_unpickling__' attribute with a true value. I'm not sure 15258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters why it does this, but I've sure seen this complaint often enough when 15268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I didn't want to <wink>. 15278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 15288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='BUILD', 15308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='b', 15318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 15328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[anyobject, anyobject], 15338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[anyobject], 15348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 15358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Finish building an object, via __setstate__ or dict update. 15368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack before: ... anyobject argument 15388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack after: ... anyobject 15398ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15408ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters where anyobject may have been mutated, as follows: 15418ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters If the object has a __setstate__ method, 15438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters anyobject.__setstate__(argument) 15458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters is called. 15478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Else the argument must be a dict, the object must have a __dict__, and 15498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters the object is updated via 15508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters anyobject.__dict__.update(argument) 15528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This may raise RuntimeError in restricted execution mode (which 15548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters disallows access to __dict__ directly); in that case, the object 15558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters is updated instead via 15568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters for k, v in argument.items(): 15588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters anyobject[k] = v 15598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 15608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='INST', 15628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='i', 15638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=stringnl_noescape_pair, 15648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[markobject, stackslice], 15658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[anyobject], 15668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 15678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Build a class instance. 15688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This is the protocol 0 version of protocol 1's OBJ opcode. 15708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters INST is followed by two newline-terminated strings, giving a 15718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters module and class name, just as for the GLOBAL opcode (and see 15728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters GLOBAL for more details about that). self.find_class(module, name) 15738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters is used to get a class object. 15748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters In addition, all the objects on the stack following the topmost 15768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters markobject are gathered into a tuple and popped (along with the 15778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters topmost markobject), just as for the TUPLE opcode. 15788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Now it gets complicated. If all of these are true: 15808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters + The argtuple is empty (markobject was at the top of the stack 15828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters at the start). 15838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters + It's an old-style class object (the type of the class object is 15858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ClassType). 15868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters + The class object does not have a __getinitargs__ attribute. 15888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 15898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters then we want to create an old-style class instance without invoking 15908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters its __init__() method (pickle has waffled on this over the years; not 15918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters calling __init__() is current wisdom). In this case, an instance of 15928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters an old-style dummy class is created, and then we try to rebind its 15938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters __class__ attribute to the desired class object. If this succeeds, 15948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters the new instance object is pushed on the stack, and we're done. In 15958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters restricted execution mode it can fail (assignment to __class__ is 15968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters disallowed), and I'm not really sure what happens then -- it looks 15978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters like the code ends up calling the class object's __init__ anyway, 15988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters via falling into the next case. 15998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 16008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Else (the argtuple is not empty, it's not an old-style class object, 16018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters or the class object does have a __getinitargs__ attribute), the code 16028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters first insists that the class object have a __safe_for_unpickling__ 16038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters attribute. Unlike as for the __safe_for_unpickling__ check in REDUCE, 16048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters it doesn't matter whether this attribute has a true or false value, it 1605ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van Rossum only matters whether it exists (XXX this is a bug; cPickle 1606ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van Rossum requires the attribute to be true). If __safe_for_unpickling__ 1607ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van Rossum doesn't exist, UnpicklingError is raised. 16088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 16098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Else (the class object does have a __safe_for_unpickling__ attr), 16108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters the class object obtained from INST's arguments is applied to the 16118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters argtuple obtained from the stack, and the resulting instance object 16128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters is pushed on the stack. 16138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 16148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 16158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='OBJ', 16168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='o', 16178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 16188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[markobject, anyobject, stackslice], 16198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[anyobject], 16208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 16218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Build a class instance. 16228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 16238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters This is the protocol 1 version of protocol 0's INST opcode, and is 16248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters very much like it. The major difference is that the class object 16258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters is taken off the stack, allowing it to be retrieved from the memo 16268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters repeatedly if several instances of the same class are created. This 16278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters can be much more efficient (in both time and space) than repeatedly 16288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters embedding the module and class names in INST opcodes. 16298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 16308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Unlike INST, OBJ takes no arguments from the opcode stream. Instead 16318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters the class object is taken off the stack, immediately above the 16328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters topmost markobject: 16338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 16348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack before: ... markobject classobject stackslice 16358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Stack after: ... new_instance_object 16368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 16378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters As for INST, the remainder of the stack above the markobject is 16388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters gathered into an argument tuple, and then the logic seems identical, 1639ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van Rossum except that no __safe_for_unpickling__ check is done (XXX this is 1640ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van Rossum a bug; cPickle does test __safe_for_unpickling__). See INST for 1641ecb1104342af0e2dca191f7666c60d5ca65069a8Guido van Rossum the gory details. 16428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 16438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1644fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name='NEWOBJ', 1645fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='\x81', 1646fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=None, 1647fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[anyobject, anyobject], 1648fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[anyobject], 1649fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=2, 1650fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""Build an object instance. 1651fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1652fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters The stack before should be thought of as containing a class 1653fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters object followed by an argument tuple (the tuple being the stack 1654fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters top). Call these cls and args. They are popped off the stack, 1655fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters and the value returned by cls.__new__(cls, *args) is pushed back 1656fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters onto the stack. 1657fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters """), 1658fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 16598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Machine control. 16608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1661fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters I(name='PROTO', 1662fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters code='\x80', 1663fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters arg=uint1, 1664fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_before=[], 1665fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters stack_after=[], 1666fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters proto=2, 1667fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters doc="""Protocol version indicator. 1668fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 1669fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters For protocol 2 and above, a pickle must start with this opcode. 1670fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters The argument is the protocol version, an int in range(2, 256). 1671fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters """), 1672fdc03462b3e0796ae6474da6f0f9844773d1da8fTim Peters 16738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='STOP', 16748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='.', 16758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 16768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[anyobject], 16778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[], 16788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 16798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Stop the unpickling machine. 16808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 16818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Every pickle ends with this opcode. The object at the top of the stack 16828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters is popped, and that's the result of unpickling. The stack should be 16838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters empty then. 16848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 16858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 16868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Ways to deal with persistent IDs. 16878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 16888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='PERSID', 16898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='P', 16908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=stringnl_noescape, 16918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[], 16928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[anyobject], 16938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=0, 16948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push an object identified by a persistent ID. 16958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 16968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The pickle module doesn't define what a persistent ID means. PERSID's 16978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters argument is a newline-terminated str-style (no embedded escapes, no 16988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters bracketing quote characters) string, which *is* "the persistent ID". 16998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters The unpickler passes this string to self.persistent_load(). Whatever 17008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters object that returns is pushed on the stack. There is no implementation 17018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters of persistent_load() in Python's unpickler: it must be supplied by an 17028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters unpickler subclass. 17038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 17048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters I(name='BINPERSID', 17068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code='Q', 17078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg=None, 17088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_before=[anyobject], 17098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stack_after=[anyobject], 17108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters proto=1, 17118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters doc="""Push an object identified by a persistent ID. 17128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Like PERSID, except the persistent ID is popped off the stack (instead 17148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters of being a string embedded in the opcode bytestream). The persistent 17158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters ID is passed to self.persistent_load(), and whatever object that 17168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters returns is pushed on the stack. See PERSID for more detail. 17178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """), 17188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters] 17198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdel I 17208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# Verify uniqueness of .name and .code members. 17228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersname2i = {} 17238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterscode2i = {} 17248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersfor i, d in enumerate(opcodes): 17268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if d.name in name2i: 17278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("repeated name %r at indices %d and %d" % 17288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters (d.name, name2i[d.name], i)) 17298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if d.code in code2i: 17308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("repeated code %r at indices %d and %d" % 17318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters (d.code, code2i[d.code], i)) 17328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name2i[d.name] = i 17348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code2i[d.code] = i 17358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdel name2i, code2i, i, d 17378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters############################################################################## 17398ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# Build a code2op dict, mapping opcode characters to OpcodeInfo records. 17408ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# Also ensure we've got the same stuff as pickle.py, although the 17418ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# introspection here is dicey. 17428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterscode2op = {} 17448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersfor d in opcodes: 17458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code2op[d.code] = d 17468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdel d 17478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef assure_pickle_consistency(verbose=False): 17498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters import pickle, re 17508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters copy = code2op.copy() 17528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters for name in pickle.__all__: 17538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if not re.match("[A-Z][A-Z0-9_]+$", name): 17548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if verbose: 17558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters print "skipping %r: it doesn't look like an opcode name" % name 17568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters continue 17578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters picklecode = getattr(pickle, name) 17588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if not isinstance(picklecode, str) or len(picklecode) != 1: 17598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if verbose: 17608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters print ("skipping %r: value %r doesn't look like a pickle " 17618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters "code" % (name, picklecode)) 17628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters continue 17638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if picklecode in copy: 17648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if verbose: 17658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters print "checking name %r w/ code %r for consistency" % ( 17668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name, picklecode) 17678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters d = copy[picklecode] 17688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if d.name != name: 17698ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("for pickle code %r, pickle.py uses name %r " 17708ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters "but we're using name %r" % (picklecode, 17718ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters name, 17728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters d.name)) 17738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # Forget this one. Any left over in copy at the end are a problem 17748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # of a different kind. 17758ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters del copy[picklecode] 17768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters else: 17778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("pickle.py appears to have a pickle opcode with " 17788ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters "name %r and code %r, but we don't" % 17798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters (name, picklecode)) 17808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if copy: 17818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters msg = ["we appear to have pickle opcodes that pickle.py doesn't have:"] 17828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters for code, d in copy.items(): 17838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters msg.append(" name %r with code %r" % (d.name, code)) 17848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("\n".join(msg)) 17858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersassure_pickle_consistency() 1787c0c12b57070a5b494662bebc418e3958bf5bdbeeTim Petersdel assure_pickle_consistency 17888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters############################################################################## 17908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# A pickle opcode generator. 17918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef genops(pickle): 1793a72ded9bc81bd278bf119e92feaeaea2aa16f02dGuido van Rossum """Generate all the opcodes in a pickle. 17948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'pickle' is a file-like object, or string, containing the pickle. 17968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 17978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Each opcode in the pickle is generated, from the current pickle position, 17988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters stopping after a STOP opcode is delivered. A triple is generated for 17998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters each opcode: 18008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters opcode, arg, pos 18028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters opcode is an OpcodeInfo record, describing the current opcode. 18048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters If the opcode has an argument embedded in the pickle, arg is its decoded 18068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters value, as a Python object. If the opcode doesn't have an argument, arg 18078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters is None. 18088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters If the pickle has a tell() method, pos was the value of pickle.tell() 18108ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters before reading the current opcode. If the pickle is a string object, 18118ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters it's wrapped in a StringIO object, and the latter's tell() result is 18128ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters used. Else (the pickle doesn't have a tell(), and it's not obvious how 18138ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters to query its current position) pos is None. 18148ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 18158ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18168ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters import cStringIO as StringIO 18178ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18188ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if isinstance(pickle, str): 18198ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters pickle = StringIO.StringIO(pickle) 18208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18218ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if hasattr(pickle, "tell"): 18228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters getpos = pickle.tell 18238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters else: 18248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters getpos = lambda: None 18258ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18268ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters while True: 18278ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters pos = getpos() 18288ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code = pickle.read(1) 18298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters opcode = code2op.get(code) 18308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if opcode is None: 18318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if code == "": 18328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("pickle exhausted before seeing STOP") 18338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters else: 18348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters raise ValueError("at position %s, opcode %r unknown" % ( 18358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters pos is None and "<unknown>" or pos, 18368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters code)) 18378ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if opcode.arg is None: 18388ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg = None 18398ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters else: 18408ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters arg = opcode.arg.reader(pickle) 18418ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters yield opcode, arg, pos 18428ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if code == '.': 18438ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert opcode.name == 'STOP' 18448ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters break 18458ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters############################################################################## 18478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters# A symbolic pickle disassembler. 18488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef dis(pickle, out=None, indentlevel=4): 18508ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """Produce a symbolic disassembly of a pickle. 18518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18528ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 'pickle' is a file-like object, or string, containing a (at least one) 18538ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters pickle. The pickle is disassembled from the current position, through 18548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters the first STOP opcode encountered. 18558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18568ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Optional arg 'out' is a file-like object to which the disassembly is 18578ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters printed. It defaults to sys.stdout. 18588ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18598ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters Optional arg indentlevel is the number of blanks by which to indent 18608ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters a new MARK level. It defaults to 4. 18618ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters """ 18628ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18638ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters markstack = [] 18648ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters indentchunk = ' ' * indentlevel 18658ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters for opcode, arg, pos in genops(pickle): 18668ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if pos is not None: 18678ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters print >> out, "%5d:" % pos, 18688ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1869d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters line = "%-4s %s%s" % (repr(opcode.code)[1:-1], 1870d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters indentchunk * len(markstack), 1871d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters opcode.name) 18728ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18738ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters markmsg = None 18748ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if markstack and markobject in opcode.stack_before: 18752c60f7a13697bbc19c4d5ef0b052c34cf1848244Tim Peters assert markobject not in opcode.stack_after 18762c60f7a13697bbc19c4d5ef0b052c34cf1848244Tim Peters markpos = markstack.pop() 18772c60f7a13697bbc19c4d5ef0b052c34cf1848244Tim Peters if markpos is not None: 18782c60f7a13697bbc19c4d5ef0b052c34cf1848244Tim Peters markmsg = "(MARK at %d)" % markpos 18798ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18808ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if arg is not None or markmsg: 18818ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters # make a mild effort to align arguments 18828ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters line += ' ' * (10 - len(opcode.name)) 18838ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if arg is not None: 18848ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters line += ' ' + repr(arg) 18858ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if markmsg: 18868ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters line += ' ' + markmsg 18878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters print >> out, line 18888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18898ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters if markobject in opcode.stack_after: 18908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters assert markobject not in opcode.stack_before 18918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters markstack.append(pos) 18928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 18938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 189403e35327f26d353db126a8f840a8890b3593f18aGuido van Rossum_dis_test = r""" 18958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> import pickle 18968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> x = [1, 2, (3, 4), {'abc': u"def"}] 1897570283584af6a9aff47d2341d6154055572aaff5Guido van Rossum>>> pkl = pickle.dumps(x, 0) 1898570283584af6a9aff47d2341d6154055572aaff5Guido van Rossum>>> dis(pkl) 1899d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 0: ( MARK 1900d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 1: l LIST (MARK at 0) 1901d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 2: p PUT 0 1902d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 5: I INT 1 1903d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 8: a APPEND 1904d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 9: I INT 2 1905d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 12: a APPEND 1906d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 13: ( MARK 1907d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 14: I INT 3 1908d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 17: I INT 4 1909d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 20: t TUPLE (MARK at 13) 1910d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 21: p PUT 1 1911d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 24: a APPEND 1912d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 25: ( MARK 1913d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 26: d DICT (MARK at 25) 1914d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 27: p PUT 2 1915d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 30: S STRING 'abc' 1916d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 37: p PUT 3 1917d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 40: V UNICODE u'def' 1918d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 45: p PUT 4 1919d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 48: s SETITEM 1920d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 49: a APPEND 1921d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 50: . STOP 19228ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 19238ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersTry again with a "binary" pickle. 19248ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 1925570283584af6a9aff47d2341d6154055572aaff5Guido van Rossum>>> pkl = pickle.dumps(x, 1) 1926570283584af6a9aff47d2341d6154055572aaff5Guido van Rossum>>> dis(pkl) 1927d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 0: ] EMPTY_LIST 1928d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 1: q BINPUT 0 1929d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 3: ( MARK 1930d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 4: K BININT1 1 1931d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 6: K BININT1 2 1932d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 8: ( MARK 1933d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 9: K BININT1 3 1934d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 11: K BININT1 4 1935d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 13: t TUPLE (MARK at 8) 1936d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 14: q BINPUT 1 1937d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 16: } EMPTY_DICT 1938d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 17: q BINPUT 2 1939d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 19: U SHORT_BINSTRING 'abc' 1940d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 24: q BINPUT 3 1941d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 26: X BINUNICODE u'def' 1942d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 34: q BINPUT 4 1943d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 36: s SETITEM 1944d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 37: e APPENDS (MARK at 3) 1945d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 38: . STOP 19468ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 19478ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersExercise the INST/OBJ/BUILD family. 19488ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 19498ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> import random 1950f29d3d6011e41b40282994375454f2020a429d79Guido van Rossum>>> dis(pickle.dumps(random.random, 0)) 1951d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 0: c GLOBAL 'random random' 1952d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 15: p PUT 0 1953d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 18: . STOP 19548ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 19558ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> x = [pickle.PicklingError()] * 2 1956f29d3d6011e41b40282994375454f2020a429d79Guido van Rossum>>> dis(pickle.dumps(x, 0)) 1957d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 0: ( MARK 1958d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 1: l LIST (MARK at 0) 1959d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 2: p PUT 0 1960d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 5: ( MARK 1961d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 6: i INST 'pickle PicklingError' (MARK at 5) 1962d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 28: p PUT 1 1963d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 31: ( MARK 1964d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 32: d DICT (MARK at 31) 1965d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 33: p PUT 2 1966d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 36: S STRING 'args' 1967d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 44: p PUT 3 1968d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 47: ( MARK 1969d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 48: t TUPLE (MARK at 47) 1970d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 49: s SETITEM 1971d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 50: b BUILD 1972d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 51: a APPEND 1973d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 52: g GET 1 1974d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 55: a APPEND 1975d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 56: . STOP 19768ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 19778ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> dis(pickle.dumps(x, 1)) 1978d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 0: ] EMPTY_LIST 1979d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 1: q BINPUT 0 1980d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 3: ( MARK 1981d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 4: ( MARK 1982d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 5: c GLOBAL 'pickle PicklingError' 1983d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 27: q BINPUT 1 1984d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 29: o OBJ (MARK at 4) 1985d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 30: q BINPUT 2 1986d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 32: } EMPTY_DICT 1987d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 33: q BINPUT 3 1988d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 35: U SHORT_BINSTRING 'args' 1989d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 41: q BINPUT 4 1990d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 43: ) EMPTY_TUPLE 1991d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 44: s SETITEM 1992d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 45: b BUILD 1993d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 46: h BINGET 2 1994d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 48: e APPENDS (MARK at 3) 1995d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 49: . STOP 19968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 19978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersTry "the canonical" recursive-object test. 19988ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 19998ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> L = [] 20008ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> T = L, 20018ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> L.append(T) 20028ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> L[0] is T 20038ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersTrue 20048ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> T[0] is L 20058ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersTrue 20068ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> L[0][0] is L 20078ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersTrue 20088ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> T[0][0] is T 20098ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersTrue 2010f29d3d6011e41b40282994375454f2020a429d79Guido van Rossum>>> dis(pickle.dumps(L, 0)) 2011d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 0: ( MARK 2012d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 1: l LIST (MARK at 0) 2013d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 2: p PUT 0 2014d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 5: ( MARK 2015d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 6: g GET 0 2016d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 9: t TUPLE (MARK at 5) 2017d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 10: p PUT 1 2018d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 13: a APPEND 2019d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 14: . STOP 20208ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> dis(pickle.dumps(L, 1)) 2021d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 0: ] EMPTY_LIST 2022d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 1: q BINPUT 0 2023d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 3: ( MARK 2024d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 4: h BINGET 0 2025d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 6: t TUPLE (MARK at 3) 2026d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 7: q BINPUT 1 2027d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 9: a APPEND 2028d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 10: . STOP 20298ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 20308ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersThe protocol 0 pickle of the tuple causes the disassembly to get confused, 20318ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersas it doesn't realize that the POP opcode at 16 gets rid of the MARK at 0 20328ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters(so the output remains indented until the end). The protocol 1 pickle 20338ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdoesn't trigger this glitch, because the disassembler realizes that 20348ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim PetersPOP_MARK gets rid of the MARK. Doing a better job on the protocol 0 20358ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peterspickle would require the disassembler to emulate the stack. 20368ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2037f29d3d6011e41b40282994375454f2020a429d79Guido van Rossum>>> dis(pickle.dumps(T, 0)) 2038d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 0: ( MARK 2039d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 1: ( MARK 2040d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 2: l LIST (MARK at 1) 2041d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 3: p PUT 0 2042d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 6: ( MARK 2043d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 7: g GET 0 2044d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 10: t TUPLE (MARK at 6) 2045d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 11: p PUT 1 2046d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 14: a APPEND 2047d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 15: 0 POP 2048d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 16: 0 POP 2049d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 17: g GET 1 2050d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 20: . STOP 20518ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters>>> dis(pickle.dumps(T, 1)) 2052d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 0: ( MARK 2053d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 1: ] EMPTY_LIST 2054d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 2: q BINPUT 0 2055d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 4: ( MARK 2056d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 5: h BINGET 0 2057d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 7: t TUPLE (MARK at 4) 2058d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 8: q BINPUT 1 2059d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 10: a APPEND 2060d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 11: 1 POP_MARK (MARK at 0) 2061d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 12: h BINGET 1 2062d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 14: . STOP 2063d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 2064d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim PetersTry protocol 2. 2065d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 2066d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters>>> dis(pickle.dumps(L, 2)) 2067d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 0: \x80 PROTO 2 2068d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 2: ] EMPTY_LIST 2069d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 3: q BINPUT 0 2070d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 5: h BINGET 0 2071d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 7: \x85 TUPLE1 2072d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 8: q BINPUT 1 2073d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 10: a APPEND 2074d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 11: . STOP 2075d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 2076d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters>>> dis(pickle.dumps(T, 2)) 2077d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 0: \x80 PROTO 2 2078d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 2: ] EMPTY_LIST 2079d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 3: q BINPUT 0 2080d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 5: h BINGET 0 2081d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 7: \x85 TUPLE1 2082d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 8: q BINPUT 1 2083d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 10: a APPEND 2084d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 11: 0 POP 2085d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 12: h BINGET 1 2086d0f7c86a20e9cef8cbf2c8fb676fcc8d8a7000b4Tim Peters 14: . STOP 20878ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters""" 20888ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 2089570283584af6a9aff47d2341d6154055572aaff5Guido van Rossum__test__ = {'disassembler_test': _dis_test, 20908ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters } 20918ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 20928ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersdef _test(): 20938ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters import doctest 20948ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters return doctest.testmod() 20958ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters 20968ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Petersif __name__ == "__main__": 20978ecfc8ef9d6ffe0d9c732a438cb36e1e11480a19Tim Peters _test() 2098