15821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# Protocol Buffers - Google's data interchange format 25821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# Copyright 2008 Google Inc. All rights reserved. 35821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# http://code.google.com/p/protobuf/ 45821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# 55821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# Redistribution and use in source and binary forms, with or without 65821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# modification, are permitted provided that the following conditions are 75821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# met: 85821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# 95821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# * Redistributions of source code must retain the above copyright 105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# notice, this list of conditions and the following disclaimer. 115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# * Redistributions in binary form must reproduce the above 125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# copyright notice, this list of conditions and the following disclaimer 135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# in the documentation and/or other materials provided with the 145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# distribution. 155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# * Neither the name of Google Inc. nor the names of its 165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# contributors may be used to endorse or promote products derived from 175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# this software without specific prior written permission. 185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# 195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)"""Code for decoding protocol buffer primitives. 325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)This code is very similar to encoder.py -- read the docs for that module first. 345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)A "decoder" is a function with the signature: 365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Decode(buffer, pos, end, message, field_dict) 375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)The arguments are: 385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) buffer: The string containing the encoded message. 395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos: The current position in the string. 405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) end: The position in the string where the current message ends. May be 415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) less than len(buffer) if we're reading a sub-message. 425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) message: The message object into which we're parsing. 435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) field_dict: message._fields (avoids a hashtable lookup). 445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)The decoder reads the field and stores it into field_dict, returning the new 455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)buffer position. A decoder for a repeated field may proactively decode all of 465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)the elements of that field, if they appear consecutively. 475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)Note that decoders may throw any of the following: 495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) IndexError: Indicates a truncated message. 505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) struct.error: Unpacking of a fixed-width field failed. 515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) message.DecodeError: Other errors. 525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)Decoders are expected to raise an exception if they are called with pos > end. 545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)This allows callers to be lax about bounds checking: it's fineto read past 555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)"end" as long as you are sure that someone else will notice and throw an 565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)exception later on. 575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)Something up the call stack is expected to catch IndexError and struct.error 595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)and convert them to message.DecodeError. 605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)Decoders are constructed using decoder constructors with the signature: 625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) MakeDecoder(field_number, is_repeated, is_packed, key, new_default) 635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)The arguments are: 645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) field_number: The field number of the field we want to decode. 655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) is_repeated: Is the field a repeated field? (bool) 665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) is_packed: Is the field a packed field? (bool) 675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) key: The key to use when looking up the field within field_dict. 685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (This is actually the FieldDescriptor but nothing in this 695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) file should depend on that.) 705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_default: A function which takes a message object as a parameter and 715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) returns a new instance of the default value for this field. 725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (This is called for repeated fields and sub-messages, when an 735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) instance does not already exist.) 745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)As with encoders, we define a decoder constructor for every type of field. 765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)Then, for every field of every message class we construct an actual decoder. 775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)That decoder goes into a dict indexed by tag, so when we decode a message 785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)we repeatedly read a tag, look up the corresponding decoder, and invoke it. 795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)""" 805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)__author__ = 'kenton@google.com (Kenton Varda)' 825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)import struct 845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)from google.protobuf.internal import encoder 855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)from google.protobuf.internal import wire_format 865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)from google.protobuf import message 875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# This will overflow and thus become IEEE-754 "infinity". We would use 905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# "float('inf')" but it doesn't work on Windows pre-Python-2.6. 915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)_POS_INF = 1e10000 925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)_NEG_INF = -_POS_INF 935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)_NAN = _POS_INF * 0 945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# This is not for optimization, but rather to avoid conflicts with local 975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# variables named "message". 985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)_DecodeError = message.DecodeError 995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _VarintDecoder(mask): 1025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Return an encoder for a basic varint value (does not include tag). 1035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Decoded values will be bitwise-anded with the given mask before being 1055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) returned, e.g. to limit them to 32 bits. The returned decoder does not 1065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) take the usual "end" parameter -- the caller is expected to do bounds checking 1075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) after the fact (often the caller can defer such checking until later). The 1085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) decoder returns a (value, new_pos) pair. 1095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """ 1105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_ord = ord 1125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeVarint(buffer, pos): 1135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) result = 0 1145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) shift = 0 1155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while 1: 1165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) b = local_ord(buffer[pos]) 1175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) result |= ((b & 0x7f) << shift) 1185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos += 1 1195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if not (b & 0x80): 1205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) result &= mask 1215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return (result, pos) 1225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) shift += 7 1235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if shift >= 64: 1245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Too many bytes when decoding varint.') 1255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeVarint 1265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _SignedVarintDecoder(mask): 1295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Like _VarintDecoder() but decodes signed values.""" 1305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_ord = ord 1325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeVarint(buffer, pos): 1335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) result = 0 1345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) shift = 0 1355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while 1: 1365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) b = local_ord(buffer[pos]) 1375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) result |= ((b & 0x7f) << shift) 1385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos += 1 1395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if not (b & 0x80): 1405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if result > 0x7fffffffffffffff: 1415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) result -= (1 << 64) 1425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) result |= ~mask 1435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) else: 1445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) result &= mask 1455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return (result, pos) 1465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) shift += 7 1475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if shift >= 64: 1485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Too many bytes when decoding varint.') 1495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeVarint 1505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)_DecodeVarint = _VarintDecoder((1 << 64) - 1) 1535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)_DecodeSignedVarint = _SignedVarintDecoder((1 << 64) - 1) 1545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# Use these versions for values which must be limited to 32 bits. 1565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)_DecodeVarint32 = _VarintDecoder((1 << 32) - 1) 1575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)_DecodeSignedVarint32 = _SignedVarintDecoder((1 << 32) - 1) 1585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def ReadTag(buffer, pos): 1615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Read a tag from the buffer, and return a (tag_bytes, new_pos) tuple. 1625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) We return the raw bytes of the tag rather than decoding them. The raw 1645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) bytes can then be used to look up the proper decoder. This effectively allows 1655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) us to trade some work that would be done in pure-python (decoding a varint) 1665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) for work that is done in C (searching for a byte string in a hash table). 1675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) In a low-level language it would be much cheaper to decode the varint and 1685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) use that, but not in Python. 1695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """ 1705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) start = pos 1725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while ord(buffer[pos]) & 0x80: 1735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos += 1 1745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos += 1 1755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return (buffer[start:pos], pos) 1765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# -------------------------------------------------------------------- 1795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _SimpleDecoder(wire_type, decode_value): 1825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Return a constructor for a decoder for fields of a particular type. 1835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Args: 1855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_type: The field's wire type. 1865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) decode_value: A function which decodes an individual value, e.g. 1875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) _DecodeVarint() 1885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """ 1895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def SpecificDecoder(field_number, is_repeated, is_packed, key, new_default): 1915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if is_packed: 1925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_DecodeVarint = _DecodeVarint 1935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodePackedField(buffer, pos, end, message, field_dict): 1945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.get(key) 1955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value is None: 1965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.setdefault(key, new_default(message)) 1975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (endpoint, pos) = local_DecodeVarint(buffer, pos) 1985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) endpoint += pos 1995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if endpoint > end: 2005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated message.') 2015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while pos < endpoint: 2025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (element, pos) = decode_value(buffer, pos) 2035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value.append(element) 2045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if pos > endpoint: 2055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) del value[-1] # Discard corrupt value. 2065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Packed element was truncated.') 2075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return pos 2085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodePackedField 2095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) elif is_repeated: 2105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) tag_bytes = encoder.TagBytes(field_number, wire_type) 2115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) tag_len = len(tag_bytes) 2125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeRepeatedField(buffer, pos, end, message, field_dict): 2135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.get(key) 2145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value is None: 2155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.setdefault(key, new_default(message)) 2165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while 1: 2175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (element, new_pos) = decode_value(buffer, pos) 2185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value.append(element) 2195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Predict that the next tag is another copy of the same repeated 2205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # field. 2215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos = new_pos + tag_len 2225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if buffer[new_pos:pos] != tag_bytes or new_pos >= end: 2235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Prediction failed. Return. 2245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if new_pos > end: 2255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated message.') 2265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return new_pos 2275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeRepeatedField 2285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) else: 2295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeField(buffer, pos, end, message, field_dict): 2305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (field_dict[key], pos) = decode_value(buffer, pos) 2315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if pos > end: 2325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) del field_dict[key] # Discard corrupt value. 2335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated message.') 2345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return pos 2355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeField 2365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return SpecificDecoder 2385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _ModifiedDecoder(wire_type, decode_value, modify_value): 2415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Like SimpleDecoder but additionally invokes modify_value on every value 2425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) before storing it. Usually modify_value is ZigZagDecode. 2435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """ 2445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Reusing _SimpleDecoder is slightly slower than copying a bunch of code, but 2465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # not enough to make a significant difference. 2475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def InnerDecode(buffer, pos): 2495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (result, new_pos) = decode_value(buffer, pos) 2505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return (modify_value(result), new_pos) 2515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return _SimpleDecoder(wire_type, InnerDecode) 2525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _StructPackDecoder(wire_type, format): 2555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Return a constructor for a decoder for a fixed-width field. 2565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Args: 2585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_type: The field's wire type. 2595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) format: The format string to pass to struct.unpack(). 2605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """ 2615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value_size = struct.calcsize(format) 2635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_unpack = struct.unpack 2645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Reusing _SimpleDecoder is slightly slower than copying a bunch of code, but 2665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # not enough to make a significant difference. 2675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Note that we expect someone up-stack to catch struct.error and convert 2695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # it to _DecodeError -- this way we don't have to set up exception- 2705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # handling blocks every time we parse one value. 2715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def InnerDecode(buffer, pos): 2735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_pos = pos + value_size 2745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) result = local_unpack(format, buffer[pos:new_pos])[0] 2755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return (result, new_pos) 2765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return _SimpleDecoder(wire_type, InnerDecode) 2775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _FloatDecoder(): 2805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Returns a decoder for a float field. 2815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) This code works around a bug in struct.unpack for non-finite 32-bit 2835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) floating-point values. 2845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """ 2855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_unpack = struct.unpack 2875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def InnerDecode(buffer, pos): 2895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # We expect a 32-bit value in little-endian byte order. Bit 1 is the sign 2905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # bit, bits 2-9 represent the exponent, and bits 10-32 are the significand. 2915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_pos = pos + 4 2925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) float_bytes = buffer[pos:new_pos] 2935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # If this value has all its exponent bits set, then it's non-finite. 2955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # In Python 2.4, struct.unpack will convert it to a finite 64-bit value. 2965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # To avoid that, we parse it specially. 2975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if ((float_bytes[3] in '\x7F\xFF') 2985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) and (float_bytes[2] >= '\x80')): 2995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # If at least one significand bit is set... 3005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if float_bytes[0:3] != '\x00\x00\x80': 3015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return (_NAN, new_pos) 3025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # If sign bit is set... 3035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if float_bytes[3] == '\xFF': 3045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return (_NEG_INF, new_pos) 3055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return (_POS_INF, new_pos) 3065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Note that we expect someone up-stack to catch struct.error and convert 3085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # it to _DecodeError -- this way we don't have to set up exception- 3095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # handling blocks every time we parse one value. 3105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) result = local_unpack('<f', float_bytes)[0] 3115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return (result, new_pos) 3125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return _SimpleDecoder(wire_format.WIRETYPE_FIXED32, InnerDecode) 3135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _DoubleDecoder(): 3165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Returns a decoder for a double field. 3175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) This code works around a bug in struct.unpack for not-a-number. 3195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """ 3205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_unpack = struct.unpack 3225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def InnerDecode(buffer, pos): 3245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # We expect a 64-bit value in little-endian byte order. Bit 1 is the sign 3255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # bit, bits 2-12 represent the exponent, and bits 13-64 are the significand. 3265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_pos = pos + 8 3275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) double_bytes = buffer[pos:new_pos] 3285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # If this value has all its exponent bits set and at least one significand 3305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # bit set, it's not a number. In Python 2.4, struct.unpack will treat it 3315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # as inf or -inf. To avoid that, we treat it specially. 3325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if ((double_bytes[7] in '\x7F\xFF') 3335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) and (double_bytes[6] >= '\xF0') 3345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) and (double_bytes[0:7] != '\x00\x00\x00\x00\x00\x00\xF0')): 3355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return (_NAN, new_pos) 3365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Note that we expect someone up-stack to catch struct.error and convert 3385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # it to _DecodeError -- this way we don't have to set up exception- 3395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # handling blocks every time we parse one value. 3405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) result = local_unpack('<d', double_bytes)[0] 3415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return (result, new_pos) 3425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return _SimpleDecoder(wire_format.WIRETYPE_FIXED64, InnerDecode) 3435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# -------------------------------------------------------------------- 3465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)Int32Decoder = EnumDecoder = _SimpleDecoder( 3495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_format.WIRETYPE_VARINT, _DecodeSignedVarint32) 3505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)Int64Decoder = _SimpleDecoder( 3525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_format.WIRETYPE_VARINT, _DecodeSignedVarint) 3535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)UInt32Decoder = _SimpleDecoder(wire_format.WIRETYPE_VARINT, _DecodeVarint32) 3555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)UInt64Decoder = _SimpleDecoder(wire_format.WIRETYPE_VARINT, _DecodeVarint) 3565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)SInt32Decoder = _ModifiedDecoder( 3585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_format.WIRETYPE_VARINT, _DecodeVarint32, wire_format.ZigZagDecode) 3595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)SInt64Decoder = _ModifiedDecoder( 3605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_format.WIRETYPE_VARINT, _DecodeVarint, wire_format.ZigZagDecode) 3615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# Note that Python conveniently guarantees that when using the '<' prefix on 3635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# formats, they will also have the same size across all platforms (as opposed 3645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# to without the prefix, where their sizes depend on the C compiler's basic 3655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# type sizes). 3665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)Fixed32Decoder = _StructPackDecoder(wire_format.WIRETYPE_FIXED32, '<I') 3675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)Fixed64Decoder = _StructPackDecoder(wire_format.WIRETYPE_FIXED64, '<Q') 3685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)SFixed32Decoder = _StructPackDecoder(wire_format.WIRETYPE_FIXED32, '<i') 3695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)SFixed64Decoder = _StructPackDecoder(wire_format.WIRETYPE_FIXED64, '<q') 3705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)FloatDecoder = _FloatDecoder() 3715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)DoubleDecoder = _DoubleDecoder() 3725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)BoolDecoder = _ModifiedDecoder( 3745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_format.WIRETYPE_VARINT, _DecodeVarint, bool) 3755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def StringDecoder(field_number, is_repeated, is_packed, key, new_default): 3785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Returns a decoder for a string field.""" 3795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_DecodeVarint = _DecodeVarint 3815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_unicode = unicode 3825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) assert not is_packed 3845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if is_repeated: 3855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) tag_bytes = encoder.TagBytes(field_number, 3865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_format.WIRETYPE_LENGTH_DELIMITED) 3875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) tag_len = len(tag_bytes) 3885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeRepeatedField(buffer, pos, end, message, field_dict): 3895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.get(key) 3905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value is None: 3915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.setdefault(key, new_default(message)) 3925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while 1: 3935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (size, pos) = local_DecodeVarint(buffer, pos) 3945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_pos = pos + size 3955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if new_pos > end: 3965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated string.') 3975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value.append(local_unicode(buffer[pos:new_pos], 'utf-8')) 3985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Predict that the next tag is another copy of the same repeated field. 3995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos = new_pos + tag_len 4005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if buffer[new_pos:pos] != tag_bytes or new_pos == end: 4015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Prediction failed. Return. 4025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return new_pos 4035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeRepeatedField 4045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) else: 4055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeField(buffer, pos, end, message, field_dict): 4065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (size, pos) = local_DecodeVarint(buffer, pos) 4075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_pos = pos + size 4085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if new_pos > end: 4095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated string.') 4105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) field_dict[key] = local_unicode(buffer[pos:new_pos], 'utf-8') 4115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return new_pos 4125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeField 4135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def BytesDecoder(field_number, is_repeated, is_packed, key, new_default): 4165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Returns a decoder for a bytes field.""" 4175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_DecodeVarint = _DecodeVarint 4195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) assert not is_packed 4215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if is_repeated: 4225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) tag_bytes = encoder.TagBytes(field_number, 4235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_format.WIRETYPE_LENGTH_DELIMITED) 4245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) tag_len = len(tag_bytes) 4255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeRepeatedField(buffer, pos, end, message, field_dict): 4265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.get(key) 4275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value is None: 4285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.setdefault(key, new_default(message)) 4295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while 1: 4305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (size, pos) = local_DecodeVarint(buffer, pos) 4315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_pos = pos + size 4325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if new_pos > end: 4335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated string.') 4345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value.append(buffer[pos:new_pos]) 4355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Predict that the next tag is another copy of the same repeated field. 4365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos = new_pos + tag_len 4375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if buffer[new_pos:pos] != tag_bytes or new_pos == end: 4385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Prediction failed. Return. 4395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return new_pos 4405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeRepeatedField 4415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) else: 4425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeField(buffer, pos, end, message, field_dict): 4435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (size, pos) = local_DecodeVarint(buffer, pos) 4445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_pos = pos + size 4455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if new_pos > end: 4465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated string.') 4475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) field_dict[key] = buffer[pos:new_pos] 4485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return new_pos 4495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeField 4505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def GroupDecoder(field_number, is_repeated, is_packed, key, new_default): 4535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Returns a decoder for a group field.""" 4545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) end_tag_bytes = encoder.TagBytes(field_number, 4565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_format.WIRETYPE_END_GROUP) 4575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) end_tag_len = len(end_tag_bytes) 4585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) assert not is_packed 4605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if is_repeated: 4615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) tag_bytes = encoder.TagBytes(field_number, 4625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_format.WIRETYPE_START_GROUP) 4635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) tag_len = len(tag_bytes) 4645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeRepeatedField(buffer, pos, end, message, field_dict): 4655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.get(key) 4665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value is None: 4675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.setdefault(key, new_default(message)) 4685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while 1: 4695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.get(key) 4705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value is None: 4715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.setdefault(key, new_default(message)) 4725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Read sub-message. 4735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos = value.add()._InternalParse(buffer, pos, end) 4745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Read end tag. 4755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_pos = pos+end_tag_len 4765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if buffer[pos:new_pos] != end_tag_bytes or new_pos > end: 4775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Missing group end tag.') 4785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Predict that the next tag is another copy of the same repeated field. 4795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos = new_pos + tag_len 4805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if buffer[new_pos:pos] != tag_bytes or new_pos == end: 4815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Prediction failed. Return. 4825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return new_pos 4835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeRepeatedField 4845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) else: 4855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeField(buffer, pos, end, message, field_dict): 4865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.get(key) 4875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value is None: 4885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.setdefault(key, new_default(message)) 4895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Read sub-message. 4905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos = value._InternalParse(buffer, pos, end) 4915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Read end tag. 4925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_pos = pos+end_tag_len 4935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if buffer[pos:new_pos] != end_tag_bytes or new_pos > end: 4945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Missing group end tag.') 4955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return new_pos 4965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeField 4975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def MessageDecoder(field_number, is_repeated, is_packed, key, new_default): 5005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Returns a decoder for a message field.""" 5015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_DecodeVarint = _DecodeVarint 5035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) assert not is_packed 5055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if is_repeated: 5065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) tag_bytes = encoder.TagBytes(field_number, 5075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_format.WIRETYPE_LENGTH_DELIMITED) 5085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) tag_len = len(tag_bytes) 5095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeRepeatedField(buffer, pos, end, message, field_dict): 5105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.get(key) 5115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value is None: 5125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.setdefault(key, new_default(message)) 5135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while 1: 5145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.get(key) 5155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value is None: 5165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.setdefault(key, new_default(message)) 5175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Read length. 5185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (size, pos) = local_DecodeVarint(buffer, pos) 5195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_pos = pos + size 5205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if new_pos > end: 5215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated message.') 5225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Read sub-message. 5235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value.add()._InternalParse(buffer, pos, new_pos) != new_pos: 5245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # The only reason _InternalParse would return early is if it 5255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # encountered an end-group tag. 5265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Unexpected end-group tag.') 5275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Predict that the next tag is another copy of the same repeated field. 5285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos = new_pos + tag_len 5295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if buffer[new_pos:pos] != tag_bytes or new_pos == end: 5305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Prediction failed. Return. 5315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return new_pos 5325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeRepeatedField 5335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) else: 5345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeField(buffer, pos, end, message, field_dict): 5355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.get(key) 5365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value is None: 5375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.setdefault(key, new_default(message)) 5385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Read length. 5395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (size, pos) = local_DecodeVarint(buffer, pos) 5405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_pos = pos + size 5415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if new_pos > end: 5425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated message.') 5435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Read sub-message. 5445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value._InternalParse(buffer, pos, new_pos) != new_pos: 5455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # The only reason _InternalParse would return early is if it encountered 5465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # an end-group tag. 5475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Unexpected end-group tag.') 5485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return new_pos 5495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeField 5505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# -------------------------------------------------------------------- 5535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)MESSAGE_SET_ITEM_TAG = encoder.TagBytes(1, wire_format.WIRETYPE_START_GROUP) 5555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def MessageSetItemDecoder(extensions_by_number): 5575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Returns a decoder for a MessageSet item. 5585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) The parameter is the _extensions_by_number map for the message class. 5605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) The message set message looks like this: 5625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) message MessageSet { 5635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) repeated group Item = 1 { 5645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) required int32 type_id = 2; 5655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) required string message = 3; 5665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 5675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 5685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """ 5695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) type_id_tag_bytes = encoder.TagBytes(2, wire_format.WIRETYPE_VARINT) 5715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) message_tag_bytes = encoder.TagBytes(3, wire_format.WIRETYPE_LENGTH_DELIMITED) 5725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) item_end_tag_bytes = encoder.TagBytes(1, wire_format.WIRETYPE_END_GROUP) 5735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_ReadTag = ReadTag 5755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_DecodeVarint = _DecodeVarint 5765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_SkipField = SkipField 5775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def DecodeItem(buffer, pos, end, message, field_dict): 579ba5b9a6411cb1792fd21f0a078d7a25cd1ceec16Ben Murdoch message_set_item_start = pos 5805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) type_id = -1 5815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) message_start = -1 5825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) message_end = -1 5835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # Technically, type_id and message can appear in any order, so we need 5855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # a little loop here. 5865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while 1: 5875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (tag_bytes, pos) = local_ReadTag(buffer, pos) 5885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if tag_bytes == type_id_tag_bytes: 5895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (type_id, pos) = local_DecodeVarint(buffer, pos) 5905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) elif tag_bytes == message_tag_bytes: 5915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (size, message_start) = local_DecodeVarint(buffer, pos) 5925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos = message_end = message_start + size 5935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) elif tag_bytes == item_end_tag_bytes: 5945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) break 5955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) else: 5965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos = SkipField(buffer, pos, end, tag_bytes) 5975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if pos == -1: 5985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Missing group end tag.') 5995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if pos > end: 6015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated message.') 6025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if type_id == -1: 6045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('MessageSet item missing type_id.') 6055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if message_start == -1: 6065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('MessageSet item missing message.') 6075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) extension = extensions_by_number.get(type_id) 6095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if extension is not None: 6105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.get(extension) 6115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value is None: 6125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) value = field_dict.setdefault( 6135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) extension, extension.message_type._concrete_class()) 6145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if value._InternalParse(buffer, message_start,message_end) != message_end: 6155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # The only reason _InternalParse would return early is if it encountered 6165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # an end-group tag. 6175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Unexpected end-group tag.') 618ba5b9a6411cb1792fd21f0a078d7a25cd1ceec16Ben Murdoch else: 619ba5b9a6411cb1792fd21f0a078d7a25cd1ceec16Ben Murdoch if not message._unknown_fields: 620ba5b9a6411cb1792fd21f0a078d7a25cd1ceec16Ben Murdoch message._unknown_fields = [] 621ba5b9a6411cb1792fd21f0a078d7a25cd1ceec16Ben Murdoch message._unknown_fields.append((MESSAGE_SET_ITEM_TAG, 622ba5b9a6411cb1792fd21f0a078d7a25cd1ceec16Ben Murdoch buffer[message_set_item_start:pos])) 6235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return pos 6255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return DecodeItem 6275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# -------------------------------------------------------------------- 6295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# Optimization is not as heavy here because calls to SkipField() are rare, 6305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# except for handling end-group tags. 6315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _SkipVarint(buffer, pos, end): 6335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Skip a varint value. Returns the new position.""" 6345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while ord(buffer[pos]) & 0x80: 6365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos += 1 6375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos += 1 6385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if pos > end: 6395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated message.') 6405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return pos 6415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _SkipFixed64(buffer, pos, end): 6435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Skip a fixed64 value. Returns the new position.""" 6445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos += 8 6465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if pos > end: 6475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated message.') 6485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return pos 6495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _SkipLengthDelimited(buffer, pos, end): 6515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Skip a length-delimited value. Returns the new position.""" 6525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (size, pos) = _DecodeVarint(buffer, pos) 6545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos += size 6555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if pos > end: 6565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated message.') 6575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return pos 6585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _SkipGroup(buffer, pos, end): 6605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Skip sub-group. Returns the new position.""" 6615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while 1: 6635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (tag_bytes, pos) = ReadTag(buffer, pos) 6645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new_pos = SkipField(buffer, pos, end, tag_bytes) 6655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if new_pos == -1: 6665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return pos 6675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos = new_pos 6685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _EndGroup(buffer, pos, end): 6705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Skipping an END_GROUP tag returns -1 to tell the parent loop to break.""" 6715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return -1 6735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _SkipFixed32(buffer, pos, end): 6755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Skip a fixed32 value. Returns the new position.""" 6765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) pos += 4 6785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if pos > end: 6795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Truncated message.') 6805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return pos 6815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _RaiseInvalidWireType(buffer, pos, end): 6835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Skip function for unknown wire types. Raises an exception.""" 6845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) raise _DecodeError('Tag had invalid wire type.') 6865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)def _FieldSkipper(): 6885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Constructs the SkipField function.""" 6895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) WIRETYPE_TO_SKIPPER = [ 6915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) _SkipVarint, 6925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) _SkipFixed64, 6935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) _SkipLengthDelimited, 6945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) _SkipGroup, 6955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) _EndGroup, 6965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) _SkipFixed32, 6975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) _RaiseInvalidWireType, 6985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) _RaiseInvalidWireType, 6995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) ] 7005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 7015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wiretype_mask = wire_format.TAG_TYPE_MASK 7025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) local_ord = ord 7035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 7045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) def SkipField(buffer, pos, end, tag_bytes): 7055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """Skips a field with the specified tag. 7065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 7075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) |pos| should point to the byte immediately after the tag. 7085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 7095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Returns: 7105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) The new position (after the tag value), or -1 if the tag is an end-group 7115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) tag (in which case the calling loop should break). 7125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) """ 7135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 7145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) # The wire type is always in the first byte since varints are little-endian. 7155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) wire_type = local_ord(tag_bytes[0]) & wiretype_mask 7165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return WIRETYPE_TO_SKIPPER[wire_type](buffer, pos, end) 7175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 7185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return SkipField 7195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 7205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)SkipField = _FieldSkipper() 721