1fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# Protocol Buffers - Google's data interchange format 2fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# Copyright 2008 Google Inc. All rights reserved. 3afb4b72037e3f13db208590fc782c4bc8e27f862Jeff Davidson# https://developers.google.com/protocol-buffers/ 4fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# 5fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# Redistribution and use in source and binary forms, with or without 6fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# modification, are permitted provided that the following conditions are 7fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# met: 8fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# 9fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# * Redistributions of source code must retain the above copyright 10fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# notice, this list of conditions and the following disclaimer. 11fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# * Redistributions in binary form must reproduce the above 12fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# copyright notice, this list of conditions and the following disclaimer 13fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# in the documentation and/or other materials provided with the 14fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# distribution. 15fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# * Neither the name of Google Inc. nor the names of its 16fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# contributors may be used to endorse or promote products derived from 17fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# this software without specific prior written permission. 18fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# 19fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 20fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 21fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 22fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 23fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 24fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 25fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 26fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 27fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 28fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville 31a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson#PY25 compatible for GAE. 32a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson# 33a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson# Copyright 2009 Google Inc. All Rights Reserved. 34a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson 35d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville"""Code for encoding protocol message primitives. 36fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville 37fbaaef999ba563838ebd00874ed8a1c01fbf286dWink SavilleContains the logic for encoding every logical protocol field type 38fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Savilleinto one of the 5 physical wire types. 39d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 40d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleThis code is designed to push the Python interpreter's performance to the 41d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savillelimits. 42d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 43d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleThe basic idea is that at startup time, for every field (i.e. every 44d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleFieldDescriptor) we construct two functions: a "sizer" and an "encoder". The 45d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savillesizer takes a value of this field's type and computes its byte size. The 46d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilleencoder takes a writer function and a value. It encodes the value into byte 47d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savillestrings and invokes the writer function to write those strings. Typically the 48d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savillewriter function is the write() method of a cStringIO. 49d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 50d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleWe try to do as much work as possible when constructing the writer and the 51d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savillesizer rather than when calling them. In particular: 52d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville* We copy any needed global functions to local variables, so that we do not need 53d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville to do costly global table lookups at runtime. 54d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville* Similarly, we try to do any attribute lookups at startup time if possible. 55d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville* Every field's tag is encoded to bytes at startup, since it can't change at 56d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville runtime. 57d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville* Whatever component of the field size we can compute at startup, we do. 58d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville* We *avoid* sharing code if doing so would make the code slower and not sharing 59d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville does not burden us too much. For example, encoders for repeated fields do 60d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville not just call the encoders for singular fields in a loop because this would 61d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville add an extra function call overhead for every loop iteration; instead, we 62d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville manually inline the single-value encoder into the loop. 63d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville* If a Python function lacks a return statement, Python actually generates 64d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville instructions to pop the result of the last statement off the stack, push 65d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville None onto the stack, and then return that. If we really don't care what 66d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville value is returned, then we can save two instructions by returning the 67d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result of the last statement. It looks funny but it helps. 68d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville* We assume that type and bounds checking has happened at a higher level. 69fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville""" 70fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville 71d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville__author__ = 'kenton@google.com (Kenton Varda)' 72fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville 73fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Savilleimport struct 74a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidsonimport sys ##PY25 75a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson_PY2 = sys.version_info[0] < 3 ##PY25 76fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Savillefrom google.protobuf.internal import wire_format 77d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 78d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 79a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson# This will overflow and thus become IEEE-754 "infinity". We would use 80a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson# "float('inf')" but it doesn't work on Windows pre-Python-2.6. 81a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson_POS_INF = 1e10000 82a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson_NEG_INF = -_POS_INF 83a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson 84a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson 85d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef _VarintSize(value): 86d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Compute the size of a varint value.""" 87d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x7f: return 1 88d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x3fff: return 2 89d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x1fffff: return 3 90d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0xfffffff: return 4 91d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x7ffffffff: return 5 92d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x3ffffffffff: return 6 93d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x1ffffffffffff: return 7 94d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0xffffffffffffff: return 8 95d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x7fffffffffffffff: return 9 96d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return 10 97d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 98d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 99d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef _SignedVarintSize(value): 100d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Compute the size of a signed varint value.""" 101d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value < 0: return 10 102d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x7f: return 1 103d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x3fff: return 2 104d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x1fffff: return 3 105d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0xfffffff: return 4 106d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x7ffffffff: return 5 107d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x3ffffffffff: return 6 108d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x1ffffffffffff: return 7 109d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0xffffffffffffff: return 8 110d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value <= 0x7fffffffffffffff: return 9 111d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return 10 112d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 113d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 114d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef _TagSize(field_number): 115d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Returns the number of bytes required to serialize a tag with this field 116d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville number.""" 117d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville # Just pass in type 0, since the type won't affect the tag+type size. 118d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return _VarintSize(wire_format.PackTag(field_number, 0)) 119d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 120d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 121d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# -------------------------------------------------------------------- 122d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# In this section we define some generic sizers. Each of these functions 123d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# takes parameters specific to a particular field type, e.g. int32 or fixed64. 124d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# It returns another function which in turn takes parameters specific to a 125d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# particular field, e.g. the field number and whether it is repeated or packed. 126d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# Look at the next section to see how these are used. 127d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 128d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 129d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef _SimpleSizer(compute_value_size): 130d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """A sizer which uses the function compute_value_size to compute the size of 131d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville each value. Typically compute_value_size is _VarintSize.""" 132d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 133d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def SpecificSizer(field_number, is_repeated, is_packed): 134d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_size = _TagSize(field_number) 135d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_packed: 136d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_VarintSize = _VarintSize 137d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def PackedFieldSize(value): 138d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result = 0 139d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 140d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result += compute_value_size(element) 141d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return result + local_VarintSize(result) + tag_size 142d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return PackedFieldSize 143d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville elif is_repeated: 144d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def RepeatedFieldSize(value): 145d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result = tag_size * len(value) 146d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 147d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result += compute_value_size(element) 148d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return result 149d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return RepeatedFieldSize 150d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 151d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def FieldSize(value): 152d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return tag_size + compute_value_size(value) 153d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return FieldSize 154d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 155d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return SpecificSizer 156d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 157d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 158d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef _ModifiedSizer(compute_value_size, modify_value): 159d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Like SimpleSizer, but modify_value is invoked on each value before it is 160d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville passed to compute_value_size. modify_value is typically ZigZagEncode.""" 161d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 162d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def SpecificSizer(field_number, is_repeated, is_packed): 163d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_size = _TagSize(field_number) 164d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_packed: 165d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_VarintSize = _VarintSize 166d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def PackedFieldSize(value): 167d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result = 0 168d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 169d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result += compute_value_size(modify_value(element)) 170d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return result + local_VarintSize(result) + tag_size 171d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return PackedFieldSize 172d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville elif is_repeated: 173d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def RepeatedFieldSize(value): 174d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result = tag_size * len(value) 175d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 176d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result += compute_value_size(modify_value(element)) 177d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return result 178d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return RepeatedFieldSize 179d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 180d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def FieldSize(value): 181d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return tag_size + compute_value_size(modify_value(value)) 182d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return FieldSize 183d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 184d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return SpecificSizer 185d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 186d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 187d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef _FixedSizer(value_size): 188d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Like _SimpleSizer except for a fixed-size field. The input is the size 189d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville of one value.""" 190d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 191d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def SpecificSizer(field_number, is_repeated, is_packed): 192d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_size = _TagSize(field_number) 193d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_packed: 194d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_VarintSize = _VarintSize 195d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def PackedFieldSize(value): 196d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result = len(value) * value_size 197d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return result + local_VarintSize(result) + tag_size 198d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return PackedFieldSize 199d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville elif is_repeated: 200d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville element_size = value_size + tag_size 201d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def RepeatedFieldSize(value): 202d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return len(value) * element_size 203d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return RepeatedFieldSize 204d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 205d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville field_size = value_size + tag_size 206d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def FieldSize(value): 207d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return field_size 208d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return FieldSize 209d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 210d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return SpecificSizer 211d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 212d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 213d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# ==================================================================== 214d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# Here we declare a sizer constructor for each field type. Each "sizer 215d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# constructor" is a function that takes (field_number, is_repeated, is_packed) 216d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# as parameters and returns a sizer, which in turn takes a field value as 217d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# a parameter and returns its encoded size. 218d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 219d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 220d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleInt32Sizer = Int64Sizer = EnumSizer = _SimpleSizer(_SignedVarintSize) 221d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 222d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleUInt32Sizer = UInt64Sizer = _SimpleSizer(_VarintSize) 223d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 224d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleSInt32Sizer = SInt64Sizer = _ModifiedSizer( 225d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville _SignedVarintSize, wire_format.ZigZagEncode) 226d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 227d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleFixed32Sizer = SFixed32Sizer = FloatSizer = _FixedSizer(4) 228d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleFixed64Sizer = SFixed64Sizer = DoubleSizer = _FixedSizer(8) 229d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 230d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleBoolSizer = _FixedSizer(1) 231d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 232d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 233d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef StringSizer(field_number, is_repeated, is_packed): 234d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Returns a sizer for a string field.""" 235d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 236d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_size = _TagSize(field_number) 237d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_VarintSize = _VarintSize 238d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_len = len 239d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville assert not is_packed 240d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_repeated: 241d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def RepeatedFieldSize(value): 242d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result = tag_size * len(value) 243d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 244d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville l = local_len(element.encode('utf-8')) 245d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result += local_VarintSize(l) + l 246d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return result 247d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return RepeatedFieldSize 248d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 249d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def FieldSize(value): 250d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville l = local_len(value.encode('utf-8')) 251d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return tag_size + local_VarintSize(l) + l 252d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return FieldSize 253d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 254d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 255d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef BytesSizer(field_number, is_repeated, is_packed): 256d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Returns a sizer for a bytes field.""" 257d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 258d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_size = _TagSize(field_number) 259d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_VarintSize = _VarintSize 260d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_len = len 261d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville assert not is_packed 262d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_repeated: 263d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def RepeatedFieldSize(value): 264d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result = tag_size * len(value) 265d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 266d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville l = local_len(element) 267d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result += local_VarintSize(l) + l 268d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return result 269d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return RepeatedFieldSize 270d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 271d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def FieldSize(value): 272d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville l = local_len(value) 273d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return tag_size + local_VarintSize(l) + l 274d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return FieldSize 275d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 276d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 277d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef GroupSizer(field_number, is_repeated, is_packed): 278d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Returns a sizer for a group field.""" 279d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 280d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_size = _TagSize(field_number) * 2 281d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville assert not is_packed 282d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_repeated: 283d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def RepeatedFieldSize(value): 284d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result = tag_size * len(value) 285d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 286d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result += element.ByteSize() 287d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return result 288d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return RepeatedFieldSize 289d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 290d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def FieldSize(value): 291d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return tag_size + value.ByteSize() 292d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return FieldSize 293d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 294d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 295d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef MessageSizer(field_number, is_repeated, is_packed): 296d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Returns a sizer for a message field.""" 297d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 298d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_size = _TagSize(field_number) 299d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_VarintSize = _VarintSize 300d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville assert not is_packed 301d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_repeated: 302d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def RepeatedFieldSize(value): 303d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result = tag_size * len(value) 304d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 305d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville l = element.ByteSize() 306d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville result += local_VarintSize(l) + l 307d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return result 308d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return RepeatedFieldSize 309d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 310d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def FieldSize(value): 311d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville l = value.ByteSize() 312d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return tag_size + local_VarintSize(l) + l 313d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return FieldSize 314d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 315d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 316d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# -------------------------------------------------------------------- 317d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# MessageSet is special. 318d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 319d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 320d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef MessageSetItemSizer(field_number): 321d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Returns a sizer for extensions of MessageSet. 322d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 323d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville The message set message looks like this: 324d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville message MessageSet { 325d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville repeated group Item = 1 { 326d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville required int32 type_id = 2; 327d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville required string message = 3; 328d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville } 329d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville } 330d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """ 331d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville static_size = (_TagSize(1) * 2 + _TagSize(2) + _VarintSize(field_number) + 332d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville _TagSize(3)) 333d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_VarintSize = _VarintSize 334d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 335d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def FieldSize(value): 336d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville l = value.ByteSize() 337d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return static_size + local_VarintSize(l) + l 338d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 339d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return FieldSize 340d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 341d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 342d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# ==================================================================== 343d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# Encoders! 344d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 345d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 346d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef _VarintEncoder(): 347d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Return an encoder for a basic varint value (does not include tag).""" 348d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 349a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson local_chr = _PY2 and chr or (lambda x: bytes((x,))) ##PY25 350a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson##!PY25 local_chr = chr if bytes is str else lambda x: bytes((x,)) 351d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeVarint(write, value): 352d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville bits = value & 0x7f 353d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville value >>= 7 354d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville while value: 355d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(local_chr(0x80|bits)) 356d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville bits = value & 0x7f 357d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville value >>= 7 358d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return write(local_chr(bits)) 359d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 360d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeVarint 361d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 362d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 363d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef _SignedVarintEncoder(): 364d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Return an encoder for a basic signed varint value (does not include 365d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag).""" 366d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 367a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson local_chr = _PY2 and chr or (lambda x: bytes((x,))) ##PY25 368a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson##!PY25 local_chr = chr if bytes is str else lambda x: bytes((x,)) 369d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeSignedVarint(write, value): 370d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value < 0: 371d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville value += (1 << 64) 372d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville bits = value & 0x7f 373d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville value >>= 7 374d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville while value: 375d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(local_chr(0x80|bits)) 376d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville bits = value & 0x7f 377d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville value >>= 7 378d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return write(local_chr(bits)) 379d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 380d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeSignedVarint 381d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 382d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 383d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville_EncodeVarint = _VarintEncoder() 384d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville_EncodeSignedVarint = _SignedVarintEncoder() 385d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 386d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 387d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef _VarintBytes(value): 388d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Encode the given integer as a varint and return the bytes. This is only 389d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville called at startup time so it doesn't need to be fast.""" 390d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 391d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville pieces = [] 392d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville _EncodeVarint(pieces.append, value) 393a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson return "".encode("latin1").join(pieces) ##PY25 394a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson##!PY25 return b"".join(pieces) 395d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 396d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 397d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef TagBytes(field_number, wire_type): 398d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Encode the given tag and return the bytes. Only called at startup.""" 399d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 400d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return _VarintBytes(wire_format.PackTag(field_number, wire_type)) 401d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 402d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# -------------------------------------------------------------------- 403d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# As with sizers (see above), we have a number of common encoder 404d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# implementations. 405d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 406d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 407d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef _SimpleEncoder(wire_type, encode_value, compute_value_size): 408d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Return a constructor for an encoder for fields of a particular type. 409d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 410d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville Args: 411d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville wire_type: The field's wire type, for encoding tags. 412d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville encode_value: A function which encodes an individual value, e.g. 413d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville _EncodeVarint(). 414d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville compute_value_size: A function which computes the size of an individual 415d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville value, e.g. _VarintSize(). 416d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """ 417d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 418d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def SpecificEncoder(field_number, is_repeated, is_packed): 419d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_packed: 420d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_bytes = TagBytes(field_number, wire_format.WIRETYPE_LENGTH_DELIMITED) 421d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint = _EncodeVarint 422d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodePackedField(write, value): 423d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag_bytes) 424d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville size = 0 425d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 426d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville size += compute_value_size(element) 427d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint(write, size) 428d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 429d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville encode_value(write, element) 430d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodePackedField 431d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville elif is_repeated: 432d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_bytes = TagBytes(field_number, wire_type) 433d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeRepeatedField(write, value): 434d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 435d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag_bytes) 436d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville encode_value(write, element) 437d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeRepeatedField 438d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 439d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_bytes = TagBytes(field_number, wire_type) 440d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeField(write, value): 441d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag_bytes) 442d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return encode_value(write, value) 443d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeField 444d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 445d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return SpecificEncoder 446d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 447d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 448d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef _ModifiedEncoder(wire_type, encode_value, compute_value_size, modify_value): 449d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Like SimpleEncoder but additionally invokes modify_value on every value 450d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville before passing it to encode_value. Usually modify_value is ZigZagEncode.""" 451d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 452d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def SpecificEncoder(field_number, is_repeated, is_packed): 453d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_packed: 454d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_bytes = TagBytes(field_number, wire_format.WIRETYPE_LENGTH_DELIMITED) 455d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint = _EncodeVarint 456d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodePackedField(write, value): 457d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag_bytes) 458d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville size = 0 459d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 460d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville size += compute_value_size(modify_value(element)) 461d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint(write, size) 462d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 463d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville encode_value(write, modify_value(element)) 464d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodePackedField 465d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville elif is_repeated: 466d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_bytes = TagBytes(field_number, wire_type) 467d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeRepeatedField(write, value): 468d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 469d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag_bytes) 470d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville encode_value(write, modify_value(element)) 471d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeRepeatedField 472d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 473d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_bytes = TagBytes(field_number, wire_type) 474d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeField(write, value): 475d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag_bytes) 476d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return encode_value(write, modify_value(value)) 477d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeField 478d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 479d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return SpecificEncoder 480d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 481d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 482d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef _StructPackEncoder(wire_type, format): 483d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Return a constructor for an encoder for a fixed-width field. 484d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 485d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville Args: 486d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville wire_type: The field's wire type, for encoding tags. 487d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville format: The format string to pass to struct.pack(). 488d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """ 489d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 490d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville value_size = struct.calcsize(format) 491d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 492d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def SpecificEncoder(field_number, is_repeated, is_packed): 493d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_struct_pack = struct.pack 494d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_packed: 495d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_bytes = TagBytes(field_number, wire_format.WIRETYPE_LENGTH_DELIMITED) 496d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint = _EncodeVarint 497d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodePackedField(write, value): 498d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag_bytes) 499d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint(write, len(value) * value_size) 500d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 501d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(local_struct_pack(format, element)) 502d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodePackedField 503d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville elif is_repeated: 504d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_bytes = TagBytes(field_number, wire_type) 505d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeRepeatedField(write, value): 506d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 507d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag_bytes) 508d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(local_struct_pack(format, element)) 509d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeRepeatedField 510d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 511d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_bytes = TagBytes(field_number, wire_type) 512d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeField(write, value): 513d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag_bytes) 514d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return write(local_struct_pack(format, value)) 515d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeField 516d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 517d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return SpecificEncoder 518d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 519d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 520a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidsondef _FloatingPointEncoder(wire_type, format): 521a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson """Return a constructor for an encoder for float fields. 522a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson 523a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson This is like StructPackEncoder, but catches errors that may be due to 524a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson passing non-finite floating-point values to struct.pack, and makes a 525a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson second attempt to encode those values. 526a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson 527a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson Args: 528a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson wire_type: The field's wire type, for encoding tags. 529a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson format: The format string to pass to struct.pack(). 530a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson """ 531a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson 532a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson b = _PY2 and (lambda x:x) or (lambda x:x.encode('latin1')) ##PY25 533a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson value_size = struct.calcsize(format) 534a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson if value_size == 4: 535a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson def EncodeNonFiniteOrRaise(write, value): 536a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson # Remember that the serialized form uses little-endian byte order. 537a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson if value == _POS_INF: 538a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson write(b('\x00\x00\x80\x7F')) ##PY25 539a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson##!PY25 write(b'\x00\x00\x80\x7F') 540a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson elif value == _NEG_INF: 541a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson write(b('\x00\x00\x80\xFF')) ##PY25 542a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson##!PY25 write(b'\x00\x00\x80\xFF') 543a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson elif value != value: # NaN 544a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson write(b('\x00\x00\xC0\x7F')) ##PY25 545a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson##!PY25 write(b'\x00\x00\xC0\x7F') 546a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson else: 547a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson raise 548a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson elif value_size == 8: 549a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson def EncodeNonFiniteOrRaise(write, value): 550a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson if value == _POS_INF: 551a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson write(b('\x00\x00\x00\x00\x00\x00\xF0\x7F')) ##PY25 552a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson##!PY25 write(b'\x00\x00\x00\x00\x00\x00\xF0\x7F') 553a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson elif value == _NEG_INF: 554a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson write(b('\x00\x00\x00\x00\x00\x00\xF0\xFF')) ##PY25 555a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson##!PY25 write(b'\x00\x00\x00\x00\x00\x00\xF0\xFF') 556a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson elif value != value: # NaN 557a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson write(b('\x00\x00\x00\x00\x00\x00\xF8\x7F')) ##PY25 558a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson##!PY25 write(b'\x00\x00\x00\x00\x00\x00\xF8\x7F') 559a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson else: 560a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson raise 561a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson else: 562a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson raise ValueError('Can\'t encode floating-point values that are ' 563a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson '%d bytes long (only 4 or 8)' % value_size) 564a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson 565a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson def SpecificEncoder(field_number, is_repeated, is_packed): 566a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson local_struct_pack = struct.pack 567a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson if is_packed: 568a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson tag_bytes = TagBytes(field_number, wire_format.WIRETYPE_LENGTH_DELIMITED) 569a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson local_EncodeVarint = _EncodeVarint 570a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson def EncodePackedField(write, value): 571a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson write(tag_bytes) 572a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson local_EncodeVarint(write, len(value) * value_size) 573a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson for element in value: 574a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson # This try/except block is going to be faster than any code that 575a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson # we could write to check whether element is finite. 576a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson try: 577a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson write(local_struct_pack(format, element)) 578a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson except SystemError: 579a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson EncodeNonFiniteOrRaise(write, element) 580a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson return EncodePackedField 581a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson elif is_repeated: 582a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson tag_bytes = TagBytes(field_number, wire_type) 583a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson def EncodeRepeatedField(write, value): 584a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson for element in value: 585a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson write(tag_bytes) 586a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson try: 587a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson write(local_struct_pack(format, element)) 588a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson except SystemError: 589a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson EncodeNonFiniteOrRaise(write, element) 590a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson return EncodeRepeatedField 591a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson else: 592a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson tag_bytes = TagBytes(field_number, wire_type) 593a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson def EncodeField(write, value): 594a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson write(tag_bytes) 595a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson try: 596a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson write(local_struct_pack(format, value)) 597a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson except SystemError: 598a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson EncodeNonFiniteOrRaise(write, value) 599a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson return EncodeField 600a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson 601a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson return SpecificEncoder 602a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson 603a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson 604d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# ==================================================================== 605d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# Here we declare an encoder constructor for each field type. These work 606d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# very similarly to sizer constructors, described earlier. 607d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 608d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 609d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleInt32Encoder = Int64Encoder = EnumEncoder = _SimpleEncoder( 610d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville wire_format.WIRETYPE_VARINT, _EncodeSignedVarint, _SignedVarintSize) 611d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 612d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleUInt32Encoder = UInt64Encoder = _SimpleEncoder( 613d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville wire_format.WIRETYPE_VARINT, _EncodeVarint, _VarintSize) 614d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 615d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleSInt32Encoder = SInt64Encoder = _ModifiedEncoder( 616d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville wire_format.WIRETYPE_VARINT, _EncodeVarint, _VarintSize, 617d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville wire_format.ZigZagEncode) 618d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 619d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# Note that Python conveniently guarantees that when using the '<' prefix on 620d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# formats, they will also have the same size across all platforms (as opposed 621d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# to without the prefix, where their sizes depend on the C compiler's basic 622d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# type sizes). 623d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleFixed32Encoder = _StructPackEncoder(wire_format.WIRETYPE_FIXED32, '<I') 624d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleFixed64Encoder = _StructPackEncoder(wire_format.WIRETYPE_FIXED64, '<Q') 625d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleSFixed32Encoder = _StructPackEncoder(wire_format.WIRETYPE_FIXED32, '<i') 626d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink SavilleSFixed64Encoder = _StructPackEncoder(wire_format.WIRETYPE_FIXED64, '<q') 627a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff DavidsonFloatEncoder = _FloatingPointEncoder(wire_format.WIRETYPE_FIXED32, '<f') 628a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff DavidsonDoubleEncoder = _FloatingPointEncoder(wire_format.WIRETYPE_FIXED64, '<d') 629d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 630d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 631d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef BoolEncoder(field_number, is_repeated, is_packed): 632d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Returns an encoder for a boolean field.""" 633d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 634a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson##!PY25 false_byte = b'\x00' 635a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson##!PY25 true_byte = b'\x01' 636a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson false_byte = '\x00'.encode('latin1') ##PY25 637a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson true_byte = '\x01'.encode('latin1') ##PY25 638d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_packed: 639d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_bytes = TagBytes(field_number, wire_format.WIRETYPE_LENGTH_DELIMITED) 640d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint = _EncodeVarint 641d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodePackedField(write, value): 642d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag_bytes) 643d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint(write, len(value)) 644d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 645d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if element: 646d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(true_byte) 647d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 648d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(false_byte) 649d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodePackedField 650d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville elif is_repeated: 651d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_bytes = TagBytes(field_number, wire_format.WIRETYPE_VARINT) 652d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeRepeatedField(write, value): 653d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 654d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag_bytes) 655d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if element: 656d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(true_byte) 657d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 658d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(false_byte) 659d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeRepeatedField 660d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 661d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag_bytes = TagBytes(field_number, wire_format.WIRETYPE_VARINT) 662d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeField(write, value): 663d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag_bytes) 664d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if value: 665d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return write(true_byte) 666d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return write(false_byte) 667d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeField 668d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 669d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 670d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef StringEncoder(field_number, is_repeated, is_packed): 671d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Returns an encoder for a string field.""" 672d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 673d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag = TagBytes(field_number, wire_format.WIRETYPE_LENGTH_DELIMITED) 674d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint = _EncodeVarint 675d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_len = len 676d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville assert not is_packed 677d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_repeated: 678d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeRepeatedField(write, value): 679d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 680d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville encoded = element.encode('utf-8') 681d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag) 682d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint(write, local_len(encoded)) 683d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(encoded) 684d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeRepeatedField 685d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 686d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeField(write, value): 687d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville encoded = value.encode('utf-8') 688d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag) 689d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint(write, local_len(encoded)) 690d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return write(encoded) 691d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeField 692d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 693d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 694d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef BytesEncoder(field_number, is_repeated, is_packed): 695d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Returns an encoder for a bytes field.""" 696d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 697d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag = TagBytes(field_number, wire_format.WIRETYPE_LENGTH_DELIMITED) 698d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint = _EncodeVarint 699d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_len = len 700d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville assert not is_packed 701d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_repeated: 702d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeRepeatedField(write, value): 703d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 704d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag) 705d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint(write, local_len(element)) 706d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(element) 707d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeRepeatedField 708d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 709d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeField(write, value): 710d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag) 711d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint(write, local_len(value)) 712d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return write(value) 713d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeField 714d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 715d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 716d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef GroupEncoder(field_number, is_repeated, is_packed): 717d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Returns an encoder for a group field.""" 718d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 719d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville start_tag = TagBytes(field_number, wire_format.WIRETYPE_START_GROUP) 720d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville end_tag = TagBytes(field_number, wire_format.WIRETYPE_END_GROUP) 721d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville assert not is_packed 722d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_repeated: 723d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeRepeatedField(write, value): 724d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 725d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(start_tag) 726d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville element._InternalSerialize(write) 727d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(end_tag) 728d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeRepeatedField 729d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 730d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeField(write, value): 731d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(start_tag) 732d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville value._InternalSerialize(write) 733d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return write(end_tag) 734d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeField 735d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 736d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 737d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef MessageEncoder(field_number, is_repeated, is_packed): 738d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Returns an encoder for a message field.""" 739d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 740d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville tag = TagBytes(field_number, wire_format.WIRETYPE_LENGTH_DELIMITED) 741d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint = _EncodeVarint 742d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville assert not is_packed 743d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville if is_repeated: 744d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeRepeatedField(write, value): 745d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville for element in value: 746d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag) 747d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint(write, element.ByteSize()) 748d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville element._InternalSerialize(write) 749d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeRepeatedField 750d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville else: 751d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeField(write, value): 752d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(tag) 753d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint(write, value.ByteSize()) 754d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return value._InternalSerialize(write) 755d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeField 756d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 757d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 758d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# -------------------------------------------------------------------- 759d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville# As before, MessageSet is special. 760d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 761d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 762d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Savilledef MessageSetItemEncoder(field_number): 763d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """Encoder for extensions of MessageSet. 764d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 765d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville The message set message looks like this: 766d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville message MessageSet { 767d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville repeated group Item = 1 { 768d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville required int32 type_id = 2; 769d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville required string message = 3; 770fbaaef999ba563838ebd00874ed8a1c01fbf286dWink Saville } 771d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville } 772d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville """ 773a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson start_bytes = "".encode("latin1").join([ ##PY25 774a3b2a6da25a76f17c73d31def3952feb0fd2296eJeff Davidson##!PY25 start_bytes = b"".join([ 775d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville TagBytes(1, wire_format.WIRETYPE_START_GROUP), 776d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville TagBytes(2, wire_format.WIRETYPE_VARINT), 777d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville _VarintBytes(field_number), 778d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville TagBytes(3, wire_format.WIRETYPE_LENGTH_DELIMITED)]) 779d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville end_bytes = TagBytes(1, wire_format.WIRETYPE_END_GROUP) 780d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint = _EncodeVarint 781d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 782d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville def EncodeField(write, value): 783d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville write(start_bytes) 784d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville local_EncodeVarint(write, value.ByteSize()) 785d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville value._InternalSerialize(write) 786d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return write(end_bytes) 787d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville 788d0332953cda33fb4f8e24ebff9c49159b69c43d6Wink Saville return EncodeField 789