1a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)# Copyright 2014 The Chromium Authors. All rights reserved. 2a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)# Use of this source code is governed by a BSD-style license that can be 3a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)# found in the LICENSE file. 4a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 5a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)import collections 6a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)import datetime 7a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)import logging 8a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)import multiprocessing 9a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)import os 10a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)import posixpath 11a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)import Queue 12a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)import re 13a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)import subprocess 14a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)import sys 15a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)import threading 16a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 17a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 1846d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles)# addr2line builds a possibly infinite memory cache that can exhaust 1946d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles)# the computer's memory if allowed to grow for too long. This constant 2046d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles)# controls how many lookups we do before restarting the process. 4000 2146d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles)# gives near peak performance without extreme memory usage. 2246d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles)ADDR2LINE_RECYCLE_LIMIT = 4000 2346d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) 2446d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) 25a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)class ELFSymbolizer(object): 26a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """An uber-fast (multiprocessing, pipelined and asynchronous) ELF symbolizer. 27a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 28a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) This class is a frontend for addr2line (part of GNU binutils), designed to 29a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) symbolize batches of large numbers of symbols for a given ELF file. It 30a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) supports sharding symbolization against many addr2line instances and 31a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) pipelining of multiple requests per each instance (in order to hide addr2line 32a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) internals and OS pipe latencies). 33a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 34a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) The interface exhibited by this class is a very simple asynchronous interface, 35a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) which is based on the following three methods: 36a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) - SymbolizeAsync(): used to request (enqueue) resolution of a given address. 37a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) - The |callback| method: used to communicated back the symbol information. 38a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) - Join(): called to conclude the batch to gather the last outstanding results. 39a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) In essence, before the Join method returns, this class will have issued as 40a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) many callbacks as the number of SymbolizeAsync() calls. In this regard, note 41a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) that due to multiprocess sharding, callbacks can be delivered out of order. 42a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 43a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) Some background about addr2line: 44a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) - it is invoked passing the elf path in the cmdline, piping the addresses in 45a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) its stdin and getting results on its stdout. 46a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) - it has pretty large response times for the first requests, but it 47a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) works very well in streaming mode once it has been warmed up. 48a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) - it doesn't scale by itself (on more cores). However, spawning multiple 49a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) instances at the same time on the same file is pretty efficient as they 50a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) keep hitting the pagecache and become mostly CPU bound. 51a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) - it might hang or crash, mostly for OOM. This class deals with both of these 52a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) problems. 53a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 54a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) Despite the "scary" imports and the multi* words above, (almost) no multi- 55a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) threading/processing is involved from the python viewpoint. Concurrency 56a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) here is achieved by spawning several addr2line subprocesses and handling their 57a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) output pipes asynchronously. Therefore, all the code here (with the exception 58a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) of the Queue instance in Addr2Line) should be free from mind-blowing 59a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) thread-safety concerns. 60a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 61a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) The multiprocess sharding works as follows: 62a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) The symbolizer tries to use the lowest number of addr2line instances as 63a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) possible (with respect of |max_concurrent_jobs|) and enqueue all the requests 64a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) in a single addr2line instance. For few symbols (i.e. dozens) sharding isn't 65a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) worth the startup cost. 66a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) The multiprocess logic kicks in as soon as the queues for the existing 67a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) instances grow. Specifically, once all the existing instances reach the 68a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) |max_queue_size| bound, a new addr2line instance is kicked in. 69a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) In the case of a very eager producer (i.e. all |max_concurrent_jobs| instances 70a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) have a backlog of |max_queue_size|), back-pressure is applied on the caller by 71a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) blocking the SymbolizeAsync method. 72a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 73a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) This module has been deliberately designed to be dependency free (w.r.t. of 74a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) other modules in this project), to allow easy reuse in external projects. 75a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """ 76a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 77a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def __init__(self, elf_file_path, addr2line_path, callback, inlines=False, 78a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) max_concurrent_jobs=None, addr2line_timeout=30, max_queue_size=50): 79a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """Args: 80a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) elf_file_path: path of the elf file to be symbolized. 81a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) addr2line_path: path of the toolchain's addr2line binary. 82a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) callback: a callback which will be invoked for each resolved symbol with 83a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) the two args (sym_info, callback_arg). The former is an instance of 84a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) |ELFSymbolInfo| and contains the symbol information. The latter is an 85a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) embedder-provided argument which is passed to SymbolizeAsync(). 86a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) inlines: when True, the ELFSymbolInfo will contain also the details about 87a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) the outer inlining functions. When False, only the innermost function 88a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) will be provided. 89a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) max_concurrent_jobs: Max number of addr2line instances spawned. 90a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) Parallelize responsibly, addr2line is a memory and I/O monster. 91a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) max_queue_size: Max number of outstanding requests per addr2line instance. 92a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) addr2line_timeout: Max time (in seconds) to wait for a addr2line response. 93a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) After the timeout, the instance will be considered hung and respawned. 94a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """ 95a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) assert(os.path.isfile(addr2line_path)), 'Cannot find ' + addr2line_path 96a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.elf_file_path = elf_file_path 97a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.addr2line_path = addr2line_path 98a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.callback = callback 99a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.inlines = inlines 100a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.max_concurrent_jobs = (max_concurrent_jobs or 101a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) min(multiprocessing.cpu_count(), 4)) 102a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.max_queue_size = max_queue_size 103a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.addr2line_timeout = addr2line_timeout 104a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.requests_counter = 0 # For generating monotonic request IDs. 105a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._a2l_instances = [] # Up to |max_concurrent_jobs| _Addr2Line inst. 106a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 107a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # Create one addr2line instance. More instances will be created on demand 108a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # (up to |max_concurrent_jobs|) depending on the rate of the requests. 109a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._CreateNewA2LInstance() 110a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 111a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def SymbolizeAsync(self, addr, callback_arg=None): 112a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """Requests symbolization of a given address. 113a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 114a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) This method is not guaranteed to return immediately. It generally does, but 115a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) in some scenarios (e.g. all addr2line instances have full queues) it can 116a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) block to create back-pressure. 117a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 118a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) Args: 119a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) addr: address to symbolize. 120a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) callback_arg: optional argument which will be passed to the |callback|.""" 121a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) assert(isinstance(addr, int)) 122a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 123a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # Process all the symbols that have been resolved in the meanwhile. 124a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # Essentially, this drains all the addr2line(s) out queues. 125a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) for a2l_to_purge in self._a2l_instances: 126a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) a2l_to_purge.ProcessAllResolvedSymbolsInQueue() 12746d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) a2l_to_purge.RecycleIfNecessary() 128a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 129a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # Find the best instance according to this logic: 130a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # 1. Find an existing instance with the shortest queue. 131a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # 2. If all of instances' queues are full, but there is room in the pool, 132a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # (i.e. < |max_concurrent_jobs|) create a new instance. 133a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # 3. If there were already |max_concurrent_jobs| instances and all of them 134a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # had full queues, make back-pressure. 135a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 136a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # 1. 137a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def _SortByQueueSizeAndReqID(a2l): 138a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) return (a2l.queue_size, a2l.first_request_id) 139a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) a2l = min(self._a2l_instances, key=_SortByQueueSizeAndReqID) 140a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 141a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # 2. 142a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if (a2l.queue_size >= self.max_queue_size and 143a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) len(self._a2l_instances) < self.max_concurrent_jobs): 144a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) a2l = self._CreateNewA2LInstance() 145a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 146a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # 3. 147a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if a2l.queue_size >= self.max_queue_size: 148a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) a2l.WaitForNextSymbolInQueue() 149a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 150a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) a2l.EnqueueRequest(addr, callback_arg) 151a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 152a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def Join(self): 153a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """Waits for all the outstanding requests to complete and terminates.""" 154a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) for a2l in self._a2l_instances: 155a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) a2l.WaitForIdle() 156a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) a2l.Terminate() 157a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 158a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def _CreateNewA2LInstance(self): 159a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) assert(len(self._a2l_instances) < self.max_concurrent_jobs) 160a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) a2l = ELFSymbolizer.Addr2Line(self) 161a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._a2l_instances.append(a2l) 162a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) return a2l 163a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 164a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 165a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) class Addr2Line(object): 166a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """A python wrapper around an addr2line instance. 167a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 168a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) The communication with the addr2line process looks as follows: 169a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) [STDIN] [STDOUT] (from addr2line's viewpoint) 170a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) > f001111 171a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) > f002222 172a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) < Symbol::Name(foo, bar) for f001111 173a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) < /path/to/source/file.c:line_number 174a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) > f003333 175a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) < Symbol::Name2() for f002222 176a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) < /path/to/source/file.c:line_number 177a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) < Symbol::Name3() for f003333 178a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) < /path/to/source/file.c:line_number 179a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """ 180a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 181a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) SYM_ADDR_RE = re.compile(r'([^:]+):(\?|\d+).*') 182a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 183a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def __init__(self, symbolizer): 184a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._symbolizer = symbolizer 185a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._lib_file_name = posixpath.basename(symbolizer.elf_file_path) 186a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 187a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # The request queue (i.e. addresses pushed to addr2line's stdin and not 188a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # yet retrieved on stdout) 189a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._request_queue = collections.deque() 190a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 191a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # This is essentially len(self._request_queue). It has been optimized to a 192a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # separate field because turned out to be a perf hot-spot. 193a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.queue_size = 0 194a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 19546d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) # Keep track of the number of symbols a process has processed to 19646d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) # avoid a single process growing too big and using all the memory. 19746d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) self._processed_symbols_count = 0 19846d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) 199a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # Objects required to handle the addr2line subprocess. 200a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._proc = None # Subprocess.Popen(...) instance. 201a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._thread = None # Threading.thread instance. 202a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._out_queue = None # Queue.Queue instance (for buffering a2l stdout). 203a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._RestartAddr2LineProcess() 204a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 205a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def EnqueueRequest(self, addr, callback_arg): 206a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """Pushes an address to addr2line's stdin (and keeps track of it).""" 207a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._symbolizer.requests_counter += 1 # For global "age" of requests. 208a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) req_idx = self._symbolizer.requests_counter 209a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._request_queue.append((addr, callback_arg, req_idx)) 210a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.queue_size += 1 211a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._WriteToA2lStdin(addr) 212a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 213a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def WaitForIdle(self): 214a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """Waits until all the pending requests have been symbolized.""" 215a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) while self.queue_size > 0: 216a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.WaitForNextSymbolInQueue() 217a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 218a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def WaitForNextSymbolInQueue(self): 219a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """Waits for the next pending request to be symbolized.""" 220a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if not self.queue_size: 221a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) return 222a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 223a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # This outer loop guards against a2l hanging (detecting stdout timeout). 224a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) while True: 225a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) start_time = datetime.datetime.now() 226a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) timeout = datetime.timedelta(seconds=self._symbolizer.addr2line_timeout) 227a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 228a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # The inner loop guards against a2l crashing (checking if it exited). 229a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) while (datetime.datetime.now() - start_time < timeout): 230a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # poll() returns !None if the process exited. a2l should never exit. 231a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if self._proc.poll(): 232a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) logging.warning('addr2line crashed, respawning (lib: %s).' % 233a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._lib_file_name) 234a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._RestartAddr2LineProcess() 235a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # TODO(primiano): the best thing to do in this case would be 236a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # shrinking the pool size as, very likely, addr2line is crashed 237a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # due to low memory (and the respawned one will die again soon). 238a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 239a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) try: 240a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) lines = self._out_queue.get(block=True, timeout=0.25) 241a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) except Queue.Empty: 242a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # On timeout (1/4 s.) repeat the inner loop and check if either the 243a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # addr2line process did crash or we waited its output for too long. 244a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) continue 245a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 246a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # In nominal conditions, we get straight to this point. 247a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._ProcessSymbolOutput(lines) 248a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) return 249a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 250a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # If this point is reached, we waited more than |addr2line_timeout|. 251a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) logging.warning('Hung addr2line process, respawning (lib: %s).' % 252a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._lib_file_name) 253a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._RestartAddr2LineProcess() 254a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 255a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def ProcessAllResolvedSymbolsInQueue(self): 256a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """Consumes all the addr2line output lines produced (without blocking).""" 257a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if not self.queue_size: 258a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) return 259a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) while True: 260a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) try: 261a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) lines = self._out_queue.get_nowait() 262a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) except Queue.Empty: 263a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) break 264a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._ProcessSymbolOutput(lines) 265a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 26646d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) def RecycleIfNecessary(self): 26746d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) """Restarts the process if it has been used for too long. 26846d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) 26946d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) A long running addr2line process will consume excessive amounts 27046d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) of memory without any gain in performance.""" 27146d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) if self._processed_symbols_count >= ADDR2LINE_RECYCLE_LIMIT: 27246d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) self._RestartAddr2LineProcess() 27346d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) 27446d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) 275a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def Terminate(self): 276a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """Kills the underlying addr2line process. 277a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 278a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) The poller |_thread| will terminate as well due to the broken pipe.""" 279a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) try: 280a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._proc.kill() 281a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._proc.communicate() # Essentially wait() without risking deadlock. 282a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) except Exception: # An exception while terminating? How interesting. 283a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) pass 284a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._proc = None 285a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 286a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def _WriteToA2lStdin(self, addr): 287a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._proc.stdin.write('%s\n' % hex(addr)) 288a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if self._symbolizer.inlines: 289a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # In the case of inlines we output an extra blank line, which causes 290a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # addr2line to emit a (??,??:0) tuple that we use as a boundary marker. 291a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._proc.stdin.write('\n') 292a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._proc.stdin.flush() 293a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 294a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def _ProcessSymbolOutput(self, lines): 295a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """Parses an addr2line symbol output and triggers the client callback.""" 296a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) (_, callback_arg, _) = self._request_queue.popleft() 297a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.queue_size -= 1 298a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 299a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) innermost_sym_info = None 300a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) sym_info = None 301a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) for (line1, line2) in lines: 302a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) prev_sym_info = sym_info 303a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) name = line1 if not line1.startswith('?') else None 304a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) source_path = None 305a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) source_line = None 306a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) m = ELFSymbolizer.Addr2Line.SYM_ADDR_RE.match(line2) 307a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if m: 308a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if not m.group(1).startswith('?'): 309a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) source_path = m.group(1) 310cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) if not m.group(2).startswith('?'): 311cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) source_line = int(m.group(2)) 312a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) else: 313a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) logging.warning('Got invalid symbol path from addr2line: %s' % line2) 314a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 315a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) sym_info = ELFSymbolInfo(name, source_path, source_line) 316a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if prev_sym_info: 317a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) prev_sym_info.inlined_by = sym_info 318a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if not innermost_sym_info: 319a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) innermost_sym_info = sym_info 320a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 32146d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) self._processed_symbols_count += 1 322a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._symbolizer.callback(innermost_sym_info, callback_arg) 323a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 324a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def _RestartAddr2LineProcess(self): 325a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if self._proc: 326a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.Terminate() 327a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 328a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # The only reason of existence of this Queue (and the corresponding 329a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # Thread below) is the lack of a subprocess.stdout.poll_avail_lines(). 330a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # Essentially this is a pipe able to extract a couple of lines atomically. 331a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._out_queue = Queue.Queue() 332a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 333a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # Start the underlying addr2line process in line buffered mode. 334a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 335a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) cmd = [self._symbolizer.addr2line_path, '--functions', '--demangle', 336a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) '--exe=' + self._symbolizer.elf_file_path] 337a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if self._symbolizer.inlines: 338a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) cmd += ['--inlines'] 339a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._proc = subprocess.Popen(cmd, bufsize=1, stdout=subprocess.PIPE, 340a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) stdin=subprocess.PIPE, stderr=sys.stderr, close_fds=True) 341a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 342a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # Start the poller thread, which simply moves atomically the lines read 343a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # from the addr2line's stdout to the |_out_queue|. 344a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._thread = threading.Thread( 345a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) target=ELFSymbolizer.Addr2Line.StdoutReaderThread, 346a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) args=(self._proc.stdout, self._out_queue, self._symbolizer.inlines)) 347a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._thread.daemon = True # Don't prevent early process exit. 348a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._thread.start() 349a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 35046d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) self._processed_symbols_count = 0 35146d4c2bc3267f3f028f39e7e311b0f89aba2e4fdTorne (Richard Coles) 352a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # Replay the pending requests on the new process (only for the case 353a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # of a hung addr2line timing out during the game). 354a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) for (addr, _, _) in self._request_queue: 355a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self._WriteToA2lStdin(addr) 356a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 357a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) @staticmethod 358a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def StdoutReaderThread(process_pipe, queue, inlines): 359a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """The poller thread fn, which moves the addr2line stdout to the |queue|. 360a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 361a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) This is the only piece of code not running on the main thread. It merely 362a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) writes to a Queue, which is thread-safe. In the case of inlines, it 363a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) detects the ??,??:0 marker and sends the lines atomically, such that the 364a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) main thread always receives all the lines corresponding to one symbol in 365a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) one shot.""" 366a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) try: 367a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) lines_for_one_symbol = [] 368a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) while True: 369a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) line1 = process_pipe.readline().rstrip('\r\n') 370a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) line2 = process_pipe.readline().rstrip('\r\n') 371a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if not line1 or not line2: 372a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) break 373a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) inline_has_more_lines = inlines and (len(lines_for_one_symbol) == 0 or 374a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) (line1 != '??' and line2 != '??:0')) 375a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if not inlines or inline_has_more_lines: 376a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) lines_for_one_symbol += [(line1, line2)] 377a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) if inline_has_more_lines: 378a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) continue 379a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) queue.put(lines_for_one_symbol) 380a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) lines_for_one_symbol = [] 381a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) process_pipe.close() 382a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 383a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # Every addr2line processes will die at some point, please die silently. 384a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) except (IOError, OSError): 385a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) pass 386a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 387a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) @property 388a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def first_request_id(self): 389a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """Returns the request_id of the oldest pending request in the queue.""" 390a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) return self._request_queue[0][2] if self._request_queue else 0 391a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 392a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 393a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles)class ELFSymbolInfo(object): 394a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """The result of the symbolization passed as first arg. of each callback.""" 395a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 396a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def __init__(self, name, source_path, source_line): 397a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) """All the fields here can be None (if addr2line replies with '??').""" 398a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.name = name 399a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.source_path = source_path 400a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.source_line = source_line 401a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # In the case of |inlines|=True, the |inlined_by| points to the outer 402a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) # function inlining the current one (and so on, to form a chain). 403a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.inlined_by = None 404a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) 405a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) def __str__(self): 406a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) return '%s [%s:%d]' % ( 407a1401311d1ab56c4ed0a474bd38c108f75cb0cd9Torne (Richard Coles) self.name or '??', self.source_path or '??', self.source_line or 0) 408