15821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)/* Copyright (c) 2006, Google Inc. 25821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * All rights reserved. 35821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * 45821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * Redistribution and use in source and binary forms, with or without 55821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * modification, are permitted provided that the following conditions are 65821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * met: 75821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * 85821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * * Redistributions of source code must retain the above copyright 95821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * notice, this list of conditions and the following disclaimer. 105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * * Redistributions in binary form must reproduce the above 115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * copyright notice, this list of conditions and the following disclaimer 125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * in the documentation and/or other materials provided with the 135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * distribution. 145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * * Neither the name of Google Inc. nor the names of its 155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * contributors may be used to endorse or promote products derived from 165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * this software without specific prior written permission. 175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * 185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * 305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * --- 315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * Author: Maxim Lifantsev 325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) */ 335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Background and key design points of MemoryRegionMap. 365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// MemoryRegionMap is a low-level module with quite atypical requirements that 385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// result in some degree of non-triviality of the implementation and design. 395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// MemoryRegionMap collects info about *all* memory regions created with 415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// mmap, munmap, mremap, sbrk. 425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// They key word above is 'all': all that are happening in a process 435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// during its lifetime frequently starting even before global object 445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// constructor execution. 455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// This is needed by the primary client of MemoryRegionMap: 475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// HeapLeakChecker uses the regions and the associated stack traces 485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// to figure out what part of the memory is the heap: 495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// if MemoryRegionMap were to miss some (early) regions, leak checking would 505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// stop working correctly. 515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// To accomplish the goal of functioning before/during global object 535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// constructor execution MemoryRegionMap is done as a singleton service 545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// that relies on own on-demand initialized static constructor-less data, 555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// and only relies on other low-level modules that can also function properly 565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// even before global object constructors run. 575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Accomplishing the goal of collecting data about all mmap, munmap, mremap, 595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// sbrk occurrences is a more involved: conceptually to do this one needs to 605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// record some bits of data in particular about any mmap or sbrk call, 615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// but to do that one needs to allocate memory for that data at some point, 625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// but all memory allocations in the end themselves come from an mmap 635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// or sbrk call (that's how the address space of the process grows). 645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Also note that we need to do all the above recording from 665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// within an mmap/sbrk hook which is sometimes/frequently is made by a memory 675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// allocator, including the allocator MemoryRegionMap itself must rely on. 685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// In the case of heap-checker usage this includes even the very first 695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// mmap/sbrk call happening in the program: heap-checker gets activated due to 705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// a link-time installed mmap/sbrk hook and it initializes MemoryRegionMap 715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// and asks it to record info about this very first call right from that 725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// very first hook invocation. 735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// MemoryRegionMap is doing its memory allocations via LowLevelAlloc: 755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// unlike more complex standard memory allocator, LowLevelAlloc cooperates with 765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// MemoryRegionMap by not holding any of its own locks while it calls mmap 775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// to get memory, thus we are able to call LowLevelAlloc from 785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// our mmap/sbrk hooks without causing a deadlock in it. 795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// For the same reason of deadlock prevention the locking in MemoryRegionMap 805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// itself is write-recursive which is an exception to Google's mutex usage. 815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// We still need to break the infinite cycle of mmap calling our hook, 835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// which asks LowLevelAlloc for memory to record this mmap, 845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// which (sometimes) causes mmap, which calls our hook, and so on. 855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// We do this as follows: on a recursive call of MemoryRegionMap's 865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// mmap/sbrk/mremap hook we record the data about the allocation in a 872a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)// static fixed-sized stack (saved_regions and saved_buckets), when the 882a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)// recursion unwinds but before returning from the outer hook call we unwind 892a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)// this stack and move the data from saved_regions and saved_buckets to its 902a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)// permanent place in the RegionSet and "bucket_table" respectively, 915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// which can cause more allocations and mmap-s and recursion and unwinding, 925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// but the whole process ends eventually due to the fact that for the small 935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// allocations we are doing LowLevelAlloc reuses one mmap call and parcels out 945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// the memory it created to satisfy several of our allocation requests. 955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// ========================================================================= // 985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <config.h> 1005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#ifdef HAVE_UNISTD_H 1025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <unistd.h> 1035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#endif 1045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#ifdef HAVE_INTTYPES_H 1055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <inttypes.h> 1065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#endif 1075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#ifdef HAVE_MMAP 1085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <sys/mman.h> 1095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#elif !defined(MAP_FAILED) 1105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#define MAP_FAILED -1 // the only thing we need from mman.h 1115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#endif 1125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#ifdef HAVE_PTHREAD 1135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <pthread.h> // for pthread_t, pthread_self() 1145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#endif 1155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <stddef.h> 1165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <algorithm> 1185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <set> 1195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include "memory_region_map.h" 1215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include "base/logging.h" 1235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include "base/low_level_alloc.h" 1245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include "malloc_hook-inl.h" 1255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <gperftools/stacktrace.h> 1275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <gperftools/malloc_hook.h> 1285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// MREMAP_FIXED is a linux extension. How it's used in this file, 1305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// setting it to 0 is equivalent to saying, "This feature isn't 1315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// supported", which is right. 1325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#ifndef MREMAP_FIXED 1335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# define MREMAP_FIXED 0 1345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#endif 1355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)using std::max; 1375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// ========================================================================= // 1395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)int MemoryRegionMap::client_count_ = 0; 1415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)int MemoryRegionMap::max_stack_depth_ = 0; 1425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)MemoryRegionMap::RegionSet* MemoryRegionMap::regions_ = NULL; 1435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)LowLevelAlloc::Arena* MemoryRegionMap::arena_ = NULL; 1445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)SpinLock MemoryRegionMap::lock_(SpinLock::LINKER_INITIALIZED); 1455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)SpinLock MemoryRegionMap::owner_lock_( // ACQUIRED_AFTER(lock_) 1465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) SpinLock::LINKER_INITIALIZED); 1475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)int MemoryRegionMap::recursion_count_ = 0; // GUARDED_BY(owner_lock_) 1485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)pthread_t MemoryRegionMap::lock_owner_tid_; // GUARDED_BY(owner_lock_) 1495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)int64 MemoryRegionMap::map_size_ = 0; 1505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)int64 MemoryRegionMap::unmap_size_ = 0; 1512a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)HeapProfileBucket** MemoryRegionMap::bucket_table_ = NULL; // GUARDED_BY(lock_) 1522a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)int MemoryRegionMap::num_buckets_ = 0; // GUARDED_BY(lock_) 1532a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)int MemoryRegionMap::saved_buckets_count_ = 0; // GUARDED_BY(lock_) 1542a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)HeapProfileBucket MemoryRegionMap::saved_buckets_[20]; // GUARDED_BY(lock_) 1552a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) 1562a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)// GUARDED_BY(lock_) 1572a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)const void* MemoryRegionMap::saved_buckets_keys_[20][kMaxStackDepth]; 1585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// ========================================================================= // 1605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Simple hook into execution of global object constructors, 1625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// so that we do not call pthread_self() when it does not yet work. 1635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static bool libpthread_initialized = false; 1645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static bool initializer = (libpthread_initialized = true, true); 1655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static inline bool current_thread_is(pthread_t should_be) { 1675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Before main() runs, there's only one thread, so we're always that thread 1685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (!libpthread_initialized) return true; 1695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // this starts working only sometime well into global constructor execution: 1705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return pthread_equal(pthread_self(), should_be); 1715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 1725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// ========================================================================= // 1745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Constructor-less place-holder to store a RegionSet in. 1765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)union MemoryRegionMap::RegionSetRep { 1775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) char rep[sizeof(RegionSet)]; 1785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) void* align_it; // do not need a better alignment for 'rep' than this 1795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RegionSet* region_set() { return reinterpret_cast<RegionSet*>(rep); } 1805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)}; 1815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// The bytes where MemoryRegionMap::regions_ will point to. 1835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// We use RegionSetRep with noop c-tor so that global construction 1845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// does not interfere. 1855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static MemoryRegionMap::RegionSetRep regions_rep; 1865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// ========================================================================= // 1885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Has InsertRegionLocked been called recursively 1905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// (or rather should we *not* use regions_ to record a hooked mmap). 1915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static bool recursive_insert = false; 1925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1932a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)void MemoryRegionMap::Init(int max_stack_depth, bool use_buckets) { 1945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MemoryRegionMap Init"); 1955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(max_stack_depth >= 0, ""); 1965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Make sure we don't overflow the memory in region stacks: 1975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(max_stack_depth <= kMaxStackDepth, 1985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "need to increase kMaxStackDepth?"); 1995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Lock(); 2005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) client_count_ += 1; 2015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) max_stack_depth_ = max(max_stack_depth_, max_stack_depth); 2025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (client_count_ > 1) { 2035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // not first client: already did initialization-proper 2045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 2055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MemoryRegionMap Init increment done"); 2065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return; 2075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 2085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Set our hooks and make sure they were installed: 2095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::AddMmapHook(&MmapHook), ""); 2105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::AddMremapHook(&MremapHook), ""); 2115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::AddSbrkHook(&SbrkHook), ""); 2125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::AddMunmapHook(&MunmapHook), ""); 2135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // We need to set recursive_insert since the NewArena call itself 2145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // will already do some allocations with mmap which our hooks will catch 2155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // recursive_insert allows us to buffer info about these mmap calls. 2165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Note that Init() can be (and is) sometimes called 2175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // already from within an mmap/sbrk hook. 2185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursive_insert = true; 2195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) arena_ = LowLevelAlloc::NewArena(0, LowLevelAlloc::DefaultArena()); 2205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursive_insert = false; 2215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) HandleSavedRegionsLocked(&InsertRegionLocked); // flush the buffered ones 2225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Can't instead use HandleSavedRegionsLocked(&DoInsertRegionLocked) before 2235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // recursive_insert = false; as InsertRegionLocked will also construct 2245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // regions_ on demand for us. 2252a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) if (use_buckets) { 2262a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) const int table_bytes = kHashTableSize * sizeof(*bucket_table_); 2272a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) recursive_insert = true; 2282a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket_table_ = static_cast<HeapProfileBucket**>( 2292a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) MyAllocator::Allocate(table_bytes)); 2302a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) recursive_insert = false; 2312a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) memset(bucket_table_, 0, table_bytes); 2322a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) num_buckets_ = 0; 2332a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 234b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles) if (regions_ == NULL) // init regions_ 235b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles) InitRegionSetLocked(); 2365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 2375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MemoryRegionMap Init done"); 2385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 2395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)bool MemoryRegionMap::Shutdown() { 2415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MemoryRegionMap Shutdown"); 2425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Lock(); 2435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(client_count_ > 0, ""); 2445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) client_count_ -= 1; 2455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (client_count_ != 0) { // not last client; need not really shutdown 2465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 2475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MemoryRegionMap Shutdown decrement done"); 2485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return true; 2495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 2502a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) if (bucket_table_ != NULL) { 2512a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) for (int i = 0; i < kHashTableSize; i++) { 2522a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) for (HeapProfileBucket* curr = bucket_table_[i]; curr != 0; /**/) { 2532a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) HeapProfileBucket* bucket = curr; 2542a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) curr = curr->next; 2552a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) MyAllocator::Free(bucket->stack, 0); 2562a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) MyAllocator::Free(bucket, 0); 2572a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 2582a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 2592a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) MyAllocator::Free(bucket_table_, 0); 2602a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) num_buckets_ = 0; 2612a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket_table_ = NULL; 2622a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 2635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::RemoveMmapHook(&MmapHook), ""); 2645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::RemoveMremapHook(&MremapHook), ""); 2655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::RemoveSbrkHook(&SbrkHook), ""); 2665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::RemoveMunmapHook(&MunmapHook), ""); 2675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (regions_) regions_->~RegionSet(); 2685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_ = NULL; 2695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) bool deleted_arena = LowLevelAlloc::DeleteArena(arena_); 2705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (deleted_arena) { 2715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) arena_ = 0; 2725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else { 2735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_LOG(WARNING, "Can't delete LowLevelAlloc arena: it's being used"); 2745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 2755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 2765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MemoryRegionMap Shutdown done"); 2775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return deleted_arena; 2785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 2795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2802a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)bool MemoryRegionMap::IsRecordingLocked() { 2812a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 2822a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) return client_count_ > 0; 2832a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)} 2842a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) 2855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Invariants (once libpthread_initialized is true): 2865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// * While lock_ is not held, recursion_count_ is 0 (and 2875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// lock_owner_tid_ is the previous owner, but we don't rely on 2885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// that). 2895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// * recursion_count_ and lock_owner_tid_ are only written while 2905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// both lock_ and owner_lock_ are held. They may be read under 2915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// just owner_lock_. 2925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// * At entry and exit of Lock() and Unlock(), the current thread 2935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// owns lock_ iff pthread_equal(lock_owner_tid_, pthread_self()) 2945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// && recursion_count_ > 0. 2955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::Lock() { 2965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) { 2975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) SpinLockHolder l(&owner_lock_); 2985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (recursion_count_ > 0 && current_thread_is(lock_owner_tid_)) { 2995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(lock_.IsHeld(), "Invariants violated"); 3005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursion_count_++; 3015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(recursion_count_ <= 5, 3025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "recursive lock nesting unexpectedly deep"); 3035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return; 3045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) lock_.Lock(); 3075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) { 3085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) SpinLockHolder l(&owner_lock_); 3095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(recursion_count_ == 0, 3105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "Last Unlock didn't reset recursion_count_"); 3115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (libpthread_initialized) 3125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) lock_owner_tid_ = pthread_self(); 3135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursion_count_ = 1; 3145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 3165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::Unlock() { 3185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) SpinLockHolder l(&owner_lock_); 3195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(recursion_count_ > 0, "unlock when not held"); 3205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(lock_.IsHeld(), 3215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "unlock when not held, and recursion_count_ is wrong"); 3225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(current_thread_is(lock_owner_tid_), "unlock by non-holder"); 3235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursion_count_--; 3245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (recursion_count_ == 0) { 3255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) lock_.Unlock(); 3265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 3285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)bool MemoryRegionMap::LockIsHeld() { 3305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) SpinLockHolder l(&owner_lock_); 3315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return lock_.IsHeld() && current_thread_is(lock_owner_tid_); 3325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 3335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)const MemoryRegionMap::Region* 3355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)MemoryRegionMap::DoFindRegionLocked(uintptr_t addr) { 3365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 3375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (regions_ != NULL) { 3385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region sample; 3395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) sample.SetRegionSetKey(addr); 3405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RegionSet::iterator region = regions_->lower_bound(sample); 3415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (region != regions_->end()) { 3425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(addr <= region->end_addr, ""); 3435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (region->start_addr <= addr && addr < region->end_addr) { 3445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return &(*region); 3455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return NULL; 3495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 3505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)bool MemoryRegionMap::FindRegion(uintptr_t addr, Region* result) { 3525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Lock(); 3535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const Region* region = DoFindRegionLocked(addr); 3545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (region != NULL) *result = *region; // create it as an independent copy 3555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 3565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return region != NULL; 3575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 3585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)bool MemoryRegionMap::FindAndMarkStackRegion(uintptr_t stack_top, 3605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region* result) { 3615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Lock(); 3625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const Region* region = DoFindRegionLocked(stack_top); 3635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (region != NULL) { 3645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "Stack at %p is inside region %p..%p", 3655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(stack_top), 3665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->start_addr), 3675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->end_addr)); 3685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const_cast<Region*>(region)->set_is_stack(); // now we know 3695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // cast is safe (set_is_stack does not change the set ordering key) 3705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) *result = *region; // create *result as an independent copy 3715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 3735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return region != NULL; 3745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 3755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3762a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)HeapProfileBucket* MemoryRegionMap::GetBucket(int depth, 3772a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) const void* const key[]) { 3782a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 3792a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) // Make hash-value 3802a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) uintptr_t hash = 0; 3812a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) for (int i = 0; i < depth; i++) { 3822a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) hash += reinterpret_cast<uintptr_t>(key[i]); 3832a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) hash += hash << 10; 3842a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) hash ^= hash >> 6; 3852a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 3862a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) hash += hash << 3; 3872a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) hash ^= hash >> 11; 3882a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) 3892a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) // Lookup stack trace in table 3902a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) unsigned int hash_index = (static_cast<unsigned int>(hash)) % kHashTableSize; 3912a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) for (HeapProfileBucket* bucket = bucket_table_[hash_index]; 3922a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket != 0; 3932a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket = bucket->next) { 3942a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) if ((bucket->hash == hash) && (bucket->depth == depth) && 3952a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) std::equal(key, key + depth, bucket->stack)) { 3962a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) return bucket; 3972a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 3982a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 3992a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) 4002a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) // Create new bucket 4012a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) const size_t key_size = sizeof(key[0]) * depth; 4022a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) HeapProfileBucket* bucket; 4032a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) if (recursive_insert) { // recursion: save in saved_buckets_ 4042a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) const void** key_copy = saved_buckets_keys_[saved_buckets_count_]; 4052a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) std::copy(key, key + depth, key_copy); 4062a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket = &saved_buckets_[saved_buckets_count_]; 4072a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) memset(bucket, 0, sizeof(*bucket)); 4082a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) ++saved_buckets_count_; 4092a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket->stack = key_copy; 4102a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket->next = NULL; 4112a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } else { 4122a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) recursive_insert = true; 4132a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) const void** key_copy = static_cast<const void**>( 4142a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) MyAllocator::Allocate(key_size)); 4152a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) recursive_insert = false; 4162a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) std::copy(key, key + depth, key_copy); 4172a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) recursive_insert = true; 4182a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket = static_cast<HeapProfileBucket*>( 4192a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) MyAllocator::Allocate(sizeof(HeapProfileBucket))); 4202a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) recursive_insert = false; 4212a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) memset(bucket, 0, sizeof(*bucket)); 4222a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket->stack = key_copy; 4232a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket->next = bucket_table_[hash_index]; 4242a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 4252a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket->hash = hash; 4262a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket->depth = depth; 4272a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket_table_[hash_index] = bucket; 4282a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) ++num_buckets_; 4292a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) return bucket; 4302a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)} 4312a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) 4325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)MemoryRegionMap::RegionIterator MemoryRegionMap::BeginRegionLocked() { 4335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 4345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(regions_ != NULL, ""); 4355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return regions_->begin(); 4365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 4375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)MemoryRegionMap::RegionIterator MemoryRegionMap::EndRegionLocked() { 4395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 4405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(regions_ != NULL, ""); 4415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return regions_->end(); 4425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 4435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)inline void MemoryRegionMap::DoInsertRegionLocked(const Region& region) { 4455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Inserting region %p..%p from %p", 4465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.start_addr), 4475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.end_addr), 4485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.caller())); 4495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RegionSet::const_iterator i = regions_->lower_bound(region); 4505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (i != regions_->end() && i->start_addr <= region.start_addr) { 4515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_DCHECK(region.end_addr <= i->end_addr, ""); // lower_bound ensures this 4525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return; // 'region' is a subset of an already recorded region; do nothing 4535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // We can be stricter and allow this only when *i has been created via 4545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // an mmap with MAP_NORESERVE flag set. 4555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 4565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (DEBUG_MODE) { 4575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(i == regions_->end() || !region.Overlaps(*i), 4585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "Wow, overlapping memory regions"); 4595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region sample; 4605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) sample.SetRegionSetKey(region.start_addr); 4615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) i = regions_->lower_bound(sample); 4625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(i == regions_->end() || !region.Overlaps(*i), 4635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "Wow, overlapping memory regions"); 4645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 4655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) region.AssertIsConsistent(); // just making sure 4665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // This inserts and allocates permanent storage for region 4675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // and its call stack data: it's safe to do it now: 4685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_->insert(region); 4695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Inserted region %p..%p :", 4705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.start_addr), 4715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.end_addr)); 4725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (VLOG_IS_ON(12)) LogAllLocked(); 4735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 4745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// These variables are local to MemoryRegionMap::InsertRegionLocked() 4765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// and MemoryRegionMap::HandleSavedRegionsLocked() 4775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// and are file-level to ensure that they are initialized at load time. 4785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Number of unprocessed region inserts. 4805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static int saved_regions_count = 0; 4815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Unprocessed inserts (must be big enough to hold all allocations that can 4835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// be caused by a InsertRegionLocked call). 4845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Region has no constructor, so that c-tor execution does not interfere 4855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// with the any-time use of the static memory behind saved_regions. 4865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static MemoryRegionMap::Region saved_regions[20]; 4875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)inline void MemoryRegionMap::HandleSavedRegionsLocked( 4895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) void (*insert_func)(const Region& region)) { 4905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while (saved_regions_count > 0) { 4915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Making a local-var copy of the region argument to insert_func 4925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // including its stack (w/o doing any memory allocations) is important: 4935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // in many cases the memory in saved_regions 4945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // will get written-to during the (*insert_func)(r) call below. 4955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region r = saved_regions[--saved_regions_count]; 4965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (*insert_func)(r); 4975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 4985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 4995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5002a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)void MemoryRegionMap::RestoreSavedBucketsLocked() { 5012a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 5022a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) while (saved_buckets_count_ > 0) { 5032a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) HeapProfileBucket bucket = saved_buckets_[--saved_buckets_count_]; 5042a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) unsigned int hash_index = 5052a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) static_cast<unsigned int>(bucket.hash) % kHashTableSize; 5062a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bool is_found = false; 5072a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) for (HeapProfileBucket* curr = bucket_table_[hash_index]; 5082a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) curr != 0; 5092a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) curr = curr->next) { 5102a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) if ((curr->hash == bucket.hash) && (curr->depth == bucket.depth) && 5112a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) std::equal(bucket.stack, bucket.stack + bucket.depth, curr->stack)) { 5122a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) curr->allocs += bucket.allocs; 5132a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) curr->alloc_size += bucket.alloc_size; 5142a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) curr->frees += bucket.frees; 5152a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) curr->free_size += bucket.free_size; 5162a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) is_found = true; 5172a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) break; 5182a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 5192a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 5202a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) if (is_found) continue; 5212a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) 5222a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) const size_t key_size = sizeof(bucket.stack[0]) * bucket.depth; 5232a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) const void** key_copy = static_cast<const void**>( 5242a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) MyAllocator::Allocate(key_size)); 5252a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) std::copy(bucket.stack, bucket.stack + bucket.depth, key_copy); 5262a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) HeapProfileBucket* new_bucket = static_cast<HeapProfileBucket*>( 5272a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) MyAllocator::Allocate(sizeof(HeapProfileBucket))); 5282a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) memset(new_bucket, 0, sizeof(*new_bucket)); 5292a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) new_bucket->hash = bucket.hash; 5302a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) new_bucket->depth = bucket.depth; 5312a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) new_bucket->stack = key_copy; 5322a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) new_bucket->next = bucket_table_[hash_index]; 5332a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) bucket_table_[hash_index] = new_bucket; 5342a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) ++num_buckets_; 5352a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 5362a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)} 5372a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) 538b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles)inline void MemoryRegionMap::InitRegionSetLocked() { 539b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles) RAW_VLOG(12, "Initializing region set"); 540b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles) regions_ = regions_rep.region_set(); 541b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles) recursive_insert = true; 542b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles) new(regions_) RegionSet(); 543b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles) HandleSavedRegionsLocked(&DoInsertRegionLocked); 544b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles) recursive_insert = false; 545b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles)} 546b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles) 5475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)inline void MemoryRegionMap::InsertRegionLocked(const Region& region) { 5485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 5495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // We can be called recursively, because RegionSet constructor 5505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // and DoInsertRegionLocked() (called below) can call the allocator. 5515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // recursive_insert tells us if that's the case. When this happens, 5525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // region insertion information is recorded in saved_regions[], 5535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // and taken into account when the recursion unwinds. 5545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Do the insert: 5555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (recursive_insert) { // recursion: save in saved_regions 5565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Saving recursive insert of region %p..%p from %p", 5575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.start_addr), 5585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.end_addr), 5595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.caller())); 5605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(saved_regions_count < arraysize(saved_regions), ""); 5615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Copy 'region' to saved_regions[saved_regions_count] 5625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // together with the contents of its call_stack, 5635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // then increment saved_regions_count. 5645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) saved_regions[saved_regions_count++] = region; 5655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else { // not a recusrive call 566b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles) if (regions_ == NULL) // init regions_ 567b2df76ea8fec9e32f6f3718986dba0d95315b29cTorne (Richard Coles) InitRegionSetLocked(); 5685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursive_insert = true; 5695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Do the actual insertion work to put new regions into regions_: 5705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) DoInsertRegionLocked(region); 5715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) HandleSavedRegionsLocked(&DoInsertRegionLocked); 5725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursive_insert = false; 5735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 5745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 5755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// We strip out different number of stack frames in debug mode 5775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// because less inlining happens in that case 5785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#ifdef NDEBUG 5795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static const int kStripFrames = 1; 5805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#else 5815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static const int kStripFrames = 3; 5825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#endif 5835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::RecordRegionAddition(const void* start, size_t size) { 5855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Record start/end info about this memory acquisition call in a new region: 5865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region region; 5875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) region.Create(start, size); 5885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // First get the call stack info into the local varible 'region': 5895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const int depth = 5905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) max_stack_depth_ > 0 5915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) ? MallocHook::GetCallerStackTrace(const_cast<void**>(region.call_stack), 5925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) max_stack_depth_, kStripFrames + 1) 5935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) : 0; 5945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) region.set_call_stack_depth(depth); // record stack info fully 5955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "New global region %p..%p from %p", 5965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.start_addr), 5975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.end_addr), 5985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.caller())); 5995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Note: none of the above allocates memory. 6005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Lock(); // recursively lock 6015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) map_size_ += size; 6025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) InsertRegionLocked(region); 6035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // This will (eventually) allocate storage for and copy over the stack data 6045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // from region.call_stack_data_ that is pointed by region.call_stack(). 6052a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) if (bucket_table_ != NULL) { 6062a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) HeapProfileBucket* b = GetBucket(depth, region.call_stack); 6072a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) ++b->allocs; 6082a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) b->alloc_size += size; 6092a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) if (!recursive_insert) { 6102a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) recursive_insert = true; 6112a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) RestoreSavedBucketsLocked(); 6122a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) recursive_insert = false; 6132a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 6142a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) } 6155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 6165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 6175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::RecordRegionRemoval(const void* start, size_t size) { 6195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Lock(); 6205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (recursive_insert) { 6215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // First remove the removed region from saved_regions, if it's 6225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // there, to prevent overrunning saved_regions in recursive 6235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // map/unmap call sequences, and also from later inserting regions 6245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // which have already been unmapped. 6255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) uintptr_t start_addr = reinterpret_cast<uintptr_t>(start); 6265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) uintptr_t end_addr = start_addr + size; 6275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) int put_pos = 0; 6285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) int old_count = saved_regions_count; 6295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) for (int i = 0; i < old_count; ++i, ++put_pos) { 6305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region& r = saved_regions[i]; 6315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (r.start_addr == start_addr && r.end_addr == end_addr) { 6325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // An exact match, so it's safe to remove. 6332a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) RecordRegionRemovalInBucket(r.call_stack_depth, r.call_stack, size); 6345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) --saved_regions_count; 6355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) --put_pos; 6365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, ("Insta-Removing saved region %p..%p; " 6375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "now have %d saved regions"), 6385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(start_addr), 6395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(end_addr), 6405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) saved_regions_count); 6415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else { 6425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (put_pos < i) { 6435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) saved_regions[put_pos] = saved_regions[i]; 6445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 6455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 6465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 6475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 6485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (regions_ == NULL) { // We must have just unset the hooks, 6495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // but this thread was already inside the hook. 6505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 6515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return; 6525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 6535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (!recursive_insert) { 6545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) HandleSavedRegionsLocked(&InsertRegionLocked); 6555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 6565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // first handle adding saved regions if any 6575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) uintptr_t start_addr = reinterpret_cast<uintptr_t>(start); 6585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) uintptr_t end_addr = start_addr + size; 6595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // subtract start_addr, end_addr from all the regions 660558790d6acca3451cf3a6b497803a5f07d0bec58Ben Murdoch RAW_VLOG(10, "Removing global region %p..%p; have %" PRIuS " regions", 6615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(start_addr), 6625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(end_addr), 6635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_->size()); 6645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region sample; 6655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) sample.SetRegionSetKey(start_addr); 6665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Only iterate over the regions that might overlap start_addr..end_addr: 6675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) for (RegionSet::iterator region = regions_->lower_bound(sample); 6685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) region != regions_->end() && region->start_addr < end_addr; 6695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) /*noop*/) { 6705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(13, "Looking at region %p..%p", 6715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->start_addr), 6725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->end_addr)); 6735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (start_addr <= region->start_addr && 6745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) region->end_addr <= end_addr) { // full deletion 6755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Deleting region %p..%p", 6765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->start_addr), 6775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->end_addr)); 6782a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) RecordRegionRemovalInBucket(region->call_stack_depth, region->call_stack, 6792a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) region->end_addr - region->start_addr); 6805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RegionSet::iterator d = region; 6815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) ++region; 6825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_->erase(d); 6835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) continue; 6845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else if (region->start_addr < start_addr && 6855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) end_addr < region->end_addr) { // cutting-out split 6865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Splitting region %p..%p in two", 6875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->start_addr), 6885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->end_addr)); 6892a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) RecordRegionRemovalInBucket(region->call_stack_depth, region->call_stack, 6902a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) end_addr - start_addr); 6915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Make another region for the start portion: 6925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // The new region has to be the start portion because we can't 6935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // just modify region->end_addr as it's the sorting key. 6945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region r = *region; 6955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) r.set_end_addr(start_addr); 6965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) InsertRegionLocked(r); 6975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // cut *region from start: 6985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const_cast<Region&>(*region).set_start_addr(end_addr); 6995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else if (end_addr > region->start_addr && 7005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) start_addr <= region->start_addr) { // cut from start 7015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Start-chopping region %p..%p", 7025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->start_addr), 7035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->end_addr)); 7042a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) RecordRegionRemovalInBucket(region->call_stack_depth, region->call_stack, 7052a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) end_addr - region->start_addr); 7065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const_cast<Region&>(*region).set_start_addr(end_addr); 7075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else if (start_addr > region->start_addr && 7085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) start_addr < region->end_addr) { // cut from end 7095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "End-chopping region %p..%p", 7105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->start_addr), 7115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->end_addr)); 7122a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) RecordRegionRemovalInBucket(region->call_stack_depth, region->call_stack, 7132a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) region->end_addr - start_addr); 7145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Can't just modify region->end_addr (it's the sorting key): 7155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region r = *region; 7165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) r.set_end_addr(start_addr); 7175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RegionSet::iterator d = region; 7185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) ++region; 7195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // It's safe to erase before inserting since r is independent of *d: 7205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // r contains an own copy of the call stack: 7215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_->erase(d); 7225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) InsertRegionLocked(r); 7235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) continue; 7245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 7255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) ++region; 7265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 727558790d6acca3451cf3a6b497803a5f07d0bec58Ben Murdoch RAW_VLOG(12, "Removed region %p..%p; have %" PRIuS " regions", 7285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(start_addr), 7295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(end_addr), 7305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_->size()); 7315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (VLOG_IS_ON(12)) LogAllLocked(); 7325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) unmap_size_ += size; 7335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 7345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 7355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 7362a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)void MemoryRegionMap::RecordRegionRemovalInBucket(int depth, 7372a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) const void* const stack[], 7382a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) size_t size) { 7392a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 7402a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) if (bucket_table_ == NULL) return; 7412a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) HeapProfileBucket* b = GetBucket(depth, stack); 7422a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) ++b->frees; 7432a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) b->free_size += size; 7442a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles)} 7452a99a7e74a7f215066514fe81d2bfa6639d9edddTorne (Richard Coles) 7465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::MmapHook(const void* result, 7475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const void* start, size_t size, 7485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) int prot, int flags, 7495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) int fd, off_t offset) { 7505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // TODO(maxim): replace all 0x%"PRIxS" by %p when RAW_VLOG uses a safe 7515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // snprintf reimplementation that does not malloc to pretty-print NULL 752558790d6acca3451cf3a6b497803a5f07d0bec58Ben Murdoch RAW_VLOG(10, "MMap = 0x%" PRIxPTR " of %" PRIuS " at %" PRIu64 " " 753558790d6acca3451cf3a6b497803a5f07d0bec58Ben Murdoch "prot %d flags %d fd %d offs %" PRId64, 7545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<uintptr_t>(result), size, 7555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<uint64>(start), prot, flags, fd, 7565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) static_cast<int64>(offset)); 7575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (result != reinterpret_cast<void*>(MAP_FAILED) && size != 0) { 7585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RecordRegionAddition(result, size); 7595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 7605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 7615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 7625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::MunmapHook(const void* ptr, size_t size) { 763558790d6acca3451cf3a6b497803a5f07d0bec58Ben Murdoch RAW_VLOG(10, "MUnmap of %p %" PRIuS, ptr, size); 7645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (size != 0) { 7655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RecordRegionRemoval(ptr, size); 7665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 7675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 7685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 7695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::MremapHook(const void* result, 7705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const void* old_addr, size_t old_size, 7715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) size_t new_size, int flags, 7725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const void* new_addr) { 773558790d6acca3451cf3a6b497803a5f07d0bec58Ben Murdoch RAW_VLOG(10, "MRemap = 0x%" PRIxPTR " of 0x%" PRIxPTR " %" PRIuS " " 774558790d6acca3451cf3a6b497803a5f07d0bec58Ben Murdoch "to %" PRIuS " flags %d new_addr=0x%" PRIxPTR, 7755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (uintptr_t)result, (uintptr_t)old_addr, 7765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) old_size, new_size, flags, 7775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) flags & MREMAP_FIXED ? (uintptr_t)new_addr : 0); 7785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (result != reinterpret_cast<void*>(-1)) { 7795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RecordRegionRemoval(old_addr, old_size); 7805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RecordRegionAddition(result, new_size); 7815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 7825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 7835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 7845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)extern "C" void* __sbrk(ptrdiff_t increment); // defined in libc 7855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 7865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::SbrkHook(const void* result, ptrdiff_t increment) { 787558790d6acca3451cf3a6b497803a5f07d0bec58Ben Murdoch RAW_VLOG(10, "Sbrk = 0x%" PRIxPTR " of %" PRIdS, 788558790d6acca3451cf3a6b497803a5f07d0bec58Ben Murdoch (uintptr_t)result, increment); 7895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (result != reinterpret_cast<void*>(-1)) { 7905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (increment > 0) { 7915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) void* new_end = sbrk(0); 7925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RecordRegionAddition(result, reinterpret_cast<uintptr_t>(new_end) - 7935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<uintptr_t>(result)); 7945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else if (increment < 0) { 7955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) void* new_end = sbrk(0); 7965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RecordRegionRemoval(new_end, reinterpret_cast<uintptr_t>(result) - 7975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<uintptr_t>(new_end)); 7985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 7995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 8005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 8015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 8025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::LogAllLocked() { 8035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 8045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_LOG(INFO, "List of regions:"); 8055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) uintptr_t previous = 0; 8065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) for (RegionSet::const_iterator r = regions_->begin(); 8075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) r != regions_->end(); ++r) { 808558790d6acca3451cf3a6b497803a5f07d0bec58Ben Murdoch RAW_LOG(INFO, "Memory region 0x%" PRIxPTR "..0x%" PRIxPTR " " 809558790d6acca3451cf3a6b497803a5f07d0bec58Ben Murdoch "from 0x%" PRIxPTR " stack=%d", 8105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) r->start_addr, r->end_addr, r->caller(), r->is_stack); 8115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(previous < r->end_addr, "wow, we messed up the set order"); 8125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // this must be caused by uncontrolled recursive operations on regions_ 8135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) previous = r->end_addr; 8145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 8155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_LOG(INFO, "End of regions list"); 8165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 817