15821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)/* Copyright (c) 2006, Google Inc. 25821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * All rights reserved. 35821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * 45821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * Redistribution and use in source and binary forms, with or without 55821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * modification, are permitted provided that the following conditions are 65821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * met: 75821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * 85821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * * Redistributions of source code must retain the above copyright 95821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * notice, this list of conditions and the following disclaimer. 105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * * Redistributions in binary form must reproduce the above 115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * copyright notice, this list of conditions and the following disclaimer 125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * in the documentation and/or other materials provided with the 135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * distribution. 145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * * Neither the name of Google Inc. nor the names of its 155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * contributors may be used to endorse or promote products derived from 165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * this software without specific prior written permission. 175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * 185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * 305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * --- 315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) * Author: Maxim Lifantsev 325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) */ 335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Background and key design points of MemoryRegionMap. 365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// MemoryRegionMap is a low-level module with quite atypical requirements that 385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// result in some degree of non-triviality of the implementation and design. 395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// MemoryRegionMap collects info about *all* memory regions created with 415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// mmap, munmap, mremap, sbrk. 425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// They key word above is 'all': all that are happening in a process 435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// during its lifetime frequently starting even before global object 445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// constructor execution. 455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// This is needed by the primary client of MemoryRegionMap: 475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// HeapLeakChecker uses the regions and the associated stack traces 485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// to figure out what part of the memory is the heap: 495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// if MemoryRegionMap were to miss some (early) regions, leak checking would 505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// stop working correctly. 515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// To accomplish the goal of functioning before/during global object 535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// constructor execution MemoryRegionMap is done as a singleton service 545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// that relies on own on-demand initialized static constructor-less data, 555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// and only relies on other low-level modules that can also function properly 565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// even before global object constructors run. 575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Accomplishing the goal of collecting data about all mmap, munmap, mremap, 595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// sbrk occurrences is a more involved: conceptually to do this one needs to 605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// record some bits of data in particular about any mmap or sbrk call, 615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// but to do that one needs to allocate memory for that data at some point, 625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// but all memory allocations in the end themselves come from an mmap 635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// or sbrk call (that's how the address space of the process grows). 645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Also note that we need to do all the above recording from 665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// within an mmap/sbrk hook which is sometimes/frequently is made by a memory 675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// allocator, including the allocator MemoryRegionMap itself must rely on. 685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// In the case of heap-checker usage this includes even the very first 695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// mmap/sbrk call happening in the program: heap-checker gets activated due to 705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// a link-time installed mmap/sbrk hook and it initializes MemoryRegionMap 715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// and asks it to record info about this very first call right from that 725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// very first hook invocation. 735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// MemoryRegionMap is doing its memory allocations via LowLevelAlloc: 755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// unlike more complex standard memory allocator, LowLevelAlloc cooperates with 765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// MemoryRegionMap by not holding any of its own locks while it calls mmap 775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// to get memory, thus we are able to call LowLevelAlloc from 785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// our mmap/sbrk hooks without causing a deadlock in it. 795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// For the same reason of deadlock prevention the locking in MemoryRegionMap 805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// itself is write-recursive which is an exception to Google's mutex usage. 815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// We still need to break the infinite cycle of mmap calling our hook, 835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// which asks LowLevelAlloc for memory to record this mmap, 845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// which (sometimes) causes mmap, which calls our hook, and so on. 855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// We do this as follows: on a recursive call of MemoryRegionMap's 865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// mmap/sbrk/mremap hook we record the data about the allocation in a 875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// static fixed-sized stack (saved_regions), when the recursion unwinds 885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// but before returning from the outer hook call we unwind this stack and 895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// move the data from saved_regions to its permanent place in the RegionSet, 905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// which can cause more allocations and mmap-s and recursion and unwinding, 915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// but the whole process ends eventually due to the fact that for the small 925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// allocations we are doing LowLevelAlloc reuses one mmap call and parcels out 935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// the memory it created to satisfy several of our allocation requests. 945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// 955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// ========================================================================= // 975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <config.h> 995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#ifdef HAVE_UNISTD_H 1015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <unistd.h> 1025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#endif 1035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#ifdef HAVE_INTTYPES_H 1045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <inttypes.h> 1055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#endif 1065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#ifdef HAVE_MMAP 1075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <sys/mman.h> 1085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#elif !defined(MAP_FAILED) 1095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#define MAP_FAILED -1 // the only thing we need from mman.h 1105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#endif 1115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#ifdef HAVE_PTHREAD 1125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <pthread.h> // for pthread_t, pthread_self() 1135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#endif 1145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <stddef.h> 1155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <algorithm> 1175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <set> 1185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include "memory_region_map.h" 1205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include "base/logging.h" 1225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include "base/low_level_alloc.h" 1235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include "malloc_hook-inl.h" 1245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <gperftools/stacktrace.h> 1265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#include <gperftools/malloc_hook.h> 1275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// MREMAP_FIXED is a linux extension. How it's used in this file, 1295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// setting it to 0 is equivalent to saying, "This feature isn't 1305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// supported", which is right. 1315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#ifndef MREMAP_FIXED 1325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)# define MREMAP_FIXED 0 1335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#endif 1345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)using std::max; 1365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// ========================================================================= // 1385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)int MemoryRegionMap::client_count_ = 0; 1405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)int MemoryRegionMap::max_stack_depth_ = 0; 1415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)MemoryRegionMap::RegionSet* MemoryRegionMap::regions_ = NULL; 1425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)LowLevelAlloc::Arena* MemoryRegionMap::arena_ = NULL; 1435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)SpinLock MemoryRegionMap::lock_(SpinLock::LINKER_INITIALIZED); 1445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)SpinLock MemoryRegionMap::owner_lock_( // ACQUIRED_AFTER(lock_) 1455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) SpinLock::LINKER_INITIALIZED); 1465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)int MemoryRegionMap::recursion_count_ = 0; // GUARDED_BY(owner_lock_) 1475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)pthread_t MemoryRegionMap::lock_owner_tid_; // GUARDED_BY(owner_lock_) 1485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)int64 MemoryRegionMap::map_size_ = 0; 1495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)int64 MemoryRegionMap::unmap_size_ = 0; 1505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// ========================================================================= // 1525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Simple hook into execution of global object constructors, 1545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// so that we do not call pthread_self() when it does not yet work. 1555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static bool libpthread_initialized = false; 1565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static bool initializer = (libpthread_initialized = true, true); 1575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static inline bool current_thread_is(pthread_t should_be) { 1595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Before main() runs, there's only one thread, so we're always that thread 1605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (!libpthread_initialized) return true; 1615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // this starts working only sometime well into global constructor execution: 1625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return pthread_equal(pthread_self(), should_be); 1635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 1645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// ========================================================================= // 1665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Constructor-less place-holder to store a RegionSet in. 1685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)union MemoryRegionMap::RegionSetRep { 1695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) char rep[sizeof(RegionSet)]; 1705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) void* align_it; // do not need a better alignment for 'rep' than this 1715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RegionSet* region_set() { return reinterpret_cast<RegionSet*>(rep); } 1725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)}; 1735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// The bytes where MemoryRegionMap::regions_ will point to. 1755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// We use RegionSetRep with noop c-tor so that global construction 1765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// does not interfere. 1775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static MemoryRegionMap::RegionSetRep regions_rep; 1785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// ========================================================================= // 1805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Has InsertRegionLocked been called recursively 1825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// (or rather should we *not* use regions_ to record a hooked mmap). 1835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static bool recursive_insert = false; 1845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 1855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::Init(int max_stack_depth) { 1865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MemoryRegionMap Init"); 1875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(max_stack_depth >= 0, ""); 1885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Make sure we don't overflow the memory in region stacks: 1895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(max_stack_depth <= kMaxStackDepth, 1905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "need to increase kMaxStackDepth?"); 1915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Lock(); 1925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) client_count_ += 1; 1935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) max_stack_depth_ = max(max_stack_depth_, max_stack_depth); 1945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (client_count_ > 1) { 1955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // not first client: already did initialization-proper 1965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 1975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MemoryRegionMap Init increment done"); 1985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return; 1995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 2005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Set our hooks and make sure they were installed: 2015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::AddMmapHook(&MmapHook), ""); 2025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::AddMremapHook(&MremapHook), ""); 2035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::AddSbrkHook(&SbrkHook), ""); 2045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::AddMunmapHook(&MunmapHook), ""); 2055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // We need to set recursive_insert since the NewArena call itself 2065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // will already do some allocations with mmap which our hooks will catch 2075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // recursive_insert allows us to buffer info about these mmap calls. 2085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Note that Init() can be (and is) sometimes called 2095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // already from within an mmap/sbrk hook. 2105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursive_insert = true; 2115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) arena_ = LowLevelAlloc::NewArena(0, LowLevelAlloc::DefaultArena()); 2125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursive_insert = false; 2135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) HandleSavedRegionsLocked(&InsertRegionLocked); // flush the buffered ones 2145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Can't instead use HandleSavedRegionsLocked(&DoInsertRegionLocked) before 2155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // recursive_insert = false; as InsertRegionLocked will also construct 2165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // regions_ on demand for us. 2175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 2185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MemoryRegionMap Init done"); 2195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 2205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)bool MemoryRegionMap::Shutdown() { 2225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MemoryRegionMap Shutdown"); 2235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Lock(); 2245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(client_count_ > 0, ""); 2255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) client_count_ -= 1; 2265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (client_count_ != 0) { // not last client; need not really shutdown 2275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 2285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MemoryRegionMap Shutdown decrement done"); 2295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return true; 2305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 2315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::RemoveMmapHook(&MmapHook), ""); 2325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::RemoveMremapHook(&MremapHook), ""); 2335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::RemoveSbrkHook(&SbrkHook), ""); 2345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(MallocHook::RemoveMunmapHook(&MunmapHook), ""); 2355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (regions_) regions_->~RegionSet(); 2365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_ = NULL; 2375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) bool deleted_arena = LowLevelAlloc::DeleteArena(arena_); 2385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (deleted_arena) { 2395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) arena_ = 0; 2405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else { 2415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_LOG(WARNING, "Can't delete LowLevelAlloc arena: it's being used"); 2425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 2435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 2445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MemoryRegionMap Shutdown done"); 2455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return deleted_arena; 2465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 2475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Invariants (once libpthread_initialized is true): 2495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// * While lock_ is not held, recursion_count_ is 0 (and 2505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// lock_owner_tid_ is the previous owner, but we don't rely on 2515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// that). 2525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// * recursion_count_ and lock_owner_tid_ are only written while 2535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// both lock_ and owner_lock_ are held. They may be read under 2545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// just owner_lock_. 2555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// * At entry and exit of Lock() and Unlock(), the current thread 2565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// owns lock_ iff pthread_equal(lock_owner_tid_, pthread_self()) 2575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// && recursion_count_ > 0. 2585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::Lock() { 2595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) { 2605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) SpinLockHolder l(&owner_lock_); 2615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (recursion_count_ > 0 && current_thread_is(lock_owner_tid_)) { 2625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(lock_.IsHeld(), "Invariants violated"); 2635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursion_count_++; 2645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(recursion_count_ <= 5, 2655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "recursive lock nesting unexpectedly deep"); 2665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return; 2675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 2685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 2695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) lock_.Lock(); 2705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) { 2715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) SpinLockHolder l(&owner_lock_); 2725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(recursion_count_ == 0, 2735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "Last Unlock didn't reset recursion_count_"); 2745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (libpthread_initialized) 2755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) lock_owner_tid_ = pthread_self(); 2765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursion_count_ = 1; 2775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 2785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 2795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::Unlock() { 2815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) SpinLockHolder l(&owner_lock_); 2825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(recursion_count_ > 0, "unlock when not held"); 2835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(lock_.IsHeld(), 2845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "unlock when not held, and recursion_count_ is wrong"); 2855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(current_thread_is(lock_owner_tid_), "unlock by non-holder"); 2865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursion_count_--; 2875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (recursion_count_ == 0) { 2885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) lock_.Unlock(); 2895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 2905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 2915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)bool MemoryRegionMap::LockIsHeld() { 2935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) SpinLockHolder l(&owner_lock_); 2945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return lock_.IsHeld() && current_thread_is(lock_owner_tid_); 2955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 2965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 2975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)const MemoryRegionMap::Region* 2985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)MemoryRegionMap::DoFindRegionLocked(uintptr_t addr) { 2995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 3005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (regions_ != NULL) { 3015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region sample; 3025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) sample.SetRegionSetKey(addr); 3035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RegionSet::iterator region = regions_->lower_bound(sample); 3045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (region != regions_->end()) { 3055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(addr <= region->end_addr, ""); 3065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (region->start_addr <= addr && addr < region->end_addr) { 3075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return &(*region); 3085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return NULL; 3125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 3135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)bool MemoryRegionMap::FindRegion(uintptr_t addr, Region* result) { 3155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Lock(); 3165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const Region* region = DoFindRegionLocked(addr); 3175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (region != NULL) *result = *region; // create it as an independent copy 3185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 3195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return region != NULL; 3205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 3215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)bool MemoryRegionMap::FindAndMarkStackRegion(uintptr_t stack_top, 3235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region* result) { 3245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Lock(); 3255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const Region* region = DoFindRegionLocked(stack_top); 3265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (region != NULL) { 3275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "Stack at %p is inside region %p..%p", 3285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(stack_top), 3295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->start_addr), 3305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->end_addr)); 3315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const_cast<Region*>(region)->set_is_stack(); // now we know 3325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // cast is safe (set_is_stack does not change the set ordering key) 3335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) *result = *region; // create *result as an independent copy 3345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 3365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return region != NULL; 3375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 3385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)MemoryRegionMap::RegionIterator MemoryRegionMap::BeginRegionLocked() { 3405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 3415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(regions_ != NULL, ""); 3425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return regions_->begin(); 3435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 3445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)MemoryRegionMap::RegionIterator MemoryRegionMap::EndRegionLocked() { 3465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 3475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(regions_ != NULL, ""); 3485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return regions_->end(); 3495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 3505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)inline void MemoryRegionMap::DoInsertRegionLocked(const Region& region) { 3525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Inserting region %p..%p from %p", 3535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.start_addr), 3545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.end_addr), 3555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.caller())); 3565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RegionSet::const_iterator i = regions_->lower_bound(region); 3575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (i != regions_->end() && i->start_addr <= region.start_addr) { 3585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_DCHECK(region.end_addr <= i->end_addr, ""); // lower_bound ensures this 3595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return; // 'region' is a subset of an already recorded region; do nothing 3605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // We can be stricter and allow this only when *i has been created via 3615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // an mmap with MAP_NORESERVE flag set. 3625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (DEBUG_MODE) { 3645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(i == regions_->end() || !region.Overlaps(*i), 3655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "Wow, overlapping memory regions"); 3665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region sample; 3675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) sample.SetRegionSetKey(region.start_addr); 3685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) i = regions_->lower_bound(sample); 3695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(i == regions_->end() || !region.Overlaps(*i), 3705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "Wow, overlapping memory regions"); 3715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 3725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) region.AssertIsConsistent(); // just making sure 3735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // This inserts and allocates permanent storage for region 3745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // and its call stack data: it's safe to do it now: 3755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_->insert(region); 3765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Inserted region %p..%p :", 3775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.start_addr), 3785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.end_addr)); 3795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (VLOG_IS_ON(12)) LogAllLocked(); 3805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 3815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// These variables are local to MemoryRegionMap::InsertRegionLocked() 3835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// and MemoryRegionMap::HandleSavedRegionsLocked() 3845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// and are file-level to ensure that they are initialized at load time. 3855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Number of unprocessed region inserts. 3875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static int saved_regions_count = 0; 3885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Unprocessed inserts (must be big enough to hold all allocations that can 3905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// be caused by a InsertRegionLocked call). 3915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// Region has no constructor, so that c-tor execution does not interfere 3925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// with the any-time use of the static memory behind saved_regions. 3935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static MemoryRegionMap::Region saved_regions[20]; 3945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 3955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)inline void MemoryRegionMap::HandleSavedRegionsLocked( 3965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) void (*insert_func)(const Region& region)) { 3975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) while (saved_regions_count > 0) { 3985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Making a local-var copy of the region argument to insert_func 3995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // including its stack (w/o doing any memory allocations) is important: 4005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // in many cases the memory in saved_regions 4015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // will get written-to during the (*insert_func)(r) call below. 4025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region r = saved_regions[--saved_regions_count]; 4035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (*insert_func)(r); 4045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 4055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 4065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)inline void MemoryRegionMap::InsertRegionLocked(const Region& region) { 4085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 4095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // We can be called recursively, because RegionSet constructor 4105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // and DoInsertRegionLocked() (called below) can call the allocator. 4115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // recursive_insert tells us if that's the case. When this happens, 4125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // region insertion information is recorded in saved_regions[], 4135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // and taken into account when the recursion unwinds. 4145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Do the insert: 4155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (recursive_insert) { // recursion: save in saved_regions 4165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Saving recursive insert of region %p..%p from %p", 4175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.start_addr), 4185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.end_addr), 4195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.caller())); 4205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(saved_regions_count < arraysize(saved_regions), ""); 4215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Copy 'region' to saved_regions[saved_regions_count] 4225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // together with the contents of its call_stack, 4235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // then increment saved_regions_count. 4245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) saved_regions[saved_regions_count++] = region; 4255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else { // not a recusrive call 4265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (regions_ == NULL) { // init regions_ 4275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Initializing region set"); 4285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_ = regions_rep.region_set(); 4295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursive_insert = true; 4305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) new(regions_) RegionSet(); 4315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) HandleSavedRegionsLocked(&DoInsertRegionLocked); 4325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursive_insert = false; 4335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 4345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursive_insert = true; 4355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Do the actual insertion work to put new regions into regions_: 4365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) DoInsertRegionLocked(region); 4375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) HandleSavedRegionsLocked(&DoInsertRegionLocked); 4385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) recursive_insert = false; 4395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 4405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 4415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// We strip out different number of stack frames in debug mode 4435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)// because less inlining happens in that case 4445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#ifdef NDEBUG 4455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static const int kStripFrames = 1; 4465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#else 4475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)static const int kStripFrames = 3; 4485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)#endif 4495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::RecordRegionAddition(const void* start, size_t size) { 4515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Record start/end info about this memory acquisition call in a new region: 4525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region region; 4535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) region.Create(start, size); 4545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // First get the call stack info into the local varible 'region': 4555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const int depth = 4565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) max_stack_depth_ > 0 4575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) ? MallocHook::GetCallerStackTrace(const_cast<void**>(region.call_stack), 4585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) max_stack_depth_, kStripFrames + 1) 4595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) : 0; 4605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) region.set_call_stack_depth(depth); // record stack info fully 4615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "New global region %p..%p from %p", 4625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.start_addr), 4635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.end_addr), 4645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region.caller())); 4655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Note: none of the above allocates memory. 4665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Lock(); // recursively lock 4675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) map_size_ += size; 4685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) InsertRegionLocked(region); 4695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // This will (eventually) allocate storage for and copy over the stack data 4705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // from region.call_stack_data_ that is pointed by region.call_stack(). 4715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 4725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 4735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 4745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::RecordRegionRemoval(const void* start, size_t size) { 4755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Lock(); 4765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (recursive_insert) { 4775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // First remove the removed region from saved_regions, if it's 4785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // there, to prevent overrunning saved_regions in recursive 4795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // map/unmap call sequences, and also from later inserting regions 4805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // which have already been unmapped. 4815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) uintptr_t start_addr = reinterpret_cast<uintptr_t>(start); 4825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) uintptr_t end_addr = start_addr + size; 4835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) int put_pos = 0; 4845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) int old_count = saved_regions_count; 4855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) for (int i = 0; i < old_count; ++i, ++put_pos) { 4865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region& r = saved_regions[i]; 4875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (r.start_addr == start_addr && r.end_addr == end_addr) { 4885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // An exact match, so it's safe to remove. 4895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) --saved_regions_count; 4905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) --put_pos; 4915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, ("Insta-Removing saved region %p..%p; " 4925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "now have %d saved regions"), 4935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(start_addr), 4945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(end_addr), 4955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) saved_regions_count); 4965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else { 4975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (put_pos < i) { 4985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) saved_regions[put_pos] = saved_regions[i]; 4995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 5005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 5015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 5025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 5035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (regions_ == NULL) { // We must have just unset the hooks, 5045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // but this thread was already inside the hook. 5055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 5065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) return; 5075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 5085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (!recursive_insert) { 5095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) HandleSavedRegionsLocked(&InsertRegionLocked); 5105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 5115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // first handle adding saved regions if any 5125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) uintptr_t start_addr = reinterpret_cast<uintptr_t>(start); 5135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) uintptr_t end_addr = start_addr + size; 5145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // subtract start_addr, end_addr from all the regions 5155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "Removing global region %p..%p; have %"PRIuS" regions", 5165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(start_addr), 5175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(end_addr), 5185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_->size()); 5195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region sample; 5205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) sample.SetRegionSetKey(start_addr); 5215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Only iterate over the regions that might overlap start_addr..end_addr: 5225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) for (RegionSet::iterator region = regions_->lower_bound(sample); 5235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) region != regions_->end() && region->start_addr < end_addr; 5245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) /*noop*/) { 5255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(13, "Looking at region %p..%p", 5265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->start_addr), 5275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->end_addr)); 5285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (start_addr <= region->start_addr && 5295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) region->end_addr <= end_addr) { // full deletion 5305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Deleting region %p..%p", 5315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->start_addr), 5325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->end_addr)); 5335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RegionSet::iterator d = region; 5345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) ++region; 5355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_->erase(d); 5365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) continue; 5375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else if (region->start_addr < start_addr && 5385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) end_addr < region->end_addr) { // cutting-out split 5395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Splitting region %p..%p in two", 5405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->start_addr), 5415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->end_addr)); 5425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Make another region for the start portion: 5435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // The new region has to be the start portion because we can't 5445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // just modify region->end_addr as it's the sorting key. 5455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region r = *region; 5465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) r.set_end_addr(start_addr); 5475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) InsertRegionLocked(r); 5485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // cut *region from start: 5495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const_cast<Region&>(*region).set_start_addr(end_addr); 5505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else if (end_addr > region->start_addr && 5515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) start_addr <= region->start_addr) { // cut from start 5525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Start-chopping region %p..%p", 5535821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->start_addr), 5545821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->end_addr)); 5555821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const_cast<Region&>(*region).set_start_addr(end_addr); 5565821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else if (start_addr > region->start_addr && 5575821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) start_addr < region->end_addr) { // cut from end 5585821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "End-chopping region %p..%p", 5595821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->start_addr), 5605821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(region->end_addr)); 5615821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // Can't just modify region->end_addr (it's the sorting key): 5625821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Region r = *region; 5635821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) r.set_end_addr(start_addr); 5645821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RegionSet::iterator d = region; 5655821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) ++region; 5665821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // It's safe to erase before inserting since r is independent of *d: 5675821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // r contains an own copy of the call stack: 5685821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_->erase(d); 5695821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) InsertRegionLocked(r); 5705821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) continue; 5715821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 5725821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) ++region; 5735821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 5745821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(12, "Removed region %p..%p; have %"PRIuS" regions", 5755821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(start_addr), 5765821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<void*>(end_addr), 5775821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) regions_->size()); 5785821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (VLOG_IS_ON(12)) LogAllLocked(); 5795821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) unmap_size_ += size; 5805821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) Unlock(); 5815821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 5825821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5835821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::MmapHook(const void* result, 5845821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const void* start, size_t size, 5855821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) int prot, int flags, 5865821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) int fd, off_t offset) { 5875821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // TODO(maxim): replace all 0x%"PRIxS" by %p when RAW_VLOG uses a safe 5885821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // snprintf reimplementation that does not malloc to pretty-print NULL 5895821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MMap = 0x%"PRIxPTR" of %"PRIuS" at %"PRIu64" " 5905821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "prot %d flags %d fd %d offs %"PRId64, 5915821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<uintptr_t>(result), size, 5925821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<uint64>(start), prot, flags, fd, 5935821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) static_cast<int64>(offset)); 5945821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (result != reinterpret_cast<void*>(MAP_FAILED) && size != 0) { 5955821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RecordRegionAddition(result, size); 5965821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 5975821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 5985821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 5995821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::MunmapHook(const void* ptr, size_t size) { 6005821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MUnmap of %p %"PRIuS"", ptr, size); 6015821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (size != 0) { 6025821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RecordRegionRemoval(ptr, size); 6035821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 6045821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 6055821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6065821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::MremapHook(const void* result, 6075821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const void* old_addr, size_t old_size, 6085821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) size_t new_size, int flags, 6095821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) const void* new_addr) { 6105821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "MRemap = 0x%"PRIxPTR" of 0x%"PRIxPTR" %"PRIuS" " 6115821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "to %"PRIuS" flags %d new_addr=0x%"PRIxPTR, 6125821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) (uintptr_t)result, (uintptr_t)old_addr, 6135821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) old_size, new_size, flags, 6145821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) flags & MREMAP_FIXED ? (uintptr_t)new_addr : 0); 6155821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (result != reinterpret_cast<void*>(-1)) { 6165821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RecordRegionRemoval(old_addr, old_size); 6175821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RecordRegionAddition(result, new_size); 6185821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 6195821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 6205821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6215821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)extern "C" void* __sbrk(ptrdiff_t increment); // defined in libc 6225821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6235821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::SbrkHook(const void* result, ptrdiff_t increment) { 6245821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_VLOG(10, "Sbrk = 0x%"PRIxPTR" of %"PRIdS"", (uintptr_t)result, increment); 6255821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (result != reinterpret_cast<void*>(-1)) { 6265821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) if (increment > 0) { 6275821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) void* new_end = sbrk(0); 6285821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RecordRegionAddition(result, reinterpret_cast<uintptr_t>(new_end) - 6295821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<uintptr_t>(result)); 6305821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } else if (increment < 0) { 6315821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) void* new_end = sbrk(0); 6325821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RecordRegionRemoval(new_end, reinterpret_cast<uintptr_t>(result) - 6335821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) reinterpret_cast<uintptr_t>(new_end)); 6345821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 6355821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 6365821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 6375821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) 6385821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)void MemoryRegionMap::LogAllLocked() { 6395821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(LockIsHeld(), "should be held (by this thread)"); 6405821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_LOG(INFO, "List of regions:"); 6415821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) uintptr_t previous = 0; 6425821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) for (RegionSet::const_iterator r = regions_->begin(); 6435821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) r != regions_->end(); ++r) { 6445821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_LOG(INFO, "Memory region 0x%"PRIxPTR"..0x%"PRIxPTR" " 6455821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) "from 0x%"PRIxPTR" stack=%d", 6465821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) r->start_addr, r->end_addr, r->caller(), r->is_stack); 6475821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_CHECK(previous < r->end_addr, "wow, we messed up the set order"); 6485821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) // this must be caused by uncontrolled recursive operations on regions_ 6495821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) previous = r->end_addr; 6505821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) } 6515821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles) RAW_LOG(INFO, "End of regions list"); 6525821806d5e7f356e8fa4b058a389a808ea183019Torne (Richard Coles)} 653