1c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 2c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath This is a version (aka dlmalloc) of malloc/free/realloc written by 3c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Doug Lea and released to the public domain, as explained at 4c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath http://creativecommons.org/licenses/publicdomain. Send questions, 5c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath comments, complaints, performance data, etc to dl@cs.oswego.edu 6c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 7c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath* Version 2.8.4 Wed May 27 09:56:23 2009 Doug Lea (dl at gee) 8c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 9c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Note: There may be an updated version of this malloc obtainable at 10c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ftp://gee.cs.oswego.edu/pub/misc/malloc.c 11c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Check before installing! 12c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 13c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath* Quickstart 14c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 15c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath This library is all in one file to simplify the most common usage: 16c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ftp it, compile it (-O3), and link it into another program. All of 17c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the compile-time options default to reasonable values for use on 18c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath most platforms. You might later want to step through various 19c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath compile-time and dynamic tuning options. 20c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 21c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath For convenience, an include file for code using this malloc is at: 22c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ftp://gee.cs.oswego.edu/pub/misc/malloc-2.8.4.h 23c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath You don't really need this .h file unless you call functions not 24c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath defined in your system include files. The .h file contains only the 25c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath excerpts from this file needed for using this malloc on ANSI C/C++ 26c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath systems, so long as you haven't changed compile-time options about 27c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath naming and tuning parameters. If you do, then you can create your 28c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath own malloc.h that does include all settings by cutting at the point 29c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath indicated below. Note that you may already by default be using a C 30c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath library containing a malloc that is based on some version of this 31c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc (for example in linux). You might still want to use the one 32c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath in this file to customize settings or to avoid overheads associated 33c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath with library versions. 34c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 35c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath* Vital statistics: 36c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 37c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Supported pointer/size_t representation: 4 or 8 bytes 38c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t MUST be an unsigned type of the same width as 39c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pointers. (If you are using an ancient system that declares 40c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t as a signed type, or need it to be a different width 41c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath than pointers, you can use a previous release of this malloc 42c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (e.g. 2.7.2) supporting these.) 43c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 44c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Alignment: 8 bytes (default) 45c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath This suffices for nearly all current machines and C compilers. 46c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath However, you can define MALLOC_ALIGNMENT to be wider than this 47c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if necessary (up to 128bytes), at the expense of using more space. 48c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 49c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Minimum overhead per allocated chunk: 4 or 8 bytes (if 4byte sizes) 50c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 8 or 16 bytes (if 8byte sizes) 51c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Each malloced chunk has a hidden word of overhead holding size 52c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and status information, and additional cross-check word 53c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if FOOTERS is defined. 54c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 55c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Minimum allocated size: 4-byte ptrs: 16 bytes (including overhead) 56c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 8-byte ptrs: 32 bytes (including overhead) 57c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 58c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Even a request for zero bytes (i.e., malloc(0)) returns a 59c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pointer to something of the minimum allocatable size. 60c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The maximum overhead wastage (i.e., number of extra bytes 61c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath allocated than were requested in malloc) is less than or equal 62c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath to the minimum size, except for requests >= mmap_threshold that 63c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath are serviced via mmap(), where the worst case wastage is about 64c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 32 bytes plus the remainder from a system page (the minimal 65c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mmap unit); typically 4096 or 8192 bytes. 66c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 67c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Security: static-safe; optionally more or less 68c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The "security" of malloc refers to the ability of malicious 69c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath code to accentuate the effects of errors (for example, freeing 70c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath space that is not currently malloc'ed or overwriting past the 71c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ends of chunks) in code that calls malloc. This malloc 72c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath guarantees not to modify any memory locations below the base of 73c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath heap, i.e., static variables, even in the presence of usage 74c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath errors. The routines additionally detect most improper frees 75c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and reallocs. All this holds as long as the static bookkeeping 76c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for malloc itself is not corrupted by some other means. This 77c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath is only one aspect of security -- these checks do not, and 78c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath cannot, detect all possible programming errors. 79c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 80c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If FOOTERS is defined nonzero, then each allocated chunk 81c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath carries an additional check word to verify that it was malloced 82c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath from its space. These check words are the same within each 83c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath execution of a program using malloc, but differ across 84c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath executions, so externally crafted fake chunks cannot be 85c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath freed. This improves security by rejecting frees/reallocs that 86c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath could corrupt heap memory, in addition to the checks preventing 87c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath writes to statics that are always on. This may further improve 88c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath security at the expense of time and space overhead. (Note that 89c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath FOOTERS may also be worth using with MSPACES.) 90c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 91c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath By default detected errors cause the program to abort (calling 92c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath "abort()"). You can override this to instead proceed past 93c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath errors by defining PROCEED_ON_ERROR. In this case, a bad free 94c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath has no effect, and a malloc that encounters a bad address 95c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath caused by user overwrites will ignore the bad address by 96c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath dropping pointers and indices to all known memory. This may 97c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath be appropriate for programs that should continue if at all 98c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath possible in the face of programming errors, although they may 99c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath run out of memory because dropped memory is never reclaimed. 100c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 101c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If you don't like either of these options, you can define 102c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION and USAGE_ERROR_ACTION to do anything 103c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else. And if if you are sure that your program using malloc has 104c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath no errors or vulnerabilities, you can define INSECURE to 1, 105c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath which might (or might not) provide a small performance improvement. 106c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 107c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Thread-safety: NOT thread-safe unless USE_LOCKS defined 108c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath When USE_LOCKS is defined, each public call to malloc, free, 109c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath etc is surrounded with either a pthread mutex or a win32 110c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath spinlock (depending on WIN32). This is not especially fast, and 111c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath can be a major bottleneck. It is designed only to provide 112c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath minimal protection in concurrent environments, and to provide a 113c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath basis for extensions. If you are using malloc in a concurrent 114c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath program, consider instead using nedmalloc 115c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (http://www.nedprod.com/programs/portable/nedmalloc/) or 116c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ptmalloc (See http://www.malloc.de), which are derived 117c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath from versions of this malloc. 118c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 119c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath System requirements: Any combination of MORECORE and/or MMAP/MUNMAP 120c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath This malloc can use unix sbrk or any emulation (invoked using 121c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the CALL_MORECORE macro) and/or mmap/munmap or any emulation 122c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (invoked using CALL_MMAP/CALL_MUNMAP) to get and release system 123c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memory. On most unix systems, it tends to work best if both 124c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MORECORE and MMAP are enabled. On Win32, it uses emulations 125c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath based on VirtualAlloc. It also uses common C library functions 126c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath like memset. 127c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 128c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Compliance: I believe it is compliant with the Single Unix Specification 129c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (See http://www.unix.org). Also SVID/XPG, ANSI C, and probably 130c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath others as well. 131c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 132c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath* Overview of algorithms 133c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 134c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath This is not the fastest, most space-conserving, most portable, or 135c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath most tunable malloc ever written. However it is among the fastest 136c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while also being among the most space-conserving, portable and 137c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tunable. Consistent balance across these factors results in a good 138c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath general-purpose allocator for malloc-intensive programs. 139c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 140c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath In most ways, this malloc is a best-fit allocator. Generally, it 141c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chooses the best-fitting existing chunk for a request, with ties 142c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath broken in approximately least-recently-used order. (This strategy 143c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath normally maintains low fragmentation.) However, for requests less 144c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath than 256bytes, it deviates from best-fit when there is not an 145c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath exactly fitting available chunk by preferring to use space adjacent 146c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath to that used for the previous small request, as well as by breaking 147c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ties in approximately most-recently-used order. (These enhance 148c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath locality of series of small allocations.) And for very large requests 149c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (>= 256Kb by default), it relies on system memory mapping 150c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath facilities, if supported. (This helps avoid carrying around and 151c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath possibly fragmenting memory used only for large chunks.) 152c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 153c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath All operations (except malloc_stats and mallinfo) have execution 154c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath times that are bounded by a constant factor of the number of bits in 155c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath a size_t, not counting any clearing in calloc or copying in realloc, 156c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath or actions surrounding MORECORE and MMAP that have times 157c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath proportional to the number of non-contiguous regions returned by 158c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath system allocation routines, which is often just 1. In real-time 159c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath applications, you can optionally suppress segment traversals using 160c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath NO_SEGMENT_TRAVERSAL, which assures bounded execution even when 161c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath system allocators return non-contiguous spaces, at the typical 162c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath expense of carrying around more memory and increased fragmentation. 163c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 164c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The implementation is not very modular and seriously overuses 165c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath macros. Perhaps someday all C compilers will do as good a job 166c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath inlining modular code as can now be done by brute-force expansion, 167c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath but now, enough of them seem not to. 168c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 169c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Some compilers issue a lot of warnings about code that is 170c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath dead/unreachable only on some platforms, and also about intentional 171c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath uses of negation on unsigned types. All known cases of each can be 172c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ignored. 173c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 174c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath For a longer but out of date high-level description, see 175c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath http://gee.cs.oswego.edu/dl/html/malloc.html 176c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 177c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath* MSPACES 178c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If MSPACES is defined, then in addition to malloc, free, etc., 179c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath this file also defines mspace_malloc, mspace_free, etc. These 180c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath are versions of malloc routines that take an "mspace" argument 181c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath obtained using create_mspace, to control all internal bookkeeping. 182c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If ONLY_MSPACES is defined, only these versions are compiled. 183c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath So if you would like to use this allocator for only some allocations, 184c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and your system malloc for others, you can compile with 185c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ONLY_MSPACES and then do something like... 186c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath static mspace mymspace = create_mspace(0,0); // for example 187c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define mymalloc(bytes) mspace_malloc(mymspace, bytes) 188c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 189c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (Note: If you only need one instance of an mspace, you can instead 190c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath use "USE_DL_PREFIX" to relabel the global malloc.) 191c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 192c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath You can similarly create thread-local allocators by storing 193c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspaces as thread-locals. For example: 194c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath static __thread mspace tlms = 0; 195c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* tlmalloc(size_t bytes) { 196c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (tlms == 0) tlms = create_mspace(0, 0); 197c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return mspace_malloc(tlms, bytes); 198c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 199c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void tlfree(void* mem) { mspace_free(tlms, mem); } 200c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 201c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Unless FOOTERS is defined, each mspace is completely independent. 202c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath You cannot allocate from one and free to another (although 203c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath conformance is only weakly checked, so usage errors are not always 204c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath caught). If FOOTERS is defined, then each chunk carries around a tag 205c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath indicating its originating mspace, and frees are directed to their 206c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath originating spaces. 207c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 208c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ------------------------- Compile-time options --------------------------- 209c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 210c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathBe careful in setting #define values for numerical constants of type 211c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t. On some systems, literal values are not automatically extended 212c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathto size_t precision unless they are explicitly casted. You can also 213c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathuse the symbolic values MAX_SIZE_T, SIZE_T_ONE, etc below. 214c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 215c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathWIN32 default: defined if _WIN32 defined 216c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Defining WIN32 sets up defaults for MS environment and compilers. 217c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Otherwise defaults are for unix. Beware that there seem to be some 218c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath cases where this malloc might not be a pure drop-in replacement for 219c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Win32 malloc: Random-looking failures from Win32 GDI API's (eg; 220c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath SetDIBits()) may be due to bugs in some video driver implementations 221c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath when pixel buffers are malloc()ed, and the region spans more than 222c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath one VirtualAlloc()ed region. Because dlmalloc uses a small (64Kb) 223c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath default granularity, pixel buffers may straddle virtual allocation 224c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath regions more often than when using the Microsoft allocator. You can 225c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath avoid this by using VirtualAlloc() and VirtualFree() for all pixel 226c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath buffers rather than using malloc(). If this is not possible, 227c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath recompile this malloc with a larger DEFAULT_GRANULARITY. 228c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 229c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathMALLOC_ALIGNMENT default: (size_t)8 230c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Controls the minimum alignment for malloc'ed chunks. It must be a 231c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath power of two and at least 8, even on machines for which smaller 232c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath alignments would suffice. It may be defined as larger than this 233c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath though. Note however that code and data structures are optimized for 234c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the case of 8-byte alignment. 235c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 236c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathMSPACES default: 0 (false) 237c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If true, compile in support for independent allocation spaces. 238c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath This is only supported if HAVE_MMAP is true. 239c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 240c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathONLY_MSPACES default: 0 (false) 241c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If true, only compile in mspace versions, not regular versions. 242c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 243c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathUSE_LOCKS default: 0 (false) 244c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Causes each call to each public routine to be surrounded with 245c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pthread or WIN32 mutex lock/unlock. (If set true, this can be 246c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath overridden on a per-mspace basis for mspace versions.) If set to a 247c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath non-zero value other than 1, locks are used, but their 248c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath implementation is left out, so lock functions must be supplied manually, 249c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath as described below. 250c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 251c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathUSE_SPIN_LOCKS default: 1 iff USE_LOCKS and on x86 using gcc or MSC 252c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If true, uses custom spin locks for locking. This is currently 253c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath supported only for x86 platforms using gcc or recent MS compilers. 254c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Otherwise, posix locks or win32 critical sections are used. 255c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 256c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathFOOTERS default: 0 257c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If true, provide extra checking and dispatching by placing 258c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath information in the footers of allocated chunks. This adds 259c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath space and time overhead. 260c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 261c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathINSECURE default: 0 262c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If true, omit checks for usage errors and heap space overwrites. 263c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 264c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathUSE_DL_PREFIX default: NOT defined 265c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Causes compiler to prefix all public routines with the string 'dl'. 266c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath This can be useful when you only want to use this malloc in one part 267c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath of a program, using your regular system malloc elsewhere. 268c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 269c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathABORT default: defined as abort() 270c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Defines how to abort on failed checks. On most systems, a failed 271c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check cannot die with an "assert" or even print an informative 272c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath message, because the underlying print routines in turn call malloc, 273c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath which will fail again. Generally, the best policy is to simply call 274c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath abort(). It's not very useful to do more than this because many 275c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath errors due to overwriting will show up as address faults (null, odd 276c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath addresses etc) rather than malloc-triggered checks, so will also 277c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath abort. Also, most compilers know that abort() does not return, so 278c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath can better optimize code conditionally calling it. 279c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 280c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathPROCEED_ON_ERROR default: defined as 0 (false) 281c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Controls whether detected bad addresses cause them to bypassed 282c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath rather than aborting. If set, detected bad arguments to free and 283c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath realloc are ignored. And all bookkeeping information is zeroed out 284c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath upon a detected overwrite of freed heap space, thus losing the 285c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ability to ever return it from malloc again, but enabling the 286c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath application to proceed. If PROCEED_ON_ERROR is defined, the 287c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath static variable malloc_corruption_error_count is compiled in 288c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and can be examined to see if errors have occurred. This option 289c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath generates slower code than the default abort policy. 290c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 291c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathDEBUG default: NOT defined 292c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The DEBUG setting is mainly intended for people trying to modify 293c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath this code or diagnose problems when porting to new platforms. 294c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath However, it may also be able to better isolate user errors than just 295c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath using runtime checks. The assertions in the check routines spell 296c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath out in more detail the assumptions and invariants underlying the 297c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath algorithms. The checking is fairly extensive, and will slow down 298c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath execution noticeably. Calling malloc_stats or mallinfo with DEBUG 299c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set will attempt to check every non-mmapped allocated and free chunk 300c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath in the course of computing the summaries. 301c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 302c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathABORT_ON_ASSERT_FAILURE default: defined as 1 (true) 303c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Debugging assertion failures can be nearly impossible if your 304c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath version of the assert macro causes malloc to be called, which will 305c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath lead to a cascade of further failures, blowing the runtime stack. 306c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ABORT_ON_ASSERT_FAILURE cause assertions failures to call abort(), 307c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath which will usually make debugging easier. 308c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 309c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathMALLOC_FAILURE_ACTION default: sets errno to ENOMEM, or no-op on win32 310c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The action to take before "return 0" when malloc fails to be able to 311c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return memory because there is none available. 312c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 313c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathHAVE_MORECORE default: 1 (true) unless win32 or ONLY_MSPACES 314c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath True if this system supports sbrk or an emulation of it. 315c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 316c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathMORECORE default: sbrk 317c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The name of the sbrk-style system routine to call to obtain more 318c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memory. See below for guidance on writing custom MORECORE 319c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath functions. The type of the argument to sbrk/MORECORE varies across 320c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath systems. It cannot be size_t, because it supports negative 321c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath arguments, so it is normally the signed type of the same width as 322c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t (sometimes declared as "intptr_t"). It doesn't much matter 323c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath though. Internally, we only call it with arguments less than half 324c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the max value of a size_t, which should work across all reasonable 325c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath possibilities, although sometimes generating compiler warnings. 326c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 327c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathMORECORE_CONTIGUOUS default: 1 (true) if HAVE_MORECORE 328c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If true, take advantage of fact that consecutive calls to MORECORE 329c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath with positive arguments always return contiguous increasing 330c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath addresses. This is true of unix sbrk. It does not hurt too much to 331c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set it true anyway, since malloc copes with non-contiguities. 332c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Setting it false when definitely non-contiguous saves time 333c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and possibly wasted space it would take to discover this though. 334c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 335c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathMORECORE_CANNOT_TRIM default: NOT defined 336c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath True if MORECORE cannot release space back to the system when given 337c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath negative arguments. This is generally necessary only if you are 338c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath using a hand-crafted MORECORE function that cannot handle negative 339c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath arguments. 340c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 341c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathNO_SEGMENT_TRAVERSAL default: 0 342c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If non-zero, suppresses traversals of memory segments 343c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath returned by either MORECORE or CALL_MMAP. This disables 344c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath merging of segments that are contiguous, and selectively 345c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath releasing them to the OS if unused, but bounds execution times. 346c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 347c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathHAVE_MMAP default: 1 (true) 348c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath True if this system supports mmap or an emulation of it. If so, and 349c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath HAVE_MORECORE is not true, MMAP is used for all system 350c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath allocation. If set and HAVE_MORECORE is true as well, MMAP is 351c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath primarily used to directly allocate very large blocks. It is also 352c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath used as a backup strategy in cases where MORECORE fails to provide 353c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath space from system. Note: A single call to MUNMAP is assumed to be 354c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath able to unmap memory that may have be allocated using multiple calls 355c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath to MMAP, so long as they are adjacent. 356c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 357c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathHAVE_MREMAP default: 1 on linux, else 0 358c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If true realloc() uses mremap() to re-allocate large blocks and 359c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath extend or shrink allocation spaces. 360c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 361c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathMMAP_CLEARS default: 1 except on WINCE. 362c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath True if mmap clears memory so calloc doesn't need to. This is true 363c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for standard unix mmap using /dev/zero and on WIN32 except for WINCE. 364c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 365c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathUSE_BUILTIN_FFS default: 0 (i.e., not used) 366c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Causes malloc to use the builtin ffs() function to compute indices. 367c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Some compilers may recognize and intrinsify ffs to be faster than the 368c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath supplied C version. Also, the case of x86 using gcc is special-cased 369c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath to an asm instruction, so is already as fast as it can be, and so 370c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath this setting has no effect. Similarly for Win32 under recent MS compilers. 371c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (On most x86s, the asm version is only slightly faster than the C version.) 372c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 373c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathmalloc_getpagesize default: derive from system includes, or 4096. 374c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The system page size. To the extent possible, this malloc manages 375c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memory from the system in page-size units. This may be (and 376c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath usually is) a function rather than a constant. This is ignored 377c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if WIN32, where page size is determined using getSystemInfo during 378c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath initialization. 379c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 380c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathUSE_DEV_RANDOM default: 0 (i.e., not used) 381c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Causes malloc to use /dev/random to initialize secure magic seed for 382c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath stamping footers. Otherwise, the current time is used. 383c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 384c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathNO_MALLINFO default: 0 385c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If defined, don't compile "mallinfo". This can be a simple way 386c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath of dealing with mismatches between system declarations and 387c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath those in this file. 388c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 389c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathMALLINFO_FIELD_TYPE default: size_t 390c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The type of the fields in the mallinfo struct. This was originally 391c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath defined as "int" in SVID etc, but is more usefully defined as 392c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t. The value is used only if HAVE_USR_INCLUDE_MALLOC_H is not set 393c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 394c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathREALLOC_ZERO_BYTES_FREES default: not defined 395c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath This should be set if a call to realloc with zero bytes should 396c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath be the same as a call to free. Some people think it should. Otherwise, 397c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath since this malloc returns a unique pointer for malloc(0), so does 398c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath realloc(p, 0). 399c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 400c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathLACKS_UNISTD_H, LACKS_FCNTL_H, LACKS_SYS_PARAM_H, LACKS_SYS_MMAN_H 401c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathLACKS_STRINGS_H, LACKS_STRING_H, LACKS_SYS_TYPES_H, LACKS_ERRNO_H 402c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathLACKS_STDLIB_H default: NOT defined unless on WIN32 403c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Define these if your system does not have these header files. 404c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath You might need to manually insert some of the declarations they provide. 405c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 406c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathDEFAULT_GRANULARITY default: page size if MORECORE_CONTIGUOUS, 407c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath system_info.dwAllocationGranularity in WIN32, 408c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath otherwise 64K. 409c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Also settable using mallopt(M_GRANULARITY, x) 410c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The unit for allocating and deallocating memory from the system. On 411c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath most systems with contiguous MORECORE, there is no reason to 412c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath make this more than a page. However, systems with MMAP tend to 413c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath either require or encourage larger granularities. You can increase 414c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath this value to prevent system allocation functions to be called so 415c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath often, especially if they are slow. The value must be at least one 416c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath page and must be a power of two. Setting to 0 causes initialization 417c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath to either page size or win32 region size. (Note: In previous 418c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath versions of malloc, the equivalent of this option was called 419c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath "TOP_PAD") 420c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 421c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathDEFAULT_TRIM_THRESHOLD default: 2MB 422c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Also settable using mallopt(M_TRIM_THRESHOLD, x) 423c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The maximum amount of unused top-most memory to keep before 424c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath releasing via malloc_trim in free(). Automatic trimming is mainly 425c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath useful in long-lived programs using contiguous MORECORE. Because 426c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath trimming via sbrk can be slow on some systems, and can sometimes be 427c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath wasteful (in cases where programs immediately afterward allocate 428c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath more large chunks) the value should be high enough so that your 429c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath overall system performance would improve by releasing this much 430c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memory. As a rough guide, you might set to a value close to the 431c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath average size of a process (program) running on your system. 432c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Releasing this much memory would allow such a process to run in 433c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memory. Generally, it is worth tuning trim thresholds when a 434c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath program undergoes phases where several large chunks are allocated 435c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and released in ways that can reuse each other's storage, perhaps 436c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mixed with phases where there are no such chunks at all. The trim 437c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath value must be greater than page size to have any useful effect. To 438c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath disable trimming completely, you can set to MAX_SIZE_T. Note that the trick 439c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath some people use of mallocing a huge space and then freeing it at 440c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath program startup, in an attempt to reserve system memory, doesn't 441c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath have the intended effect under automatic trimming, since that memory 442c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath will immediately be returned to the system. 443c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 444c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathDEFAULT_MMAP_THRESHOLD default: 256K 445c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Also settable using mallopt(M_MMAP_THRESHOLD, x) 446c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The request size threshold for using MMAP to directly service a 447c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath request. Requests of at least this size that cannot be allocated 448c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath using already-existing space will be serviced via mmap. (If enough 449c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath normal freed space already exists it is used instead.) Using mmap 450c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath segregates relatively large chunks of memory so that they can be 451c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath individually obtained and released from the host system. A request 452c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath serviced through mmap is never reused by any other request (at least 453c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath not directly; the system may just so happen to remap successive 454c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath requests to the same locations). Segregating space in this way has 455c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the benefits that: Mmapped space can always be individually released 456c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath back to the system, which helps keep the system level memory demands 457c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath of a long-lived program low. Also, mapped memory doesn't become 458c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath `locked' between other chunks, as can happen with normally allocated 459c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunks, which means that even trimming via malloc_trim would not 460c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath release them. However, it has the disadvantage that the space 461c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath cannot be reclaimed, consolidated, and then used to service later 462c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath requests, as happens with normal chunks. The advantages of mmap 463c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nearly always outweigh disadvantages for "large" chunks, but the 464c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath value of "large" may vary across systems. The default is an 465c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath empirically derived value that works well in most systems. You can 466c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath disable mmap by setting to MAX_SIZE_T. 467c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 468c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathMAX_RELEASE_CHECK_RATE default: 4095 unless not HAVE_MMAP 469c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The number of consolidated frees between checks to release 470c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unused segments when freeing. When using non-contiguous segments, 471c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath especially with multiple mspaces, checking only for topmost space 472c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath doesn't always suffice to trigger trimming. To compensate for this, 473c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath free() will, with a period of MAX_RELEASE_CHECK_RATE (or the 474c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath current number of segments, if greater) try to release unused 475c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath segments to the OS when freeing chunks that result in 476c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath consolidation. The best value for this parameter is a compromise 477c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath between slowing down frees with relatively costly checks that 478c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath rarely trigger versus holding on to unused memory. To effectively 479c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath disable, set to MAX_SIZE_T. This may lead to a very slight speed 480c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath improvement at the expense of carrying around more memory. 481c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 482c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 483c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define USE_DL_PREFIX 484c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath//#define HAVE_USR_INCLUDE_MALLOC_H 485c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath//#define MSPACES 1 486c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define NO_SEGMENT_TRAVERSAL 1 487c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 488c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Version identifier to allow people to support multiple versions */ 489c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef DLMALLOC_VERSION 490c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define DLMALLOC_VERSION 20804 491c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* DLMALLOC_VERSION */ 492c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 493c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef WIN32 494c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef _WIN32 495c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define WIN32 1 496c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* _WIN32 */ 497c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef _WIN32_WCE 498c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define LACKS_FCNTL_H 499c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define WIN32 1 500c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* _WIN32_WCE */ 501c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* WIN32 */ 502c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef WIN32 503c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define WIN32_LEAN_AND_MEAN 504c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <windows.h> 505c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define HAVE_MMAP 1 506c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define HAVE_MORECORE 0 507c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define LACKS_UNISTD_H 508c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define LACKS_SYS_PARAM_H 509c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define LACKS_SYS_MMAN_H 510c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define LACKS_STRING_H 511c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define LACKS_STRINGS_H 512c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define LACKS_SYS_TYPES_H 513c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define LACKS_ERRNO_H 514c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef MALLOC_FAILURE_ACTION 515c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MALLOC_FAILURE_ACTION 516c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MALLOC_FAILURE_ACTION */ 517c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef _WIN32_WCE /* WINCE reportedly does not clear */ 518c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MMAP_CLEARS 0 519c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else 520c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MMAP_CLEARS 1 521c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* _WIN32_WCE */ 522c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* WIN32 */ 523c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 524c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if defined(DARWIN) || defined(_DARWIN) 525c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Mac OSX docs advise not to use sbrk; it seems better to use mmap */ 526c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef HAVE_MORECORE 527c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define HAVE_MORECORE 0 528c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define HAVE_MMAP 1 529c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* OSX allocators provide 16 byte alignment */ 530c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef MALLOC_ALIGNMENT 531c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MALLOC_ALIGNMENT ((size_t)16U) 532c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 533c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MORECORE */ 534c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* DARWIN */ 535c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 536c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef LACKS_SYS_TYPES_H 537c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <sys/types.h> /* For size_t */ 538c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* LACKS_SYS_TYPES_H */ 539c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 540c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if (defined(__GNUC__) && ((defined(__i386__) || defined(__x86_64__)))) || (defined(_MSC_VER) && _MSC_VER>=1310) 541c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SPIN_LOCKS_AVAILABLE 1 542c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else 543c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SPIN_LOCKS_AVAILABLE 0 544c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 545c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 546c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* The maximum possible size_t value has all bits set */ 547c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MAX_SIZE_T (~(size_t)0) 548c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 549c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef ONLY_MSPACES 550c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ONLY_MSPACES 0 /* define to a value */ 551c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else 552c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ONLY_MSPACES 1 553c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* ONLY_MSPACES */ 554c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef MSPACES 555c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if ONLY_MSPACES 556c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MSPACES 1 557c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* ONLY_MSPACES */ 558c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MSPACES 0 559c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* ONLY_MSPACES */ 560c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MSPACES */ 561c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef MALLOC_ALIGNMENT 562c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MALLOC_ALIGNMENT ((size_t)8U) 563c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MALLOC_ALIGNMENT */ 564c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef FOOTERS 565c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define FOOTERS 0 566c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* FOOTERS */ 567c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef ABORT 568c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ABORT abort() 569c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* ABORT */ 570c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef ABORT_ON_ASSERT_FAILURE 571c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ABORT_ON_ASSERT_FAILURE 1 572c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* ABORT_ON_ASSERT_FAILURE */ 573c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef PROCEED_ON_ERROR 574c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define PROCEED_ON_ERROR 0 575c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* PROCEED_ON_ERROR */ 576c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef USE_LOCKS 577c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define USE_LOCKS 0 578c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_LOCKS */ 579c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef USE_SPIN_LOCKS 580c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if USE_LOCKS && SPIN_LOCKS_AVAILABLE 581c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define USE_SPIN_LOCKS 1 582c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else 583c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define USE_SPIN_LOCKS 0 584c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_LOCKS && SPIN_LOCKS_AVAILABLE. */ 585c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_SPIN_LOCKS */ 586c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef INSECURE 587c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define INSECURE 0 588c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* INSECURE */ 589c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef HAVE_MMAP 590c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define HAVE_MMAP 1 591c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MMAP */ 592c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef MMAP_CLEARS 593c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MMAP_CLEARS 1 594c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MMAP_CLEARS */ 595c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef HAVE_MREMAP 596c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef linux 597c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define HAVE_MREMAP 1 598c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* linux */ 599c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define HAVE_MREMAP 0 600c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* linux */ 601c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MREMAP */ 602c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef MALLOC_FAILURE_ACTION 603c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MALLOC_FAILURE_ACTION errno = ENOMEM; 604c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MALLOC_FAILURE_ACTION */ 605c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef HAVE_MORECORE 606c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if ONLY_MSPACES 607c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define HAVE_MORECORE 0 608c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* ONLY_MSPACES */ 609c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define HAVE_MORECORE 1 610c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* ONLY_MSPACES */ 611c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MORECORE */ 612c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !HAVE_MORECORE 613c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MORECORE_CONTIGUOUS 0 614c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* !HAVE_MORECORE */ 615c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MORECORE_DEFAULT sbrk 616c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef MORECORE_CONTIGUOUS 617c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MORECORE_CONTIGUOUS 1 618c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MORECORE_CONTIGUOUS */ 619c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MORECORE */ 620c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef DEFAULT_GRANULARITY 621c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if (MORECORE_CONTIGUOUS || defined(WIN32)) 622c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define DEFAULT_GRANULARITY (0) /* 0 means to compute in init_mparams */ 623c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* MORECORE_CONTIGUOUS */ 624c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define DEFAULT_GRANULARITY ((size_t)64U * (size_t)1024U) 625c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MORECORE_CONTIGUOUS */ 626c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* DEFAULT_GRANULARITY */ 627c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef DEFAULT_TRIM_THRESHOLD 628c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef MORECORE_CANNOT_TRIM 629c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define DEFAULT_TRIM_THRESHOLD ((size_t)2U * (size_t)1024U * (size_t)1024U) 630c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* MORECORE_CANNOT_TRIM */ 631c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define DEFAULT_TRIM_THRESHOLD MAX_SIZE_T 632c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MORECORE_CANNOT_TRIM */ 633c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* DEFAULT_TRIM_THRESHOLD */ 634c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef DEFAULT_MMAP_THRESHOLD 635c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if HAVE_MMAP 636c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define DEFAULT_MMAP_THRESHOLD ((size_t)256U * (size_t)1024U) 637c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* HAVE_MMAP */ 638c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define DEFAULT_MMAP_THRESHOLD MAX_SIZE_T 639c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MMAP */ 640c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* DEFAULT_MMAP_THRESHOLD */ 641c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef MAX_RELEASE_CHECK_RATE 642c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if HAVE_MMAP 643c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MAX_RELEASE_CHECK_RATE 4095 644c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else 645c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MAX_RELEASE_CHECK_RATE MAX_SIZE_T 646c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MMAP */ 647c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MAX_RELEASE_CHECK_RATE */ 648c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef USE_BUILTIN_FFS 649c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define USE_BUILTIN_FFS 0 650c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_BUILTIN_FFS */ 651c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef USE_DEV_RANDOM 652c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define USE_DEV_RANDOM 0 653c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_DEV_RANDOM */ 654c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef NO_MALLINFO 655c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define NO_MALLINFO 0 656c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* NO_MALLINFO */ 657c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef MALLINFO_FIELD_TYPE 658c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MALLINFO_FIELD_TYPE size_t 659c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MALLINFO_FIELD_TYPE */ 660c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef NO_SEGMENT_TRAVERSAL 661c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define NO_SEGMENT_TRAVERSAL 0 662c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* NO_SEGMENT_TRAVERSAL */ 663c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 664c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 665c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mallopt tuning options. SVID/XPG defines four standard parameter 666c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath numbers for mallopt, normally defined in malloc.h. None of these 667c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath are used in this malloc, so setting them has no effect. But this 668c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc does support the following options. 669c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 670c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 671c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define M_TRIM_THRESHOLD (-1) 672c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define M_GRANULARITY (-2) 673c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define M_MMAP_THRESHOLD (-3) 674c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 675c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ------------------------ Mallinfo declarations ------------------------ */ 676c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 677c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !NO_MALLINFO 678c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 679c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath This version of malloc supports the standard SVID/XPG mallinfo 680c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath routine that returns a struct containing usage properties and 681c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath statistics. It should work on any system that has a 682c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /usr/include/malloc.h defining struct mallinfo. The main 683c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath declaration needed is the mallinfo struct that is returned (by-copy) 684c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath by mallinfo(). The malloinfo struct contains a bunch of fields that 685c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath are not even meaningful in this version of malloc. These fields are 686c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath are instead filled by mallinfo() with other numbers that might be of 687c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath interest. 688c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 689c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath HAVE_USR_INCLUDE_MALLOC_H should be set if you have a 690c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /usr/include/malloc.h file that includes a declaration of struct 691c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mallinfo. If so, it is included; else a compliant version is 692c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath declared below. These must be precisely the same for mallinfo() to 693c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath work. The original SVID version of this struct, defined on most 694c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath systems with mallinfo, declares all fields as ints. But some others 695c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath define as unsigned long. If your system defines the fields using a 696c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath type of different width than listed here, you MUST #include your 697c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath system version and #define HAVE_USR_INCLUDE_MALLOC_H. 698c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 699c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 700c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* #define HAVE_USR_INCLUDE_MALLOC_H */ 701c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 702c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef HAVE_USR_INCLUDE_MALLOC_H 703c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include "/usr/include/malloc.h" 704c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* HAVE_USR_INCLUDE_MALLOC_H */ 705c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef STRUCT_MALLINFO_DECLARED 706c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define STRUCT_MALLINFO_DECLARED 1 707c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstruct mallinfo { 708c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLINFO_FIELD_TYPE arena; /* non-mmapped space allocated from system */ 709c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLINFO_FIELD_TYPE ordblks; /* number of free chunks */ 710c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLINFO_FIELD_TYPE smblks; /* always 0 */ 711c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLINFO_FIELD_TYPE hblks; /* always 0 */ 712c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLINFO_FIELD_TYPE hblkhd; /* space in mmapped regions */ 713c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLINFO_FIELD_TYPE usmblks; /* maximum total allocated space */ 714c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLINFO_FIELD_TYPE fsmblks; /* always 0 */ 715c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLINFO_FIELD_TYPE uordblks; /* total allocated space */ 716c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLINFO_FIELD_TYPE fordblks; /* total free space */ 717c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLINFO_FIELD_TYPE keepcost; /* releasable (via malloc_trim) space */ 718c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath}; 719c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* STRUCT_MALLINFO_DECLARED */ 720c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_USR_INCLUDE_MALLOC_H */ 721c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* NO_MALLINFO */ 722c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 723c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 724c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Try to persuade compilers to inline. The most critical functions for 725c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath inlining are defined as macros, so these aren't used for them. 726c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 727c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 728c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef FORCEINLINE 729c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #if defined(__GNUC__) 730c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define FORCEINLINE __inline __attribute__ ((always_inline)) 731c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #elif defined(_MSC_VER) 732c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define FORCEINLINE __forceinline 733c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #endif 734c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 735c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef NOINLINE 736c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #if defined(__GNUC__) 737c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define NOINLINE __attribute__ ((noinline)) 738c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #elif defined(_MSC_VER) 739c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define NOINLINE __declspec(noinline) 740c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #else 741c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define NOINLINE 742c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #endif 743c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 744c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 745c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef __cplusplus 746c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathextern "C" { 747c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef FORCEINLINE 748c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define FORCEINLINE inline 749c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 750c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* __cplusplus */ 751c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef FORCEINLINE 752c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define FORCEINLINE 753c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 754c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 755c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !ONLY_MSPACES 756c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 757c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ------------------- Declarations of public routines ------------------- */ 758c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 759c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef USE_DL_PREFIX 760c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlcalloc calloc 761c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlfree free 762c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlmalloc malloc 763c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlmemalign memalign 764c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlrealloc realloc 765c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlvalloc valloc 766c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlpvalloc pvalloc 767c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlmallinfo mallinfo 768c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlmallopt mallopt 769c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlmalloc_trim malloc_trim 770c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlmalloc_stats malloc_stats 771c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlmalloc_usable_size malloc_usable_size 772c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlmalloc_footprint malloc_footprint 773c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlmalloc_max_footprint malloc_max_footprint 774c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlindependent_calloc independent_calloc 775c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define dlindependent_comalloc independent_comalloc 776c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_DL_PREFIX */ 777c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 778c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 779c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 780c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc(size_t n) 781c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Returns a pointer to a newly allocated chunk of at least n bytes, or 782c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath null if no space is available, in which case errno is set to ENOMEM 783c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath on ANSI C systems. 784c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 785c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If n is zero, malloc returns a minimum-sized chunk. (The minimum 786c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size is 16 bytes on most 32bit systems, and 32 bytes on 64bit 787c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath systems.) Note that size_t is an unsigned type, so calls with 788c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath arguments that would be negative if signed are interpreted as 789c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath requests for huge amounts of space, which will often fail. The 790c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath maximum supported value of n differs across systems, but is in all 791c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath cases less than the maximum representable value of a size_t. 792c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 793c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* dlmalloc(size_t); 794c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 795c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 796c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath free(void* p) 797c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Releases the chunk of memory pointed to by p, that had been previously 798c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath allocated using malloc or a related routine such as realloc. 799c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath It has no effect if p is null. If p was not malloced or already 800c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath freed, free(p) will by default cause the current program to abort. 801c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 802c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid dlfree(void*); 803c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 804c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 805c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath calloc(size_t n_elements, size_t element_size); 806c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Returns a pointer to n_elements * element_size bytes, with all locations 807c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set to zero. 808c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 809c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* dlcalloc(size_t, size_t); 810c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 811c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 812c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath realloc(void* p, size_t n) 813c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Returns a pointer to a chunk of size n that contains the same data 814c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath as does chunk p up to the minimum of (n, p's size) bytes, or null 815c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if no space is available. 816c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 817c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The returned pointer may or may not be the same as p. The algorithm 818c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath prefers extending p in most cases when possible, otherwise it 819c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath employs the equivalent of a malloc-copy-free sequence. 820c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 821c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If p is null, realloc is equivalent to malloc. 822c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 823c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If space is not available, realloc returns null, errno is set (if on 824c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ANSI) and p is NOT freed. 825c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 826c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if n is for fewer bytes than already held by p, the newly unused 827c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath space is lopped off and freed if possible. realloc with a size 828c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath argument of zero (re)allocates a minimum-sized chunk. 829c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 830c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The old unix realloc convention of allowing the last-free'd chunk 831c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath to be used as an argument to realloc is not supported. 832c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 833c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 834c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* dlrealloc(void*, size_t); 835c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 836c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 837c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memalign(size_t alignment, size_t n); 838c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Returns a pointer to a newly allocated chunk of n bytes, aligned 839c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath in accord with the alignment argument. 840c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 841c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The alignment argument should be a power of two. If the argument is 842c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath not a power of two, the nearest greater power is used. 843c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 8-byte alignment is guaranteed by normal malloc calls, so don't 844c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bother calling memalign with an argument of 8 or less. 845c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 846c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Overreliance on memalign is a sure way to fragment space. 847c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 848c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* dlmemalign(size_t, size_t); 849c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 850c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 851c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath valloc(size_t n); 852c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Equivalent to memalign(pagesize, n), where pagesize is the page 853c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size of the system. If the pagesize is unknown, 4096 is used. 854c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 855c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* dlvalloc(size_t); 856c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 857c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 858c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mallopt(int parameter_number, int parameter_value) 859c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Sets tunable parameters The format is to provide a 860c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (parameter-number, parameter-value) pair. mallopt then sets the 861c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath corresponding parameter to the argument value if it can (i.e., so 862c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath long as the value is meaningful), and returns 1 if successful else 863c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 0. To workaround the fact that mallopt is specified to use int, 864c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath not size_t parameters, the value -1 is specially treated as the 865c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath maximum unsigned size_t value. 866c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 867c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath SVID/XPG/ANSI defines four standard param numbers for mallopt, 868c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath normally defined in malloc.h. None of these are use in this malloc, 869c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath so setting them has no effect. But this malloc also supports other 870c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath options in mallopt. See below for details. Briefly, supported 871c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath parameters are as follows (listed defaults are for "typical" 872c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath configurations). 873c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 874c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Symbol param # default allowed param values 875c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath M_TRIM_THRESHOLD -1 2*1024*1024 any (-1 disables) 876c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath M_GRANULARITY -2 page size any power of 2 >= page size 877c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath M_MMAP_THRESHOLD -3 256*1024 any (or 0 if no MMAP support) 878c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 879c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathint dlmallopt(int, int); 880c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 881c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 882c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc_footprint(); 883c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Returns the number of bytes obtained from the system. The total 884c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath number of bytes allocated by malloc, realloc etc., is less than this 885c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath value. Unlike mallinfo, this function returns only a precomputed 886c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath result, so can be called frequently to monitor memory consumption. 887c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Even if locks are otherwise defined, this function does not use them, 888c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath so results might not be up to date. 889c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 890c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t dlmalloc_footprint(void); 891c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 892c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 893c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc_max_footprint(); 894c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Returns the maximum number of bytes obtained from the system. This 895c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath value will be greater than current footprint if deallocated space 896c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath has been reclaimed by the system. The peak number of bytes allocated 897c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath by malloc, realloc etc., is less than this value. Unlike mallinfo, 898c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath this function returns only a precomputed result, so can be called 899c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath frequently to monitor memory consumption. Even if locks are 900c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath otherwise defined, this function does not use them, so results might 901c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath not be up to date. 902c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 903c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t dlmalloc_max_footprint(void); 904c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 905c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !NO_MALLINFO 906c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 907c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mallinfo() 908c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Returns (by copy) a struct containing various summary statistics: 909c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 910c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath arena: current total non-mmapped bytes allocated from system 911c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ordblks: the number of free chunks 912c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath smblks: always zero. 913c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath hblks: current number of mmapped regions 914c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath hblkhd: total bytes held in mmapped regions 915c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath usmblks: the maximum total allocated space. This will be greater 916c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath than current total if trimming has occurred. 917c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fsmblks: always zero 918c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath uordblks: current total allocated space (normal or mmapped) 919c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fordblks: total free space 920c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath keepcost: the maximum number of bytes that could ideally be released 921c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath back to system via malloc_trim. ("ideally" means that 922c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath it ignores page restrictions etc.) 923c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 924c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Because these fields are ints, but internal bookkeeping may 925c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath be kept as longs, the reported values may wrap around zero and 926c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath thus be inaccurate. 927c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 928c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstruct mallinfo dlmallinfo(void); 929c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* NO_MALLINFO */ 930c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 931c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 932c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath independent_calloc(size_t n_elements, size_t element_size, void* chunks[]); 933c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 934c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath independent_calloc is similar to calloc, but instead of returning a 935c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath single cleared space, it returns an array of pointers to n_elements 936c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath independent elements that can hold contents of size elem_size, each 937c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath of which starts out cleared, and can be independently freed, 938c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath realloc'ed etc. The elements are guaranteed to be adjacently 939c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath allocated (this is not guaranteed to occur with multiple callocs or 940c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mallocs), which may also improve cache locality in some 941c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath applications. 942c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 943c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The "chunks" argument is optional (i.e., may be null, which is 944c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath probably the most typical usage). If it is null, the returned array 945c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath is itself dynamically allocated and should also be freed when it is 946c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath no longer needed. Otherwise, the chunks array must be of at least 947c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath n_elements in length. It is filled in with the pointers to the 948c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunks. 949c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 950c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath In either case, independent_calloc returns this pointer array, or 951c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath null if the allocation failed. If n_elements is zero and "chunks" 952c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath is null, it returns a chunk representing an array with zero elements 953c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (which should be freed if not wanted). 954c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 955c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Each element must be individually freed when it is no longer 956c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath needed. If you'd like to instead be able to free all at once, you 957c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath should instead use regular calloc and assign pointers into this 958c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath space to represent elements. (In this case though, you cannot 959c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath independently free elements.) 960c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 961c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath independent_calloc simplifies and speeds up implementations of many 962c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath kinds of pools. It may also be useful when constructing large data 963c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath structures that initially have a fixed number of fixed-sized nodes, 964c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath but the number is not known at compile time, and some of the nodes 965c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath may later need to be freed. For example: 966c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 967c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct Node { int item; struct Node* next; }; 968c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 969c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct Node* build_list() { 970c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct Node** pool; 971c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int n = read_number_of_nodes_needed(); 972c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (n <= 0) return 0; 973c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pool = (struct Node**)(independent_calloc(n, sizeof(struct Node), 0); 974c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (pool == 0) die(); 975c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath // organize into a linked list... 976c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct Node* first = pool[0]; 977c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (i = 0; i < n-1; ++i) 978c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pool[i]->next = pool[i+1]; 979c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath free(pool); // Can now free the array (or not, if it is needed later) 980c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return first; 981c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 982c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 983c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid** dlindependent_calloc(size_t, size_t, void**); 984c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 985c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 986c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath independent_comalloc(size_t n_elements, size_t sizes[], void* chunks[]); 987c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 988c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath independent_comalloc allocates, all at once, a set of n_elements 989c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunks with sizes indicated in the "sizes" array. It returns 990c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath an array of pointers to these elements, each of which can be 991c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath independently freed, realloc'ed etc. The elements are guaranteed to 992c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath be adjacently allocated (this is not guaranteed to occur with 993c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath multiple callocs or mallocs), which may also improve cache locality 994c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath in some applications. 995c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 996c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The "chunks" argument is optional (i.e., may be null). If it is null 997c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the returned array is itself dynamically allocated and should also 998c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath be freed when it is no longer needed. Otherwise, the chunks array 999c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath must be of at least n_elements in length. It is filled in with the 1000c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pointers to the chunks. 1001c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1002c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath In either case, independent_comalloc returns this pointer array, or 1003c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath null if the allocation failed. If n_elements is zero and chunks is 1004c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath null, it returns a chunk representing an array with zero elements 1005c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (which should be freed if not wanted). 1006c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1007c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Each element must be individually freed when it is no longer 1008c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath needed. If you'd like to instead be able to free all at once, you 1009c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath should instead use a single regular malloc, and assign pointers at 1010c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath particular offsets in the aggregate space. (In this case though, you 1011c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath cannot independently free elements.) 1012c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1013c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath independent_comallac differs from independent_calloc in that each 1014c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath element may have a different size, and also that it does not 1015c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath automatically clear elements. 1016c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1017c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath independent_comalloc can be used to speed up allocation in cases 1018c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath where several structs or objects must always be allocated at the 1019c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath same time. For example: 1020c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1021c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct Head { ... } 1022c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct Foot { ... } 1023c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1024c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void send_message(char* msg) { 1025c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int msglen = strlen(msg); 1026c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sizes[3] = { sizeof(struct Head), msglen, sizeof(struct Foot) }; 1027c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* chunks[3]; 1028c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (independent_comalloc(3, sizes, chunks) == 0) 1029c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath die(); 1030c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct Head* head = (struct Head*)(chunks[0]); 1031c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* body = (char*)(chunks[1]); 1032c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct Foot* foot = (struct Foot*)(chunks[2]); 1033c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath // ... 1034c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1035c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1036c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath In general though, independent_comalloc is worth using only for 1037c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath larger values of n_elements. For small values, you probably won't 1038c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath detect enough difference from series of malloc calls to bother. 1039c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1040c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Overuse of independent_comalloc can increase overall memory usage, 1041c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath since it cannot reuse existing noncontiguous small chunks that 1042c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath might be available for some of the elements. 1043c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1044c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid** dlindependent_comalloc(size_t, size_t*, void**); 1045c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1046c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1047c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1048c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pvalloc(size_t n); 1049c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Equivalent to valloc(minimum-page-that-holds(n)), that is, 1050c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath round up n to nearest pagesize. 1051c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath */ 1052c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* dlpvalloc(size_t); 1053c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1054c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1055c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc_trim(size_t pad); 1056c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1057c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If possible, gives memory back to the system (via negative arguments 1058c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath to sbrk) if there is unused memory at the `high' end of the malloc 1059c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pool or in unused MMAP segments. You can call this after freeing 1060c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath large blocks of memory to potentially reduce the system-level memory 1061c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath requirements of a program. However, it cannot guarantee to reduce 1062c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memory. Under some allocation patterns, some large free blocks of 1063c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memory will be locked between two used chunks, so they cannot be 1064c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath given back to the system. 1065c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1066c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The `pad' argument to malloc_trim represents the amount of free 1067c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath trailing space to leave untrimmed. If this argument is zero, only 1068c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the minimum amount of memory to maintain internal data structures 1069c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath will be left. Non-zero arguments can be supplied to maintain enough 1070c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath trailing space to service future expected allocations without having 1071c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath to re-obtain memory from the system. 1072c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1073c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Malloc_trim returns 1 if it actually released any memory, else 0. 1074c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1075c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathint dlmalloc_trim(size_t); 1076c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1077c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1078c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc_stats(); 1079c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Prints on stderr the amount of space obtained from the system (both 1080c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath via sbrk and mmap), the maximum amount (which may be more than 1081c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath current if malloc_trim and/or munmap got called), and the current 1082c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath number of bytes allocated via malloc (or realloc, etc) but not yet 1083c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath freed. Note that this is the number of bytes allocated, not the 1084c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath number requested. It will be larger than the number requested 1085c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath because of alignment and bookkeeping overhead. Because it includes 1086c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath alignment wastage as being in use, this figure may be greater than 1087c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath zero even when no user-level chunks are allocated. 1088c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1089c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The reported current and maximum system memory can be inaccurate if 1090c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath a program makes other calls to system memory allocation functions 1091c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (normally sbrk) outside of malloc. 1092c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1093c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc_stats prints only the most commonly interesting statistics. 1094c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath More information can be obtained by calling mallinfo. 1095c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1096c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid dlmalloc_stats(void); 1097c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1098c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* ONLY_MSPACES */ 1099c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1100c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1101c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc_usable_size(void* p); 1102c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1103c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Returns the number of bytes you can actually use in 1104c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath an allocated chunk, which may be more than you requested (although 1105c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath often not) due to alignment and minimum size constraints. 1106c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath You can use this many bytes without worrying about 1107c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath overwriting other allocated objects. This is not a particularly great 1108c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath programming practice. malloc_usable_size can be more useful in 1109c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath debugging and assertions, for example: 1110c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1111c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p = malloc(n); 1112c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(malloc_usable_size(p) >= 256); 1113c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1114c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t dlmalloc_usable_size(void*); 1115c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1116c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1117c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if MSPACES 1118c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1119c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1120c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace is an opaque type representing an independent 1121c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath region of space that supports mspace_malloc, etc. 1122c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1123c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef void* mspace; 1124c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1125c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1126c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath create_mspace creates and returns a new independent space with the 1127c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath given initial capacity, or, if 0, the default granularity size. It 1128c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath returns null if there is no system memory available to create the 1129c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath space. If argument locked is non-zero, the space uses a separate 1130c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath lock to control access. The capacity of the space will grow 1131c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath dynamically as needed to service mspace_malloc requests. You can 1132c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath control the sizes of incremental increases of this space by 1133c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath compiling with a different DEFAULT_GRANULARITY or dynamically 1134c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath setting with mallopt(M_GRANULARITY, value). 1135c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1136c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathmspace create_mspace(size_t capacity, int locked); 1137c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1138c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1139c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath destroy_mspace destroys the given space, and attempts to return all 1140c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath of its memory back to the system, returning the total number of 1141c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bytes freed. After destruction, the results of access to all memory 1142c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath used by the space become undefined. 1143c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1144c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t destroy_mspace(mspace msp); 1145c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1146c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1147c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath create_mspace_with_base uses the memory supplied as the initial base 1148c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath of a new mspace. Part (less than 128*sizeof(size_t) bytes) of this 1149c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath space is used for bookkeeping, so the capacity must be at least this 1150c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath large. (Otherwise 0 is returned.) When this initial space is 1151c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath exhausted, additional memory will be obtained from the system. 1152c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Destroying this space will deallocate all additionally allocated 1153c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath space (if possible) but not the initial base. 1154c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1155c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathmspace create_mspace_with_base(void* base, size_t capacity, int locked); 1156c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1157c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1158c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_track_large_chunks controls whether requests for large chunks 1159c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath are allocated in their own untracked mmapped regions, separate from 1160c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath others in this mspace. By default large chunks are not tracked, 1161c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath which reduces fragmentation. However, such chunks are not 1162c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath necessarily released to the system upon destroy_mspace. Enabling 1163c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tracking by setting to true may increase fragmentation, but avoids 1164c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath leakage when relying on destroy_mspace to release all memory 1165c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath allocated using this space. The function returns the previous 1166c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath setting. 1167c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1168c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathint mspace_track_large_chunks(mspace msp, int enable); 1169c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1170c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1171c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1172c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_malloc behaves as malloc, but operates within 1173c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the given space. 1174c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1175c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* mspace_malloc(mspace msp, size_t bytes); 1176c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1177c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1178c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_free behaves as free, but operates within 1179c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the given space. 1180c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1181c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If compiled with FOOTERS==1, mspace_free is not actually needed. 1182c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath free may be called instead of mspace_free because freed chunks from 1183c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath any space are handled by their originating spaces. 1184c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1185c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid mspace_free(mspace msp, void* mem); 1186c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1187c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1188c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_realloc behaves as realloc, but operates within 1189c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the given space. 1190c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1191c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If compiled with FOOTERS==1, mspace_realloc is not actually 1192c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath needed. realloc may be called instead of mspace_realloc because 1193c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath realloced chunks from any space are handled by their originating 1194c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath spaces. 1195c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1196c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* mspace_realloc(mspace msp, void* mem, size_t newsize); 1197c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1198c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1199c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_calloc behaves as calloc, but operates within 1200c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the given space. 1201c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1202c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* mspace_calloc(mspace msp, size_t n_elements, size_t elem_size); 1203c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1204c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1205c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_memalign behaves as memalign, but operates within 1206c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the given space. 1207c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1208c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* mspace_memalign(mspace msp, size_t alignment, size_t bytes); 1209c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1210c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1211c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_independent_calloc behaves as independent_calloc, but 1212c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath operates within the given space. 1213c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1214c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid** mspace_independent_calloc(mspace msp, size_t n_elements, 1215c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t elem_size, void* chunks[]); 1216c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1217c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1218c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_independent_comalloc behaves as independent_comalloc, but 1219c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath operates within the given space. 1220c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1221c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid** mspace_independent_comalloc(mspace msp, size_t n_elements, 1222c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sizes[], void* chunks[]); 1223c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1224c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1225c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_footprint() returns the number of bytes obtained from the 1226c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath system for this space. 1227c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1228c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t mspace_footprint(mspace msp); 1229c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1230c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1231c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_max_footprint() returns the peak number of bytes obtained from the 1232c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath system for this space. 1233c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1234c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t mspace_max_footprint(mspace msp); 1235c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1236c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1237c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !NO_MALLINFO 1238c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1239c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_mallinfo behaves as mallinfo, but reports properties of 1240c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the given space. 1241c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1242c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstruct mallinfo mspace_mallinfo(mspace msp); 1243c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* NO_MALLINFO */ 1244c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1245c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1246c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc_usable_size(void* p) behaves the same as malloc_usable_size; 1247c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1248c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t mspace_usable_size(void* mem); 1249c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1250c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1251c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_malloc_stats behaves as malloc_stats, but reports 1252c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath properties of the given space. 1253c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1254c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid mspace_malloc_stats(mspace msp); 1255c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1256c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1257c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_trim behaves as malloc_trim, but 1258c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath operates within the given space. 1259c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1260c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathint mspace_trim(mspace msp, size_t pad); 1261c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1262c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1263c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath An alias for mallopt. 1264c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1265c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathint mspace_mallopt(int, int); 1266c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1267c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MSPACES */ 1268c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1269c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef __cplusplus 1270c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} /* end of extern "C" */ 1271c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* __cplusplus */ 1272c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1273c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1274c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ======================================================================== 1275c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath To make a fully customizable malloc.h header file, cut everything 1276c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath above this line, put into file malloc.h, edit to suit, and #include it 1277c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath on the next line, as well as in programs that use this malloc. 1278c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ======================================================================== 1279c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1280c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1281c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* #include "malloc.h" */ 1282c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1283c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/*------------------------------ internal #includes ---------------------- */ 1284c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1285c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef WIN32 1286c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#pragma warning( disable : 4146 ) /* no "unsigned" warnings */ 1287c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* WIN32 */ 1288c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1289c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <stdio.h> /* for printing in malloc_stats */ 1290c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1291c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef LACKS_ERRNO_H 1292c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <errno.h> /* for MALLOC_FAILURE_ACTION */ 1293c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* LACKS_ERRNO_H */ 1294c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/*#if FOOTERS || DEBUG 1295c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1296c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <time.h> /* for magic initialization */ 1297c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/*#endif*/ /* FOOTERS */ 1298c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef LACKS_STDLIB_H 1299c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <stdlib.h> /* for abort() */ 1300c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* LACKS_STDLIB_H */ 1301c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef DEBUG 1302c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if ABORT_ON_ASSERT_FAILURE 1303c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#undef assert 1304c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define assert(x) if(!(x)) ABORT 1305c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* ABORT_ON_ASSERT_FAILURE */ 1306c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <assert.h> 1307c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* ABORT_ON_ASSERT_FAILURE */ 1308c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* DEBUG */ 1309c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef assert 1310c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define assert(x) 1311c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 1312c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define DEBUG 0 1313c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* DEBUG */ 1314c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef LACKS_STRING_H 1315c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <string.h> /* for memset etc */ 1316c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* LACKS_STRING_H */ 1317c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if USE_BUILTIN_FFS 1318c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef LACKS_STRINGS_H 1319c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <strings.h> /* for ffs */ 1320c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* LACKS_STRINGS_H */ 1321c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_BUILTIN_FFS */ 1322c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if HAVE_MMAP 1323c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef LACKS_SYS_MMAN_H 1324c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* On some versions of linux, mremap decl in mman.h needs __USE_GNU set */ 1325c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if (defined(linux) && !defined(__USE_GNU)) 1326c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define __USE_GNU 1 1327c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <sys/mman.h> /* for mmap */ 1328c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#undef __USE_GNU 1329c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else 1330c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <sys/mman.h> /* for mmap */ 1331c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* linux */ 1332c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* LACKS_SYS_MMAN_H */ 1333c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef LACKS_FCNTL_H 1334c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <fcntl.h> 1335c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* LACKS_FCNTL_H */ 1336c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MMAP */ 1337c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef LACKS_UNISTD_H 1338c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <unistd.h> /* for sbrk, sysconf */ 1339c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* LACKS_UNISTD_H */ 1340c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !defined(__FreeBSD__) && !defined(__OpenBSD__) && !defined(__NetBSD__) 1341c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathextern void* sbrk(ptrdiff_t); 1342c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* FreeBSD etc */ 1343c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* LACKS_UNISTD_H */ 1344c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1345c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Declarations for locking */ 1346c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if USE_LOCKS 1347c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef WIN32 1348c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <pthread.h> 1349c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if defined (__SVR4) && defined (__sun) /* solaris */ 1350c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#include <thread.h> 1351c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* solaris */ 1352c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else 1353c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef _M_AMD64 1354c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* These are already defined on AMD64 builds */ 1355c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef __cplusplus 1356c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathextern "C" { 1357c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* __cplusplus */ 1358c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathLONG __cdecl _InterlockedCompareExchange(LONG volatile *Dest, LONG Exchange, LONG Comp); 1359c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathLONG __cdecl _InterlockedExchange(LONG volatile *Target, LONG Value); 1360c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef __cplusplus 1361c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1362c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* __cplusplus */ 1363c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* _M_AMD64 */ 1364c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#pragma intrinsic (_InterlockedCompareExchange) 1365c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#pragma intrinsic (_InterlockedExchange) 1366c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define interlockedcompareexchange _InterlockedCompareExchange 1367c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define interlockedexchange _InterlockedExchange 1368c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* Win32 */ 1369c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_LOCKS */ 1370c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1371c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Declarations for bit scanning on win32 */ 1372c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if defined(_MSC_VER) && _MSC_VER>=1300 1373c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef BitScanForward /* Try to avoid pulling in WinNT.h */ 1374c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef __cplusplus 1375c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathextern "C" { 1376c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* __cplusplus */ 1377c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathunsigned char _BitScanForward(unsigned long *index, unsigned long mask); 1378c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathunsigned char _BitScanReverse(unsigned long *index, unsigned long mask); 1379c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef __cplusplus 1380c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1381c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* __cplusplus */ 1382c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1383c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define BitScanForward _BitScanForward 1384c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define BitScanReverse _BitScanReverse 1385c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#pragma intrinsic(_BitScanForward) 1386c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#pragma intrinsic(_BitScanReverse) 1387c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* BitScanForward */ 1388c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* defined(_MSC_VER) && _MSC_VER>=1300 */ 1389c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1390c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef WIN32 1391c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef malloc_getpagesize 1392c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# ifdef _SC_PAGESIZE /* some SVR4 systems omit an underscore */ 1393c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# ifndef _SC_PAGE_SIZE 1394c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# define _SC_PAGE_SIZE _SC_PAGESIZE 1395c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# endif 1396c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# endif 1397c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# ifdef _SC_PAGE_SIZE 1398c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# define malloc_getpagesize sysconf(_SC_PAGE_SIZE) 1399c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# else 1400c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# if defined(BSD) || defined(DGUX) || defined(HAVE_GETPAGESIZE) 1401c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath extern size_t getpagesize(); 1402c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# define malloc_getpagesize getpagesize() 1403c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# else 1404c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# ifdef WIN32 /* use supplied emulation of getpagesize */ 1405c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# define malloc_getpagesize getpagesize() 1406c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# else 1407c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# ifndef LACKS_SYS_PARAM_H 1408c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# include <sys/param.h> 1409c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# endif 1410c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# ifdef EXEC_PAGESIZE 1411c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# define malloc_getpagesize EXEC_PAGESIZE 1412c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# else 1413c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# ifdef NBPG 1414c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# ifndef CLSIZE 1415c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# define malloc_getpagesize NBPG 1416c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# else 1417c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# define malloc_getpagesize (NBPG * CLSIZE) 1418c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# endif 1419c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# else 1420c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# ifdef NBPC 1421c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# define malloc_getpagesize NBPC 1422c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# else 1423c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# ifdef PAGESIZE 1424c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# define malloc_getpagesize PAGESIZE 1425c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# else /* just guess */ 1426c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# define malloc_getpagesize ((size_t)4096U) 1427c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# endif 1428c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# endif 1429c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# endif 1430c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# endif 1431c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# endif 1432c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# endif 1433c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath# endif 1434c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 1435c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 1436c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1437c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1438c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1439c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ------------------- size_t and alignment properties -------------------- */ 1440c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1441c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* The byte and bit size of a size_t */ 1442c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SIZE_T_SIZE (sizeof(size_t)) 1443c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SIZE_T_BITSIZE (sizeof(size_t) << 3) 1444c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1445c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Some constants coerced to size_t */ 1446c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Annoying but necessary to avoid errors on some platforms */ 1447c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SIZE_T_ZERO ((size_t)0) 1448c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SIZE_T_ONE ((size_t)1) 1449c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SIZE_T_TWO ((size_t)2) 1450c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SIZE_T_FOUR ((size_t)4) 1451c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define TWO_SIZE_T_SIZES (SIZE_T_SIZE<<1) 1452c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define FOUR_SIZE_T_SIZES (SIZE_T_SIZE<<2) 1453c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SIX_SIZE_T_SIZES (FOUR_SIZE_T_SIZES+TWO_SIZE_T_SIZES) 1454c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define HALF_MAX_SIZE_T (MAX_SIZE_T / 2U) 1455c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1456c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* The bit mask value corresponding to MALLOC_ALIGNMENT */ 1457c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define CHUNK_ALIGN_MASK (MALLOC_ALIGNMENT - SIZE_T_ONE) 1458c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1459c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* True if address a has acceptable alignment */ 1460c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define is_aligned(A) (((size_t)((A)) & (CHUNK_ALIGN_MASK)) == 0) 1461c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1462c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* the number of bytes to offset an address to align it */ 1463c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define align_offset(A)\ 1464c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((((size_t)(A) & CHUNK_ALIGN_MASK) == 0)? 0 :\ 1465c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((MALLOC_ALIGNMENT - ((size_t)(A) & CHUNK_ALIGN_MASK)) & CHUNK_ALIGN_MASK)) 1466c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1467c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* -------------------------- MMAP preliminaries ------------------------- */ 1468c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1469c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1470c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If HAVE_MORECORE or HAVE_MMAP are false, we just define calls and 1471c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath checks to fail so compiler optimizer can delete code rather than 1472c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath using so many "#if"s. 1473c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1474c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1475c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1476c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* MORECORE and MMAP must return MFAIL on failure */ 1477c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MFAIL ((void*)(MAX_SIZE_T)) 1478c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define CMFAIL ((char*)(MFAIL)) /* defined for convenience */ 1479c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1480c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if HAVE_MMAP 1481c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1482c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef WIN32 1483c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MUNMAP_DEFAULT(a, s) munmap((a), (s)) 1484c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MMAP_PROT (PROT_READ|PROT_WRITE) 1485c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !defined(MAP_ANONYMOUS) && defined(MAP_ANON) 1486c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MAP_ANONYMOUS MAP_ANON 1487c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MAP_ANON */ 1488c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef MAP_ANONYMOUS 1489c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MMAP_FLAGS (MAP_PRIVATE|MAP_ANONYMOUS) 1490c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MMAP_DEFAULT(s) mmap(0, (s), MMAP_PROT, MMAP_FLAGS, -1, 0) 1491c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* MAP_ANONYMOUS */ 1492c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1493c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Nearly all versions of mmap support MAP_ANONYMOUS, so the following 1494c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath is unlikely to be needed, but is supplied just in case. 1495c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1496c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MMAP_FLAGS (MAP_PRIVATE) 1497c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic int dev_zero_fd = -1; /* Cached file descriptor for /dev/zero. */ 1498c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MMAP_DEFAULT(s) ((dev_zero_fd < 0) ? \ 1499c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (dev_zero_fd = open("/dev/zero", O_RDWR), \ 1500c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mmap(0, (s), MMAP_PROT, MMAP_FLAGS, dev_zero_fd, 0)) : \ 1501c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mmap(0, (s), MMAP_PROT, MMAP_FLAGS, dev_zero_fd, 0)) 1502c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MAP_ANONYMOUS */ 1503c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1504c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define DIRECT_MMAP_DEFAULT(s) MMAP_DEFAULT(s) 1505c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1506c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* WIN32 */ 1507c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1508c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Win32 MMAP via VirtualAlloc */ 1509c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic FORCEINLINE void* win32mmap(size_t size) { 1510c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* ptr = VirtualAlloc(0, size, MEM_RESERVE|MEM_COMMIT, PAGE_READWRITE); 1511c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return (ptr != 0)? ptr: MFAIL; 1512c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1513c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1514c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* For direct MMAP, use MEM_TOP_DOWN to minimize interference */ 1515c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic FORCEINLINE void* win32direct_mmap(size_t size) { 1516c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* ptr = VirtualAlloc(0, size, MEM_RESERVE|MEM_COMMIT|MEM_TOP_DOWN, 1517c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath PAGE_READWRITE); 1518c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return (ptr != 0)? ptr: MFAIL; 1519c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1520c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1521c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* This function supports releasing coalesed segments */ 1522c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic FORCEINLINE int win32munmap(void* ptr, size_t size) { 1523c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MEMORY_BASIC_INFORMATION minfo; 1524c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* cptr = (char*)ptr; 1525c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (size) { 1526c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (VirtualQuery(cptr, &minfo, sizeof(minfo)) == 0) 1527c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return -1; 1528c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (minfo.BaseAddress != cptr || minfo.AllocationBase != cptr || 1529c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath minfo.State != MEM_COMMIT || minfo.RegionSize > size) 1530c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return -1; 1531c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (VirtualFree(cptr, 0, MEM_RELEASE) == 0) 1532c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return -1; 1533c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath cptr += minfo.RegionSize; 1534c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size -= minfo.RegionSize; 1535c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1536c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 1537c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1538c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1539c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MMAP_DEFAULT(s) win32mmap(s) 1540c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MUNMAP_DEFAULT(a, s) win32munmap((a), (s)) 1541c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define DIRECT_MMAP_DEFAULT(s) win32direct_mmap(s) 1542c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* WIN32 */ 1543c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MMAP */ 1544c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1545c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if HAVE_MREMAP 1546c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef WIN32 1547c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MREMAP_DEFAULT(addr, osz, nsz, mv) mremap((addr), (osz), (nsz), (mv)) 1548c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* WIN32 */ 1549c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MREMAP */ 1550c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1551c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1552c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/** 1553c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Define CALL_MORECORE 1554c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath */ 1555c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if HAVE_MORECORE 1556c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #ifdef MORECORE 1557c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_MORECORE(S) MORECORE(S) 1558c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #else /* MORECORE */ 1559c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_MORECORE(S) MORECORE_DEFAULT(S) 1560c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #endif /* MORECORE */ 1561c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* HAVE_MORECORE */ 1562c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_MORECORE(S) MFAIL 1563c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MORECORE */ 1564c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1565c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/** 1566c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Define CALL_MMAP/CALL_MUNMAP/CALL_DIRECT_MMAP 1567c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath */ 1568c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if HAVE_MMAP 1569c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define USE_MMAP_BIT (SIZE_T_ONE) 1570c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1571c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #ifdef MMAP 1572c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_MMAP(s) MMAP(s) 1573c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #else /* MMAP */ 1574c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_MMAP(s) MMAP_DEFAULT(s) 1575c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #endif /* MMAP */ 1576c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #ifdef MUNMAP 1577c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_MUNMAP(a, s) MUNMAP((a), (s)) 1578c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #else /* MUNMAP */ 1579c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_MUNMAP(a, s) MUNMAP_DEFAULT((a), (s)) 1580c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #endif /* MUNMAP */ 1581c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #ifdef DIRECT_MMAP 1582c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_DIRECT_MMAP(s) DIRECT_MMAP(s) 1583c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #else /* DIRECT_MMAP */ 1584c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_DIRECT_MMAP(s) DIRECT_MMAP_DEFAULT(s) 1585c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #endif /* DIRECT_MMAP */ 1586c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* HAVE_MMAP */ 1587c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define USE_MMAP_BIT (SIZE_T_ZERO) 1588c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1589c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define MMAP(s) MFAIL 1590c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define MUNMAP(a, s) (-1) 1591c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define DIRECT_MMAP(s) MFAIL 1592c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_DIRECT_MMAP(s) DIRECT_MMAP(s) 1593c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_MMAP(s) MMAP(s) 1594c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_MUNMAP(a, s) MUNMAP((a), (s)) 1595c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MMAP */ 1596c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1597c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/** 1598c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Define CALL_MREMAP 1599c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath */ 1600c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if HAVE_MMAP && HAVE_MREMAP 1601c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #ifdef MREMAP 1602c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_MREMAP(addr, osz, nsz, mv) MREMAP((addr), (osz), (nsz), (mv)) 1603c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #else /* MREMAP */ 1604c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_MREMAP(addr, osz, nsz, mv) MREMAP_DEFAULT((addr), (osz), (nsz), (mv)) 1605c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #endif /* MREMAP */ 1606c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* HAVE_MMAP && HAVE_MREMAP */ 1607c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define CALL_MREMAP(addr, osz, nsz, mv) MFAIL 1608c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* HAVE_MMAP && HAVE_MREMAP */ 1609c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1610c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* mstate bit set if continguous morecore disabled or failed */ 1611c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define USE_NONCONTIGUOUS_BIT (4U) 1612c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1613c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* segment bit set in create_mspace_with_base */ 1614c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define EXTERN_BIT (8U) 1615c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1616c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1617c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* --------------------------- Lock preliminaries ------------------------ */ 1618c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1619c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1620c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath When locks are defined, there is one global lock, plus 1621c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath one per-mspace lock. 1622c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1623c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The global lock_ensures that mparams.magic and other unique 1624c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mparams values are initialized only once. It also protects 1625c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sequences of calls to MORECORE. In many cases sys_alloc requires 1626c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath two calls, that should not be interleaved with calls by other 1627c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath threads. This does not protect against direct calls to MORECORE 1628c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath by other threads not using this lock, so there is still code to 1629c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath cope the best we can on interference. 1630c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1631c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Per-mspace locks surround calls to malloc, free, etc. To enable use 1632c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath in layered extensions, per-mspace locks are reentrant. 1633c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1634c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Because lock-protected regions generally have bounded times, it is 1635c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath OK to use the supplied simple spinlocks in the custom versions for 1636c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath x86. Spinlocks are likely to improve performance for lightly 1637c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath contended applications, but worsen performance under heavy 1638c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath contention. 1639c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1640c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If USE_LOCKS is > 1, the definitions of lock routines here are 1641c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bypassed, in which case you will need to define the type MLOCK_T, 1642c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and at least INITIAL_LOCK, ACQUIRE_LOCK, RELEASE_LOCK and possibly 1643c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath TRY_LOCK (which is not used in this malloc, but commonly needed in 1644c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath extensions.) You must also declare a 1645c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath static MLOCK_T malloc_global_mutex = { initialization values };. 1646c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1647c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 1648c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1649c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if USE_LOCKS == 1 1650c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1651c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if USE_SPIN_LOCKS && SPIN_LOCKS_AVAILABLE 1652c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef WIN32 1653c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1654c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Custom pthread-style spin locks on x86 and x64 for gcc */ 1655c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstruct pthread_mlock_t { 1656c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath volatile unsigned int l; 1657c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int c; 1658c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pthread_t threadid; 1659c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath}; 1660c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MLOCK_T struct pthread_mlock_t 1661c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define CURRENT_THREAD pthread_self() 1662c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define INITIAL_LOCK(sl) ((sl)->threadid = 0, (sl)->l = (sl)->c = 0, 0) 1663c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ACQUIRE_LOCK(sl) pthread_acquire_lock(sl) 1664c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define RELEASE_LOCK(sl) pthread_release_lock(sl) 1665c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define TRY_LOCK(sl) pthread_try_lock(sl) 1666c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SPINS_PER_YIELD 63 1667c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1668c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic MLOCK_T malloc_global_mutex = { 0, 0, 0}; 1669c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1670c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic FORCEINLINE int pthread_acquire_lock (MLOCK_T *sl) { 1671c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int spins = 0; 1672c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath volatile unsigned int* lp = &sl->l; 1673c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (;;) { 1674c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (*lp != 0) { 1675c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (sl->threadid == CURRENT_THREAD) { 1676c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ++sl->c; 1677c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 1678c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1679c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1680c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 1681c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* place args to cmpxchgl in locals to evade oddities in some gccs */ 1682c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int cmp = 0; 1683c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int val = 1; 1684c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int ret; 1685c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath __asm__ __volatile__ ("lock; cmpxchgl %1, %2" 1686c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath : "=a" (ret) 1687c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath : "r" (val), "m" (*(lp)), "0"(cmp) 1688c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath : "memory", "cc"); 1689c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!ret) { 1690c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(!sl->threadid); 1691c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sl->threadid = CURRENT_THREAD; 1692c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sl->c = 1; 1693c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 1694c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1695c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1696c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((++spins & SPINS_PER_YIELD) == 0) { 1697c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if defined (__SVR4) && defined (__sun) /* solaris */ 1698c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath thr_yield(); 1699c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else 1700c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if defined(__linux__) || defined(__FreeBSD__) || defined(__APPLE__) 1701c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sched_yield(); 1702c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* no-op yield on unknown systems */ 1703c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ; 1704c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* __linux__ || __FreeBSD__ || __APPLE__ */ 1705c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* solaris */ 1706c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1707c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1708c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1709c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1710c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic FORCEINLINE void pthread_release_lock (MLOCK_T *sl) { 1711c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath volatile unsigned int* lp = &sl->l; 1712c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(*lp != 0); 1713c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(sl->threadid == CURRENT_THREAD); 1714c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (--sl->c == 0) { 1715c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sl->threadid = 0; 1716c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int prev = 0; 1717c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int ret; 1718c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath __asm__ __volatile__ ("lock; xchgl %0, %1" 1719c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath : "=r" (ret) 1720c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath : "m" (*(lp)), "0"(prev) 1721c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath : "memory"); 1722c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1723c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1724c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1725c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic FORCEINLINE int pthread_try_lock (MLOCK_T *sl) { 1726c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath volatile unsigned int* lp = &sl->l; 1727c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (*lp != 0) { 1728c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (sl->threadid == CURRENT_THREAD) { 1729c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ++sl->c; 1730c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 1; 1731c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1732c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1733c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 1734c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int cmp = 0; 1735c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int val = 1; 1736c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int ret; 1737c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath __asm__ __volatile__ ("lock; cmpxchgl %1, %2" 1738c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath : "=a" (ret) 1739c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath : "r" (val), "m" (*(lp)), "0"(cmp) 1740c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath : "memory", "cc"); 1741c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!ret) { 1742c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(!sl->threadid); 1743c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sl->threadid = CURRENT_THREAD; 1744c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sl->c = 1; 1745c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 1; 1746c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1747c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1748c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 1749c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1750c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1751c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1752c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* WIN32 */ 1753c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Custom win32-style spin locks on x86 and x64 for MSC */ 1754c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstruct win32_mlock_t { 1755c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath volatile long l; 1756c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int c; 1757c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath long threadid; 1758c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath}; 1759c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1760c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MLOCK_T struct win32_mlock_t 1761c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define CURRENT_THREAD GetCurrentThreadId() 1762c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define INITIAL_LOCK(sl) ((sl)->threadid = 0, (sl)->l = (sl)->c = 0, 0) 1763c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ACQUIRE_LOCK(sl) win32_acquire_lock(sl) 1764c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define RELEASE_LOCK(sl) win32_release_lock(sl) 1765c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define TRY_LOCK(sl) win32_try_lock(sl) 1766c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SPINS_PER_YIELD 63 1767c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1768c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic MLOCK_T malloc_global_mutex = { 0, 0, 0}; 1769c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1770c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic FORCEINLINE int win32_acquire_lock (MLOCK_T *sl) { 1771c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int spins = 0; 1772c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (;;) { 1773c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (sl->l != 0) { 1774c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (sl->threadid == CURRENT_THREAD) { 1775c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ++sl->c; 1776c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 1777c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1778c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1779c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 1780c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!interlockedexchange(&sl->l, 1)) { 1781c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(!sl->threadid); 1782c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sl->threadid = CURRENT_THREAD; 1783c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sl->c = 1; 1784c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 1785c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1786c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1787c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((++spins & SPINS_PER_YIELD) == 0) 1788c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath SleepEx(0, FALSE); 1789c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1790c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1791c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1792c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic FORCEINLINE void win32_release_lock (MLOCK_T *sl) { 1793c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(sl->threadid == CURRENT_THREAD); 1794c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(sl->l != 0); 1795c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (--sl->c == 0) { 1796c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sl->threadid = 0; 1797c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath interlockedexchange (&sl->l, 0); 1798c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1799c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1800c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1801c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic FORCEINLINE int win32_try_lock (MLOCK_T *sl) { 1802c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (sl->l != 0) { 1803c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (sl->threadid == CURRENT_THREAD) { 1804c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ++sl->c; 1805c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 1; 1806c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1807c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1808c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 1809c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!interlockedexchange(&sl->l, 1)){ 1810c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(!sl->threadid); 1811c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sl->threadid = CURRENT_THREAD; 1812c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sl->c = 1; 1813c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 1; 1814c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1815c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1816c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 1817c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1818c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1819c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* WIN32 */ 1820c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* USE_SPIN_LOCKS */ 1821c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1822c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef WIN32 1823c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* pthreads-based locks */ 1824c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1825c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MLOCK_T pthread_mutex_t 1826c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define CURRENT_THREAD pthread_self() 1827c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define INITIAL_LOCK(sl) pthread_init_lock(sl) 1828c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ACQUIRE_LOCK(sl) pthread_mutex_lock(sl) 1829c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define RELEASE_LOCK(sl) pthread_mutex_unlock(sl) 1830c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define TRY_LOCK(sl) (!pthread_mutex_trylock(sl)) 1831c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1832c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic MLOCK_T malloc_global_mutex = PTHREAD_MUTEX_INITIALIZER; 1833c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1834c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Cope with old-style linux recursive lock initialization by adding */ 1835c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* skipped internal declaration from pthread.h */ 1836c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef linux 1837c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef PTHREAD_MUTEX_RECURSIVE 1838c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathextern int pthread_mutexattr_setkind_np __P ((pthread_mutexattr_t *__attr, 1839c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int __kind)); 1840c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define PTHREAD_MUTEX_RECURSIVE PTHREAD_MUTEX_RECURSIVE_NP 1841c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define pthread_mutexattr_settype(x,y) pthread_mutexattr_setkind_np(x,y) 1842c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 1843c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 1844c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1845c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic int pthread_init_lock (MLOCK_T *sl) { 1846c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pthread_mutexattr_t attr; 1847c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (pthread_mutexattr_init(&attr)) return 1; 1848c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE)) return 1; 1849c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (pthread_mutex_init(sl, &attr)) return 1; 1850c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (pthread_mutexattr_destroy(&attr)) return 1; 1851c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 1852c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1853c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1854c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* WIN32 */ 1855c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Win32 critical sections */ 1856c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MLOCK_T CRITICAL_SECTION 1857c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define CURRENT_THREAD GetCurrentThreadId() 1858c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define INITIAL_LOCK(s) (!InitializeCriticalSectionAndSpinCount((s), 0x80000000|4000)) 1859c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ACQUIRE_LOCK(s) (EnterCriticalSection(sl), 0) 1860c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define RELEASE_LOCK(s) LeaveCriticalSection(sl) 1861c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define TRY_LOCK(s) TryEnterCriticalSection(sl) 1862c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define NEED_GLOBAL_LOCK_INIT 1863c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1864c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic MLOCK_T malloc_global_mutex; 1865c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic volatile long malloc_global_mutex_status; 1866c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1867c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Use spin loop to initialize global lock */ 1868c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void init_malloc_global_mutex() { 1869c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (;;) { 1870c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath long stat = malloc_global_mutex_status; 1871c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (stat > 0) 1872c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return; 1873c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* transition to < 0 while initializing, then to > 0) */ 1874c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (stat == 0 && 1875c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath interlockedcompareexchange(&malloc_global_mutex_status, -1, 0) == 0) { 1876c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath InitializeCriticalSection(&malloc_global_mutex); 1877c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath interlockedexchange(&malloc_global_mutex_status,1); 1878c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return; 1879c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1880c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath SleepEx(0, FALSE); 1881c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 1882c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 1883c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1884c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* WIN32 */ 1885c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_SPIN_LOCKS */ 1886c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_LOCKS == 1 */ 1887c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1888c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ----------------------- User-defined locks ------------------------ */ 1889c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1890c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if USE_LOCKS > 1 1891c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Define your own lock implementation here */ 1892c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* #define INITIAL_LOCK(sl) ... */ 1893c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* #define ACQUIRE_LOCK(sl) ... */ 1894c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* #define RELEASE_LOCK(sl) ... */ 1895c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* #define TRY_LOCK(sl) ... */ 1896c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* static MLOCK_T malloc_global_mutex = ... */ 1897c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_LOCKS > 1 */ 1898c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1899c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ----------------------- Lock-based state ------------------------ */ 1900c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1901c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if USE_LOCKS 1902c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define USE_LOCK_BIT (2U) 1903c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* USE_LOCKS */ 1904c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define USE_LOCK_BIT (0U) 1905c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define INITIAL_LOCK(l) 1906c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_LOCKS */ 1907c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1908c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if USE_LOCKS 1909c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef ACQUIRE_MALLOC_GLOBAL_LOCK 1910c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ACQUIRE_MALLOC_GLOBAL_LOCK() ACQUIRE_LOCK(&malloc_global_mutex); 1911c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 1912c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef RELEASE_MALLOC_GLOBAL_LOCK 1913c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define RELEASE_MALLOC_GLOBAL_LOCK() RELEASE_LOCK(&malloc_global_mutex); 1914c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 1915c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* USE_LOCKS */ 1916c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ACQUIRE_MALLOC_GLOBAL_LOCK() 1917c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define RELEASE_MALLOC_GLOBAL_LOCK() 1918c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_LOCKS */ 1919c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1920c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1921c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ----------------------- Chunk representations ------------------------ */ 1922c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1923c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 1924c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (The following includes lightly edited explanations by Colin Plumb.) 1925c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1926c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The malloc_chunk declaration below is misleading (but accurate and 1927c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath necessary). It declares a "view" into memory allowing access to 1928c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath necessary fields at known offsets from a given base. 1929c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1930c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Chunks of memory are maintained using a `boundary tag' method as 1931c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath originally described by Knuth. (See the paper by Paul Wilson 1932c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ftp://ftp.cs.utexas.edu/pub/garbage/allocsrv.ps for a survey of such 1933c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath techniques.) Sizes of free chunks are stored both in the front of 1934c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath each chunk and at the end. This makes consolidating fragmented 1935c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunks into bigger chunks fast. The head fields also hold bits 1936c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath representing whether chunks are free or in use. 1937c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1938c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Here are some pictures to make it clearer. They are "exploded" to 1939c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath show that the state of a chunk can be thought of as extending from 1940c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the high 31 bits of the head field of its header through the 1941c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath prev_foot and PINUSE_BIT bit of the following chunk header. 1942c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1943c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath A chunk that's in use looks like: 1944c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1945c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1946c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Size of previous chunk (if P = 0) | 1947c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1948c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |P| 1949c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Size of this chunk 1| +-+ 1950c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1951c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | | 1952c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +- -+ 1953c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | | 1954c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +- -+ 1955c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | : 1956c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +- size - sizeof(size_t) available payload bytes -+ 1957c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath : | 1958c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk-> +- -+ 1959c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | | 1960c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1961c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1| 1962c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Size of next chunk (may or may not be in use) | +-+ 1963c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1964c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1965c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath And if it's free, it looks like this: 1966c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1967c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk-> +- -+ 1968c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | User payload (must be in use, or we would have merged!) | 1969c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1970c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |P| 1971c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Size of this chunk 0| +-+ 1972c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1973c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Next pointer | 1974c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1975c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Prev pointer | 1976c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1977c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | : 1978c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +- size - sizeof(struct chunk) unused bytes -+ 1979c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath : | 1980c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1981c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Size of this chunk | 1982c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1983c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| 1984c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Size of next chunk (must be in use, or we would have merged)| +-+ 1985c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1986c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | : 1987c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +- User payload -+ 1988c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath : | 1989c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1990c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath |0| 1991c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+ 1992c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Note that since we always merge adjacent free chunks, the chunks 1993c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath adjacent to a free chunk must be in use. 1994c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1995c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Given a pointer to a chunk (which can be derived trivially from the 1996c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath payload pointer) we can, in O(1) time, find out whether the adjacent 1997c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunks are free, and if so, unlink them from the lists that they 1998c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath are on and merge them with the current chunk. 1999c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2000c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Chunks always begin on even word boundaries, so the mem portion 2001c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (which is returned to the user) is also on an even word boundary, and 2002c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath thus at least double-word aligned. 2003c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2004c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The P (PINUSE_BIT) bit, stored in the unused low-order bit of the 2005c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk size (which is always a multiple of two words), is an in-use 2006c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bit for the *previous* chunk. If that bit is *clear*, then the 2007c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath word before the current chunk size contains the previous chunk 2008c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size, and can be used to find the front of the previous chunk. 2009c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The very first chunk allocated always has this bit set, preventing 2010c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath access to non-existent (or non-owned) memory. If pinuse is set for 2011c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath any given chunk, then you CANNOT determine the size of the 2012c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath previous chunk, and might even get a memory addressing fault when 2013c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath trying to do so. 2014c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2015c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The C (CINUSE_BIT) bit, stored in the unused second-lowest bit of 2016c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the chunk size redundantly records whether the current chunk is 2017c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath inuse (unless the chunk is mmapped). This redundancy enables usage 2018c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath checks within free and realloc, and reduces indirection when freeing 2019c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and consolidating chunks. 2020c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2021c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Each freshly allocated chunk must have both cinuse and pinuse set. 2022c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath That is, each allocated chunk borders either a previously allocated 2023c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and still in-use chunk, or the base of its memory arena. This is 2024c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensured by making all allocations from the the `lowest' part of any 2025c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath found chunk. Further, no free chunk physically borders another one, 2026c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath so each free chunk is known to be preceded and followed by either 2027c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath inuse chunks or the ends of memory. 2028c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2029c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Note that the `foot' of the current chunk is actually represented 2030c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath as the prev_foot of the NEXT chunk. This makes it easier to 2031c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath deal with alignments etc but can be very confusing when trying 2032c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath to extend or adapt this code. 2033c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2034c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The exceptions to all this are 2035c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2036c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1. The special chunk `top' is the top-most available chunk (i.e., 2037c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the one bordering the end of available memory). It is treated 2038c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath specially. Top is never included in any bin, is used only if 2039c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath no other chunk is available, and is released back to the 2040c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath system if it is very large (see M_TRIM_THRESHOLD). In effect, 2041c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the top chunk is treated as larger (and thus less well 2042c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fitting) than any other available chunk. The top chunk 2043c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath doesn't update its trailing size field since there is no next 2044c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath contiguous chunk that would have to index off it. However, 2045c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath space is still allocated for it (TOP_FOOT_SIZE) to enable 2046c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath separation or merging when space is extended. 2047c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2048c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3. Chunks allocated via mmap, have both cinuse and pinuse bits 2049c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath cleared in their head fields. Because they are allocated 2050c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath one-by-one, each must carry its own prev_foot field, which is 2051c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath also used to hold the offset this chunk has within its mmapped 2052c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath region, which is needed to preserve alignment. Each mmapped 2053c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk is trailed by the first two fields of a fake next-chunk 2054c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for sake of usage checks. 2055c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2056c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 2057c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2058c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstruct malloc_chunk { 2059c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t prev_foot; /* Size of previous chunk (if free). */ 2060c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t head; /* Size and inuse bits. */ 2061c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct malloc_chunk* fd; /* double links -- used only if free. */ 2062c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct malloc_chunk* bk; 2063c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath}; 2064c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2065c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef struct malloc_chunk mchunk; 2066c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef struct malloc_chunk* mchunkptr; 2067c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef struct malloc_chunk* sbinptr; /* The type of bins of chunks */ 2068c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef unsigned int bindex_t; /* Described below */ 2069c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef unsigned int binmap_t; /* Described below */ 2070c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef unsigned int flag_t; /* The type of various bit flag sets */ 2071c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2072c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ------------------- Chunks sizes and alignments ----------------------- */ 2073c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2074c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MCHUNK_SIZE (sizeof(mchunk)) 2075c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2076c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if FOOTERS 2077c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define CHUNK_OVERHEAD (TWO_SIZE_T_SIZES) 2078c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* FOOTERS */ 2079c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define CHUNK_OVERHEAD (SIZE_T_SIZE) 2080c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* FOOTERS */ 2081c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2082c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* MMapped chunks need a second word of overhead ... */ 2083c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MMAP_CHUNK_OVERHEAD (TWO_SIZE_T_SIZES) 2084c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ... and additional padding for fake next-chunk at foot */ 2085c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MMAP_FOOT_PAD (FOUR_SIZE_T_SIZES) 2086c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2087c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* The smallest size we can malloc is an aligned minimal chunk */ 2088c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MIN_CHUNK_SIZE\ 2089c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((MCHUNK_SIZE + CHUNK_ALIGN_MASK) & ~CHUNK_ALIGN_MASK) 2090c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2091c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* conversion from malloc headers to user pointers, and back */ 2092c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define chunk2mem(p) ((void*)((char*)(p) + TWO_SIZE_T_SIZES)) 2093c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define mem2chunk(mem) ((mchunkptr)((char*)(mem) - TWO_SIZE_T_SIZES)) 2094c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* chunk associated with aligned address A */ 2095c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define align_as_chunk(A) (mchunkptr)((A) + align_offset(chunk2mem(A))) 2096c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2097c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Bounds on request (not chunk) sizes. */ 2098c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MAX_REQUEST ((-MIN_CHUNK_SIZE) << 2) 2099c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MIN_REQUEST (MIN_CHUNK_SIZE - CHUNK_OVERHEAD - SIZE_T_ONE) 2100c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2101c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* pad request bytes into a usable size */ 2102c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define pad_request(req) \ 2103c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (((req) + CHUNK_OVERHEAD + CHUNK_ALIGN_MASK) & ~CHUNK_ALIGN_MASK) 2104c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2105c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* pad request, checking for minimum (but not maximum) */ 2106c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define request2size(req) \ 2107c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (((req) < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(req)) 2108c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2109c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2110c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ------------------ Operations on head and foot fields ----------------- */ 2111c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2112c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 2113c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The head field of a chunk is or'ed with PINUSE_BIT when previous 2114c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath adjacent chunk in use, and or'ed with CINUSE_BIT if this chunk is in 2115c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath use, unless mmapped, in which case both bits are cleared. 2116c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2117c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath FLAG4_BIT is not used by this malloc, but might be useful in extensions. 2118c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 2119c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2120c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define PINUSE_BIT (SIZE_T_ONE) 2121c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define CINUSE_BIT (SIZE_T_TWO) 2122c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define FLAG4_BIT (SIZE_T_FOUR) 2123c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define INUSE_BITS (PINUSE_BIT|CINUSE_BIT) 2124c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define FLAG_BITS (PINUSE_BIT|CINUSE_BIT|FLAG4_BIT) 2125c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2126c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Head value for fenceposts */ 2127c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define FENCEPOST_HEAD (INUSE_BITS|SIZE_T_SIZE) 2128c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2129c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* extraction of fields from head words */ 2130c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define cinuse(p) ((p)->head & CINUSE_BIT) 2131c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define pinuse(p) ((p)->head & PINUSE_BIT) 2132c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define is_inuse(p) (((p)->head & INUSE_BITS) != PINUSE_BIT) 2133c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define is_mmapped(p) (((p)->head & INUSE_BITS) == 0) 2134c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2135c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define chunksize(p) ((p)->head & ~(FLAG_BITS)) 2136c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2137c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define clear_pinuse(p) ((p)->head &= ~PINUSE_BIT) 2138c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2139c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Treat space at ptr +/- offset as a chunk */ 2140c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define chunk_plus_offset(p, s) ((mchunkptr)(((char*)(p)) + (s))) 2141c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define chunk_minus_offset(p, s) ((mchunkptr)(((char*)(p)) - (s))) 2142c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2143c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Ptr to next or previous physical malloc_chunk. */ 2144c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define next_chunk(p) ((mchunkptr)( ((char*)(p)) + ((p)->head & ~FLAG_BITS))) 2145c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define prev_chunk(p) ((mchunkptr)( ((char*)(p)) - ((p)->prev_foot) )) 2146c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2147c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* extract next chunk's pinuse bit */ 2148c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define next_pinuse(p) ((next_chunk(p)->head) & PINUSE_BIT) 2149c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2150c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Get/set size at footer */ 2151c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define get_foot(p, s) (((mchunkptr)((char*)(p) + (s)))->prev_foot) 2152c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define set_foot(p, s) (((mchunkptr)((char*)(p) + (s)))->prev_foot = (s)) 2153c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2154c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Set size, pinuse bit, and foot */ 2155c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define set_size_and_pinuse_of_free_chunk(p, s)\ 2156c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((p)->head = (s|PINUSE_BIT), set_foot(p, s)) 2157c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2158c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Set size, pinuse bit, foot, and clear next pinuse */ 2159c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define set_free_with_pinuse(p, s, n)\ 2160c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (clear_pinuse(n), set_size_and_pinuse_of_free_chunk(p, s)) 2161c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2162c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Get the internal overhead associated with chunk p */ 2163c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define overhead_for(p)\ 2164c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (is_mmapped(p)? MMAP_CHUNK_OVERHEAD : CHUNK_OVERHEAD) 2165c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2166c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Return true if malloced space is not necessarily cleared */ 2167c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if MMAP_CLEARS 2168c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define calloc_must_clear(p) (!is_mmapped(p)) 2169c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* MMAP_CLEARS */ 2170c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define calloc_must_clear(p) (1) 2171c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MMAP_CLEARS */ 2172c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2173c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ---------------------- Overlaid data structures ----------------------- */ 2174c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2175c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 2176c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath When chunks are not in use, they are treated as nodes of either 2177c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath lists or trees. 2178c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2179c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath "Small" chunks are stored in circular doubly-linked lists, and look 2180c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath like this: 2181c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2182c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2183c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Size of previous chunk | 2184c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2185c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath `head:' | Size of chunk, in bytes |P| 2186c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2187c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Forward pointer to next chunk in list | 2188c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2189c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Back pointer to previous chunk in list | 2190c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2191c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Unused space (may be 0 bytes long) . 2192c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath . . 2193c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath . | 2194c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathnextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2195c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath `foot:' | Size of chunk, in bytes | 2196c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2197c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2198c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Larger chunks are kept in a form of bitwise digital trees (aka 2199c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tries) keyed on chunksizes. Because malloc_tree_chunks are only for 2200c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath free chunks greater than 256 bytes, their size doesn't impose any 2201c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath constraints on user chunk sizes. Each node looks like: 2202c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2203c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2204c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Size of previous chunk | 2205c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2206c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath `head:' | Size of chunk, in bytes |P| 2207c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2208c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Forward pointer to next chunk of same size | 2209c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2210c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Back pointer to previous chunk of same size | 2211c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2212c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Pointer to left child (child[0]) | 2213c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2214c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Pointer to right child (child[1]) | 2215c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2216c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Pointer to parent | 2217c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2218c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | bin index of this chunk | 2219c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2220c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath | Unused space . 2221c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath . | 2222c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathnextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2223c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath `foot:' | Size of chunk, in bytes | 2224c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2225c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2226c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Each tree holding treenodes is a tree of unique chunk sizes. Chunks 2227c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath of the same size are arranged in a circularly-linked list, with only 2228c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the oldest chunk (the next to be used, in our FIFO ordering) 2229c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath actually in the tree. (Tree members are distinguished by a non-null 2230c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath parent pointer.) If a chunk with the same size an an existing node 2231c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath is inserted, it is linked off the existing node using pointers that 2232c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath work in the same way as fd/bk pointers of small chunks. 2233c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2234c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Each tree contains a power of 2 sized range of chunk sizes (the 2235c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath smallest is 0x100 <= x < 0x180), which is is divided in half at each 2236c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tree level, with the chunks in the smaller half of the range (0x100 2237c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath <= x < 0x140 for the top nose) in the left subtree and the larger 2238c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath half (0x140 <= x < 0x180) in the right subtree. This is, of course, 2239c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath done by inspecting individual bits. 2240c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2241c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Using these rules, each node's left subtree contains all smaller 2242c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sizes than its right subtree. However, the node at the root of each 2243c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath subtree has no particular ordering relationship to either. (The 2244c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath dividing line between the subtree sizes is based on trie relation.) 2245c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If we remove the last chunk of a given size from the interior of the 2246c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tree, we need to replace it with a leaf node. The tree ordering 2247c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath rules permit a node to be replaced by any leaf below it. 2248c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2249c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The smallest chunk in a tree (a common operation in a best-fit 2250c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath allocator) can be found by walking a path to the leftmost leaf in 2251c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the tree. Unlike a usual binary tree, where we follow left child 2252c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pointers until we reach a null, here we follow the right child 2253c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pointer any time the left one is null, until we reach a leaf with 2254c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath both child pointers null. The smallest chunk in the tree will be 2255c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath somewhere along that path. 2256c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2257c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The worst case number of steps to add, find, or remove a node is 2258c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bounded by the number of bits differentiating chunks within 2259c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bins. Under current bin calculations, this ranges from 6 up to 21 2260c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (for 32 bit sizes) or up to 53 (for 64 bit sizes). The typical case 2261c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath is of course much better. 2262c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 2263c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2264c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstruct malloc_tree_chunk { 2265c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* The first four fields must be compatible with malloc_chunk */ 2266c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t prev_foot; 2267c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t head; 2268c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct malloc_tree_chunk* fd; 2269c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct malloc_tree_chunk* bk; 2270c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2271c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct malloc_tree_chunk* child[2]; 2272c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct malloc_tree_chunk* parent; 2273c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t index; 2274c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath}; 2275c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2276c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef struct malloc_tree_chunk tchunk; 2277c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef struct malloc_tree_chunk* tchunkptr; 2278c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef struct malloc_tree_chunk* tbinptr; /* The type of bins of trees */ 2279c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2280c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* A little helper macro for trees */ 2281c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define leftmost_child(t) ((t)->child[0] != 0? (t)->child[0] : (t)->child[1]) 2282c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2283c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ----------------------------- Segments -------------------------------- */ 2284c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2285c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 2286c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Each malloc space may include non-contiguous segments, held in a 2287c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath list headed by an embedded malloc_segment record representing the 2288c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath top-most space. Segments also include flags holding properties of 2289c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the space. Large chunks that are directly allocated by mmap are not 2290c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath included in this list. They are instead independently created and 2291c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath destroyed without otherwise keeping track of them. 2292c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2293c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Segment management mainly comes into play for spaces allocated by 2294c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MMAP. Any call to MMAP might or might not return memory that is 2295c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath adjacent to an existing segment. MORECORE normally contiguously 2296c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath extends the current space, so this space is almost always adjacent, 2297c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath which is simpler and faster to deal with. (This is why MORECORE is 2298c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath used preferentially to MMAP when both are available -- see 2299c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sys_alloc.) When allocating using MMAP, we don't use any of the 2300c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath hinting mechanisms (inconsistently) supported in various 2301c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath implementations of unix mmap, or distinguish reserving from 2302c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath committing memory. Instead, we just ask for space, and exploit 2303c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath contiguity when we get it. It is probably possible to do 2304c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath better than this on some systems, but no general scheme seems 2305c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath to be significantly better. 2306c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2307c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Management entails a simpler variant of the consolidation scheme 2308c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath used for chunks to reduce fragmentation -- new adjacent memory is 2309c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath normally prepended or appended to an existing segment. However, 2310c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath there are limitations compared to chunk consolidation that mostly 2311c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath reflect the fact that segment processing is relatively infrequent 2312c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (occurring only when getting memory from system) and that we 2313c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath don't expect to have huge numbers of segments: 2314c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2315c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Segments are not indexed, so traversal requires linear scans. (It 2316c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath would be possible to index these, but is not worth the extra 2317c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath overhead and complexity for most programs on most platforms.) 2318c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * New segments are only appended to old ones when holding top-most 2319c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memory; if they cannot be prepended to others, they are held in 2320c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath different segments. 2321c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2322c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Except for the top-most segment of an mstate, each segment record 2323c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath is kept at the tail of its segment. Segments are added by pushing 2324c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath segment records onto the list headed by &mstate.seg for the 2325c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath containing mstate. 2326c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2327c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Segment flags control allocation/merge/deallocation policies: 2328c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * If EXTERN_BIT set, then we did not allocate this segment, 2329c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and so should not try to deallocate or merge with others. 2330c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (This currently holds only for the initial segment passed 2331c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath into create_mspace_with_base.) 2332c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * If USE_MMAP_BIT set, the segment may be merged with 2333c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath other surrounding mmapped segments and trimmed/de-allocated 2334c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath using munmap. 2335c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * If neither bit is set, then the segment was obtained using 2336c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MORECORE so can be merged with surrounding MORECORE'd segments 2337c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and deallocated/trimmed using MORECORE with negative arguments. 2338c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 2339c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2340c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstruct malloc_segment { 2341c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* base; /* base address */ 2342c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t size; /* allocated size */ 2343c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct malloc_segment* next; /* ptr to next segment */ 2344c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath flag_t sflags; /* mmap and extern flag */ 2345c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath}; 2346c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2347c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define is_mmapped_segment(S) ((S)->sflags & USE_MMAP_BIT) 2348c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define is_extern_segment(S) ((S)->sflags & EXTERN_BIT) 2349c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2350c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef struct malloc_segment msegment; 2351c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef struct malloc_segment* msegmentptr; 2352c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2353c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ---------------------------- malloc_state ----------------------------- */ 2354c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2355c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 2356c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath A malloc_state holds all of the bookkeeping for a space. 2357c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The main fields are: 2358c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2359c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Top 2360c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The topmost chunk of the currently active segment. Its size is 2361c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath cached in topsize. The actual size of topmost space is 2362c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath topsize+TOP_FOOT_SIZE, which includes space reserved for adding 2363c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fenceposts and segment records if necessary when getting more 2364c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath space from the system. The size at which to autotrim top is 2365c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath cached from mparams in trim_check, except that it is disabled if 2366c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath an autotrim fails. 2367c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2368c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Designated victim (dv) 2369c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath This is the preferred chunk for servicing small requests that 2370c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath don't have exact fits. It is normally the chunk split off most 2371c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath recently to service another small request. Its size is cached in 2372c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath dvsize. The link fields of this chunk are not maintained since it 2373c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath is not kept in a bin. 2374c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2375c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath SmallBins 2376c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath An array of bin headers for free chunks. These bins hold chunks 2377c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath with sizes less than MIN_LARGE_SIZE bytes. Each bin contains 2378c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunks of all the same size, spaced 8 bytes apart. To simplify 2379c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath use in double-linked lists, each bin header acts as a malloc_chunk 2380c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pointing to the real first node, if it exists (else pointing to 2381c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath itself). This avoids special-casing for headers. But to avoid 2382c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath waste, we allocate only the fd/bk pointers of bins, and then use 2383c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath repositioning tricks to treat these as the fields of a chunk. 2384c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2385c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath TreeBins 2386c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Treebins are pointers to the roots of trees holding a range of 2387c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sizes. There are 2 equally spaced treebins for each power of two 2388c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath from TREE_SHIFT to TREE_SHIFT+16. The last bin holds anything 2389c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath larger. 2390c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2391c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Bin maps 2392c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath There is one bit map for small bins ("smallmap") and one for 2393c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath treebins ("treemap). Each bin sets its bit when non-empty, and 2394c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath clears the bit when empty. Bit operations are then used to avoid 2395c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bin-by-bin searching -- nearly all "search" is done without ever 2396c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath looking at bins that won't be selected. The bit maps 2397c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath conservatively use 32 bits per map word, even if on 64bit system. 2398c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath For a good description of some of the bit-based techniques used 2399c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath here, see Henry S. Warren Jr's book "Hacker's Delight" (and 2400c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath supplement at http://hackersdelight.org/). Many of these are 2401c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath intended to reduce the branchiness of paths through malloc etc, as 2402c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath well as to reduce the number of memory locations read or written. 2403c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2404c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Segments 2405c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath A list of segments headed by an embedded malloc_segment record 2406c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath representing the initial space. 2407c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2408c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Address check support 2409c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The least_addr field is the least address ever obtained from 2410c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MORECORE or MMAP. Attempted frees and reallocs of any address less 2411c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath than this are trapped (unless INSECURE is defined). 2412c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2413c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Magic tag 2414c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath A cross-check field that should always hold same value as mparams.magic. 2415c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2416c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Flags 2417c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Bits recording whether to use MMAP, locks, or contiguous MORECORE 2418c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2419c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Statistics 2420c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Each space keeps track of current and maximum system memory 2421c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath obtained via MORECORE or MMAP. 2422c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2423c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Trim support 2424c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Fields holding the amount of unused topmost memory that should trigger 2425c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath timming, and a counter to force periodic scanning to release unused 2426c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath non-topmost segments. 2427c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2428c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Locking 2429c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If USE_LOCKS is defined, the "mutex" lock is acquired and released 2430c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath around every public call using this mspace. 2431c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2432c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Extension support 2433c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath A void* pointer and a size_t field that can be used to help implement 2434c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath extensions to this malloc. 2435c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 2436c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2437c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Bin types, widths and sizes */ 2438c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define NSMALLBINS (32U) 2439c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define NTREEBINS (32U) 2440c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SMALLBIN_SHIFT (3U) 2441c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SMALLBIN_WIDTH (SIZE_T_ONE << SMALLBIN_SHIFT) 2442c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define TREEBIN_SHIFT (8U) 2443c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MIN_LARGE_SIZE (SIZE_T_ONE << TREEBIN_SHIFT) 2444c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MAX_SMALL_SIZE (MIN_LARGE_SIZE - SIZE_T_ONE) 2445c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MAX_SMALL_REQUEST (MAX_SMALL_SIZE - CHUNK_ALIGN_MASK - CHUNK_OVERHEAD) 2446c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2447c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstruct malloc_state { 2448c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath binmap_t smallmap; 2449c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath binmap_t treemap; 2450c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t dvsize; 2451c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t topsize; 2452c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* least_addr; 2453c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr dv; 2454c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr top; 2455c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t trim_check; 2456c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t release_checks; 2457c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t magic; 2458c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr smallbins[(NSMALLBINS+1)*2]; 2459c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tbinptr treebins[NTREEBINS]; 2460c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t footprint; 2461c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t max_footprint; 2462c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath flag_t mflags; 2463c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if USE_LOCKS 2464c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MLOCK_T mutex; /* locate lock among fields that rarely change */ 2465c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_LOCKS */ 2466c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegment seg; 2467c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* extp; /* Unused but available for extensions */ 2468c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t exts; 2469c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath}; 2470c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2471c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathtypedef struct malloc_state* mstate; 2472c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2473c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ------------- Global malloc_state and malloc_params ------------------- */ 2474c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2475c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 2476c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc_params holds global properties, including those that can be 2477c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath dynamically set using mallopt. There is a single instance, mparams, 2478c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath initialized in init_mparams. Note that the non-zeroness of "magic" 2479c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath also serves as an initialization flag. 2480c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 2481c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2482c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstruct malloc_params { 2483c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath volatile size_t magic; 2484c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t page_size; 2485c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t granularity; 2486c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t mmap_threshold; 2487c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t trim_threshold; 2488c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath flag_t default_mflags; 2489c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath}; 2490c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2491c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic struct malloc_params mparams; 2492c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2493c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Ensure mparams initialized */ 2494c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ensure_initialization() (void)(mparams.magic != 0 || init_mparams()) 2495c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2496c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !ONLY_MSPACES 2497c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2498c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* The global malloc_state used for all non-"mspace" calls */ 2499c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic struct malloc_state _gm_; 2500c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define gm (&_gm_) 2501c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define is_global(M) ((M) == &_gm_) 2502c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2503c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* !ONLY_MSPACES */ 2504c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2505c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define is_initialized(M) ((M)->top != 0) 2506c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2507c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* -------------------------- system alloc setup ------------------------- */ 2508c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2509c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Operations on mflags */ 2510c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2511c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define use_lock(M) ((M)->mflags & USE_LOCK_BIT) 2512c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define enable_lock(M) ((M)->mflags |= USE_LOCK_BIT) 2513c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define disable_lock(M) ((M)->mflags &= ~USE_LOCK_BIT) 2514c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2515c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define use_mmap(M) ((M)->mflags & USE_MMAP_BIT) 2516c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define enable_mmap(M) ((M)->mflags |= USE_MMAP_BIT) 2517c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define disable_mmap(M) ((M)->mflags &= ~USE_MMAP_BIT) 2518c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2519c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define use_noncontiguous(M) ((M)->mflags & USE_NONCONTIGUOUS_BIT) 2520c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define disable_contiguous(M) ((M)->mflags |= USE_NONCONTIGUOUS_BIT) 2521c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2522c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define set_lock(M,L)\ 2523c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((M)->mflags = (L)?\ 2524c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((M)->mflags | USE_LOCK_BIT) :\ 2525c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((M)->mflags & ~USE_LOCK_BIT)) 2526c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2527c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* page-align a size */ 2528c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define page_align(S)\ 2529c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (((S) + (mparams.page_size - SIZE_T_ONE)) & ~(mparams.page_size - SIZE_T_ONE)) 2530c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2531c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* granularity-align a size */ 2532c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define granularity_align(S)\ 2533c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (((S) + (mparams.granularity - SIZE_T_ONE))\ 2534c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath & ~(mparams.granularity - SIZE_T_ONE)) 2535c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2536c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2537c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* For mmap, use granularity alignment on windows, else page-align */ 2538c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef WIN32 2539c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define mmap_align(S) granularity_align(S) 2540c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else 2541c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define mmap_align(S) page_align(S) 2542c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 2543c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2544c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* For sys_alloc, enough padding to ensure can malloc request on success */ 2545c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define SYS_ALLOC_PADDING (TOP_FOOT_SIZE + MALLOC_ALIGNMENT) 2546c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2547c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define is_page_aligned(S)\ 2548c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (((size_t)(S) & (mparams.page_size - SIZE_T_ONE)) == 0) 2549c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define is_granularity_aligned(S)\ 2550c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (((size_t)(S) & (mparams.granularity - SIZE_T_ONE)) == 0) 2551c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2552c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* True if segment S holds address A */ 2553c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define segment_holds(S, A)\ 2554c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((char*)(A) >= S->base && (char*)(A) < S->base + S->size) 2555c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2556c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Return segment holding given address */ 2557c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic msegmentptr segment_holding(mstate m, char* addr) { 2558c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr sp = &m->seg; 2559c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (;;) { 2560c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (addr >= sp->base && addr < sp->base + sp->size) 2561c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return sp; 2562c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((sp = sp->next) == 0) 2563c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 2564c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 2565c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 2566c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2567c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Return true if segment contains a segment link */ 2568c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic int has_segment_link(mstate m, msegmentptr ss) { 2569c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr sp = &m->seg; 2570c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (;;) { 2571c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((char*)sp >= ss->base && (char*)sp < ss->base + ss->size) 2572c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 1; 2573c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((sp = sp->next) == 0) 2574c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 2575c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 2576c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 2577c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2578c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef MORECORE_CANNOT_TRIM 2579c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define should_trim(M,s) ((s) > (M)->trim_check) 2580c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* MORECORE_CANNOT_TRIM */ 2581c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define should_trim(M,s) (0) 2582c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MORECORE_CANNOT_TRIM */ 2583c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2584c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 2585c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath TOP_FOOT_SIZE is padding at the end of a segment, including space 2586c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath that may be needed to place segment records and fenceposts when new 2587c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath noncontiguous segments are added. 2588c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 2589c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define TOP_FOOT_SIZE\ 2590c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (align_offset(chunk2mem(0))+pad_request(sizeof(struct malloc_segment))+MIN_CHUNK_SIZE) 2591c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2592c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2593c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ------------------------------- Hooks -------------------------------- */ 2594c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2595c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 2596c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath PREACTION should be defined to return 0 on success, and nonzero on 2597c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath failure. If you are not using locking, you can redefine these to do 2598c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath anything you like. 2599c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 2600c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2601c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if USE_LOCKS 2602c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2603c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define PREACTION(M) ((use_lock(M))? ACQUIRE_LOCK(&(M)->mutex) : 0) 2604c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define POSTACTION(M) { if (use_lock(M)) RELEASE_LOCK(&(M)->mutex); } 2605c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* USE_LOCKS */ 2606c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2607c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef PREACTION 2608c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define PREACTION(M) (0) 2609c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* PREACTION */ 2610c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2611c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef POSTACTION 2612c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define POSTACTION(M) 2613c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* POSTACTION */ 2614c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2615c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_LOCKS */ 2616c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2617c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 2618c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION is triggered upon detected bad addresses. 2619c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION is triggered on detected bad frees and 2620c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath reallocs. The argument p is an address that might have triggered the 2621c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fault. It is ignored by the two predefined actions, but might be 2622c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath useful in custom actions that try to help diagnose errors. 2623c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 2624c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2625c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if PROCEED_ON_ERROR 2626c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2627c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* A count of the number of corruption errors causing resets */ 2628c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathint malloc_corruption_error_count; 2629c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2630c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* default corruption action */ 2631c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void reset_on_error(mstate m); 2632c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2633c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define CORRUPTION_ERROR_ACTION(m) reset_on_error(m) 2634c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define USAGE_ERROR_ACTION(m, p) 2635c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2636c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* PROCEED_ON_ERROR */ 2637c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2638c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef CORRUPTION_ERROR_ACTION 2639c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define CORRUPTION_ERROR_ACTION(m) ABORT 2640c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* CORRUPTION_ERROR_ACTION */ 2641c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2642c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef USAGE_ERROR_ACTION 2643c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define USAGE_ERROR_ACTION(m,p) ABORT 2644c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USAGE_ERROR_ACTION */ 2645c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2646c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* PROCEED_ON_ERROR */ 2647c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2648c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* -------------------------- Debugging setup ---------------------------- */ 2649c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2650c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if ! DEBUG 2651c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2652c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define check_free_chunk(M,P) 2653c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define check_inuse_chunk(M,P) 2654c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define check_malloced_chunk(M,P,N) 2655c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define check_mmapped_chunk(M,P) 2656c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define check_malloc_state(M) 2657c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define check_top_chunk(M,P) 2658c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2659c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* DEBUG */ 2660c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define check_free_chunk(M,P) do_check_free_chunk(M,P) 2661c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define check_inuse_chunk(M,P) do_check_inuse_chunk(M,P) 2662c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define check_top_chunk(M,P) do_check_top_chunk(M,P) 2663c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define check_malloced_chunk(M,P,N) do_check_malloced_chunk(M,P,N) 2664c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define check_mmapped_chunk(M,P) do_check_mmapped_chunk(M,P) 2665c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define check_malloc_state(M) do_check_malloc_state(M) 2666c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2667c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_any_chunk(mstate m, mchunkptr p); 2668c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_top_chunk(mstate m, mchunkptr p); 2669c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_mmapped_chunk(mstate m, mchunkptr p); 2670c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_inuse_chunk(mstate m, mchunkptr p); 2671c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_free_chunk(mstate m, mchunkptr p); 2672c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_malloced_chunk(mstate m, void* mem, size_t s); 2673c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_tree(mstate m, tchunkptr t); 2674c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_treebin(mstate m, bindex_t i); 2675c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_smallbin(mstate m, bindex_t i); 2676c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_malloc_state(mstate m); 2677c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic int bin_find(mstate m, mchunkptr x); 2678c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic size_t traverse_and_check(mstate m); 2679c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* DEBUG */ 2680c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2681c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ---------------------------- Indexing Bins ---------------------------- */ 2682c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2683c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define is_small(s) (((s) >> SMALLBIN_SHIFT) < NSMALLBINS) 2684c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define small_index(s) ((s) >> SMALLBIN_SHIFT) 2685c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define small_index2size(i) ((i) << SMALLBIN_SHIFT) 2686c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define MIN_SMALL_INDEX (small_index(MIN_CHUNK_SIZE)) 2687c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2688c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* addressing by index. See above about smallbin repositioning */ 2689c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define smallbin_at(M, i) ((sbinptr)((char*)&((M)->smallbins[(i)<<1]))) 2690c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define treebin_at(M,i) (&((M)->treebins[i])) 2691c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2692c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* assign tree index for size S to variable I. Use x86 asm if possible */ 2693c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__)) 2694c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define compute_tree_index(S, I)\ 2695c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath{\ 2696c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int X = S >> TREEBIN_SHIFT;\ 2697c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (X == 0)\ 2698c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = 0;\ 2699c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (X > 0xFFFF)\ 2700c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = NTREEBINS-1;\ 2701c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 2702c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int K;\ 2703c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath __asm__("bsrl\t%1, %0\n\t" : "=r" (K) : "g" (X));\ 2704c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = (bindex_t)((K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1)));\ 2705c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 2706c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 2707c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2708c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#elif defined (__INTEL_COMPILER) 2709c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define compute_tree_index(S, I)\ 2710c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath{\ 2711c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t X = S >> TREEBIN_SHIFT;\ 2712c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (X == 0)\ 2713c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = 0;\ 2714c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (X > 0xFFFF)\ 2715c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = NTREEBINS-1;\ 2716c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 2717c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int K = _bit_scan_reverse (X); \ 2718c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = (bindex_t)((K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1)));\ 2719c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 2720c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 2721c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2722c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#elif defined(_MSC_VER) && _MSC_VER>=1300 2723c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define compute_tree_index(S, I)\ 2724c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath{\ 2725c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t X = S >> TREEBIN_SHIFT;\ 2726c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (X == 0)\ 2727c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = 0;\ 2728c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (X > 0xFFFF)\ 2729c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = NTREEBINS-1;\ 2730c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 2731c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int K;\ 2732c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath _BitScanReverse((DWORD *) &K, X);\ 2733c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = (bindex_t)((K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1)));\ 2734c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 2735c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 2736c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2737c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* GNUC */ 2738c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define compute_tree_index(S, I)\ 2739c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath{\ 2740c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t X = S >> TREEBIN_SHIFT;\ 2741c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (X == 0)\ 2742c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = 0;\ 2743c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (X > 0xFFFF)\ 2744c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = NTREEBINS-1;\ 2745c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 2746c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int Y = (unsigned int)X;\ 2747c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int N = ((Y - 0x100) >> 16) & 8;\ 2748c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int K = (((Y <<= N) - 0x1000) >> 16) & 4;\ 2749c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath N += K;\ 2750c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath N += K = (((Y <<= K) - 0x4000) >> 16) & 2;\ 2751c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath K = 14 - N + ((Y <<= K) >> 15);\ 2752c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = (K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1));\ 2753c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 2754c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 2755c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* GNUC */ 2756c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2757c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Bit representing maximum resolved size in a treebin at i */ 2758c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define bit_for_tree_index(i) \ 2759c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (i == NTREEBINS-1)? (SIZE_T_BITSIZE-1) : (((i) >> 1) + TREEBIN_SHIFT - 2) 2760c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2761c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Shift placing maximum resolved bit in a treebin at i as sign bit */ 2762c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define leftshift_for_tree_index(i) \ 2763c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((i == NTREEBINS-1)? 0 : \ 2764c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((SIZE_T_BITSIZE-SIZE_T_ONE) - (((i) >> 1) + TREEBIN_SHIFT - 2))) 2765c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2766c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* The size of the smallest chunk held in bin with index i */ 2767c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define minsize_for_tree_index(i) \ 2768c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((SIZE_T_ONE << (((i) >> 1) + TREEBIN_SHIFT)) | \ 2769c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (((size_t)((i) & SIZE_T_ONE)) << (((i) >> 1) + TREEBIN_SHIFT - 1))) 2770c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2771c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2772c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ------------------------ Operations on bin maps ----------------------- */ 2773c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2774c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* bit corresponding to given index */ 2775c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define idx2bit(i) ((binmap_t)(1) << (i)) 2776c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2777c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Mark/Clear bits with given index */ 2778c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define mark_smallmap(M,i) ((M)->smallmap |= idx2bit(i)) 2779c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define clear_smallmap(M,i) ((M)->smallmap &= ~idx2bit(i)) 2780c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define smallmap_is_marked(M,i) ((M)->smallmap & idx2bit(i)) 2781c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2782c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define mark_treemap(M,i) ((M)->treemap |= idx2bit(i)) 2783c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define clear_treemap(M,i) ((M)->treemap &= ~idx2bit(i)) 2784c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define treemap_is_marked(M,i) ((M)->treemap & idx2bit(i)) 2785c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2786c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* isolate the least set bit of a bitmap */ 2787c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define least_bit(x) ((x) & -(x)) 2788c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2789c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* mask with all bits to left of least bit of x on */ 2790c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define left_bits(x) ((x<<1) | -(x<<1)) 2791c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2792c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* mask with all bits to left of or equal to least bit of x on */ 2793c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define same_or_left_bits(x) ((x) | -(x)) 2794c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2795c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* index corresponding to given bit. Use x86 asm if possible */ 2796c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2797c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__)) 2798c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define compute_bit2idx(X, I)\ 2799c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath{\ 2800c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int J;\ 2801c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath __asm__("bsfl\t%1, %0\n\t" : "=r" (J) : "g" (X));\ 2802c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = (bindex_t)J;\ 2803c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 2804c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2805c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#elif defined (__INTEL_COMPILER) 2806c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define compute_bit2idx(X, I)\ 2807c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath{\ 2808c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int J;\ 2809c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath J = _bit_scan_forward (X); \ 2810c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = (bindex_t)J;\ 2811c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 2812c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2813c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#elif defined(_MSC_VER) && _MSC_VER>=1300 2814c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define compute_bit2idx(X, I)\ 2815c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath{\ 2816c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int J;\ 2817c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath _BitScanForward((DWORD *) &J, X);\ 2818c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = (bindex_t)J;\ 2819c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 2820c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2821c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#elif USE_BUILTIN_FFS 2822c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define compute_bit2idx(X, I) I = ffs(X)-1 2823c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2824c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else 2825c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define compute_bit2idx(X, I)\ 2826c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath{\ 2827c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int Y = X - 1;\ 2828c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int K = Y >> (16-4) & 16;\ 2829c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int N = K; Y >>= K;\ 2830c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath N += K = Y >> (8-3) & 8; Y >>= K;\ 2831c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath N += K = Y >> (4-2) & 4; Y >>= K;\ 2832c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath N += K = Y >> (2-1) & 2; Y >>= K;\ 2833c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath N += K = Y >> (1-0) & 1; Y >>= K;\ 2834c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath I = (bindex_t)(N + Y);\ 2835c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 2836c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* GNUC */ 2837c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2838c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2839c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ----------------------- Runtime Check Support ------------------------- */ 2840c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2841c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 2842c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath For security, the main invariant is that malloc/free/etc never 2843c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath writes to a static address other than malloc_state, unless static 2844c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc_state itself has been corrupted, which cannot occur via 2845c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc (because of these checks). In essence this means that we 2846c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath believe all pointers, sizes, maps etc held in malloc_state, but 2847c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check all of those linked or offsetted from other embedded data 2848c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath structures. These checks are interspersed with main code in a way 2849c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath that tends to minimize their run-time cost. 2850c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2851c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath When FOOTERS is defined, in addition to range checking, we also 2852c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath verify footer fields of inuse chunks, which can be used guarantee 2853c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath that the mstate controlling malloc/free is intact. This is a 2854c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath streamlined version of the approach described by William Robertson 2855c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath et al in "Run-time Detection of Heap-based Overflows" LISA'03 2856c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath http://www.usenix.org/events/lisa03/tech/robertson.html The footer 2857c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath of an inuse chunk holds the xor of its mstate and a random seed, 2858c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath that is checked upon calls to free() and realloc(). This is 2859c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (probablistically) unguessable from outside the program, but can be 2860c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath computed by any code successfully malloc'ing any chunk, so does not 2861c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath itself provide protection against code that has already broken 2862c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath security through some other means. Unlike Robertson et al, we 2863c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath always dynamically check addresses of all offset chunks (previous, 2864c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath next, etc). This turns out to be cheaper than relying on hashes. 2865c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 2866c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2867c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !INSECURE 2868c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check if address a is at least as high as any from MORECORE or MMAP */ 2869c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ok_address(M, a) ((char*)(a) >= (M)->least_addr) 2870c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check if address of next chunk n is higher than base chunk p */ 2871c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ok_next(p, n) ((char*)(p) < (char*)(n)) 2872c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check if p has inuse status */ 2873c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ok_inuse(p) is_inuse(p) 2874c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check if p has its pinuse bit on */ 2875c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ok_pinuse(p) pinuse(p) 2876c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2877c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* !INSECURE */ 2878c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ok_address(M, a) (1) 2879c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ok_next(b, n) (1) 2880c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ok_inuse(p) (1) 2881c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ok_pinuse(p) (1) 2882c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* !INSECURE */ 2883c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2884c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if (FOOTERS && !INSECURE) 2885c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check if (alleged) mstate m has expected magic field */ 2886c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ok_magic(M) ((M)->magic == mparams.magic) 2887c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* (FOOTERS && !INSECURE) */ 2888c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define ok_magic(M) (1) 2889c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* (FOOTERS && !INSECURE) */ 2890c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2891c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2892c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* In gcc, use __builtin_expect to minimize impact of checks */ 2893c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !INSECURE 2894c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if defined(__GNUC__) && __GNUC__ >= 3 2895c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define RTCHECK(e) __builtin_expect(e, 1) 2896c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* GNUC */ 2897c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define RTCHECK(e) (e) 2898c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* GNUC */ 2899c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* !INSECURE */ 2900c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define RTCHECK(e) (1) 2901c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* !INSECURE */ 2902c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2903c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* macros to set up inuse chunks with or without footers */ 2904c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2905c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !FOOTERS 2906c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2907c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define mark_inuse_foot(M,p,s) 2908c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2909c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Macros for setting head/foot of non-mmapped chunks */ 2910c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2911c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Set cinuse bit and pinuse bit of next chunk */ 2912c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define set_inuse(M,p,s)\ 2913c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((p)->head = (((p)->head & PINUSE_BIT)|s|CINUSE_BIT),\ 2914c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT) 2915c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2916c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Set cinuse and pinuse of this chunk and pinuse of next chunk */ 2917c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define set_inuse_and_pinuse(M,p,s)\ 2918c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((p)->head = (s|PINUSE_BIT|CINUSE_BIT),\ 2919c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT) 2920c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2921c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Set size, cinuse and pinuse bit of this chunk */ 2922c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define set_size_and_pinuse_of_inuse_chunk(M, p, s)\ 2923c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((p)->head = (s|PINUSE_BIT|CINUSE_BIT)) 2924c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2925c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* FOOTERS */ 2926c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2927c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Set foot of inuse chunk to be xor of mstate and seed */ 2928c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define mark_inuse_foot(M,p,s)\ 2929c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (((mchunkptr)((char*)(p) + (s)))->prev_foot = ((size_t)(M) ^ mparams.magic)) 2930c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2931c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define get_mstate_for(p)\ 2932c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((mstate)(((mchunkptr)((char*)(p) +\ 2933c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (chunksize(p))))->prev_foot ^ mparams.magic)) 2934c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2935c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define set_inuse(M,p,s)\ 2936c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((p)->head = (((p)->head & PINUSE_BIT)|s|CINUSE_BIT),\ 2937c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT), \ 2938c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mark_inuse_foot(M,p,s)) 2939c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2940c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define set_inuse_and_pinuse(M,p,s)\ 2941c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((p)->head = (s|PINUSE_BIT|CINUSE_BIT),\ 2942c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT),\ 2943c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mark_inuse_foot(M,p,s)) 2944c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2945c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define set_size_and_pinuse_of_inuse_chunk(M, p, s)\ 2946c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((p)->head = (s|PINUSE_BIT|CINUSE_BIT),\ 2947c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mark_inuse_foot(M, p, s)) 2948c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2949c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* !FOOTERS */ 2950c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2951c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ---------------------------- setting mparams -------------------------- */ 2952c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2953c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Initialize mparams */ 2954c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic int init_mparams(void) { 2955c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef NEED_GLOBAL_LOCK_INIT 2956c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (malloc_global_mutex_status <= 0) 2957c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath init_malloc_global_mutex(); 2958c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 2959c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2960c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ACQUIRE_MALLOC_GLOBAL_LOCK(); 2961c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mparams.magic == 0) { 2962c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t magic; 2963c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t psize; 2964c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t gsize; 2965c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2966c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifndef WIN32 2967c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath psize = malloc_getpagesize; 2968c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath gsize = ((DEFAULT_GRANULARITY != 0)? DEFAULT_GRANULARITY : psize); 2969c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* WIN32 */ 2970c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath { 2971c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath SYSTEM_INFO system_info; 2972c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath GetSystemInfo(&system_info); 2973c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath psize = system_info.dwPageSize; 2974c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath gsize = ((DEFAULT_GRANULARITY != 0)? 2975c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath DEFAULT_GRANULARITY : system_info.dwAllocationGranularity); 2976c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 2977c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* WIN32 */ 2978c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2979c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Sanity-check configuration: 2980c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t must be unsigned and as wide as pointer type. 2981c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ints must be at least 4 bytes. 2982c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath alignment must be at least 8. 2983c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Alignment, min chunk size, and page size must all be powers of 2. 2984c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath */ 2985c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((sizeof(size_t) != sizeof(char*)) || 2986c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (MAX_SIZE_T < MIN_CHUNK_SIZE) || 2987c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (sizeof(int) < 4) || 2988c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (MALLOC_ALIGNMENT < (size_t)8U) || 2989c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((MALLOC_ALIGNMENT & (MALLOC_ALIGNMENT-SIZE_T_ONE)) != 0) || 2990c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((MCHUNK_SIZE & (MCHUNK_SIZE-SIZE_T_ONE)) != 0) || 2991c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((gsize & (gsize-SIZE_T_ONE)) != 0) || 2992c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((psize & (psize-SIZE_T_ONE)) != 0)) 2993c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ABORT; 2994c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2995c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mparams.granularity = gsize; 2996c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mparams.page_size = psize; 2997c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mparams.mmap_threshold = DEFAULT_MMAP_THRESHOLD; 2998c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mparams.trim_threshold = DEFAULT_TRIM_THRESHOLD; 2999c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if MORECORE_CONTIGUOUS 3000c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mparams.default_mflags = USE_LOCK_BIT|USE_MMAP_BIT; 3001c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* MORECORE_CONTIGUOUS */ 3002c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mparams.default_mflags = USE_LOCK_BIT|USE_MMAP_BIT|USE_NONCONTIGUOUS_BIT; 3003c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MORECORE_CONTIGUOUS */ 3004c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3005c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !ONLY_MSPACES 3006c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Set up lock for main malloc area */ 3007c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath gm->mflags = mparams.default_mflags; 3008c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath INITIAL_LOCK(&gm->mutex); 3009c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 3010c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3011c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath { 3012c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if USE_DEV_RANDOM 3013c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int fd; 3014c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned char buf[sizeof(size_t)]; 3015c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Try to use /dev/urandom, else fall back on using time */ 3016c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((fd = open("/dev/urandom", O_RDONLY)) >= 0 && 3017c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath read(fd, buf, sizeof(buf)) == sizeof(buf)) { 3018c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath magic = *((size_t *) buf); 3019c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath close(fd); 3020c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3021c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 3022c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* USE_DEV_RANDOM */ 3023c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef WIN32 3024c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath magic = (size_t)(GetTickCount() ^ (size_t)0x55555555U); 3025c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else 3026c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath magic = (size_t)(time(0) ^ (size_t)0x55555555U); 3027c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 3028c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath magic |= (size_t)8U; /* ensure nonzero */ 3029c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath magic &= ~(size_t)7U; /* improve chances of fault for bad values */ 3030c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mparams.magic = magic; 3031c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3032c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3033c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3034c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath RELEASE_MALLOC_GLOBAL_LOCK(); 3035c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 1; 3036c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3037c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3038c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* support for mallopt */ 3039c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic int change_mparam(int param_number, int value) { 3040c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t val; 3041c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensure_initialization(); 3042c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath val = (value == -1)? MAX_SIZE_T : (size_t)value; 3043c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath switch(param_number) { 3044c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath case M_TRIM_THRESHOLD: 3045c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mparams.trim_threshold = val; 3046c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 1; 3047c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath case M_GRANULARITY: 3048c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (val >= mparams.page_size && ((val & (val-1)) == 0)) { 3049c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mparams.granularity = val; 3050c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 1; 3051c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3052c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 3053c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 3054c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath case M_MMAP_THRESHOLD: 3055c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mparams.mmap_threshold = val; 3056c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 1; 3057c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath default: 3058c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 3059c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3060c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3061c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3062c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if DEBUG 3063c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ------------------------- Debugging Support --------------------------- */ 3064c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3065c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check properties of any chunk, whether free, inuse, mmapped etc */ 3066c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_any_chunk(mstate m, mchunkptr p) { 3067c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert((is_aligned(chunk2mem(p))) || (p->head == FENCEPOST_HEAD)); 3068c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(ok_address(m, p)); 3069c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3070c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3071c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check properties of top chunk */ 3072c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_top_chunk(mstate m, mchunkptr p) { 3073c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr sp = segment_holding(m, (char*)p); 3074c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sz = p->head & ~INUSE_BITS; /* third-lowest bit can be set! */ 3075c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(sp != 0); 3076c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert((is_aligned(chunk2mem(p))) || (p->head == FENCEPOST_HEAD)); 3077c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(ok_address(m, p)); 3078c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(sz == m->topsize); 3079c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(sz > 0); 3080c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(sz == ((sp->base + sp->size) - (char*)p) - TOP_FOOT_SIZE); 3081c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(pinuse(p)); 3082c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(!pinuse(chunk_plus_offset(p, sz))); 3083c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3084c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3085c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check properties of (inuse) mmapped chunks */ 3086c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_mmapped_chunk(mstate m, mchunkptr p) { 3087c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sz = chunksize(p); 3088c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t len = (sz + (p->prev_foot) + MMAP_FOOT_PAD); 3089c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(is_mmapped(p)); 3090c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(use_mmap(m)); 3091c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert((is_aligned(chunk2mem(p))) || (p->head == FENCEPOST_HEAD)); 3092c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(ok_address(m, p)); 3093c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(!is_small(sz)); 3094c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert((len & (mparams.page_size-SIZE_T_ONE)) == 0); 3095c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(chunk_plus_offset(p, sz)->head == FENCEPOST_HEAD); 3096c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(chunk_plus_offset(p, sz+SIZE_T_SIZE)->head == 0); 3097c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3098c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3099c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check properties of inuse chunks */ 3100c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_inuse_chunk(mstate m, mchunkptr p) { 3101c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_any_chunk(m, p); 3102c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(is_inuse(p)); 3103c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(next_pinuse(p)); 3104c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* If not pinuse and not mmapped, previous chunk has OK offset */ 3105c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(is_mmapped(p) || pinuse(p) || next_chunk(prev_chunk(p)) == p); 3106c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_mmapped(p)) 3107c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_mmapped_chunk(m, p); 3108c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3109c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3110c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check properties of free chunks */ 3111c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_free_chunk(mstate m, mchunkptr p) { 3112c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sz = chunksize(p); 3113c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr next = chunk_plus_offset(p, sz); 3114c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_any_chunk(m, p); 3115c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(!is_inuse(p)); 3116c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(!next_pinuse(p)); 3117c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert (!is_mmapped(p)); 3118c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (p != m->dv && p != m->top) { 3119c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (sz >= MIN_CHUNK_SIZE) { 3120c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert((sz & CHUNK_ALIGN_MASK) == 0); 3121c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(is_aligned(chunk2mem(p))); 3122c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(next->prev_foot == sz); 3123c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(pinuse(p)); 3124c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert (next == m->top || is_inuse(next)); 3125c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(p->fd->bk == p); 3126c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(p->bk->fd == p); 3127c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3128c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else /* markers are always of size SIZE_T_SIZE */ 3129c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(sz == SIZE_T_SIZE); 3130c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3131c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3132c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3133c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check properties of malloced chunks at the point they are malloced */ 3134c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_malloced_chunk(mstate m, void* mem, size_t s) { 3135c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mem != 0) { 3136c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = mem2chunk(mem); 3137c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sz = p->head & ~INUSE_BITS; 3138c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_inuse_chunk(m, p); 3139c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert((sz & CHUNK_ALIGN_MASK) == 0); 3140c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(sz >= MIN_CHUNK_SIZE); 3141c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(sz >= s); 3142c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* unless mmapped, size is less than MIN_CHUNK_SIZE more than request */ 3143c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(is_mmapped(p) || sz < (s + MIN_CHUNK_SIZE)); 3144c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3145c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3146c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3147c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check a tree and its subtrees. */ 3148c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_tree(mstate m, tchunkptr t) { 3149c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr head = 0; 3150c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr u = t; 3151c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t tindex = t->index; 3152c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t tsize = chunksize(t); 3153c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t idx; 3154c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath compute_tree_index(tsize, idx); 3155c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(tindex == idx); 3156c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(tsize >= MIN_LARGE_SIZE); 3157c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(tsize >= minsize_for_tree_index(idx)); 3158c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert((idx == NTREEBINS-1) || (tsize < minsize_for_tree_index((idx+1)))); 3159c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3160c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do { /* traverse through chain of same-sized nodes */ 3161c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_any_chunk(m, ((mchunkptr)u)); 3162c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(u->index == tindex); 3163c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(chunksize(u) == tsize); 3164c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(!is_inuse(u)); 3165c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(!next_pinuse(u)); 3166c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(u->fd->bk == u); 3167c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(u->bk->fd == u); 3168c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (u->parent == 0) { 3169c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(u->child[0] == 0); 3170c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(u->child[1] == 0); 3171c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3172c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 3173c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(head == 0); /* only one node on chain has parent */ 3174c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath head = u; 3175c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(u->parent != u); 3176c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert (u->parent->child[0] == u || 3177c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath u->parent->child[1] == u || 3178c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath *((tbinptr*)(u->parent)) == u); 3179c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (u->child[0] != 0) { 3180c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(u->child[0]->parent == u); 3181c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(u->child[0] != u); 3182c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_tree(m, u->child[0]); 3183c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3184c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (u->child[1] != 0) { 3185c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(u->child[1]->parent == u); 3186c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(u->child[1] != u); 3187c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_tree(m, u->child[1]); 3188c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3189c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (u->child[0] != 0 && u->child[1] != 0) { 3190c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(chunksize(u->child[0]) < chunksize(u->child[1])); 3191c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3192c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3193c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath u = u->fd; 3194c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } while (u != t); 3195c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(head != 0); 3196c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3197c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3198c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check all the chunks in a treebin. */ 3199c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_treebin(mstate m, bindex_t i) { 3200c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tbinptr* tb = treebin_at(m, i); 3201c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr t = *tb; 3202c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int empty = (m->treemap & (1U << i)) == 0; 3203c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (t == 0) 3204c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(empty); 3205c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!empty) 3206c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_tree(m, t); 3207c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3208c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3209c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check all the chunks in a smallbin. */ 3210c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_smallbin(mstate m, bindex_t i) { 3211c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sbinptr b = smallbin_at(m, i); 3212c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = b->bk; 3213c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unsigned int empty = (m->smallmap & (1U << i)) == 0; 3214c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (p == b) 3215c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(empty); 3216c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!empty) { 3217c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (; p != b; p = p->bk) { 3218c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t size = chunksize(p); 3219c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr q; 3220c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* each chunk claims to be free */ 3221c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_free_chunk(m, p); 3222c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* chunk belongs in bin */ 3223c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(small_index(size) == i); 3224c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(p->bk == b || chunksize(p->bk) == chunksize(p)); 3225c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* chunk is followed by an inuse chunk */ 3226c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath q = next_chunk(p); 3227c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (q->head != FENCEPOST_HEAD) 3228c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_inuse_chunk(m, q); 3229c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3230c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3231c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3232c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3233c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Find x in a bin. Used in other check functions. */ 3234c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic int bin_find(mstate m, mchunkptr x) { 3235c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t size = chunksize(x); 3236c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_small(size)) { 3237c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t sidx = small_index(size); 3238c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sbinptr b = smallbin_at(m, sidx); 3239c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (smallmap_is_marked(m, sidx)) { 3240c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = b; 3241c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do { 3242c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (p == x) 3243c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 1; 3244c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } while ((p = p->fd) != b); 3245c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3246c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3247c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 3248c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t tidx; 3249c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath compute_tree_index(size, tidx); 3250c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (treemap_is_marked(m, tidx)) { 3251c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr t = *treebin_at(m, tidx); 3252c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sizebits = size << leftshift_for_tree_index(tidx); 3253c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (t != 0 && chunksize(t) != size) { 3254c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath t = t->child[(sizebits >> (SIZE_T_BITSIZE-SIZE_T_ONE)) & 1]; 3255c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sizebits <<= 1; 3256c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3257c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (t != 0) { 3258c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr u = t; 3259c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do { 3260c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (u == (tchunkptr)x) 3261c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 1; 3262c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } while ((u = u->fd) != t); 3263c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3264c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3265c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3266c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 3267c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3268c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3269c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Traverse each chunk and check it; return total */ 3270c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic size_t traverse_and_check(mstate m) { 3271c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sum = 0; 3272c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_initialized(m)) { 3273c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr s = &m->seg; 3274c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sum += m->topsize + TOP_FOOT_SIZE; 3275c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (s != 0) { 3276c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr q = align_as_chunk(s->base); 3277c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr lastq = 0; 3278c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(pinuse(q)); 3279c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (segment_holds(s, q) && 3280c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath q != m->top && q->head != FENCEPOST_HEAD) { 3281c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sum += chunksize(q); 3282c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_inuse(q)) { 3283c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(!bin_find(m, q)); 3284c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_inuse_chunk(m, q); 3285c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3286c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 3287c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(q == m->dv || bin_find(m, q)); 3288c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(lastq == 0 || is_inuse(lastq)); /* Not 2 consecutive free */ 3289c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_free_chunk(m, q); 3290c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3291c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath lastq = q; 3292c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath q = next_chunk(q); 3293c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3294c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath s = s->next; 3295c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3296c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3297c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return sum; 3298c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3299c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3300c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Check all properties of malloc_state. */ 3301c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void do_check_malloc_state(mstate m) { 3302c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t i; 3303c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t total; 3304c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* check bins */ 3305c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (i = 0; i < NSMALLBINS; ++i) 3306c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_smallbin(m, i); 3307c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (i = 0; i < NTREEBINS; ++i) 3308c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_treebin(m, i); 3309c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3310c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (m->dvsize != 0) { /* check dv chunk */ 3311c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_any_chunk(m, m->dv); 3312c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(m->dvsize == chunksize(m->dv)); 3313c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(m->dvsize >= MIN_CHUNK_SIZE); 3314c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(bin_find(m, m->dv) == 0); 3315c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3316c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3317c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (m->top != 0) { /* check top chunk */ 3318c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath do_check_top_chunk(m, m->top); 3319c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /*assert(m->topsize == chunksize(m->top)); redundant */ 3320c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(m->topsize > 0); 3321c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(bin_find(m, m->top) == 0); 3322c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3323c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3324c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath total = traverse_and_check(m); 3325c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(total <= m->footprint); 3326c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(m->footprint <= m->max_footprint); 3327c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3328c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* DEBUG */ 3329c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3330c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ----------------------------- statistics ------------------------------ */ 3331c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3332c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !NO_MALLINFO 3333c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic struct mallinfo internal_mallinfo(mstate m) { 3334c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath struct mallinfo nm = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }; 3335c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensure_initialization(); 3336c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!PREACTION(m)) { 3337c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloc_state(m); 3338c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_initialized(m)) { 3339c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t nfree = SIZE_T_ONE; /* top always free */ 3340c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t mfree = m->topsize + TOP_FOOT_SIZE; 3341c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sum = mfree; 3342c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr s = &m->seg; 3343c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (s != 0) { 3344c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr q = align_as_chunk(s->base); 3345c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (segment_holds(s, q) && 3346c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath q != m->top && q->head != FENCEPOST_HEAD) { 3347c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sz = chunksize(q); 3348c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sum += sz; 3349c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!is_inuse(q)) { 3350c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mfree += sz; 3351c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ++nfree; 3352c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3353c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath q = next_chunk(q); 3354c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3355c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath s = s->next; 3356c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3357c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3358c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nm.arena = sum; 3359c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nm.ordblks = nfree; 3360c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nm.hblkhd = m->footprint - sum; 3361c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nm.usmblks = m->max_footprint; 3362c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nm.uordblks = m->footprint - mfree; 3363c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nm.fordblks = mfree; 3364c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nm.keepcost = m->topsize; 3365c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3366c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3367c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(m); 3368c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3369c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return nm; 3370c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3371c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* !NO_MALLINFO */ 3372c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3373c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void internal_malloc_stats(mstate m) { 3374c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensure_initialization(); 3375c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!PREACTION(m)) { 3376c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t maxfp = 0; 3377c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t fp = 0; 3378c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t used = 0; 3379c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloc_state(m); 3380c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_initialized(m)) { 3381c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr s = &m->seg; 3382c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath maxfp = m->max_footprint; 3383c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fp = m->footprint; 3384c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath used = fp - (m->topsize + TOP_FOOT_SIZE); 3385c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3386c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (s != 0) { 3387c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr q = align_as_chunk(s->base); 3388c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (segment_holds(s, q) && 3389c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath q != m->top && q->head != FENCEPOST_HEAD) { 3390c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!is_inuse(q)) 3391c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath used -= chunksize(q); 3392c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath q = next_chunk(q); 3393c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3394c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath s = s->next; 3395c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3396c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3397c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3398c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fprintf(stderr, "max system bytes = %10lu\n", (unsigned long)(maxfp)); 3399c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fprintf(stderr, "system bytes = %10lu\n", (unsigned long)(fp)); 3400c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fprintf(stderr, "in use bytes = %10lu\n", (unsigned long)(used)); 3401c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3402c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(m); 3403c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3404c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3405c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3406c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ----------------------- Operations on smallbins ----------------------- */ 3407c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3408c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 3409c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Various forms of linking and unlinking are defined as macros. Even 3410c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the ones for trees, which are very long but have very short typical 3411c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath paths. This is ugly but reduces reliance on inlining support of 3412c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath compilers. 3413c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 3414c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3415c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Link a free chunk into a smallbin */ 3416c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define insert_small_chunk(M, P, S) {\ 3417c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t I = small_index(S);\ 3418c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr B = smallbin_at(M, I);\ 3419c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr F = B;\ 3420c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(S >= MIN_CHUNK_SIZE);\ 3421c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!smallmap_is_marked(M, I))\ 3422c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mark_smallmap(M, I);\ 3423c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (RTCHECK(ok_address(M, B->fd)))\ 3424c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath F = B->fd;\ 3425c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 3426c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(M);\ 3427c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3428c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath B->fd = P;\ 3429c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath F->bk = P;\ 3430c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath P->fd = F;\ 3431c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath P->bk = B;\ 3432c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3433c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3434c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Unlink a chunk from a smallbin */ 3435c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define unlink_small_chunk(M, P, S) {\ 3436c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr F = P->fd;\ 3437c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr B = P->bk;\ 3438c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t I = small_index(S);\ 3439c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(P != B);\ 3440c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(P != F);\ 3441c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(chunksize(P) == small_index2size(I));\ 3442c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (F == B)\ 3443c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath clear_smallmap(M, I);\ 3444c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (RTCHECK((F == smallbin_at(M,I) || ok_address(M, F)) &&\ 3445c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (B == smallbin_at(M,I) || ok_address(M, B)))) {\ 3446c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath F->bk = B;\ 3447c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath B->fd = F;\ 3448c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3449c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 3450c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(M);\ 3451c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3452c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3453c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3454c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Unlink the first chunk from a smallbin */ 3455c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define unlink_first_small_chunk(M, B, P, I) {\ 3456c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr F = P->fd;\ 3457c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(P != B);\ 3458c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(P != F);\ 3459c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(chunksize(P) == small_index2size(I));\ 3460c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (B == F)\ 3461c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath clear_smallmap(M, I);\ 3462c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (RTCHECK(ok_address(M, F))) {\ 3463c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath B->fd = F;\ 3464c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath F->bk = B;\ 3465c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3466c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 3467c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(M);\ 3468c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3469c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3470c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3471c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3472c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3473c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Replace dv node, binning the old one */ 3474c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Used only when dvsize known to be small */ 3475c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define replace_dv(M, P, S) {\ 3476c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t DVS = M->dvsize;\ 3477c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (DVS != 0) {\ 3478c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr DV = M->dv;\ 3479c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(is_small(DVS));\ 3480c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath insert_small_chunk(M, DV, DVS);\ 3481c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3482c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath M->dvsize = S;\ 3483c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath M->dv = P;\ 3484c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3485c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3486c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ------------------------- Operations on trees ------------------------- */ 3487c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3488c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Insert chunk into tree */ 3489c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define insert_large_chunk(M, X, S) {\ 3490c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tbinptr* H;\ 3491c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t I;\ 3492c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath compute_tree_index(S, I);\ 3493c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath H = treebin_at(M, I);\ 3494c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath X->index = I;\ 3495c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath X->child[0] = X->child[1] = 0;\ 3496c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!treemap_is_marked(M, I)) {\ 3497c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mark_treemap(M, I);\ 3498c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath *H = X;\ 3499c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath X->parent = (tchunkptr)H;\ 3500c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath X->fd = X->bk = X;\ 3501c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3502c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 3503c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr T = *H;\ 3504c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t K = S << leftshift_for_tree_index(I);\ 3505c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (;;) {\ 3506c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (chunksize(T) != S) {\ 3507c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr* C = &(T->child[(K >> (SIZE_T_BITSIZE-SIZE_T_ONE)) & 1]);\ 3508c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath K <<= 1;\ 3509c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (*C != 0)\ 3510c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath T = *C;\ 3511c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (RTCHECK(ok_address(M, C))) {\ 3512c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath *C = X;\ 3513c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath X->parent = T;\ 3514c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath X->fd = X->bk = X;\ 3515c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath break;\ 3516c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3517c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 3518c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(M);\ 3519c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath break;\ 3520c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3521c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3522c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 3523c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr F = T->fd;\ 3524c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(M, T) && ok_address(M, F))) {\ 3525c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath T->fd = F->bk = X;\ 3526c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath X->fd = F;\ 3527c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath X->bk = T;\ 3528c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath X->parent = 0;\ 3529c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath break;\ 3530c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3531c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 3532c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(M);\ 3533c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath break;\ 3534c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3535c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3536c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3537c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3538c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3539c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3540c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 3541c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Unlink steps: 3542c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3543c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1. If x is a chained node, unlink it from its same-sized fd/bk links 3544c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and choose its bk node as its replacement. 3545c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2. If x was the last node of its size, but not a leaf node, it must 3546c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath be replaced with a leaf node (not merely one with an open left or 3547c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath right), to make sure that lefts and rights of descendents 3548c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath correspond properly to bit masks. We use the rightmost descendent 3549c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath of x. We could use any other leaf, but this is easy to locate and 3550c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tends to counteract removal of leftmosts elsewhere, and so keeps 3551c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath paths shorter than minimally guaranteed. This doesn't loop much 3552c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath because on average a node in a tree is near the bottom. 3553c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3. If x is the base of a chain (i.e., has parent links) relink 3554c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath x's parent and children to x's replacement (or null if none). 3555c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 3556c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3557c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define unlink_large_chunk(M, X) {\ 3558c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr XP = X->parent;\ 3559c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr R;\ 3560c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (X->bk != X) {\ 3561c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr F = X->fd;\ 3562c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath R = X->bk;\ 3563c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(M, F))) {\ 3564c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath F->bk = R;\ 3565c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath R->fd = F;\ 3566c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3567c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 3568c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(M);\ 3569c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3570c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3571c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 3572c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr* RP;\ 3573c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (((R = *(RP = &(X->child[1]))) != 0) ||\ 3574c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ((R = *(RP = &(X->child[0]))) != 0)) {\ 3575c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr* CP;\ 3576c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while ((*(CP = &(R->child[1])) != 0) ||\ 3577c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (*(CP = &(R->child[0])) != 0)) {\ 3578c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath R = *(RP = CP);\ 3579c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3580c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(M, RP)))\ 3581c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath *RP = 0;\ 3582c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else {\ 3583c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(M);\ 3584c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3585c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3586c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3587c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (XP != 0) {\ 3588c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tbinptr* H = treebin_at(M, X->index);\ 3589c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (X == *H) {\ 3590c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((*H = R) == 0) \ 3591c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath clear_treemap(M, X->index);\ 3592c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3593c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (RTCHECK(ok_address(M, XP))) {\ 3594c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (XP->child[0] == X) \ 3595c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath XP->child[0] = R;\ 3596c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else \ 3597c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath XP->child[1] = R;\ 3598c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3599c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else\ 3600c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(M);\ 3601c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (R != 0) {\ 3602c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(M, R))) {\ 3603c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr C0, C1;\ 3604c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath R->parent = XP;\ 3605c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((C0 = X->child[0]) != 0) {\ 3606c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(M, C0))) {\ 3607c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath R->child[0] = C0;\ 3608c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath C0->parent = R;\ 3609c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3610c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else\ 3611c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(M);\ 3612c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3613c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((C1 = X->child[1]) != 0) {\ 3614c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(M, C1))) {\ 3615c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath R->child[1] = C1;\ 3616c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath C1->parent = R;\ 3617c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3618c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else\ 3619c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(M);\ 3620c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3621c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3622c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else\ 3623c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(M);\ 3624c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3625c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath }\ 3626c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3627c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3628c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Relays to large vs small bin operations */ 3629c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3630c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define insert_chunk(M, P, S)\ 3631c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_small(S)) insert_small_chunk(M, P, S)\ 3632c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { tchunkptr TP = (tchunkptr)(P); insert_large_chunk(M, TP, S); } 3633c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3634c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define unlink_chunk(M, P, S)\ 3635c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_small(S)) unlink_small_chunk(M, P, S)\ 3636c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { tchunkptr TP = (tchunkptr)(P); unlink_large_chunk(M, TP); } 3637c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3638c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3639c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Relays to internal calls to malloc/free from realloc, memalign etc */ 3640c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3641c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if ONLY_MSPACES 3642c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define internal_malloc(m, b) mspace_malloc(m, b) 3643c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define internal_free(m, mem) mspace_free(m,mem); 3644c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* ONLY_MSPACES */ 3645c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if MSPACES 3646c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define internal_malloc(m, b)\ 3647c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (m == gm)? dlmalloc(b) : mspace_malloc(m, b) 3648c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define internal_free(m, mem)\ 3649c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (m == gm) dlfree(mem); else mspace_free(m,mem); 3650c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* MSPACES */ 3651c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define internal_malloc(m, b) dlmalloc(b) 3652c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define internal_free(m, mem) dlfree(mem) 3653c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MSPACES */ 3654c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* ONLY_MSPACES */ 3655c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3656c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ----------------------- Direct-mmapping chunks ----------------------- */ 3657c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3658c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 3659c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Directly mmapped chunks are set up with an offset to the start of 3660c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the mmapped region stored in the prev_foot field of the chunk. This 3661c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath allows reconstruction of the required argument to MUNMAP when freed, 3662c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and also allows adjustment of the returned chunk to meet alignment 3663c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath requirements (especially in memalign). 3664c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 3665c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3666c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Malloc using mmap */ 3667c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void* mmap_alloc(mstate m, size_t nb) { 3668c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t mmsize = mmap_align(nb + SIX_SIZE_T_SIZES + CHUNK_ALIGN_MASK); 3669c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mmsize > nb) { /* Check for wrap around 0 */ 3670c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* mm = (char*)(CALL_DIRECT_MMAP(mmsize)); 3671c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mm != CMFAIL) { 3672c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t offset = align_offset(chunk2mem(mm)); 3673c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t psize = mmsize - offset - MMAP_FOOT_PAD; 3674c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = (mchunkptr)(mm + offset); 3675c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p->prev_foot = offset; 3676c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p->head = psize; 3677c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mark_inuse_foot(m, p, psize); 3678c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk_plus_offset(p, psize)->head = FENCEPOST_HEAD; 3679c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk_plus_offset(p, psize+SIZE_T_SIZE)->head = 0; 3680c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3681c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (m->least_addr == 0 || mm < m->least_addr) 3682c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->least_addr = mm; 3683c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((m->footprint += mmsize) > m->max_footprint) 3684c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->max_footprint = m->footprint; 3685c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(is_aligned(chunk2mem(p))); 3686c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_mmapped_chunk(m, p); 3687c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return chunk2mem(p); 3688c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3689c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3690c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 3691c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3692c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3693c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Realloc using mmap */ 3694c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic mchunkptr mmap_resize(mstate m, mchunkptr oldp, size_t nb) { 3695c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t oldsize = chunksize(oldp); 3696c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_small(nb)) /* Can't shrink mmap regions below small size */ 3697c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 3698c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Keep old chunk if big enough but not too big */ 3699c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (oldsize >= nb + SIZE_T_SIZE && 3700c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (oldsize - nb) <= (mparams.granularity << 1)) 3701c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return oldp; 3702c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 3703c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t offset = oldp->prev_foot; 3704c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t oldmmsize = oldsize + offset + MMAP_FOOT_PAD; 3705c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t newmmsize = mmap_align(nb + SIX_SIZE_T_SIZES + CHUNK_ALIGN_MASK); 3706c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* cp = (char*)CALL_MREMAP((char*)oldp - offset, 3707c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath oldmmsize, newmmsize, 1); 3708c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (cp != CMFAIL) { 3709c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr newp = (mchunkptr)(cp + offset); 3710c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t psize = newmmsize - offset - MMAP_FOOT_PAD; 3711c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath newp->head = psize; 3712c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mark_inuse_foot(m, newp, psize); 3713c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk_plus_offset(newp, psize)->head = FENCEPOST_HEAD; 3714c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk_plus_offset(newp, psize+SIZE_T_SIZE)->head = 0; 3715c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3716c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (cp < m->least_addr) 3717c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->least_addr = cp; 3718c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((m->footprint += newmmsize - oldmmsize) > m->max_footprint) 3719c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->max_footprint = m->footprint; 3720c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_mmapped_chunk(m, newp); 3721c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return newp; 3722c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3723c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3724c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 3725c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3726c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3727c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* -------------------------- mspace management -------------------------- */ 3728c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3729c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Initialize top chunk and its size */ 3730c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void init_top(mstate m, mchunkptr p, size_t psize) { 3731c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Ensure alignment */ 3732c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t offset = align_offset(chunk2mem(p)); 3733c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p = (mchunkptr)((char*)p + offset); 3734c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath psize -= offset; 3735c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3736c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->top = p; 3737c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->topsize = psize; 3738c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p->head = psize | PINUSE_BIT; 3739c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* set size of fake trailing chunk holding overhead space only once */ 3740c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk_plus_offset(p, psize)->head = TOP_FOOT_SIZE; 3741c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->trim_check = mparams.trim_threshold; /* reset on each update */ 3742c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3743c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3744c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Initialize bins for a new mstate that is otherwise zeroed out */ 3745c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void init_bins(mstate m) { 3746c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Establish circular links for smallbins */ 3747c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t i; 3748c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (i = 0; i < NSMALLBINS; ++i) { 3749c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sbinptr bin = smallbin_at(m,i); 3750c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bin->fd = bin->bk = bin; 3751c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3752c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3753c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3754c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if PROCEED_ON_ERROR 3755c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3756c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* default corruption action */ 3757c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void reset_on_error(mstate m) { 3758c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int i; 3759c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ++malloc_corruption_error_count; 3760c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Reinitialize fields to forget about all memory */ 3761c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->smallbins = m->treebins = 0; 3762c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->dvsize = m->topsize = 0; 3763c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.base = 0; 3764c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.size = 0; 3765c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.next = 0; 3766c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->top = m->dv = 0; 3767c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (i = 0; i < NTREEBINS; ++i) 3768c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath *treebin_at(m, i) = 0; 3769c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath init_bins(m); 3770c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3771c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* PROCEED_ON_ERROR */ 3772c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3773c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Allocate chunk and prepend remainder with chunk in successor base. */ 3774c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void* prepend_alloc(mstate m, char* newbase, char* oldbase, 3775c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t nb) { 3776c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = align_as_chunk(newbase); 3777c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr oldfirst = align_as_chunk(oldbase); 3778c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t psize = (char*)oldfirst - (char*)p; 3779c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr q = chunk_plus_offset(p, nb); 3780c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t qsize = psize - nb; 3781c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(m, p, nb); 3782c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3783c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert((char*)oldfirst > (char*)q); 3784c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(pinuse(oldfirst)); 3785c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(qsize >= MIN_CHUNK_SIZE); 3786c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3787c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* consolidate remainder with first chunk of old base */ 3788c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (oldfirst == m->top) { 3789c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t tsize = m->topsize += qsize; 3790c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->top = q; 3791c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath q->head = tsize | PINUSE_BIT; 3792c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_top_chunk(m, q); 3793c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3794c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (oldfirst == m->dv) { 3795c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t dsize = m->dvsize += qsize; 3796c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->dv = q; 3797c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_free_chunk(q, dsize); 3798c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3799c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 3800c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!is_inuse(oldfirst)) { 3801c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t nsize = chunksize(oldfirst); 3802c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unlink_chunk(m, oldfirst, nsize); 3803c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath oldfirst = chunk_plus_offset(oldfirst, nsize); 3804c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath qsize += nsize; 3805c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3806c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_free_with_pinuse(q, qsize, oldfirst); 3807c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath insert_chunk(m, q, qsize); 3808c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_free_chunk(m, q); 3809c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3810c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3811c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(m, chunk2mem(p), nb); 3812c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return chunk2mem(p); 3813c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3814c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3815c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Add a segment to hold a new noncontiguous region */ 3816c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void add_segment(mstate m, char* tbase, size_t tsize, flag_t mmapped) { 3817c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Determine locations and sizes of segment, fenceposts, old top */ 3818c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* old_top = (char*)m->top; 3819c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr oldsp = segment_holding(m, old_top); 3820c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* old_end = oldsp->base + oldsp->size; 3821c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t ssize = pad_request(sizeof(struct malloc_segment)); 3822c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* rawsp = old_end - (ssize + FOUR_SIZE_T_SIZES + CHUNK_ALIGN_MASK); 3823c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t offset = align_offset(chunk2mem(rawsp)); 3824c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* asp = rawsp + offset; 3825c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* csp = (asp < (old_top + MIN_CHUNK_SIZE))? old_top : asp; 3826c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr sp = (mchunkptr)csp; 3827c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr ss = (msegmentptr)(chunk2mem(sp)); 3828c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr tnext = chunk_plus_offset(sp, ssize); 3829c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = tnext; 3830c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int nfences = 0; 3831c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3832c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* reset top to new space */ 3833c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath init_top(m, (mchunkptr)tbase, tsize - TOP_FOOT_SIZE); 3834c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3835c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Set up segment record */ 3836c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(is_aligned(ss)); 3837c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(m, sp, ssize); 3838c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath *ss = m->seg; /* Push current record */ 3839c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.base = tbase; 3840c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.size = tsize; 3841c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.sflags = mmapped; 3842c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.next = ss; 3843c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3844c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Insert trailing fenceposts */ 3845c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (;;) { 3846c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr nextp = chunk_plus_offset(p, SIZE_T_SIZE); 3847c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p->head = FENCEPOST_HEAD; 3848c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ++nfences; 3849c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((char*)(&(nextp->head)) < old_end) 3850c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p = nextp; 3851c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 3852c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath break; 3853c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3854c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(nfences >= 2); 3855c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3856c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Insert the rest of old top into a bin as an ordinary free chunk */ 3857c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (csp != old_top) { 3858c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr q = (mchunkptr)old_top; 3859c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t psize = csp - old_top; 3860c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr tn = chunk_plus_offset(q, psize); 3861c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_free_with_pinuse(q, psize, tn); 3862c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath insert_chunk(m, q, psize); 3863c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3864c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3865c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_top_chunk(m, m->top); 3866c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 3867c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3868c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* -------------------------- System allocation -------------------------- */ 3869c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3870c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Get memory from system using MORECORE or MMAP */ 3871c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void* sys_alloc(mstate m, size_t nb) { 3872c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* tbase = CMFAIL; 3873c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t tsize = 0; 3874c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath flag_t mmap_flag = 0; 3875c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3876c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensure_initialization(); 3877c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3878c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Directly map large chunks, but only if already initialized */ 3879c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (use_mmap(m) && nb >= mparams.mmap_threshold && m->topsize != 0) { 3880c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* mem = mmap_alloc(m, nb); 3881c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mem != 0) 3882c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return mem; 3883c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3884c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3885c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* 3886c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Try getting memory in any of three ways (in most-preferred to 3887c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath least-preferred order): 3888c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1. A call to MORECORE that can normally contiguously extend memory. 3889c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (disabled if not MORECORE_CONTIGUOUS or not HAVE_MORECORE or 3890c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath or main space is mmapped or a previous contiguous call failed) 3891c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2. A call to MMAP new space (disabled if not HAVE_MMAP). 3892c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Note that under the default settings, if MORECORE is unable to 3893c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fulfill a request, and HAVE_MMAP is true, then mmap is 3894c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath used as a noncontiguous system allocator. This is a useful backup 3895c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath strategy for systems with holes in address spaces -- in this case 3896c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sbrk cannot contiguously expand the heap, but mmap may be able to 3897c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath find space. 3898c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3. A call to MORECORE that cannot usually contiguously extend memory. 3899c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (disabled if not HAVE_MORECORE) 3900c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3901c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath In all cases, we need to request enough bytes from system to ensure 3902c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath we can malloc nb bytes upon success, so pad with enough space for 3903c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath top_foot, plus alignment-pad to make sure we don't lose bytes if 3904c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath not on boundary, and round this up to a granularity unit. 3905c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath */ 3906c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3907c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (MORECORE_CONTIGUOUS && !use_noncontiguous(m)) { 3908c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* br = CMFAIL; 3909c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr ss = (m->top == 0)? 0 : segment_holding(m, (char*)m->top); 3910c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t asize = 0; 3911c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ACQUIRE_MALLOC_GLOBAL_LOCK(); 3912c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3913c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (ss == 0) { /* First time through or recovery */ 3914c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* base = (char*)CALL_MORECORE(0); 3915c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (base != CMFAIL) { 3916c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath asize = granularity_align(nb + SYS_ALLOC_PADDING); 3917c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Adjust to end on a page boundary */ 3918c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!is_page_aligned(base)) 3919c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath asize += (page_align((size_t)base) - (size_t)base); 3920c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Can't call MORECORE if size is negative when treated as signed */ 3921c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (asize < HALF_MAX_SIZE_T && 3922c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (br = (char*)(CALL_MORECORE(asize))) == base) { 3923c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tbase = base; 3924c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tsize = asize; 3925c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3926c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3927c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3928c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 3929c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Subtract out existing available top space from MORECORE request. */ 3930c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath asize = granularity_align(nb - m->topsize + SYS_ALLOC_PADDING); 3931c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Use mem here only if it did continuously extend old space */ 3932c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (asize < HALF_MAX_SIZE_T && 3933c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (br = (char*)(CALL_MORECORE(asize))) == ss->base+ss->size) { 3934c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tbase = br; 3935c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tsize = asize; 3936c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3937c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3938c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3939c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (tbase == CMFAIL) { /* Cope with partial failure */ 3940c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (br != CMFAIL) { /* Try to use/extend the space we did get */ 3941c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (asize < HALF_MAX_SIZE_T && 3942c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath asize < nb + SYS_ALLOC_PADDING) { 3943c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t esize = granularity_align(nb + SYS_ALLOC_PADDING - asize); 3944c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (esize < HALF_MAX_SIZE_T) { 3945c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* end = (char*)CALL_MORECORE(esize); 3946c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (end != CMFAIL) 3947c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath asize += esize; 3948c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { /* Can't use; try to release */ 3949c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (void) CALL_MORECORE(-asize); 3950c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath br = CMFAIL; 3951c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3952c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3953c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3954c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3955c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (br != CMFAIL) { /* Use the space we did get */ 3956c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tbase = br; 3957c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tsize = asize; 3958c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3959c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 3960c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath disable_contiguous(m); /* Don't try contiguous path in the future */ 3961c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3962c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3963c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath RELEASE_MALLOC_GLOBAL_LOCK(); 3964c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3965c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3966c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (HAVE_MMAP && tbase == CMFAIL) { /* Try MMAP */ 3967c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t rsize = granularity_align(nb + SYS_ALLOC_PADDING); 3968c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (rsize > nb) { /* Fail if wraps around zero */ 3969c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* mp = (char*)(CALL_MMAP(rsize)); 3970c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mp != CMFAIL) { 3971c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tbase = mp; 3972c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tsize = rsize; 3973c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mmap_flag = USE_MMAP_BIT; 3974c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3975c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3976c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3977c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3978c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (HAVE_MORECORE && tbase == CMFAIL) { /* Try noncontiguous MORECORE */ 3979c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t asize = granularity_align(nb + SYS_ALLOC_PADDING); 3980c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (asize < HALF_MAX_SIZE_T) { 3981c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* br = CMFAIL; 3982c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* end = CMFAIL; 3983c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ACQUIRE_MALLOC_GLOBAL_LOCK(); 3984c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath br = (char*)(CALL_MORECORE(asize)); 3985c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath end = (char*)(CALL_MORECORE(0)); 3986c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath RELEASE_MALLOC_GLOBAL_LOCK(); 3987c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (br != CMFAIL && end != CMFAIL && br < end) { 3988c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t ssize = end - br; 3989c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (ssize > nb + TOP_FOOT_SIZE) { 3990c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tbase = br; 3991c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tsize = ssize; 3992c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3993c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3994c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3995c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 3996c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3997c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (tbase != CMFAIL) { 3998c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3999c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((m->footprint += tsize) > m->max_footprint) 4000c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->max_footprint = m->footprint; 4001c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4002c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!is_initialized(m)) { /* first-time initialization */ 4003c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (m->least_addr == 0 || tbase < m->least_addr) 4004c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->least_addr = tbase; 4005c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.base = tbase; 4006c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.size = tsize; 4007c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.sflags = mmap_flag; 4008c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->magic = mparams.magic; 4009c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->release_checks = MAX_RELEASE_CHECK_RATE; 4010c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath init_bins(m); 4011c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !ONLY_MSPACES 4012c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_global(m)) 4013c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath init_top(m, (mchunkptr)tbase, tsize - TOP_FOOT_SIZE); 4014c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 4015c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 4016c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath { 4017c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Offset top by embedded malloc_state */ 4018c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr mn = next_chunk(mem2chunk(m)); 4019c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath init_top(m, mn, (size_t)((tbase + tsize) - (char*)mn) -TOP_FOOT_SIZE); 4020c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4021c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4022c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4023c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4024c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Try to merge with an existing segment */ 4025c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr sp = &m->seg; 4026c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Only consider most recent segment if traversal suppressed */ 4027c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (sp != 0 && tbase != sp->base + sp->size) 4028c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sp = (NO_SEGMENT_TRAVERSAL) ? 0 : sp->next; 4029c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (sp != 0 && 4030c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath !is_extern_segment(sp) && 4031c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (sp->sflags & USE_MMAP_BIT) == mmap_flag && 4032c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath segment_holds(sp, m->top)) { /* append */ 4033c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sp->size += tsize; 4034c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath init_top(m, m->top, m->topsize + tsize); 4035c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4036c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4037c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (tbase < m->least_addr) 4038c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->least_addr = tbase; 4039c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sp = &m->seg; 4040c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (sp != 0 && sp->base != tbase + tsize) 4041c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sp = (NO_SEGMENT_TRAVERSAL) ? 0 : sp->next; 4042c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (sp != 0 && 4043c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath !is_extern_segment(sp) && 4044c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (sp->sflags & USE_MMAP_BIT) == mmap_flag) { 4045c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* oldbase = sp->base; 4046c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sp->base = tbase; 4047c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sp->size += tsize; 4048c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return prepend_alloc(m, tbase, oldbase, nb); 4049c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4050c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 4051c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath add_segment(m, tbase, tsize, mmap_flag); 4052c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4053c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4054c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4055c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (nb < m->topsize) { /* Allocate from new or extended top space */ 4056c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t rsize = m->topsize -= nb; 4057c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = m->top; 4058c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr r = m->top = chunk_plus_offset(p, nb); 4059c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath r->head = rsize | PINUSE_BIT; 4060c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(m, p, nb); 4061c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_top_chunk(m, m->top); 4062c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(m, chunk2mem(p), nb); 4063c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return chunk2mem(p); 4064c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4065c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4066c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4067c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLOC_FAILURE_ACTION; 4068c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 4069c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4070c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4071c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ----------------------- system deallocation -------------------------- */ 4072c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4073c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* Unmap and unlink any mmapped segments that don't contain used chunks */ 4074c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic size_t release_unused_segments(mstate m) { 4075c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t released = 0; 4076c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int nsegs = 0; 4077c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr pred = &m->seg; 4078c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr sp = pred->next; 4079c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (sp != 0) { 4080c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* base = sp->base; 4081c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t size = sp->size; 4082c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr next = sp->next; 4083c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ++nsegs; 4084c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_mmapped_segment(sp) && !is_extern_segment(sp)) { 4085c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = align_as_chunk(base); 4086c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t psize = chunksize(p); 4087c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Can unmap if first chunk holds entire segment and not pinned */ 4088c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!is_inuse(p) && (char*)p + psize >= base + size - TOP_FOOT_SIZE) { 4089c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr tp = (tchunkptr)p; 4090c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(segment_holds(sp, (char*)sp)); 4091c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (p == m->dv) { 4092c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->dv = 0; 4093c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->dvsize = 0; 4094c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4095c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4096c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unlink_large_chunk(m, tp); 4097c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4098c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (CALL_MUNMAP(base, size) == 0) { 4099c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath released += size; 4100c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->footprint -= size; 4101c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* unlink obsoleted record */ 4102c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sp = pred; 4103c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sp->next = next; 4104c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4105c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { /* back out if cannot unmap */ 4106c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath insert_large_chunk(m, tp, psize); 4107c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4108c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4109c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4110c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (NO_SEGMENT_TRAVERSAL) /* scan only first segment */ 4111c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath break; 4112c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pred = sp; 4113c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sp = next; 4114c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4115c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Reset check counter */ 4116c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->release_checks = ((nsegs > MAX_RELEASE_CHECK_RATE)? 4117c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nsegs : MAX_RELEASE_CHECK_RATE); 4118c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return released; 4119c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4120c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4121c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic int sys_trim(mstate m, size_t pad) { 4122c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t released = 0; 4123c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensure_initialization(); 4124c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (pad < MAX_REQUEST && is_initialized(m)) { 4125c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pad += TOP_FOOT_SIZE; /* ensure enough room for segment overhead */ 4126c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4127c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (m->topsize > pad) { 4128c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Shrink top space in granularity-size units, keeping at least one */ 4129c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t unit = mparams.granularity; 4130c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t extra = ((m->topsize - pad + (unit - SIZE_T_ONE)) / unit - 4131c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath SIZE_T_ONE) * unit; 4132c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr sp = segment_holding(m, (char*)m->top); 4133c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4134c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!is_extern_segment(sp)) { 4135c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_mmapped_segment(sp)) { 4136c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (HAVE_MMAP && 4137c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sp->size >= extra && 4138c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath !has_segment_link(m, sp)) { /* can't shrink if pinned */ 4139c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t newsize = sp->size - extra; 4140c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Prefer mremap, fall back to munmap */ 4141c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((CALL_MREMAP(sp->base, sp->size, newsize, 0) != MFAIL) || 4142c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (CALL_MUNMAP(sp->base + newsize, extra) == 0)) { 4143c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath released = extra; 4144c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4145c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4146c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4147c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (HAVE_MORECORE) { 4148c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (extra >= HALF_MAX_SIZE_T) /* Avoid wrapping negative */ 4149c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath extra = (HALF_MAX_SIZE_T) + SIZE_T_ONE - unit; 4150c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ACQUIRE_MALLOC_GLOBAL_LOCK(); 4151c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath { 4152c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Make sure end of memory is where we last set it. */ 4153c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* old_br = (char*)(CALL_MORECORE(0)); 4154c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (old_br == sp->base + sp->size) { 4155c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* rel_br = (char*)(CALL_MORECORE(-extra)); 4156c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* new_br = (char*)(CALL_MORECORE(0)); 4157c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (rel_br != CMFAIL && new_br < old_br) 4158c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath released = old_br - new_br; 4159c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4160c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4161c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath RELEASE_MALLOC_GLOBAL_LOCK(); 4162c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4163c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4164c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4165c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (released != 0) { 4166c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sp->size -= released; 4167c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->footprint -= released; 4168c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath init_top(m, m->top, m->topsize - released); 4169c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_top_chunk(m, m->top); 4170c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4171c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4172c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4173c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Unmap any unused mmapped segments */ 4174c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (HAVE_MMAP) 4175c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath released += release_unused_segments(m); 4176c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4177c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* On failure, disable autotrim to avoid repeated failed future calls */ 4178c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (released == 0 && m->topsize > m->trim_check) 4179c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->trim_check = MAX_SIZE_T; 4180c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4181c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4182c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return (released != 0)? 1 : 0; 4183c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4184c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4185c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4186c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ---------------------------- malloc support --------------------------- */ 4187c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4188c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* allocate a large request from the best fitting chunk in a treebin */ 4189c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void* tmalloc_large(mstate m, size_t nb) { 4190c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr v = 0; 4191c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t rsize = -nb; /* Unsigned negation */ 4192c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr t; 4193c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t idx; 4194c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath compute_tree_index(nb, idx); 4195c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((t = *treebin_at(m, idx)) != 0) { 4196c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Traverse tree for this bin looking for node with size == nb */ 4197c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sizebits = nb << leftshift_for_tree_index(idx); 4198c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr rst = 0; /* The deepest untaken right subtree */ 4199c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (;;) { 4200c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr rt; 4201c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t trem = chunksize(t) - nb; 4202c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (trem < rsize) { 4203c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath v = t; 4204c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((rsize = trem) == 0) 4205c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath break; 4206c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4207c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath rt = t->child[1]; 4208c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath t = t->child[(sizebits >> (SIZE_T_BITSIZE-SIZE_T_ONE)) & 1]; 4209c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (rt != 0 && rt != t) 4210c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath rst = rt; 4211c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (t == 0) { 4212c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath t = rst; /* set t to least subtree holding sizes > nb */ 4213c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath break; 4214c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4215c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sizebits <<= 1; 4216c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4217c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4218c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (t == 0 && v == 0) { /* set t to root of next non-empty treebin */ 4219c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath binmap_t leftbits = left_bits(idx2bit(idx)) & m->treemap; 4220c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (leftbits != 0) { 4221c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t i; 4222c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath binmap_t leastbit = least_bit(leftbits); 4223c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath compute_bit2idx(leastbit, i); 4224c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath t = *treebin_at(m, i); 4225c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4226c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4227c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4228c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (t != 0) { /* find smallest of tree or subtree */ 4229c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t trem = chunksize(t) - nb; 4230c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (trem < rsize) { 4231c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath rsize = trem; 4232c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath v = t; 4233c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4234c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath t = leftmost_child(t); 4235c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4236c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4237c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* If dv is a better fit, return 0 so malloc will use it */ 4238c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (v != 0 && rsize < (size_t)(m->dvsize - nb)) { 4239c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(m, v))) { /* split */ 4240c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr r = chunk_plus_offset(v, nb); 4241c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(chunksize(v) == rsize + nb); 4242c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_next(v, r))) { 4243c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unlink_large_chunk(m, v); 4244c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (rsize < MIN_CHUNK_SIZE) 4245c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse_and_pinuse(m, v, (rsize + nb)); 4246c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4247c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(m, v, nb); 4248c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_free_chunk(r, rsize); 4249c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath insert_chunk(m, r, rsize); 4250c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4251c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return chunk2mem(v); 4252c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4253c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4254c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(m); 4255c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4256c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 4257c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4258c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4259c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* allocate a small request from the best fitting chunk in a treebin */ 4260c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void* tmalloc_small(mstate m, size_t nb) { 4261c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr t, v; 4262c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t rsize; 4263c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t i; 4264c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath binmap_t leastbit = least_bit(m->treemap); 4265c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath compute_bit2idx(leastbit, i); 4266c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath v = t = *treebin_at(m, i); 4267c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath rsize = chunksize(t) - nb; 4268c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4269c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while ((t = leftmost_child(t)) != 0) { 4270c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t trem = chunksize(t) - nb; 4271c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (trem < rsize) { 4272c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath rsize = trem; 4273c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath v = t; 4274c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4275c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4276c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4277c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(m, v))) { 4278c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr r = chunk_plus_offset(v, nb); 4279c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(chunksize(v) == rsize + nb); 4280c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_next(v, r))) { 4281c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unlink_large_chunk(m, v); 4282c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (rsize < MIN_CHUNK_SIZE) 4283c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse_and_pinuse(m, v, (rsize + nb)); 4284c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4285c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(m, v, nb); 4286c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_free_chunk(r, rsize); 4287c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath replace_dv(m, r, rsize); 4288c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4289c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return chunk2mem(v); 4290c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4291c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4292c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4293c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CORRUPTION_ERROR_ACTION(m); 4294c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 4295c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4296c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4297c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* --------------------------- realloc support --------------------------- */ 4298c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4299c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void* internal_realloc(mstate m, void* oldmem, size_t bytes) { 4300c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (bytes >= MAX_REQUEST) { 4301c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLOC_FAILURE_ACTION; 4302c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 4303c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4304c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!PREACTION(m)) { 4305c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr oldp = mem2chunk(oldmem); 4306c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t oldsize = chunksize(oldp); 4307c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr next = chunk_plus_offset(oldp, oldsize); 4308c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr newp = 0; 4309c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* extra = 0; 4310c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4311c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Try to either shrink or extend into top. Else malloc-copy-free */ 4312c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4313c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(m, oldp) && ok_inuse(oldp) && 4314c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ok_next(oldp, next) && ok_pinuse(next))) { 4315c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t nb = request2size(bytes); 4316c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_mmapped(oldp)) 4317c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath newp = mmap_resize(m, oldp, nb); 4318c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (oldsize >= nb) { /* already big enough */ 4319c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t rsize = oldsize - nb; 4320c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath newp = oldp; 4321c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (rsize >= MIN_CHUNK_SIZE) { 4322c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr remainder = chunk_plus_offset(newp, nb); 4323c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse(m, newp, nb); 4324c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse_and_pinuse(m, remainder, rsize); 4325c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath extra = chunk2mem(remainder); 4326c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4327c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4328c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (next == m->top && oldsize + m->topsize > nb) { 4329c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Expand into top */ 4330c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t newsize = oldsize + m->topsize; 4331c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t newtopsize = newsize - nb; 4332c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr newtop = chunk_plus_offset(oldp, nb); 4333c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse(m, oldp, nb); 4334c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath newtop->head = newtopsize |PINUSE_BIT; 4335c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->top = newtop; 4336c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->topsize = newtopsize; 4337c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath newp = oldp; 4338c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4339c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4340c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4341c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(m, oldmem); 4342c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(m); 4343c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 4344c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4345c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if DEBUG 4346c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (newp != 0) { 4347c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_inuse_chunk(m, newp); /* Check requires lock */ 4348c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4349c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 4350c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4351c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(m); 4352c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4353c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (newp != 0) { 4354c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (extra != 0) { 4355c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath internal_free(m, extra); 4356c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4357c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return chunk2mem(newp); 4358c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4359c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4360c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* newmem = internal_malloc(m, bytes); 4361c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (newmem != 0) { 4362c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t oc = oldsize - overhead_for(oldp); 4363c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memcpy(newmem, oldmem, (oc < bytes)? oc : bytes); 4364c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath internal_free(m, oldmem); 4365c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4366c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return newmem; 4367c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4368c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4369c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 4370c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4371c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4372c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* --------------------------- memalign support -------------------------- */ 4373c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4374c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void* internal_memalign(mstate m, size_t alignment, size_t bytes) { 4375c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (alignment <= MALLOC_ALIGNMENT) /* Can just use malloc */ 4376c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return internal_malloc(m, bytes); 4377c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (alignment < MIN_CHUNK_SIZE) /* must be at least a minimum chunk size */ 4378c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath alignment = MIN_CHUNK_SIZE; 4379c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((alignment & (alignment-SIZE_T_ONE)) != 0) {/* Ensure a power of 2 */ 4380c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t a = MALLOC_ALIGNMENT << 1; 4381c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (a < alignment) a <<= 1; 4382c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath alignment = a; 4383c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4384c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4385c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (bytes >= MAX_REQUEST - alignment) { 4386c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (m != 0) { /* Test isn't needed but avoids compiler warning */ 4387c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MALLOC_FAILURE_ACTION; 4388c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4389c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4390c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4391c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t nb = request2size(bytes); 4392c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t req = nb + alignment + MIN_CHUNK_SIZE - CHUNK_OVERHEAD; 4393c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* mem = (char*)internal_malloc(m, req); 4394c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mem != 0) { 4395c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* leader = 0; 4396c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* trailer = 0; 4397c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = mem2chunk(mem); 4398c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4399c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (PREACTION(m)) return 0; 4400c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((((size_t)(mem)) % alignment) != 0) { /* misaligned */ 4401c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* 4402c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Find an aligned spot inside chunk. Since we need to give 4403c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath back leading space in a chunk of at least MIN_CHUNK_SIZE, if 4404c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the first calculation places us at a spot with less than 4405c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MIN_CHUNK_SIZE leader, we can move to the next aligned spot. 4406c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath We've allocated enough total room so that this is always 4407c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath possible. 4408c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath */ 4409c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* br = (char*)mem2chunk((size_t)(((size_t)(mem + 4410c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath alignment - 4411c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath SIZE_T_ONE)) & 4412c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath -alignment)); 4413c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* pos = ((size_t)(br - (char*)(p)) >= MIN_CHUNK_SIZE)? 4414c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath br : br+alignment; 4415c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr newp = (mchunkptr)pos; 4416c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t leadsize = pos - (char*)(p); 4417c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t newsize = chunksize(p) - leadsize; 4418c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4419c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_mmapped(p)) { /* For mmapped chunks, just adjust offset */ 4420c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath newp->prev_foot = p->prev_foot + leadsize; 4421c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath newp->head = newsize; 4422c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4423c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { /* Otherwise, give back leader, use the rest */ 4424c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse(m, newp, newsize); 4425c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse(m, p, leadsize); 4426c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath leader = chunk2mem(p); 4427c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4428c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p = newp; 4429c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4430c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4431c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Give back spare room at the end */ 4432c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!is_mmapped(p)) { 4433c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t size = chunksize(p); 4434c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (size > nb + MIN_CHUNK_SIZE) { 4435c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t remainder_size = size - nb; 4436c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr remainder = chunk_plus_offset(p, nb); 4437c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse(m, p, nb); 4438c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse(m, remainder, remainder_size); 4439c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath trailer = chunk2mem(remainder); 4440c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4441c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4442c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4443c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert (chunksize(p) >= nb); 4444c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert((((size_t)(chunk2mem(p))) % alignment) == 0); 4445c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_inuse_chunk(m, p); 4446c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(m); 4447c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (leader != 0) { 4448c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath internal_free(m, leader); 4449c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4450c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (trailer != 0) { 4451c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath internal_free(m, trailer); 4452c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4453c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return chunk2mem(p); 4454c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4455c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4456c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 4457c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4458c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4459c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ------------------------ comalloc/coalloc support --------------------- */ 4460c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4461c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic void** ialloc(mstate m, 4462c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t n_elements, 4463c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t* sizes, 4464c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int opts, 4465c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* chunks[]) { 4466c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* 4467c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath This provides common support for independent_X routines, handling 4468c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath all of the combinations that can result. 4469c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4470c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The opts arg has: 4471c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bit 0 set if all elements are same size (using sizes[0]) 4472c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bit 1 set if elements should be zeroed 4473c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath */ 4474c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4475c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t element_size; /* chunksize of each element, if all same */ 4476c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t contents_size; /* total size of elements */ 4477c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t array_size; /* request size of pointer array */ 4478c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* mem; /* malloced aggregate space */ 4479c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p; /* corresponding chunk */ 4480c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t remainder_size; /* remaining bytes while splitting */ 4481c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void** marray; /* either "chunks" or malloced ptr array */ 4482c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr array_chunk; /* chunk for malloced ptr array */ 4483c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath flag_t was_enabled; /* to disable mmap */ 4484c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t size; 4485c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t i; 4486c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4487c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensure_initialization(); 4488c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* compute array length, if needed */ 4489c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (chunks != 0) { 4490c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (n_elements == 0) 4491c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return chunks; /* nothing to do */ 4492c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath marray = chunks; 4493c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath array_size = 0; 4494c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4495c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4496c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* if empty req, must still return chunk representing empty array */ 4497c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (n_elements == 0) 4498c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return (void**)internal_malloc(m, 0); 4499c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath marray = 0; 4500c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath array_size = request2size(n_elements * (sizeof(void*))); 4501c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4502c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4503c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* compute total element size */ 4504c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (opts & 0x1) { /* all-same-size */ 4505c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath element_size = request2size(*sizes); 4506c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath contents_size = n_elements * element_size; 4507c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4508c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { /* add up all the sizes */ 4509c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath element_size = 0; 4510c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath contents_size = 0; 4511c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (i = 0; i != n_elements; ++i) 4512c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath contents_size += request2size(sizes[i]); 4513c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4514c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4515c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size = contents_size + array_size; 4516c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4517c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* 4518c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Allocate the aggregate chunk. First disable direct-mmapping so 4519c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath malloc won't use it, since we would not be able to later 4520c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath free/realloc space internal to a segregated mmap region. 4521c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath */ 4522c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath was_enabled = use_mmap(m); 4523c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath disable_mmap(m); 4524c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = internal_malloc(m, size - CHUNK_OVERHEAD); 4525c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (was_enabled) 4526c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath enable_mmap(m); 4527c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mem == 0) 4528c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 4529c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4530c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (PREACTION(m)) return 0; 4531c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p = mem2chunk(mem); 4532c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath remainder_size = chunksize(p); 4533c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4534c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(!is_mmapped(p)); 4535c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4536c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (opts & 0x2) { /* optionally clear the elements */ 4537c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memset((size_t*)mem, 0, remainder_size - SIZE_T_SIZE - array_size); 4538c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4539c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4540c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* If not provided, allocate the pointer array as final part of chunk */ 4541c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (marray == 0) { 4542c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t array_chunk_size; 4543c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath array_chunk = chunk_plus_offset(p, contents_size); 4544c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath array_chunk_size = remainder_size - contents_size; 4545c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath marray = (void**) (chunk2mem(array_chunk)); 4546c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(m, array_chunk, array_chunk_size); 4547c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath remainder_size = contents_size; 4548c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4549c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4550c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* split out elements */ 4551c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (i = 0; ; ++i) { 4552c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath marray[i] = chunk2mem(p); 4553c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (i != n_elements-1) { 4554c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (element_size != 0) 4555c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size = element_size; 4556c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 4557c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size = request2size(sizes[i]); 4558c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath remainder_size -= size; 4559c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(m, p, size); 4560c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p = chunk_plus_offset(p, size); 4561c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4562c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { /* the final element absorbs any overallocation slop */ 4563c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(m, p, remainder_size); 4564c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath break; 4565c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4566c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4567c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4568c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if DEBUG 4569c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (marray != chunks) { 4570c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* final element must have exactly exhausted chunk */ 4571c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (element_size != 0) { 4572c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(remainder_size == element_size); 4573c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4574c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4575c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(remainder_size == request2size(sizes[i])); 4576c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4577c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_inuse_chunk(m, mem2chunk(marray)); 4578c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4579c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (i = 0; i != n_elements; ++i) 4580c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_inuse_chunk(m, mem2chunk(marray[i])); 4581c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4582c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* DEBUG */ 4583c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4584c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(m); 4585c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return marray; 4586c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4587c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4588c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4589c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* -------------------------- public routines ---------------------------- */ 4590c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4591c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !ONLY_MSPACES 4592c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4593c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* dlmalloc(size_t bytes) { 4594c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* 4595c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Basic algorithm: 4596c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath If a small request (< 256 bytes minus per-chunk overhead): 4597c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1. If one exists, use a remainderless chunk in associated smallbin. 4598c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (Remainderless means that there are too few excess bytes to 4599c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath represent as a chunk.) 4600c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2. If it is big enough, use the dv chunk, which is normally the 4601c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath chunk adjacent to the one used for the most recent small request. 4602c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3. If one exists, split the smallest available chunk in a bin, 4603c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath saving remainder in dv. 4604c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4. If it is big enough, use the top chunk. 4605c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5. If available, get memory from system and use it 4606c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Otherwise, for a large request: 4607c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 1. Find the smallest available binned chunk that fits, and use it 4608c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if it is better fitting than dv chunk, splitting if necessary. 4609c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 2. If better fitting than any binned chunk, use the dv chunk. 4610c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 3. If it is big enough, use the top chunk. 4611c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4. If request size >= mmap threshold, try to directly mmap this chunk. 4612c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5. If available, get memory from system and use it 4613c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4614c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath The ugly goto's here ensure that postaction occurs along all paths. 4615c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath */ 4616c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4617c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if USE_LOCKS 4618c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensure_initialization(); /* initialize in sys_alloc if not using locks */ 4619c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif 4620c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4621c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!PREACTION(gm)) { 4622c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* mem; 4623c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t nb; 4624c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (bytes <= MAX_SMALL_REQUEST) { 4625c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t idx; 4626c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath binmap_t smallbits; 4627c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nb = (bytes < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(bytes); 4628c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath idx = small_index(nb); 4629c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath smallbits = gm->smallmap >> idx; 4630c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4631c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((smallbits & 0x3U) != 0) { /* Remainderless fit to a smallbin. */ 4632c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr b, p; 4633c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath idx += ~smallbits & 1; /* Uses next bin if idx empty */ 4634c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath b = smallbin_at(gm, idx); 4635c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p = b->fd; 4636c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(chunksize(p) == small_index2size(idx)); 4637c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unlink_first_small_chunk(gm, b, p, idx); 4638c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse_and_pinuse(gm, p, small_index2size(idx)); 4639c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = chunk2mem(p); 4640c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(gm, mem, nb); 4641c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 4642c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4643c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4644c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (nb > gm->dvsize) { 4645c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (smallbits != 0) { /* Use chunk in next nonempty smallbin */ 4646c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr b, p, r; 4647c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t rsize; 4648c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t i; 4649c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath binmap_t leftbits = (smallbits << idx) & left_bits(idx2bit(idx)); 4650c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath binmap_t leastbit = least_bit(leftbits); 4651c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath compute_bit2idx(leastbit, i); 4652c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath b = smallbin_at(gm, i); 4653c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p = b->fd; 4654c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(chunksize(p) == small_index2size(i)); 4655c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unlink_first_small_chunk(gm, b, p, i); 4656c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath rsize = small_index2size(i) - nb; 4657c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Fit here cannot be remainderless if 4byte sizes */ 4658c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (SIZE_T_SIZE != 4 && rsize < MIN_CHUNK_SIZE) 4659c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse_and_pinuse(gm, p, small_index2size(i)); 4660c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4661c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(gm, p, nb); 4662c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath r = chunk_plus_offset(p, nb); 4663c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_free_chunk(r, rsize); 4664c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath replace_dv(gm, r, rsize); 4665c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4666c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = chunk2mem(p); 4667c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(gm, mem, nb); 4668c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 4669c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4670c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4671c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (gm->treemap != 0 && (mem = tmalloc_small(gm, nb)) != 0) { 4672c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(gm, mem, nb); 4673c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 4674c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4675c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4676c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4677c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (bytes >= MAX_REQUEST) 4678c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nb = MAX_SIZE_T; /* Too big to allocate. Force failure (in sys alloc) */ 4679c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4680c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nb = pad_request(bytes); 4681c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (gm->treemap != 0 && (mem = tmalloc_large(gm, nb)) != 0) { 4682c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(gm, mem, nb); 4683c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 4684c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4685c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4686c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4687c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (nb <= gm->dvsize) { 4688c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t rsize = gm->dvsize - nb; 4689c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = gm->dv; 4690c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (rsize >= MIN_CHUNK_SIZE) { /* split dv */ 4691c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr r = gm->dv = chunk_plus_offset(p, nb); 4692c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath gm->dvsize = rsize; 4693c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_free_chunk(r, rsize); 4694c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(gm, p, nb); 4695c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4696c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { /* exhaust dv */ 4697c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t dvs = gm->dvsize; 4698c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath gm->dvsize = 0; 4699c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath gm->dv = 0; 4700c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse_and_pinuse(gm, p, dvs); 4701c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4702c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = chunk2mem(p); 4703c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(gm, mem, nb); 4704c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 4705c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4706c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4707c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (nb < gm->topsize) { /* Split top */ 4708c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t rsize = gm->topsize -= nb; 4709c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = gm->top; 4710c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr r = gm->top = chunk_plus_offset(p, nb); 4711c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath r->head = rsize | PINUSE_BIT; 4712c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(gm, p, nb); 4713c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = chunk2mem(p); 4714c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_top_chunk(gm, gm->top); 4715c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(gm, mem, nb); 4716c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 4717c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4718c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4719c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = sys_alloc(gm, nb); 4720c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4721c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath postaction: 4722c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(gm); 4723c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return mem; 4724c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4725c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4726c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 4727c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4728c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4729c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid dlfree(void* mem) { 4730c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* 4731c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Consolidate freed chunks with preceeding or succeeding bordering 4732c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath free chunks, if they exist, and then place in a bin. Intermixed 4733c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath with special cases for top, dv, mmapped chunks, and usage errors. 4734c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath */ 4735c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4736c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mem != 0) { 4737c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = mem2chunk(mem); 4738c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if FOOTERS 4739c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate fm = get_mstate_for(p); 4740c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!ok_magic(fm)) { 4741c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(fm, p); 4742c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return; 4743c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4744c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* FOOTERS */ 4745c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#define fm gm 4746c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* FOOTERS */ 4747c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!PREACTION(fm)) { 4748c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_inuse_chunk(fm, p); 4749c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(fm, p) && ok_inuse(p))) { 4750c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t psize = chunksize(p); 4751c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr next = chunk_plus_offset(p, psize); 4752c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!pinuse(p)) { 4753c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t prevsize = p->prev_foot; 4754c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_mmapped(p)) { 4755c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath psize += prevsize + MMAP_FOOT_PAD; 4756c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (CALL_MUNMAP((char*)p - prevsize, psize) == 0) 4757c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->footprint -= psize; 4758c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 4759c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4760c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4761c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr prev = chunk_minus_offset(p, prevsize); 4762c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath psize += prevsize; 4763c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p = prev; 4764c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(fm, prev))) { /* consolidate backward */ 4765c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (p != fm->dv) { 4766c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unlink_chunk(fm, p, prevsize); 4767c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4768c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if ((next->head & INUSE_BITS) == INUSE_BITS) { 4769c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->dvsize = psize; 4770c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_free_with_pinuse(p, psize, next); 4771c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 4772c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4773c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4774c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 4775c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto erroraction; 4776c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4777c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4778c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4779c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_next(p, next) && ok_pinuse(next))) { 4780c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!cinuse(next)) { /* consolidate forward */ 4781c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (next == fm->top) { 4782c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t tsize = fm->topsize += psize; 4783c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->top = p; 4784c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p->head = tsize | PINUSE_BIT; 4785c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (p == fm->dv) { 4786c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->dv = 0; 4787c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->dvsize = 0; 4788c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4789c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (should_trim(fm, tsize)) 4790c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sys_trim(fm, 0); 4791c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 4792c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4793c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (next == fm->dv) { 4794c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t dsize = fm->dvsize += psize; 4795c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->dv = p; 4796c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_free_chunk(p, dsize); 4797c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 4798c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4799c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4800c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t nsize = chunksize(next); 4801c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath psize += nsize; 4802c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unlink_chunk(fm, next, nsize); 4803c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_free_chunk(p, psize); 4804c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (p == fm->dv) { 4805c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->dvsize = psize; 4806c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 4807c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4808c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4809c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4810c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 4811c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_free_with_pinuse(p, psize, next); 4812c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4813c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_small(psize)) { 4814c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath insert_small_chunk(fm, p, psize); 4815c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_free_chunk(fm, p); 4816c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4817c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4818c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr tp = (tchunkptr)p; 4819c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath insert_large_chunk(fm, tp, psize); 4820c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_free_chunk(fm, p); 4821c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (--fm->release_checks == 0) 4822c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath release_unused_segments(fm); 4823c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4824c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 4825c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4826c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4827c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath erroraction: 4828c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(fm, p); 4829c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath postaction: 4830c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(fm); 4831c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4832c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4833c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !FOOTERS 4834c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#undef fm 4835c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* FOOTERS */ 4836c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4837c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4838c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* dlcalloc(size_t n_elements, size_t elem_size) { 4839c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* mem; 4840c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t req = 0; 4841c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (n_elements != 0) { 4842c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath req = n_elements * elem_size; 4843c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (((n_elements | elem_size) & ~(size_t)0xffff) && 4844c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (req / n_elements != elem_size)) 4845c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath req = MAX_SIZE_T; /* force downstream failure on overflow */ 4846c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4847c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = dlmalloc(req); 4848c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mem != 0 && calloc_must_clear(mem2chunk(mem))) 4849c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memset(mem, 0, req); 4850c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return mem; 4851c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4852c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4853c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* dlrealloc(void* oldmem, size_t bytes) { 4854c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (oldmem == 0) 4855c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return dlmalloc(bytes); 4856c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef REALLOC_ZERO_BYTES_FREES 4857c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (bytes == 0) { 4858c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath dlfree(oldmem); 4859c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 4860c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4861c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* REALLOC_ZERO_BYTES_FREES */ 4862c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 4863c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if ! FOOTERS 4864c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate m = gm; 4865c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* FOOTERS */ 4866c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate m = get_mstate_for(mem2chunk(oldmem)); 4867c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!ok_magic(m)) { 4868c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(m, oldmem); 4869c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 4870c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4871c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* FOOTERS */ 4872c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return internal_realloc(m, oldmem, bytes); 4873c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4874c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4875c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4876c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* dlmemalign(size_t alignment, size_t bytes) { 4877c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return internal_memalign(gm, alignment, bytes); 4878c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4879c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4880c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid** dlindependent_calloc(size_t n_elements, size_t elem_size, 4881c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* chunks[]) { 4882c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sz = elem_size; /* serves as 1-element array */ 4883c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return ialloc(gm, n_elements, &sz, 3, chunks); 4884c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4885c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4886c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid** dlindependent_comalloc(size_t n_elements, size_t sizes[], 4887c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* chunks[]) { 4888c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return ialloc(gm, n_elements, sizes, 0, chunks); 4889c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4890c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4891c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* dlvalloc(size_t bytes) { 4892c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t pagesz; 4893c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensure_initialization(); 4894c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pagesz = mparams.page_size; 4895c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return dlmemalign(pagesz, bytes); 4896c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4897c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4898c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* dlpvalloc(size_t bytes) { 4899c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t pagesz; 4900c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensure_initialization(); 4901c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath pagesz = mparams.page_size; 4902c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return dlmemalign(pagesz, (bytes + pagesz - SIZE_T_ONE) & ~(pagesz - SIZE_T_ONE)); 4903c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4904c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4905c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathint dlmalloc_trim(size_t pad) { 4906c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int result = 0; 4907c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensure_initialization(); 4908c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!PREACTION(gm)) { 4909c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath result = sys_trim(gm, pad); 4910c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(gm); 4911c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4912c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return result; 4913c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4914c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4915c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t dlmalloc_footprint(void) { 4916c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return gm->footprint; 4917c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4918c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4919c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t dlmalloc_max_footprint(void) { 4920c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return gm->max_footprint; 4921c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4922c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4923c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !NO_MALLINFO 4924c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstruct mallinfo dlmallinfo(void) { 4925c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return internal_mallinfo(gm); 4926c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4927c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* NO_MALLINFO */ 4928c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4929c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid dlmalloc_stats() { 4930c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath internal_malloc_stats(gm); 4931c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4932c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4933c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathint dlmallopt(int param_number, int value) { 4934c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return change_mparam(param_number, value); 4935c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4936c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4937c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* !ONLY_MSPACES */ 4938c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4939c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t dlmalloc_usable_size(void* mem) { 4940c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mem != 0) { 4941c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = mem2chunk(mem); 4942c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_inuse(p)) 4943c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return chunksize(p) - overhead_for(p); 4944c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4945c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 4946c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4947c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4948c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ----------------------------- user mspaces ---------------------------- */ 4949c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4950c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if MSPACES 4951c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4952c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstatic mstate init_user_mstate(char* tbase, size_t tsize) { 4953c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t msize = pad_request(sizeof(struct malloc_state)); 4954c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr mn; 4955c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr msp = align_as_chunk(tbase); 4956c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate m = (mstate)(chunk2mem(msp)); 4957c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memset(m, 0, msize); 4958c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath INITIAL_LOCK(&m->mutex); 4959c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msp->head = (msize|INUSE_BITS); 4960c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.base = m->least_addr = tbase; 4961c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.size = m->footprint = m->max_footprint = tsize; 4962c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->magic = mparams.magic; 4963c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->release_checks = MAX_RELEASE_CHECK_RATE; 4964c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->mflags = mparams.default_mflags; 4965c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->extp = 0; 4966c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->exts = 0; 4967c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath disable_contiguous(m); 4968c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath init_bins(m); 4969c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mn = next_chunk(mem2chunk(m)); 4970c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath init_top(m, mn, (size_t)((tbase + tsize) - (char*)mn) - TOP_FOOT_SIZE); 4971c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_top_chunk(m, m->top); 4972c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return m; 4973c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4974c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4975c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathmspace create_mspace(size_t capacity, int locked) { 4976c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate m = 0; 4977c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t msize; 4978c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensure_initialization(); 4979c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msize = pad_request(sizeof(struct malloc_state)); 4980c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (capacity < (size_t) -(msize + TOP_FOOT_SIZE + mparams.page_size)) { 4981c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t rs = ((capacity == 0)? mparams.granularity : 4982c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (capacity + TOP_FOOT_SIZE + msize)); 4983c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t tsize = granularity_align(rs); 4984c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* tbase = (char*)(CALL_MMAP(tsize)); 4985c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (tbase != CMFAIL) { 4986c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m = init_user_mstate(tbase, tsize); 4987c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.sflags = USE_MMAP_BIT; 4988c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_lock(m, locked); 4989c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4990c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 4991c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return (mspace)m; 4992c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 4993c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 4994c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathmspace create_mspace_with_base(void* base, size_t capacity, int locked) { 4995c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate m = 0; 4996c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t msize; 4997c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ensure_initialization(); 4998c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msize = pad_request(sizeof(struct malloc_state)); 4999c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (capacity > msize + TOP_FOOT_SIZE && 5000c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath capacity < (size_t) -(msize + TOP_FOOT_SIZE + mparams.page_size)) { 5001c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m = init_user_mstate((char*)base, capacity); 5002c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath m->seg.sflags = EXTERN_BIT; 5003c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_lock(m, locked); 5004c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5005c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return (mspace)m; 5006c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5007c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5008c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathint mspace_track_large_chunks(mspace msp, int enable) { 5009c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int ret = 0; 5010c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5011c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!PREACTION(ms)) { 5012c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!use_mmap(ms)) 5013c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ret = 1; 5014c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!enable) 5015c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath enable_mmap(ms); 5016c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 5017c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath disable_mmap(ms); 5018c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(ms); 5019c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5020c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return ret; 5021c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5022c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5023c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t destroy_mspace(mspace msp) { 5024c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t freed = 0; 5025c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5026c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (ok_magic(ms)) { 5027c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msegmentptr sp = &ms->seg; 5028c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath while (sp != 0) { 5029c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath char* base = sp->base; 5030c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t size = sp->size; 5031c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath flag_t flag = sp->sflags; 5032c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sp = sp->next; 5033c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((flag & USE_MMAP_BIT) && !(flag & EXTERN_BIT) && 5034c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath CALL_MUNMAP(base, size) == 0) 5035c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath freed += size; 5036c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5037c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5038c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 5039c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(ms,ms); 5040c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5041c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return freed; 5042c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5043c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5044c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 5045c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace versions of routines are near-clones of the global 5046c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath versions. This is not so nice but better than the alternatives. 5047c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 5048c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5049c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5050c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* mspace_malloc(mspace msp, size_t bytes) { 5051c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5052c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!ok_magic(ms)) { 5053c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(ms,ms); 5054c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 5055c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5056c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!PREACTION(ms)) { 5057c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* mem; 5058c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t nb; 5059c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (bytes <= MAX_SMALL_REQUEST) { 5060c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t idx; 5061c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath binmap_t smallbits; 5062c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nb = (bytes < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(bytes); 5063c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath idx = small_index(nb); 5064c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath smallbits = ms->smallmap >> idx; 5065c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5066c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if ((smallbits & 0x3U) != 0) { /* Remainderless fit to a smallbin. */ 5067c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr b, p; 5068c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath idx += ~smallbits & 1; /* Uses next bin if idx empty */ 5069c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath b = smallbin_at(ms, idx); 5070c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p = b->fd; 5071c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(chunksize(p) == small_index2size(idx)); 5072c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unlink_first_small_chunk(ms, b, p, idx); 5073c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse_and_pinuse(ms, p, small_index2size(idx)); 5074c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = chunk2mem(p); 5075c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(ms, mem, nb); 5076c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 5077c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5078c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5079c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (nb > ms->dvsize) { 5080c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (smallbits != 0) { /* Use chunk in next nonempty smallbin */ 5081c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr b, p, r; 5082c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t rsize; 5083c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath bindex_t i; 5084c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath binmap_t leftbits = (smallbits << idx) & left_bits(idx2bit(idx)); 5085c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath binmap_t leastbit = least_bit(leftbits); 5086c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath compute_bit2idx(leastbit, i); 5087c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath b = smallbin_at(ms, i); 5088c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p = b->fd; 5089c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath assert(chunksize(p) == small_index2size(i)); 5090c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unlink_first_small_chunk(ms, b, p, i); 5091c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath rsize = small_index2size(i) - nb; 5092c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath /* Fit here cannot be remainderless if 4byte sizes */ 5093c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (SIZE_T_SIZE != 4 && rsize < MIN_CHUNK_SIZE) 5094c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse_and_pinuse(ms, p, small_index2size(i)); 5095c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 5096c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(ms, p, nb); 5097c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath r = chunk_plus_offset(p, nb); 5098c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_free_chunk(r, rsize); 5099c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath replace_dv(ms, r, rsize); 5100c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5101c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = chunk2mem(p); 5102c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(ms, mem, nb); 5103c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 5104c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5105c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5106c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (ms->treemap != 0 && (mem = tmalloc_small(ms, nb)) != 0) { 5107c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(ms, mem, nb); 5108c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 5109c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5110c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5111c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5112c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (bytes >= MAX_REQUEST) 5113c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nb = MAX_SIZE_T; /* Too big to allocate. Force failure (in sys alloc) */ 5114c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 5115c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nb = pad_request(bytes); 5116c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (ms->treemap != 0 && (mem = tmalloc_large(ms, nb)) != 0) { 5117c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(ms, mem, nb); 5118c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 5119c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5120c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5121c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5122c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (nb <= ms->dvsize) { 5123c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t rsize = ms->dvsize - nb; 5124c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = ms->dv; 5125c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (rsize >= MIN_CHUNK_SIZE) { /* split dv */ 5126c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr r = ms->dv = chunk_plus_offset(p, nb); 5127c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ms->dvsize = rsize; 5128c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_free_chunk(r, rsize); 5129c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(ms, p, nb); 5130c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5131c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { /* exhaust dv */ 5132c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t dvs = ms->dvsize; 5133c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ms->dvsize = 0; 5134c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ms->dv = 0; 5135c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_inuse_and_pinuse(ms, p, dvs); 5136c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5137c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = chunk2mem(p); 5138c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(ms, mem, nb); 5139c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 5140c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5141c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5142c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (nb < ms->topsize) { /* Split top */ 5143c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t rsize = ms->topsize -= nb; 5144c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = ms->top; 5145c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr r = ms->top = chunk_plus_offset(p, nb); 5146c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath r->head = rsize | PINUSE_BIT; 5147c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_inuse_chunk(ms, p, nb); 5148c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = chunk2mem(p); 5149c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_top_chunk(ms, ms->top); 5150c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_malloced_chunk(ms, mem, nb); 5151c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 5152c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5153c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5154c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = sys_alloc(ms, nb); 5155c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5156c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath postaction: 5157c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(ms); 5158c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return mem; 5159c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5160c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5161c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 5162c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5163c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5164c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid mspace_free(mspace msp, void* mem) { 5165c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mem != 0) { 5166c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = mem2chunk(mem); 5167c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if FOOTERS 5168c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate fm = get_mstate_for(p); 5169c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath msp = msp; /* placate people compiling -Wunused */ 5170c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* FOOTERS */ 5171c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate fm = (mstate)msp; 5172c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* FOOTERS */ 5173c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!ok_magic(fm)) { 5174c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(fm, p); 5175c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return; 5176c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5177c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!PREACTION(fm)) { 5178c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_inuse_chunk(fm, p); 5179c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(fm, p) && ok_inuse(p))) { 5180c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t psize = chunksize(p); 5181c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr next = chunk_plus_offset(p, psize); 5182c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!pinuse(p)) { 5183c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t prevsize = p->prev_foot; 5184c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_mmapped(p)) { 5185c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath psize += prevsize + MMAP_FOOT_PAD; 5186c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (CALL_MUNMAP((char*)p - prevsize, psize) == 0) 5187c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->footprint -= psize; 5188c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 5189c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5190c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 5191c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr prev = chunk_minus_offset(p, prevsize); 5192c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath psize += prevsize; 5193c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p = prev; 5194c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_address(fm, prev))) { /* consolidate backward */ 5195c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (p != fm->dv) { 5196c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unlink_chunk(fm, p, prevsize); 5197c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5198c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if ((next->head & INUSE_BITS) == INUSE_BITS) { 5199c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->dvsize = psize; 5200c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_free_with_pinuse(p, psize, next); 5201c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 5202c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5203c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5204c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 5205c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto erroraction; 5206c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5207c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5208c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5209c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (RTCHECK(ok_next(p, next) && ok_pinuse(next))) { 5210c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!cinuse(next)) { /* consolidate forward */ 5211c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (next == fm->top) { 5212c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t tsize = fm->topsize += psize; 5213c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->top = p; 5214c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath p->head = tsize | PINUSE_BIT; 5215c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (p == fm->dv) { 5216c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->dv = 0; 5217c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->dvsize = 0; 5218c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5219c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (should_trim(fm, tsize)) 5220c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sys_trim(fm, 0); 5221c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 5222c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5223c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (next == fm->dv) { 5224c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t dsize = fm->dvsize += psize; 5225c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->dv = p; 5226c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_free_chunk(p, dsize); 5227c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 5228c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5229c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 5230c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t nsize = chunksize(next); 5231c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath psize += nsize; 5232c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath unlink_chunk(fm, next, nsize); 5233c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_size_and_pinuse_of_free_chunk(p, psize); 5234c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (p == fm->dv) { 5235c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath fm->dvsize = psize; 5236c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 5237c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5238c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5239c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5240c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 5241c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath set_free_with_pinuse(p, psize, next); 5242c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5243c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_small(psize)) { 5244c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath insert_small_chunk(fm, p, psize); 5245c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_free_chunk(fm, p); 5246c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5247c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 5248c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath tchunkptr tp = (tchunkptr)p; 5249c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath insert_large_chunk(fm, tp, psize); 5250c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath check_free_chunk(fm, p); 5251c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (--fm->release_checks == 0) 5252c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath release_unused_segments(fm); 5253c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5254c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath goto postaction; 5255c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5256c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5257c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath erroraction: 5258c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(fm, p); 5259c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath postaction: 5260c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(fm); 5261c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5262c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5263c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5264c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5265c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* mspace_calloc(mspace msp, size_t n_elements, size_t elem_size) { 5266c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void* mem; 5267c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t req = 0; 5268c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5269c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!ok_magic(ms)) { 5270c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(ms,ms); 5271c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 5272c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5273c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (n_elements != 0) { 5274c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath req = n_elements * elem_size; 5275c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (((n_elements | elem_size) & ~(size_t)0xffff) && 5276c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (req / n_elements != elem_size)) 5277c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath req = MAX_SIZE_T; /* force downstream failure on overflow */ 5278c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5279c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mem = internal_malloc(ms, req); 5280c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mem != 0 && calloc_must_clear(mem2chunk(mem))) 5281c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memset(mem, 0, req); 5282c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return mem; 5283c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5284c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5285c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* mspace_realloc(mspace msp, void* oldmem, size_t bytes) { 5286c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (oldmem == 0) 5287c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return mspace_malloc(msp, bytes); 5288c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#ifdef REALLOC_ZERO_BYTES_FREES 5289c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (bytes == 0) { 5290c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mspace_free(msp, oldmem); 5291c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 5292c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5293c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* REALLOC_ZERO_BYTES_FREES */ 5294c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 5295c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if FOOTERS 5296c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = mem2chunk(oldmem); 5297c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = get_mstate_for(p); 5298c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#else /* FOOTERS */ 5299c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5300c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* FOOTERS */ 5301c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!ok_magic(ms)) { 5302c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(ms,ms); 5303c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 5304c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5305c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return internal_realloc(ms, oldmem, bytes); 5306c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5307c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5308c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5309c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid* mspace_memalign(mspace msp, size_t alignment, size_t bytes) { 5310c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5311c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!ok_magic(ms)) { 5312c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(ms,ms); 5313c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 5314c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5315c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return internal_memalign(ms, alignment, bytes); 5316c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5317c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5318c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid** mspace_independent_calloc(mspace msp, size_t n_elements, 5319c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t elem_size, void* chunks[]) { 5320c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sz = elem_size; /* serves as 1-element array */ 5321c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5322c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!ok_magic(ms)) { 5323c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(ms,ms); 5324c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 5325c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5326c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return ialloc(ms, n_elements, &sz, 3, chunks); 5327c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5328c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5329c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid** mspace_independent_comalloc(mspace msp, size_t n_elements, 5330c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t sizes[], void* chunks[]) { 5331c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5332c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!ok_magic(ms)) { 5333c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(ms,ms); 5334c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 5335c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5336c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return ialloc(ms, n_elements, sizes, 0, chunks); 5337c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5338c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5339c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathint mspace_trim(mspace msp, size_t pad) { 5340c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath int result = 0; 5341c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5342c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (ok_magic(ms)) { 5343c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!PREACTION(ms)) { 5344c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath result = sys_trim(ms, pad); 5345c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath POSTACTION(ms); 5346c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5347c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5348c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 5349c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(ms,ms); 5350c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5351c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return result; 5352c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5353c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5354c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathvoid mspace_malloc_stats(mspace msp) { 5355c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5356c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (ok_magic(ms)) { 5357c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath internal_malloc_stats(ms); 5358c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5359c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 5360c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(ms,ms); 5361c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5362c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5363c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5364c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t mspace_footprint(mspace msp) { 5365c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t result = 0; 5366c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5367c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (ok_magic(ms)) { 5368c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath result = ms->footprint; 5369c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5370c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 5371c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(ms,ms); 5372c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5373c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return result; 5374c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5375c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5376c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5377c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t mspace_max_footprint(mspace msp) { 5378c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size_t result = 0; 5379c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5380c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (ok_magic(ms)) { 5381c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath result = ms->max_footprint; 5382c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5383c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else { 5384c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(ms,ms); 5385c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5386c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return result; 5387c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5388c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5389c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5390c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#if !NO_MALLINFO 5391c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathstruct mallinfo mspace_mallinfo(mspace msp) { 5392c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mstate ms = (mstate)msp; 5393c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (!ok_magic(ms)) { 5394c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath USAGE_ERROR_ACTION(ms,ms); 5395c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5396c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return internal_mallinfo(ms); 5397c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5398c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* NO_MALLINFO */ 5399c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5400c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathsize_t mspace_usable_size(void* mem) { 5401c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (mem != 0) { 5402c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath mchunkptr p = mem2chunk(mem); 5403c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (is_inuse(p)) 5404c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return chunksize(p) - overhead_for(p); 5405c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5406c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return 0; 5407c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5408c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5409c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamathint mspace_mallopt(int param_number, int value) { 5410c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return change_mparam(param_number, value); 5411c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath} 5412c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5413c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath#endif /* MSPACES */ 5414c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5415c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5416c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* -------------------- Alternative MORECORE functions ------------------- */ 5417c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5418c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* 5419c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Guidelines for creating a custom version of MORECORE: 5420c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5421c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * For best performance, MORECORE should allocate in multiples of pagesize. 5422c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * MORECORE may allocate more memory than requested. (Or even less, 5423c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath but this will usually result in a malloc failure.) 5424c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * MORECORE must not allocate memory when given argument zero, but 5425c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath instead return one past the end address of memory from previous 5426c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath nonzero call. 5427c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * For best performance, consecutive calls to MORECORE with positive 5428c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath arguments should return increasing addresses, indicating that 5429c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath space has been contiguously extended. 5430c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Even though consecutive calls to MORECORE need not return contiguous 5431c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath addresses, it must be OK for malloc'ed chunks to span multiple 5432c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath regions in those cases where they do happen to be contiguous. 5433c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * MORECORE need not handle negative arguments -- it may instead 5434c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath just return MFAIL when given negative arguments. 5435c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Negative arguments are always multiples of pagesize. MORECORE 5436c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath must not misinterpret negative args as large positive unsigned 5437c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath args. You can suppress all such calls from even occurring by defining 5438c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath MORECORE_CANNOT_TRIM, 5439c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5440c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath As an example alternative MORECORE, here is a custom allocator 5441c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath kindly contributed for pre-OSX macOS. It uses virtually but not 5442c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath necessarily physically contiguous non-paged memory (locked in, 5443c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath present and won't get swapped out). You can use it by uncommenting 5444c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath this section, adding some #includes, and setting up the appropriate 5445c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath defines above: 5446c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5447c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define MORECORE osMoreCore 5448c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5449c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath There is also a shutdown routine that should somehow be called for 5450c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath cleanup upon program exit. 5451c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5452c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define MAX_POOL_ENTRIES 100 5453c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath #define MINIMUM_MORECORE_SIZE (64 * 1024U) 5454c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath static int next_os_pool; 5455c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void *our_os_pools[MAX_POOL_ENTRIES]; 5456c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5457c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void *osMoreCore(int size) 5458c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath { 5459c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void *ptr = 0; 5460c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath static void *sbrk_top = 0; 5461c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5462c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (size > 0) 5463c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath { 5464c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (size < MINIMUM_MORECORE_SIZE) 5465c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath size = MINIMUM_MORECORE_SIZE; 5466c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (CurrentExecutionLevel() == kTaskLevel) 5467c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ptr = PoolAllocateResident(size + RM_PAGE_SIZE, 0); 5468c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (ptr == 0) 5469c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath { 5470c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return (void *) MFAIL; 5471c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5472c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath // save ptrs so they can be freed during cleanup 5473c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath our_os_pools[next_os_pool] = ptr; 5474c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath next_os_pool++; 5475c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath ptr = (void *) ((((size_t) ptr) + RM_PAGE_MASK) & ~RM_PAGE_MASK); 5476c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath sbrk_top = (char *) ptr + size; 5477c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return ptr; 5478c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5479c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else if (size < 0) 5480c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath { 5481c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath // we don't currently support shrink behavior 5482c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return (void *) MFAIL; 5483c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5484c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath else 5485c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath { 5486c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath return sbrk_top; 5487c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5488c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5489c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5490c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath // cleanup any allocated memory pools 5491c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath // called as last thing before shutting down driver 5492c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5493c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void osCleanupMem(void) 5494c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath { 5495c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath void **ptr; 5496c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5497c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath for (ptr = our_os_pools; ptr < &our_os_pools[MAX_POOL_ENTRIES]; ptr++) 5498c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath if (*ptr) 5499c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath { 5500c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath PoolDeallocate(*ptr); 5501c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath *ptr = 0; 5502c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5503c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath } 5504c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5505c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 5506c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5507c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5508c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath/* ----------------------------------------------------------------------- 5509c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan KamathHistory: 5510c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.8.4 Wed May 27 09:56:23 2009 Doug Lea (dl at gee) 5511c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Use zeros instead of prev foot for is_mmapped 5512c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Add mspace_track_large_chunks; thanks to Jean Brouwers 5513c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Fix set_inuse in internal_realloc; thanks to Jean Brouwers 5514c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Fix insufficient sys_alloc padding when using 16byte alignment 5515c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Fix bad error check in mspace_footprint 5516c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Adaptations for ptmalloc; thanks to Wolfram Gloger. 5517c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Reentrant spin locks; thanks to Earl Chew and others 5518c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Win32 improvements; thanks to Niall Douglas and Earl Chew 5519c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Add NO_SEGMENT_TRAVERSAL and MAX_RELEASE_CHECK_RATE options 5520c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Extension hook in malloc_state 5521c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Various small adjustments to reduce warnings on some compilers 5522c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Various configuration extensions/changes for more platforms. Thanks 5523c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath to all who contributed these. 5524c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5525c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.8.3 Thu Sep 22 11:16:32 2005 Doug Lea (dl at gee) 5526c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Add max_footprint functions 5527c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Ensure all appropriate literals are size_t 5528c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Fix conditional compilation problem for some #define settings 5529c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Avoid concatenating segments with the one provided 5530c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath in create_mspace_with_base 5531c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Rename some variables to avoid compiler shadowing warnings 5532c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Use explicit lock initialization. 5533c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Better handling of sbrk interference. 5534c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Simplify and fix segment insertion, trimming and mspace_destroy 5535c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Reinstate REALLOC_ZERO_BYTES_FREES option from 2.7.x 5536c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Thanks especially to Dennis Flanagan for help on these. 5537c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5538c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.8.2 Sun Jun 12 16:01:10 2005 Doug Lea (dl at gee) 5539c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Fix memalign brace error. 5540c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5541c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.8.1 Wed Jun 8 16:11:46 2005 Doug Lea (dl at gee) 5542c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Fix improper #endif nesting in C++ 5543c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Add explicit casts needed for C++ 5544c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5545c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.8.0 Mon May 30 14:09:02 2005 Doug Lea (dl at gee) 5546c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Use trees for large bins 5547c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Support mspaces 5548c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Use segments to unify sbrk-based and mmap-based system allocation, 5549c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath removing need for emulation on most platforms without sbrk. 5550c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Default safety checks 5551c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Optional footer checks. Thanks to William Robertson for the idea. 5552c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Internal code refactoring 5553c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Incorporate suggestions and platform-specific changes. 5554c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Thanks to Dennis Flanagan, Colin Plumb, Niall Douglas, 5555c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Aaron Bachmann, Emery Berger, and others. 5556c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Speed up non-fastbin processing enough to remove fastbins. 5557c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Remove useless cfree() to avoid conflicts with other apps. 5558c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Remove internal memcpy, memset. Compilers handle builtins better. 5559c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Remove some options that no one ever used and rename others. 5560c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5561c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.7.2 Sat Aug 17 09:07:30 2002 Doug Lea (dl at gee) 5562c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Fix malloc_state bitmap array misdeclaration 5563c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5564c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.7.1 Thu Jul 25 10:58:03 2002 Doug Lea (dl at gee) 5565c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Allow tuning of FIRST_SORTED_BIN_SIZE 5566c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Use PTR_UINT as type for all ptr->int casts. Thanks to John Belmonte. 5567c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Better detection and support for non-contiguousness of MORECORE. 5568c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Thanks to Andreas Mueller, Conal Walsh, and Wolfram Gloger 5569c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Bypass most of malloc if no frees. Thanks To Emery Berger. 5570c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Fix freeing of old top non-contiguous chunk im sysmalloc. 5571c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Raised default trim and map thresholds to 256K. 5572c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Fix mmap-related #defines. Thanks to Lubos Lunak. 5573c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Fix copy macros; added LACKS_FCNTL_H. Thanks to Neal Walfield. 5574c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Branch-free bin calculation 5575c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Default trim and mmap thresholds now 256K. 5576c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5577c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.7.0 Sun Mar 11 14:14:06 2001 Doug Lea (dl at gee) 5578c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Introduce independent_comalloc and independent_calloc. 5579c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Thanks to Michael Pachos for motivation and help. 5580c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Make optional .h file available 5581c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Allow > 2GB requests on 32bit systems. 5582c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * new WIN32 sbrk, mmap, munmap, lock code from <Walter@GeNeSys-e.de>. 5583c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Thanks also to Andreas Mueller <a.mueller at paradatec.de>, 5584c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath and Anonymous. 5585c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Allow override of MALLOC_ALIGNMENT (Thanks to Ruud Waij for 5586c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath helping test this.) 5587c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * memalign: check alignment arg 5588c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * realloc: don't try to shift chunks backwards, since this 5589c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath leads to more fragmentation in some programs and doesn't 5590c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath seem to help in any others. 5591c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Collect all cases in malloc requiring system memory into sysmalloc 5592c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Use mmap as backup to sbrk 5593c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Place all internal state in malloc_state 5594c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Introduce fastbins (although similar to 2.5.1) 5595c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Many minor tunings and cosmetic improvements 5596c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Introduce USE_PUBLIC_MALLOC_WRAPPERS, USE_MALLOC_LOCK 5597c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Introduce MALLOC_FAILURE_ACTION, MORECORE_CONTIGUOUS 5598c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Thanks to Tony E. Bennett <tbennett@nvidia.com> and others. 5599c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Include errno.h to support default failure action. 5600c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5601c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.6.6 Sun Dec 5 07:42:19 1999 Doug Lea (dl at gee) 5602c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * return null for negative arguments 5603c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Added Several WIN32 cleanups from Martin C. Fong <mcfong at yahoo.com> 5604c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Add 'LACKS_SYS_PARAM_H' for those systems without 'sys/param.h' 5605c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (e.g. WIN32 platforms) 5606c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Cleanup header file inclusion for WIN32 platforms 5607c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Cleanup code to avoid Microsoft Visual C++ compiler complaints 5608c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Add 'USE_DL_PREFIX' to quickly allow co-existence with existing 5609c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath memory allocation routines 5610c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Set 'malloc_getpagesize' for WIN32 platforms (needs more work) 5611c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Use 'assert' rather than 'ASSERT' in WIN32 code to conform to 5612c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath usage of 'assert' in non-WIN32 code 5613c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Improve WIN32 'sbrk()' emulation's 'findRegion()' routine to 5614c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath avoid infinite loop 5615c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Always call 'fREe()' rather than 'free()' 5616c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5617c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.6.5 Wed Jun 17 15:57:31 1998 Doug Lea (dl at gee) 5618c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Fixed ordering problem with boundary-stamping 5619c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5620c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.6.3 Sun May 19 08:17:58 1996 Doug Lea (dl at gee) 5621c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Added pvalloc, as recommended by H.J. Liu 5622c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Added 64bit pointer support mainly from Wolfram Gloger 5623c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Added anonymously donated WIN32 sbrk emulation 5624c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Malloc, calloc, getpagesize: add optimizations from Raymond Nijssen 5625c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * malloc_extend_top: fix mask error that caused wastage after 5626c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath foreign sbrks 5627c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Add linux mremap support code from HJ Liu 5628c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5629c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.6.2 Tue Dec 5 06:52:55 1995 Doug Lea (dl at gee) 5630c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Integrated most documentation with the code. 5631c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Add support for mmap, with help from 5632c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Wolfram Gloger (Gloger@lrz.uni-muenchen.de). 5633c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Use last_remainder in more cases. 5634c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Pack bins using idea from colin@nyx10.cs.du.edu 5635c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Use ordered bins instead of best-fit threshhold 5636c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Eliminate block-local decls to simplify tracing and debugging. 5637c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Support another case of realloc via move into top 5638c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Fix error occuring when initial sbrk_base not word-aligned. 5639c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Rely on page size for units instead of SBRK_UNIT to 5640c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath avoid surprises about sbrk alignment conventions. 5641c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Add mallinfo, mallopt. Thanks to Raymond Nijssen 5642c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (raymond@es.ele.tue.nl) for the suggestion. 5643c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Add `pad' argument to malloc_trim and top_pad mallopt parameter. 5644c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * More precautions for cases where other routines call sbrk, 5645c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath courtesy of Wolfram Gloger (Gloger@lrz.uni-muenchen.de). 5646c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Added macros etc., allowing use in linux libc from 5647c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath H.J. Lu (hjl@gnu.ai.mit.edu) 5648c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Inverted this history list 5649c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5650c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.6.1 Sat Dec 2 14:10:57 1995 Doug Lea (dl at gee) 5651c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Re-tuned and fixed to behave more nicely with V2.6.0 changes. 5652c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Removed all preallocation code since under current scheme 5653c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the work required to undo bad preallocations exceeds 5654c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath the work saved in good cases for most test programs. 5655c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * No longer use return list or unconsolidated bins since 5656c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath no scheme using them consistently outperforms those that don't 5657c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath given above changes. 5658c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Use best fit for very large chunks to prevent some worst-cases. 5659c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Added some support for debugging 5660c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5661c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.6.0 Sat Nov 4 07:05:23 1995 Doug Lea (dl at gee) 5662c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Removed footers when chunks are in use. Thanks to 5663c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Paul Wilson (wilson@cs.texas.edu) for the suggestion. 5664c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5665c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.5.4 Wed Nov 1 07:54:51 1995 Doug Lea (dl at gee) 5666c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Added malloc_trim, with help from Wolfram Gloger 5667c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (wmglo@Dent.MED.Uni-Muenchen.DE). 5668c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5669c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.5.3 Tue Apr 26 10:16:01 1994 Doug Lea (dl at g) 5670c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5671c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.5.2 Tue Apr 5 16:20:40 1994 Doug Lea (dl at g) 5672c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * realloc: try to expand in both directions 5673c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * malloc: swap order of clean-bin strategy; 5674c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * realloc: only conditionally expand backwards 5675c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Try not to scavenge used bins 5676c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Use bin counts as a guide to preallocation 5677c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Occasionally bin return list chunks in first scan 5678c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Add a few optimizations from colin@nyx10.cs.du.edu 5679c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5680c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.5.1 Sat Aug 14 15:40:43 1993 Doug Lea (dl at g) 5681c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * faster bin computation & slightly different binning 5682c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * merged all consolidations to one part of malloc proper 5683c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath (eliminating old malloc_find_space & malloc_clean_bin) 5684c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Scan 2 returns chunks (not just 1) 5685c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Propagate failure in realloc if malloc returns 0 5686c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Add stuff to allow compilation on non-ANSI compilers 5687c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath from kpv@research.att.com 5688c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5689c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath V2.5 Sat Aug 7 07:41:59 1993 Doug Lea (dl at g.oswego.edu) 5690c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * removed potential for odd address access in prev_chunk 5691c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * removed dependency on getpagesize.h 5692c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * misc cosmetics and a bit more internal documentation 5693c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * anticosmetics: mangled names in macros to evade debugger strangeness 5694c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * tested on sparc, hp-700, dec-mips, rs6000 5695c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath with gcc & native cc (hp, dec only) allowing 5696c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Detlefs & Zorn comparison study (in SIGPLAN Notices.) 5697c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5698c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath Trial version Fri Aug 28 13:14:29 1992 Doug Lea (dl at g.oswego.edu) 5699c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath * Based loosely on libg++-1.2X malloc. (It retains some of the overall 5700c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath structure of old version, but most details differ.) 5701c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5702c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath*/ 5703c981c48f5bc9aefeffc0bcb0cc3934c2fae179ddNarayan Kamath 5704