| b5b6aebb532541bcc08c5ab8686a18d8c8babbe4 |
|
16-May-2017 |
Kevin Brodsky <kevin.brodsky@linaro.org> |
libc: ARM64: fix memset for non-standard ZVA sizes
372f19e9e27c ("libc: ARM64: update memset/strlen/memcpy/memmove to
newlib/cortex-strings") introduced a bug in memset, only occurring
on the [set_long + zero + non-standard ZVA size] path, more
specifically when DCZID_EL0 reports a size different to 64 or 128.
On platforms with such sizes reported by DCZID_EL0, various string*
unit tests fail due to memset zeroing memory before and/or after the
area it is supposed to set.
Bug: 38342450
Test: bionic-unit-tests --gtest_filter=string*
Change-Id: Idb80c0269226e40e343645a58608e3f324378468
(cherry picked from commit f19eeb8446de27e6b43c77642b03729e10facb2c)
|
| 372f19e9e27c1333c0fc1e83b53d365051e81612 |
|
17-Nov-2016 |
Jake Weinstein <xboxlover360@gmail.com> |
libc: ARM64: update memset/strlen/memcpy/memmove to newlib/cortex-strings
* Bionic benchmarks results at the bottom
* This is a squash of the following commits:
libc: ARM64: optimize memset.
This is an optimized memset for AArch64. Memset is split into 4 main
cases: small sets of up to 16 bytes, medium of 16..96 bytes which are
fully unrolled. Large memsets of more than 96 bytes align the
destination and use an unrolled loop processing 64 bytes per
iteration. Memsets of zero of more than 256 use the dc zva
instruction, and there are faster versions for the common ZVA sizes 64
or 128. STP of Q registers is used to reduce codesize without loss of
performance.
Change-Id: I0c5b5ec5ab8a1fd0f23eee8fbacada0be08e841f
libc: ARM64: improve performance in strlen
Change-Id: Ic20f93a0052a49bd76cd6795f51e8606ccfbf11c
libc: ARM64: Optimize memcpy.
This is an optimized memcpy for AArch64. Copies are split into 3 main
cases: small copies of up to 16 bytes, medium copies of 17..96 bytes
which are fully unrolled. Large copies of more than 96 bytes align
the destination and use an unrolled loop processing 64 bytes per
iteration. In order to share code with memmove, small and medium
copies read all data before writing, allowing any kind of overlap. On
a random copy test memcpy is 40.8% faster on A57 and 28.4% on A53.
Change-Id: Ibb9483e45bbc0e8ca3d5ce98a31c55dfd8a5ac28
libc: AArch64: Tune memcpy
* Further tuning for performance.
Change-Id: Id08eaab885f9743fa7575077924a947c1b88e4ff
libc: ARM64: optimize memmove for Cortex-A53
* Sadly does not work on Denver or Kryo, so can't go to generic
This is an optimized memmove for AArch64. All copies of up to 96
bytes and all backward copies are done by the new memcpy. The only
remaining case is large forward copies which are done in the same way
as the memcpy loop, but copying from the end rather than the start.
Tested on the Nextbit Robin with MSM8992 (Snapdragon 808):
Before
BM_string_memcmp/8 1000k 27 0.286 GiB/s
BM_string_memcmp/64 50M 20 3.053 GiB/s
BM_string_memcmp/512 20M 126 4.060 GiB/s
BM_string_memcmp/1024 10M 234 4.372 GiB/s
BM_string_memcmp/8Ki 1000k 1726 4.745 GiB/s
BM_string_memcmp/16Ki 500k 3711 4.415 GiB/s
BM_string_memcmp/32Ki 200k 8276 3.959 GiB/s
BM_string_memcmp/64Ki 100k 16351 4.008 GiB/s
BM_string_memcpy/8 1000k 13 0.612 GiB/s
BM_string_memcpy/64 1000k 8 7.187 GiB/s
BM_string_memcpy/512 50M 38 13.311 GiB/s
BM_string_memcpy/1024 20M 86 11.858 GiB/s
BM_string_memcpy/8Ki 5M 620 13.203 GiB/s
BM_string_memcpy/16Ki 1000k 1265 12.950 GiB/s
BM_string_memcpy/32Ki 500k 2977 11.004 GiB/s
BM_string_memcpy/64Ki 500k 8003 8.188 GiB/s
BM_string_memmove/8 1000k 11 0.684 GiB/s
BM_string_memmove/64 1000k 16 3.855 GiB/s
BM_string_memmove/512 50M 57 8.915 GiB/s
BM_string_memmove/1024 20M 117 8.720 GiB/s
BM_string_memmove/8Ki 2M 853 9.594 GiB/s
BM_string_memmove/16Ki 1000k 1731 9.462 GiB/s
BM_string_memmove/32Ki 500k 3566 9.189 GiB/s
BM_string_memmove/64Ki 500k 7708 8.501 GiB/s
BM_string_memset/8 1000k 16 0.487 GiB/s
BM_string_memset/64 1000k 16 3.995 GiB/s
BM_string_memset/512 50M 37 13.489 GiB/s
BM_string_memset/1024 50M 58 17.405 GiB/s
BM_string_memset/8Ki 5M 451 18.160 GiB/s
BM_string_memset/16Ki 2M 883 18.554 GiB/s
BM_string_memset/32Ki 1000k 2181 15.022 GiB/s
BM_string_memset/64Ki 500k 4563 14.362 GiB/s
BM_string_strlen/8 1000k 8 0.965 GiB/s
BM_string_strlen/64 1000k 16 3.855 GiB/s
BM_string_strlen/512 20M 92 5.540 GiB/s
BM_string_strlen/1024 10M 167 6.111 GiB/s
BM_string_strlen/8Ki 1000k 1237 6.620 GiB/s
BM_string_strlen/16Ki 1000k 2765 5.923 GiB/s
BM_string_strlen/32Ki 500k 6135 5.341 GiB/s
BM_string_strlen/64Ki 200k 13168 4.977 GiB/s
After
BM_string_memcmp/8 1000k 21 0.369 GiB/s
BM_string_memcmp/64 1000k 28 2.272 GiB/s
BM_string_memcmp/512 20M 128 3.983 GiB/s
BM_string_memcmp/1024 10M 234 4.375 GiB/s
BM_string_memcmp/8Ki 1000k 1732 4.728 GiB/s
BM_string_memcmp/16Ki 500k 3485 4.701 GiB/s
BM_string_memcmp/32Ki 500k 7031 4.660 GiB/s
BM_string_memcmp/64Ki 200k 14296 4.584 GiB/s
BM_string_memcpy/8 1000k 5 1.458 GiB/s
BM_string_memcpy/64 1000k 7 8.952 GiB/s
BM_string_memcpy/512 50M 36 13.907 GiB/s
BM_string_memcpy/1024 20M 80 12.750 GiB/s
BM_string_memcpy/8Ki 5M 572 14.307 GiB/s
BM_string_memcpy/16Ki 1000k 1165 14.053 GiB/s
BM_string_memcpy/32Ki 500k 3141 10.430 GiB/s
BM_string_memcpy/64Ki 500k 7008 9.351 GiB/s
BM_string_memmove/8 50M 7 1.074 GiB/s
BM_string_memmove/64 1000k 9 6.593 GiB/s
BM_string_memmove/512 50M 37 13.502 GiB/s
BM_string_memmove/1024 20M 80 12.656 GiB/s
BM_string_memmove/8Ki 5M 573 14.281 GiB/s
BM_string_memmove/16Ki 1000k 1168 14.018 GiB/s
BM_string_memmove/32Ki 1000k 2825 11.599 GiB/s
BM_string_memmove/64Ki 500k 6548 10.008 GiB/s
BM_string_memset/8 1000k 7 1.038 GiB/s
BM_string_memset/64 1000k 8 7.151 GiB/s
BM_string_memset/512 1000k 29 17.272 GiB/s
BM_string_memset/1024 50M 53 18.969 GiB/s
BM_string_memset/8Ki 5M 424 19.300 GiB/s
BM_string_memset/16Ki 2M 846 19.350 GiB/s
BM_string_memset/32Ki 1000k 2028 16.156 GiB/s
BM_string_memset/64Ki 500k 4514 14.517 GiB/s
BM_string_strlen/8 1000k 7 1.120 GiB/s
BM_string_strlen/64 1000k 16 3.918 GiB/s
BM_string_strlen/512 50M 64 7.894 GiB/s
BM_string_strlen/1024 20M 104 9.815 GiB/s
BM_string_strlen/8Ki 5M 664 12.337 GiB/s
BM_string_strlen/16Ki 1000k 1291 12.682 GiB/s
BM_string_strlen/32Ki 1000k 2940 11.143 GiB/s
BM_string_strlen/64Ki 500k 6440 10.175 GiB/s
Change-Id: I635bd2798a755256f748b2af19b1a56fb85a40c6
|
| 784609317d49e854813f1797d7a53cf7d4379643 |
|
02-Mar-2016 |
Elliott Hughes <enh@google.com> |
Mandate optimized __memset_chk for arm and arm64.
This involves actually implementing assembler __memset_chk for arm64,
but that's easily done.
Obviously I'd like this for all architectures (and all the string functions),
but this is low-hanging fruit...
Change-Id: I70ec48c91aafd1f0feb974a2555c51611de9ef82
|
| 62d92e1610080cad16342f87a7fb46d8388ee4bb |
|
19-May-2014 |
Bernhard Rosenkraenzer <Bernhard.Rosenkranzer@linaro.org> |
Add Aarch64 optimized bzero based on memset
Change-Id: I62cf25bfe23b9d811e00af7307bbd19d89937792
Signed-off-by: Bernhard Rosenkraenzer <Bernhard.Rosenkranzer@linaro.org>
|
| 7e4fa560999d07064d219a16ebb50d3691dd1b63 |
|
05-Mar-2014 |
Bernhard Rosenkraenzer <Bernhard.Rosenkranzer@linaro.org> |
Add ARMv8 optimized string handling functions based on cortex-strings
This adds ARMv8 optimized string handling functions to Bionic.
The implementations live in a generic/ directory because there will
likely be more CPU specific versions (e.g. Cortex-A53 vs. Cortex-A57)
later.
These implementations are 50%+ faster on current v8 models.
Change-Id: If3adc54a284d9519459b0d4d4390f0cd6ded8786
Signed-off-by: Bernhard Rosenkraenzer <Bernhard.Rosenkranzer@linaro.org>
|