History log of /ndk/sources/cxx-stl/llvm-libc++/libcxx/test/testit_android
Revision Date Author Comments
dfc911962c33f546a74e3ec45b48ce4c7b9d56ec 21-Jun-2014 Andrew Hsieh <andrewhsieh@google.com> misc fixes

1. Don't build wrapper *-gdb.exe" when --with-python isn't specified
2. copy arm_neon_x86.h to llvm
3. update build-mingw64-toolchain.sh to use gcc4.8.3 and binutils-2.24
4. add the following to libc++/x86_64 export list
std::__1::to_string(__float128)
std::__1::to_wstring(__float128)
5. Fix issue where Mac doesn't have readlef
6. fix make-standalone-toolchain.sh to honor gcc-vesion in --toolchain=
with *clang suffix
7. Fix tests/device/b8708181-Vector4 for 64-bit
8. Change Dl_info to "int" for TEST
9. Remove symlinks for old tests *sharedmutex* renamed to *sharedtimedmutex*
in recent libc++.
10. package-release.sh misc.tar.bz2 if exists, which contain msic stuff
not submitted yet

Change-Id: Ibe859e753e21d63f9f9de7370e6d43387ec2487b
7670977319d537cb30ed322897022f649b1bd1f9 21-Jun-2014 Andrew Hsieh <andrewhsieh@google.com> More libc++ fixes

1. Remove dependency on compiler-rt. Same test passes rate for clang,
a bit better for gcc. One less thing to package in apk, and less
worries about the incompleteness of compiler-rt to replace libgcc.
Will review and put it back when needed
2. build with libc++abi in build-target-prebuilts.sh
a). copy libc++abi cxxabi.h, etc to standalone toolchain for libc++
b). Add LIBCXX_USE_GABIXX if developers want gabi++ back. Set
LIBCXX_FORCE_REBUILD:=true because the prebuilt libc++ libraries
are compiled with libc++abi now
3. remove --gcc-version in make-standalone-toolchain.sh becasue the
value is ignored and computed at parse_toolchain_name later

4. Update export_symbols.txt to expose more symbols in libc++_shared.so
5. Fix for gcc to compiler libc++abi. Note that the prebuilt libc++
libraries are built with clang
6. Fix testit_android to
a) push libcompiler_rt_shared.so as well if libc++_shared.so depends on it
b) allow TARGET_PATH to change target subdirectory under /data/local/tmp,
to allow multiple instrance running at the same time (because each one
take 2 hrs, and there are 72 combinations to test)
c) add -lstdc++ for static test because -nodefaultlibs instruct gcc
don't add it by default

Change-Id: I225c2fd3f034d8a956ee2136cf92dbc17073622c
667dbb41059198b4039b1e6b40028a11d880ea16 17-Jun-2014 Andrew Hsieh <andrewhsieh@google.com> misc fixes for libc++

1. now that libc++ depends on libcompiler-rt, build compiler-rt before
libc++ in build-target-prebuilts.sh
2. build lib/clear_cache.c in compiler-rt
3. Fix reset of $EXTRA_CXXFLAGS which may contain -mthumb and -mhard-float
set before it
4. fix clang++ detection for adding -mllvm in testit_android
5. update black-list in tests/device/test-libc++-{static,shared}-full/jni/Android.mk

Change-Id: Ic523a9d5090e77e21957e3030cf823215c3c7063
94203c4b6f7ec3a9a3e25a22bc1171900eba2b15 14-Jun-2014 Albert J. Wong <ajwong@google.com> Make sure to emit EHABI tables.

Change-Id: I8185c52675e85fd56962e92383e6488700d19dbe
6cc6eec8fea506e16de95d4098ffbf471a1cc575 14-Jun-2014 Albert J. Wong <ajwong@google.com> Fix the link line to allow for lgcc.

Still doesn't correctly use -lm_hard for hardfloat ABI.

Change-Id: I011c63463a74d8cf3175c6a0766dfcde4ce94b98
5092c9020f6482298676951ac70a43530104d44e 19-Apr-2014 Andrew Hsieh <andrewhsieh@google.com> Misc fixes

1. Fix issue when APP_PLATFORM specified in env. var can't be adjusted
2. revert linux-x86 host compiler to the original glibc2.7
(when __USE_OLD_LINUX_HOST_GCC=yes) to generate binaries run on server
with very old libc.so
3. Fix issue about run-tests.sh --abi=armeabi-v7a-hard
4. Add 4.9 to some BROKEN_RUN
5. LOCAL_CPPFLAGS += -UNDEBUG for libc++ test becasue there are real work
done inside assert (bad, bad, bad) for some tests.

Change-Id: If9f507303186ba85fc9ed0962732d027a2dd246e
4a6fe28b54934bead656146e2571f64f4530be5b 28-Feb-2014 Andrew Hsieh <andrewhsieh@google.com> More fix for libc++ testing

1. weaken all long double math in math_support.c for armeabi-v7a-hard
which link with -lm_hard from API19 already contains all
2. Define Emplaceable(const Emplaceable&) and MoveOnly(const MoveOnly&)
for gcc4.8
3. Fixed test pointer.pass.cpp because for reason not well-understood
%p on null produces "0" in libc++
4. Define -UNDEBUG because some test use assert() which tunrs into nothing
if NDEBUG is defined (eg. in ndk-build system for release build)
5. Avoid duplicate testing by skipping test path with EQ. Note that previous
commit 1b97ee35bafeb02b7c1feabc3a99b6d004637ddc creates symlink for tests
whose path containing '='
6. Sort the *.cpp.list and directory to make test report consistent across run
7. Re-try up to 10 times if run fails due to "Text file busy". This significantly
reduce false-alarm when running on emulator

Change-Id: Ia6964442095f2e7457a02c524115ad3db0887c4d
0b823f8d0398235dc1780f6a989e91adeac769eb 27-Feb-2014 Andrew Hsieh <andrewhsieh@google.com> Try to remove libc++ dependencies on libcompiler-rt

Add --with-compiler-rt to testit_android and turn it off by default
Note that libc++ libraries are built with clang because it uses
clang-specific features. Previous assumption is that libcompiler-rt
is also needed, which turns out not true.

Rebuilding (via LIBCXX_FORCE_REBUILD:=true) for armeabi shared libc++ library
(ie. libc++_shared.so) still needs either -latomic or compiler-rt to provide
__atomic_fetch_add_4, though. See comment at the end of
sources/cxx-stl/llvm-libc++/Android.mk

Also note that traditionally clang in NDK use gcc's assembler,
linker, libgcc.a, etc, etc. This change tries to extend the tradition
to libc++ land.

Change-Id: I86601636373a6fc6c7357fa733442f5be19b5593
8654c21c31811a2c1caeac75b74325b02dc00c8c 26-Feb-2014 Andrew Hsieh <andrewhsieh@google.com> Remove "LOCAL_CPP_FEATURES := rtti exceptions" from prebuilt libc++

Similar to gnustl_static|gnustl_shared, using c++_static|c++_shared
prebuilt by default does not enable rtti and exceptions. Users
have to add "LOCAL_CPP_FEATURES := rtti exceptions" if needed

Also add armeabi-v7a-hard to libc++ testit_android, and revise
README.NDK a bit

Change-Id: I5da959498876781b1b5ee6133e8ad16c0487627f
6eac0e4b1c6d8a50320cb6fce05559619c9e198f 24-Feb-2014 Andrew Hsieh <andrewhsieh@google.com> Add static libc++ testing, and use -O2

1. Test both --static and --shared
2. -O2 testing result is identical to -O0 except for the following four when compiled
with gcc 4.8

containers/sequences/forwardlist/forwardlist.modifiers/insert_after_range.pass.cpp
I/DEBUG ( 51): Abort message: 'insert_after_range.pass.cpp:37: int main(): assertion "distance(c.begin(), c.end()) == 3" failed'
containers/sequences/list/list.ops/merge_comp.pass.cpp
I/DEBUG ( 51): Abort message: 'merge_comp.pass.cpp:29: int main(): assertion "c1 == std::list<int>(a3, a3+sizeof(a3)/sizeof(a3[0]))" failed'
containers/sequences/list/list.ops/merge.pass.cpp
I/DEBUG ( 51): Abort message: 'merge.pass.cpp:28: int main(): assertion "c1 == std::list<int>(a3, a3+sizeof(a3)/sizeof(a3[0]))" failed'
iterators/predef.iterators/insert.iterators/front.insert.iter.ops/front.insert.iter.op=/lv_value.pass.cpp
I/DEBUG ( 51): Abort message: 'lv_value.pass.cpp:28: void test(C) [with C = std::__1::list<Copyable>]: assertion "c.front() == v" failed'

Keep -O2 on to remind us those fails
3. Add -latomic as system_libs
4. Package libc++ tests for now

Change-Id: If8518ef3240050fc67910f95ca3dc443a5556bd5
5de42e6621b3d0131472c3f8838b7f0ccf3e8963 22-Oct-2013 Andrew Hsieh <andrewhsieh@google.com> Update libc++ to upstream revision r201101

To catch fixes since last update (r186119) which fails clang3.4
on containers/associative/map/map.modifiers/emplace_hint.pass.cpp
with error reads:

.../tuple:320:11: error:
rvalue reference to type 'double' cannot bind to lvalue of type 'double'
: value(__t.get())
^ ~~~~~~~~~
.../tuple:444:8: note:
in instantiation of member function 'std::__1::__tuple_leaf<1, double &&, false>::__tuple_leaf' requested here
struct __tuple_impl<__tuple_indices<_Indx...>, _Tp...>
^
emplace_hint.pass.cpp:69:42: note: in instantiation of function template specialization 'std::__1::forward_as_tuple<int, double>' requested
here
std::forward_as_tuple(2, 3.5));
^

Now clang3.4 test results are same as clang3.3

### clang3.4 armeabi-v7a
...
Target: armv5te-none-linux-androideabi
Thread model: posix
with -std=c++11 -O0 -g -march=armv7-a -mthumb -mfpu=vfpv3-d16 -march=armv7-a -mthumb -Wl,--fix-cortex-a8 -I/usr/local/google/home/andrewhsieh/mydroid/ndk/sources/cxx-stl/llvm-libc++/libcxx/test/support -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS
...
number of tests failed : 123
number of tests passed : 4522
+ ----
total number of tests : 4645

### clang3.4 x86
...
using clang version 3.4 (https://bitbucket.org/loganchien/clang c34bb12af3af42f671296cdb978b34b931062d2a) (https://bitbucket.org/loganchien/llvm d09f2eff406d17c86d51db7660bd374cf092e6ed)
Target: i686-none-linux-android
Thread model: posix
with -std=c++11 -O0 -g -I/usr/local/google/home/andrewhsieh/mydroid/ndk/sources/cxx-stl/llvm-libc++/libcxx/test/support -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS
...
number of tests failed : 125
number of tests passed : 4520
+ ----
total number of tests : 4645

### clang3.4 mips
...
Target: mipsel-none-linux-android
Thread model: posix
with -std=c++11 -O0 -g -I/usr/local/google/home/andrewhsieh/mydroid/ndk/sources/cxx-stl/llvm-libc++/libcxx/test/support -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS
...
number of tests failed : 122
number of tests passed : 4523
+ ----
total number of tests : 4645

Change-Id: I3a0aa734b9f07abf15774bfe1d687d9de5e3a4a9
8c201e8536d141e8efb711749826450067936c69 15-Nov-2013 Andrew Hsieh <andrewhsieh@google.com> Test libc++ with make-standalone-toolchain.sh --stl=libc++; misc fixes

1. previously we tested libc++ with --stl=gnustl, which may cause
multi-definition of some symbols if not incompatibility
2. also map all missing long double function to counterparts in double
3. compile 5 more new android/support/src/musl-stdio/*.c
4. compile libcompiler-rt with clang
5. Add run-libcxx.sh arm-device x86-device mips-device
eg.
./run-libcxx.sh emulator-5554 emulator-5556 emulator-5558

Current test results with clang3.3 (using the exisiting gabi++ runtime)

================ llvm-libc++
### clang3.3 armeabi-v7a
passed ...
...
Target: armv5te-none-linux-androideabi
Thread model: posix
with -std=c++11 -O0 -g -march=armv7-a -mthumb -mfpu=vfpv3-d16 -march=armv7-a -mthumb -Wl,--fix-cortex-a8 -I/usr/local/google/home/andrewhsieh/mydroid/ndk/sources/cxx-stl/llvm-libc++/libcxx/test/support -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS
----------------------------------------------------
number of tests failed : 123
number of tests passed : 4295
+ ----
total number of tests : 4418
****************************************************

### clang3.3 x86
passed ...
...
Target: i686-none-linux-android
Thread model: posix
with -std=c++11 -O0 -g -I/usr/local/google/home/andrewhsieh/mydroid/ndk/sources/cxx-stl/llvm-libc++/libcxx/test/support -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS
----------------------------------------------------
number of tests failed : 122
number of tests passed : 4296
+ ----
total number of tests : 4418
****************************************************

### clang3.3 mips
passed ...
...
Target: mipsel-none-linux-android
Thread model: posix
with -std=c++11 -O0 -g -I/usr/local/google/home/andrewhsieh/mydroid/ndk/sources/cxx-stl/llvm-libc++/libcxx/test/support -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS
number of tests failed : 126
number of tests passed : 4292
+ ----
total number of tests : 4418

Change-Id: Icaf5cb1983c3ce66f9468694dd14ef568d2f2f89
92f1ee6990752f26aebdbbfb5a87c474d3edd1e0 31-Dec-2013 Albert J. Wong <ajwong@google.com> Fix new.delete tests by correcting link methodology. Replace GABIXX_LIBCXX with LIBCXXABI.

Removed -Bsymbolic. Specified -O0 in testit_android because the optimizations
make some of the tests fail. Also replaced GABIXX_LIBCXX with LIBCXXABI
and gabi++ to reflect libc++abi, not libc++rt.

The linking of libc++_shared.so incorreclty specified -Bsymbolic
which, aside from causing issues with rtti and exception handlers,
also makes the default implementations of the array operator new,
and the single-object no-throw variant (both of which wrap the normal
single-object new) link directly to the single-object new rather than
jump through the PLT. This means that even if you override
single-object new, the default implementations of these other new
operators will NOT find the override. Bad bad bad.

Ideally, this should use -Bsymbolic-functions with
--dynamic-list-cpp-new and --dynamic-list-cpp-typeinfo so that the
linker can avoid a plt jump for most lookups, but somehow even with
the dynamic-list settings, the operator new/delete functions were
skipping the plt jump. Removed all of these options for now.

willchan, danakj, and thakis contributed to this patch.

BUG=36496

Change-Id: Iaaa25de365b6444e49ceaadaf9d0b1f442b9c85d
917347ec4e6c220e8750ae51462efdd461f45c6e 27-Sep-2013 Andrew Hsieh <andrewhsieh@google.com> [experimental] test libc++ built by clang; link tests with compiler-rt

Trying to make libc++ work with clang
1. Turns out libc++ needs some features exists in clang to work. Previously
we compile all STL in NDK with default compiler which is gcc 4.6.
In this experiment, we switch to clang3.3 (exists since r9)

./build/tools/build-cxx-stl.sh --stl=libc++ --ndk-dir=$NDK --llvm-version=3.3

2. Clang needs compiler-rt for helpers used by code expanded
from intrinsics (eg. __atomic_exchange). This patch add -lcompiler_rt*
for linker.

3. Misc fixes:
a) pass "-D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS",
accroding to llvm-libc++/libcxx/test/lit.cfg for linux
b) pass "$TOOLCHAIN_CFLAGS $TOOLCHAIN_LDFLAGS", mainly for armeabi-v7a
c) pass -mthumb for arm
d) pass -O2 -g for all by default
e) add -latomic, for __atomic_is_lock_free
f) consolidate TARGET_PATH
g) adb push *dat to TARGET_PATH because some tests depends on them

To test:
cd $NDK
./build/tools/build-compiler-rt.sh -j32 --ndk-dir=$NDK --abis=armeabi,armeabi-v7a,x86,mips \
--src-dir=/tmp/ndk-$USER/src/llvm-3.3/compiler-rt --llvm-version=3.3
./build/tools/build-cxx-stl.sh --stl=libc++ --ndk-dir=$NDK --abis=armeabi,armeabi-v7a,x86,mips \
--llvm-version=3.3

cd $NDK
./build/tools/make-standalone-toolchain.sh --platform=android-9 --llvm-version=3.3 \
--install-dir=/path/to/ndk/standalone/ndk-api9-arm --arch=arm

cd $NDK/sources/cxx-stl/llvm-libc++/libcxx/test
PATH=/path/to/ndk/standalone/ndk-api9-arm:$PATH \
./testit_android --abi=armeabi-v7a --cxx=arm-linux-androideabi-clang++

Fail tests count drop from 682 to ~126 (out of total 4418) for armeabi-v7a/x86/mips

Change-Id: I213448129f82b70542fca4e4603fb50e023a1f9e
da5b11a42de8efb020db413a6d0e79a895dc886f 26-Sep-2013 David 'Digit' Turner <digit@android.com> libc++: Improve testit_android reporting.

A small patch to improve the error reporting when running the libc++
test suite. In case of failure, the output can now be easily copy-pasted
and run directly to reproduce a specific failure.

+ Fix the handling of the --cxx=<name> option, the value was overwritten
when using --abi=<name>

Change-Id: I06f799bdd2de1b518b78e88172b6d95f8be764ef
d0722199c3b7497f9211924cb5807a564cf5e39c 07-Aug-2013 Andrew Hsieh <andrewhsieh@google.com> Fix llvm-libc++ test driver to accept different adb device

Now it is possile to test llvm-libc++ in a batch each for
different device, eg.

ADB="adb -s emulator-5554" \
PATH=$PATH:/t/ndk-arm/bin:/t/ndk-mips/bin:/t/ndk-x86/bin \
./testit_android --abi=armeabi-v7a

ADB="adb -s emulator-5556" \
PATH=$PATH:/t/ndk-arm/bin:/t/ndk-mips/bin:/t/ndk-x86/bin \
./testit_android --abi=x86

ADB="adb -s emulator-5558" \
PATH=$PATH:/t/ndk-arm/bin:/t/ndk-mips/bin:/t/ndk-x86/bin \
./testit_android --abi=mips

Change-Id: I10634fc9b8b119848cfa159cfe607f87321f9dc0
ad46d2248426e810edc5878c7cb086b7093f3cf5 11-Jul-2013 David 'Digit' Turner <digit@android.com> Update libc++ to upstream revision 186119

Nothing really drastic here. Note that this introduces
a new file under patches.android/ to deal with __config
which assumes that __linux__ means GLibc is being used.

Change-Id: Iddd56728f9cfdbcea981b1269612248e7c7af797
f27a2b381b327ca3995b580531c164248f99a072 11-Jul-2013 David 'Digit' Turner <digit@android.com> libc++: Add testit_android script.

This patch adds a new shell script that can be used to
run the whole libc++ test suite on a connected Android device.

It is a copy of the libc++ 'testit' script, customized to build
with a standalone Android toolchain, and use ADB to run each test
on a customized device.

+ Change the names of the libc++ libraries to libc++_static.a
and libc++_shared.so, both for the standalone and ndk-build
builds.

+ Fix a bug where linking against the shared library built
with build-cxx-stl.sh would fail to find nl_langinfo.

+ Fix a bug where build-cxx-stl.sh forgot to build the Android
specific locale_android.cpp support file.

To run the test suite do the following:

cd $NDK
build/tools/build-cxx-stl.sh --stl=libc++
build/tools/make-standalone-toolchain.sh --install-dir=/tmp/ndk-arm
build/tools/make-standalone-toolchain.sh --install-dir=/tmp/ndk-x86 --arch=x86
build/tools/make-standalone-toolchain.sh --install-dir=/tmp/ndk-mips --arch=mips
cd sources/cxx-stl/llvm-libc++/libcxx/test
PATH=$PATH:/tmp/ndk-arm/bin:/tmp/ndk-x86-bin:/tmp/ndk-mips/bin
./testit_android --abi=armeabi-v7a

For an Android/x86 device, replace last line with:
./testit_android --abi=x86

For an Android/MIPS one, replace last line with:
./testit_android --abi=mips

WARNING: Running the full test suite is *very* long, and several tests are
failing at the moment.

Current results for --abi=armeabi-v7a when running on a Nexus S:

****************************************************
Results for /opt/digit/repo/aosp/ndk/sources/cxx-stl/llvm-libc++/libcxx/test:
using arm-linux-androideabi-g++ (GCC) 4.6 20120106 (prerelease)
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
with -std=c++0x -I/opt/digit/repo/aosp/ndk/sources/cxx-stl/llvm-libc++/libcxx/test/support -I/opt/digit/repo/aosp/ndk/sources/cxx-stl/llvm-libc++/libcxx/include -I/opt/digit/repo/aosp/ndk/sources/android/support/include -L/opt/digit/repo/aosp/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a
----------------------------------------------------
sections without tests : 0
sections with failures : 216
sections without failures: 849
+ ----
total number of sections : 1065
----------------------------------------------------
number of tests failed : 716
number of tests passed : 3674
+ ----
total number of tests : 4390
****************************************************

real 49m45.118s
user 19m1.460s
sys 2m17.800s

Change-Id: I3f49e3a8fcea6f09f658092d864e3dcc65f33997