| 4b116a2f5d3c6e2a0a7fe39d5eb956563138d542 |
|
20-Jun-2014 |
Neil Fuller <nfuller@google.com> |
Fix the OOME in ScannerParseLargeFileBenchmarkTest
Scanner had no mechanism for recovering buffer space it didn't
need.
Now, if the buffer is more than 50% full of ignorable characters
the remaining characters are shuffled to the beginning to recover
space. For most expected usecases this means that the buffer will
stay 1k and contain up to 512 characters of useful data. A
growable, circular character buffer could have been introduced
to avoid the copy but is not worth the effort.
Previously the buffer would double in size until the data or memory
was exhausted, and each read would also double in size to fill the
empty half of the buffer. This was fine providing the input
could fit in memory.
On top of that the 1k, 2k, 4k, etc. buffer was repeatedly turned
into a String and passed to the (native) matcher, and then the
matcher was told to ignore more than half of it.
As a consequence of keeping the buffer a fixed size (and only
filling 50% of it at a time), this change may cause a performance
regression: for most usecases where delimiters are closer together
than 512 bytes, reads after the first will now usually be 512 bytes
and not the 1k, 2k, 4k, etc. it was previously.
Having fixed the test so it doesn't OOM, the test now
takes 6 minutes to pass on host and so is unsuitable for inclusion
in CTS tests and so is being suppressed.
This change also includes so tidy up changes to the test and the
Scanner class.
The implementation could no doubt be improved but the API makes
it inherently slow. It would be surprising if anybody uses the
Scanner class on Android with so many better alternatives.
Bug: 14865710
Change-Id: I40a7fc0c2bfaf6db4e42108efe417303e76bde24
|