History log of /external/clang/lib/Lex/TokenLexer.cpp
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
4967a710c84587c654b56c828382219c3937dacb 20-Sep-2016 Pirama Arumuga Nainar <pirama@google.com> Update aosp/master Clang for rebase to r275480

Bug: http://b/31320715

This merges commit ac9cc4764cf47a6c3f031687d8592e080c9f5001 from
aosp/dev.

Test: Build AOSP and run RenderScript tests (host tests for slang and
libbcc, RsTest, CTS)

Change-Id: Ic2875e5c3673c83448cd7d1013861e42947b1b55
/external/clang/lib/Lex/TokenLexer.cpp
87d948ecccffea9e9e37d0d053b246e2d6d6c47b 04-Mar-2016 Pirama Arumuga Nainar <pirama@google.com> Update aosp/master clang for rebase to r256229

http://b/26987366

Change-Id: I5d349c9843ea5c24d6e455956f8a446393b6873d
/external/clang/lib/Lex/TokenLexer.cpp
b6d6993e6e6d3daf4d9876794254d20a134e37c2 01-Jul-2015 Pirama Arumuga Nainar <pirama@google.com> Update aosp/master clang for rebase to r239765

Change-Id: I0393bcc952590a7226af8c4b58534a8ee5fd2d99
/external/clang/lib/Lex/TokenLexer.cpp
3ea9e33ea25e0c2b12db56418ba3f994eb662c04 08-Apr-2015 Pirama Arumuga Nainar <pirama@google.com> Update aosp/master clang for rebase to r233350

Change-Id: I12d4823f10bc9e445b8b86e7721b71f98d1df442
/external/clang/lib/Lex/TokenLexer.cpp
0e2c34f92f00628d48968dfea096d36381f494cb 23-Mar-2015 Stephen Hines <srhines@google.com> Update aosp/master clang for rebase to r230699.

Change-Id: I6a546ab3d4ae37119eebb735e102cca4f80ab520
/external/clang/lib/Lex/TokenLexer.cpp
176edba5311f6eff0cad2631449885ddf4fbc9ea 01-Dec-2014 Stephen Hines <srhines@google.com> Update aosp/master Clang for rebase to r222490.

Change-Id: Ic557ac55e97fbf6ee08771c7b7c3594777b0aefd
/external/clang/lib/Lex/TokenLexer.cpp
6bcf27bb9a4b5c3f79cb44c0e4654a6d7619ad89 29-May-2014 Stephen Hines <srhines@google.com> Update Clang for 3.5 rebase (r209713).

Change-Id: I8c9133b0f8f776dc915f270b60f94962e771bc83
/external/clang/lib/Lex/TokenLexer.cpp
651f13cea278ec967336033dd032faef0e9fc2ec 24-Apr-2014 Stephen Hines <srhines@google.com> Updated to Clang 3.5a.

Change-Id: I8127eb568f674c2e72635b639a3295381fe8af82
/external/clang/lib/Lex/TokenLexer.cpp
d2f9308220af22bfc1bcd3bc2cad118dbd8be013 19-Sep-2013 Eli Friedman <eli.friedman@gmail.com> Make Preprocessor::Lex non-recursive.

Before this patch, Lex() would recurse whenever the current lexer changed (e.g.
upon entry into a macro). This patch turns the recursion into a loop: the
various lex routines now don't return a token when the current lexer changes,
and at the top level Preprocessor::Lex() now loops until it finds a token.
Normally, the recursion wouldn't end up being very deep, but the recursion depth
can explode in edge cases like a bunch of consecutive macros which expand to
nothing (like in the testcase test/Preprocessor/macro_expand_empty.c in this
patch).

<rdar://problem/14569770>

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@190980 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
344472ebeded2fca2ed5013b9e87f81d09bfa908 23-Aug-2013 Robert Wilhelm <robert.wilhelm@gmx.net> Use pop_back_val() instead of both back() and pop_back().
No functionality change intended.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@189112 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
6b9240e058bf3451685df73fc8ce181b3046e92b 05-Jul-2013 Craig Topper <craig.topper@gmail.com> Use SmallVectorImpl& for function arguments instead of SmallVector.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@185715 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
11be06464a338b7c6e007898a7709498dc456db2 26-Jun-2013 Reid Kleckner <reid@kleckner.net> Match MSVC's handling of commas during macro argument expansion

This allows clang to parse the type_traits header in Visual Studio 2012,
which is included widely in practice.

This is a rework of r163022 by João Matos. The original patch broke
preprocessing of gtest headers, which this patch addresses.

Patch by Will Wilson!

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@184968 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
461f2d820ffb628ebdf6ef4a4474a43f8ff914e5 25-May-2013 Argyrios Kyrtzidis <akyrtzi@gmail.com> [Preprocessor] Prevent expansion of y in x ## y when x is empty

When x is empty, x ## is suppressed, and when y gets expanded, the fact that it follows ## is not
available in the macro expansion result. The macro definition can be checked instead, the ## will
be available there regardless of what x expands to.

Fixes http://llvm.org/PR12767

Patch by Harald van Dijk!

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182699 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
dd08a0c178329ec16cb9e494e6880f3991708b93 04-May-2013 Argyrios Kyrtzidis <akyrtzi@gmail.com> [Preprocessor] For the MacroExpands preprocessor callback, also pass the MacroArgs object that provides information about
the argument tokens for a function macro.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@181065 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
b29740ae158f9421096a28bcc7ad6af7171b1874 20-Dec-2012 Argyrios Kyrtzidis <akyrtzi@gmail.com> [preprocessor] When "merging" macro argument tokens into one SLocEntry chunk,
make sure they came from the same kind of FileIDs.

Thanks to Abramo Bagnara for providing the test case.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@170616 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
55fc873017f10f6f566b182b70f6fc22aefa3464 04-Dec-2012 Chandler Carruth <chandlerc@gmail.com> Sort all of Clang's files under 'lib', and fix up the broken headers
uncovered.

This required manually correcting all of the incorrect main-module
headers I could find, and running the new llvm/utils/sort_includes.py
script over the files.

I also manually added quite a few missing headers that were uncovered by
shuffling the order or moving headers up to be main-module-headers.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@169237 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
19c8ce0cf534caa909955ba9a17b7841f574a3be 01-Dec-2012 Eli Friedman <eli.friedman@gmail.com> Try to make the source location information for token pastes a bit more consistent.

Fixes a crash printing diagnostics on the gcc testsuite, and also makes
diagnostic range printing print nicer results for token pastes.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@169068 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
02f0022dfced0d082d979ebfd9185211f4dd0496 09-Nov-2012 Andy Gibbs <andyg1001@hotmail.co.uk> Improved support for removing the comma preceding __VA_ARGS__ where __VA_ARGS__
is empty in a variadic macro expansion. This fixes a divergence in support for
the ", ## __VA_ARGS__" GCC extension which differed in behaviour when in strict
C99 mode (note: there is no change in behaviour has been made in the gnu99 mode
that clang uses by default). In addition, there is improved support for the
Microsoft alternative extension ", __VA_ARGS__".

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@167613 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
93dec51750411678bb9c5bc1b27b259f5f5a23bd 26-Sep-2012 Nico Weber <nicolasweber@gmx.de> Revert r163022, it caused PR13924.

Add a test for PR13924. Do not revert the test added in r163022,
it surprisingly still passes even after reverting the code changes.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@164672 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
374b3837d676133fcc1eb70a25c8baf8ec4a5c4a 24-Sep-2012 Dmitri Gribenko <gribozavr@gmail.com> StringRef'ize Preprocessor::CreateString().


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@164555 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
181cd5bdebf0b88e8194c3f58da60f7f5b918b33 31-Aug-2012 Joao Matos <ripzonetriton@gmail.com> Emulate MSVC's preprocessor macro argument separator behavior by not considering commas from nested macro expansions as argument separators. Fixes parsing of VS 2012 headers.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@163022 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
c30981a563a8947cb26b1e308d122fa2ef90fceb 30-Aug-2012 Richard Smith <richard-llvm@metafoo.co.uk> Make preprocessor act in a GCC-compatible fashion when a macro is redefined
within its own argument list. The original definition is used for the immediate
expansion, but the new definition is used for any subsequent occurences within
the argument list or after the expansion.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@162906 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
bed28ac1d1463adca3ecf24fca5c30646fa9dbb2 23-Jul-2012 Sylvestre Ledru <sylvestre@debian.org> Fix a typo (the the => the)

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@160622 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
9f728cd37476c6588b06d241fa778d2df6e277da 23-Jun-2012 Richard Smith <richard-llvm@metafoo.co.uk> Minor improvements to some C99 variadic-macro-related diagnostics.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@159054 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
c5f7459b2d60ad55ed2ab3cfd281d9c718f5a8df 13-Jun-2012 Richard Smith <richard-llvm@metafoo.co.uk> Fix issue where a token paste which forms a /* or // would discard the rest of
the input: token-pasting was producing a tok::eof.

Patch by Andy Gibbs!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158412 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
4e4d08403ca5cfd4d558fa2936215d3a4e5a528d 11-Mar-2012 David Blaikie <dblaikie@gmail.com> Unify naming of LangOptions variable/get function across the Clang stack (Lex to AST).

The member variable is always "LangOpts" and the member function is always "getLangOpts".

Reviewed by Chris Lattner

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@152536 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
f7ccbad5d9949e7ddd1cbef43d482553b811e026 05-Feb-2012 Dylan Noblesmith <nobled@dreamwidth.org> Basic: import SmallString<> into clang namespace

(I was going to fix the TODO about DenseMap too, but
that would break self-host right now. See PR11922.)

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@149799 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
8fe83e1df954d72c0f4ffc15d20a5222ec151c21 04-Feb-2012 Benjamin Kramer <benny.kra@googlemail.com> Move a method from IdentifierTable.h out of line and remove the SmallString include.

Fix all the transitive include users.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@149783 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
a08529cc3f00e0b47a3c028823634129ac46847b 03-Oct-2011 Abramo Bagnara <abramo.bagnara@gmail.com> Fixed exapnsion range for # and ##.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@141012 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
a64ccefdf0ea4e03ec88805d71b0af74950c7472 19-Sep-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Rename SourceLocation::getFileLocWithOffset -> getLocWithOffset.

It already works (and is useful with) macro locs as well.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@140057 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
62ec1f2fd7368542bb926c04797fb07023547694 17-Sep-2011 Francois Pichet <pichet2000@gmail.com> Rename LangOptions::Microsoft to LangOptions::MicrosoftExt to make it clear that this flag must be used only for Microsoft extensions and not emulation; to avoid confusion with the new LangOptions::MicrosoftMode flag.

Many of the code now under LangOptions::MicrosoftExt will eventually be moved under the LangOptions::MicrosoftMode flag.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139987 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
c4b5582908e743cc05782e5220546f27286be19b 24-Aug-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Silence 'may be used uninitialized' warnings.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138475 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
b6c465e17ec37390667223a18a340e8652c212ff 23-Aug-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Amend r138129 (reduction of SLocEntries) which introduced performance regression due
to increased calls to SourceManager::getFileID. (rdar://9992664)

Use a slightly different approach that is more efficient both in terms of speed
(no extra getFileID calls) and in SLocEntries reduction.

Comparing pre-r138129 and this patch we get:

For compiling SemaExpr.cpp reduction of SLocEntries by 26%.
For the boost enum library:
-SLocEntries -34% (note that this was -5% for r138129)
-Memory consumption -50%
-PCH size -31%

Reduced SLocEntries also benefit the hot function SourceManager::getFileID,
evident by the reduced "FileID scans".

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138380 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
499ea5550d6e2fc5cfbd33b47f06d92ce25d7a13 23-Aug-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Introduce SourceManager::isInSLocAddrSpace and use it in TokenLexer instead of isInFileID
since it is a bit more efficient.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138379 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
5d579e7b0463a2e65328df85e4fed8e07799fd9e 23-Aug-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Rename SourceManager::isBeforeInSourceLocationOffset -> isBeforeInSLocAddrSpace.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138378 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
c0069d850efe74bac36c526dfec86b7dad1ede2f 23-Aug-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> TokenLexer::getExpansionLocForMacroDefLoc doesn't need to return an invalid SourceLocation.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138377 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
d60a34a4e514ec0dfddd05ef2744be104e111f45 20-Aug-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> For assigning SourceLocations to macro arg tokens, reserve a single SLocEntry
for tokens that are lexed consecutively from the same FileID, instead of creating
a SLocEntry for each token. e.g for

assert(foo == bar);

there will be a single SLocEntry for the "foo == bar" chunk and locations
for the 'foo', '==', 'bar' tokens will point inside that chunk.

For parsing SemaExpr.cpp, this reduced the number of SLocEntries by 25%.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138129 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
ec3b5389a5032433116ec2f2ee266b9666ad2eb4 20-Aug-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Rename TokenLexer::getMacroExpansionLocation -> getExpansionLocForMacroDefLoc, no functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138128 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
bf340e452339e374ea6eef78c1f0a2abdd16c5a3 26-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Rename create(MacroArg)InstantiationLoc to create(MacroArg)ExpansionLoc.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136054 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
402785357ab053dd53f4fdd858b9630a5e0f8bad 25-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Mechanically rename SourceManager::getInstantiationLoc and
FullSourceLoc::getInstantiationLoc to ...::getExpansionLoc. This is part
of the API and documentation update from 'instantiation' as the term for
macros to 'expansion'.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135914 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
5f9e272e632e951b1efe824cd16acb4d96077930 23-Jul-2011 Chris Lattner <sabre@nondot.org> remove unneeded llvm:: namespace qualifiers on some core types now that LLVM.h imports
them into the clang namespace.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135852 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
f62d43d2afe1960755a1b5813cae1e5983bcac1b 19-Jul-2011 Douglas Gregor <dgregor@apple.com> Revamp the SourceManager to separate the representation of parsed
source locations from source locations loaded from an AST/PCH file.

Previously, loading an AST/PCH file involved carefully pre-allocating
space at the beginning of the source manager for the source locations
and FileIDs that correspond to the prefix, and then appending the
source locations/FileIDs used for parsing the remaining translation
unit. This design forced us into loading PCH files early, as a prefix,
whic has become a rather significant limitation.

This patch splits the SourceManager space into two parts: for source
location "addresses", the lower values (growing upward) are used to
describe parsed code, while upper values (growing downward) are used
for source locations loaded from AST/PCH files. Similarly, positive
FileIDs are used to describe parsed code while negative FileIDs are
used to file/macro locations loaded from AST/PCH files. As a result,
we can load PCH/AST files even during parsing, making various
improvemnts in the future possible, e.g., teaching #include <foo.h> to
look for and load <foo.h.gch> if it happens to be already available.

This patch was originally written by Sebastian Redl, then brought
forward to the modern age by Jonathan Turner, and finally
polished/finished by me to be committed.




git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135484 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
d0f5e443ceed37e1a72015bd109db8e5fc6206cf 14-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Switch the TokenLexer's terminology from various forms of 'instantiate'
to 'expand' for macros. Only comments and uses local to the TokenLexer
are updated.

No functionality changed.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135137 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
b1077dcd9f40f8fb4b7a22a9bcc1dcac05b3dbae 08-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Tweak formatting.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134675 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
886ea9b4c8b20b3c550dffaa893b194792e49078 08-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Switch the token-paste source locations inside of function style macro
argument expansion to use the macro argument source locations as well.
Add a few tests to exercise this. There is still a bit more work needed
here though.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134674 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
c8d1ecca1cd3fadbd331d15c420755aa6184554b 08-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Keep track of which source locations are part of a macro argument
instantiation and improve diagnostics which are stem from macro
arguments to trace the argument itself back through the layers of macro
expansion.

This requires some tricky handling of the source locations, as the
argument appears to be expanded in the opposite direction from the
surrounding macro. This patch provides helper routines that encapsulate
the logic and explain the reasoning behind how we step through macros
during diagnostic printing.

This fixes the rest of the test cases originially in PR9279, and later
split out into PR10214 and PR10215.

There is still some more work we can do here to improve the macro
backtrace, but those will follow as separate patches.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134660 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
3582a6c0a35f202249b721e100229fde9de0d6a6 07-Jul-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Turn hashhash into tok::unkwown when it comes from expanding an argument, per Chris' suggestion.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134621 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
0fd9c48dfc6a324d8399b0b43ab4b943a1b1b843 07-Jul-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> When expanding macro arguments, treat '##' coming from an argument as a normal token.

e.g.

#define M(x) A x B
M(##) // should expand to 'A ## B', not 'AB'

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134588 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
b73377eeb3eff76be134203aebb6068244b177f3 07-Jul-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Make the Preprocessor more memory efficient and improve macro instantiation diagnostics.

When a macro instantiation occurs, reserve a SLocEntry chunk with length the
full length of the macro definition source. Set the spelling location of this chunk
to point to the start of the macro definition and any tokens that are lexed directly
from the macro definition will get a location from this chunk with the appropriate offset.

For any tokens that come from argument expansion, '##' paste operator, etc. have their
instantiation location point at the appropriate place in the instantiated macro definition
(the argument identifier and the '##' token respectively).
This improves macro instantiation diagnostics:

Before:

t.c:5:9: error: invalid operands to binary expression ('struct S' and 'int')
int y = M(/);
^~~~
t.c:5:11: note: instantiated from:
int y = M(/);
^

After:

t.c:5:9: error: invalid operands to binary expression ('struct S' and 'int')
int y = M(/);
^~~~
t.c:3:20: note: instantiated from:
\#define M(op) (foo op 3);
~~~ ^ ~
t.c:5:11: note: instantiated from:
int y = M(/);
^

The memory savings for a candidate boost library that abuses the preprocessor are:

- 32% less SLocEntries (37M -> 25M)
- 30% reduction in PCH file size (900M -> 635M)
- 50% reduction in memory usage for the SLocEntry table (1.6G -> 800M)

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134587 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
5b3284a9275a27f5c4410e25eb8933be540601d6 30-Jun-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Introduce a caching mechanism for macro expanded tokens.

Previously macro expanded tokens were added to Preprocessor's bump allocator and never released,
even after the TokenLexer that were lexing them was finished, thus they were wasting memory.
A very "useful" boost library was causing clang to eat 1 GB just for the expanded macro tokens.

Introduce a special cache that works like a stack; a TokenLexer can add the macro expanded tokens
in the cache, and when it finishes, the tokens are removed from the end of the cache.

Now consumed memory by expanded tokens for that library is ~ 1.5 MB.

Part of rdar://9327049.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134105 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
b11e43c31dc5d395a7ec6a07259c078dadd4f47b 14-Jun-2011 Chris Lattner <sabre@nondot.org> revert r133003 and fix the bug properly: the issue was that ## in a token
lexer is not a paste operator, it is a normal token. This fixes a conformance
issue shown here:
http://p99.gforge.inria.fr/c99-conformance/c99-conformance-clang-2.9.html

and it defines away the crash from before.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@133005 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
0e0b6931598be8a50ac5c6e0be595b35da276df9 14-Jun-2011 Chris Lattner <sabre@nondot.org> Fix a crash on the testcase in PR9981 / rdar://9486765.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@133003 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
28bbe4b8acc338476fe0825769b41fb32b423c72 28-Apr-2011 John Wiegley <johnw@boostpro.com> Parsing/AST support for Structured Exception Handling

Patch authored by Sohail Somani.

Provide parsing and AST support for Windows structured exception handling.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@130366 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
84021556baceb76eedf7d44be8ba71d9b8cfacce 28-Feb-2011 Peter Collingbourne <peter@pcc.me.uk> Rename tok::eom to tok::eod.

The previous name was inaccurate as this token in fact appears at
the end of every preprocessing directive, not just macro definitions.
No functionality change, except for a diagnostic tweak.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@126631 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
b2eb53d9fd973a1a02e05e67a3307b3efd12eff2 22-Feb-2011 Peter Collingbourne <peter@pcc.me.uk> Make TokenLexer capable of storing preprocessor directive tokens

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@126220 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
c4bf2b9afb7d47445a9dc6bc848657098a4e3851 22-Dec-2010 Abramo Bagnara <abramo.bagnara@gmail.com> Introduced raw_identifier token kind.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@122394 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
3248854a5d16e1de17c58e05f726bdef9f042df2 30-Oct-2010 Chris Lattner <sabre@nondot.org> Rename alignof -> alignOf to avoid irritating C++'0x compilers,
PR8423


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@117775 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
496af57256566c0db39b6a538bf7e4c6634afb2a 21-Aug-2010 Chris Lattner <sabre@nondot.org> fix PR7943, a corner case with the GNU __VA_ARGS__ comma
swallowing extension.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@111701 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
476d8b863cb65b2b5833235d97315cdb46e6f5aa 11-Aug-2010 Benjamin Kramer <benny.kra@googlemail.com> Random temporary string cleanup.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@110807 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
920bf45bd739334c25633ec9a29d903cab50be23 17-Jul-2010 Chris Lattner <sabre@nondot.org> Add another terrible VC++ compatibility hack: allow users to
allow invalid token pastes (when in -fms-extensions mode)
with -Wno-invalid-token-paste


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@108624 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
453091cc2082e207ea2c2dda645a9bc01b37fb0c 16-Mar-2010 Douglas Gregor <dgregor@apple.com> Audit all Preprocessor::getSpelling() callers, improving failure
recovery for those that need it.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98689 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
f6ac97b101c8840efa92bf29166077ce4049e293 16-Mar-2010 Benjamin Kramer <benny.kra@googlemail.com> Let SourceManager::getBufferData return StringRef instead of a pair of two const char*.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98630 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
f715ca12bfc9fddfde75f98a197424434428b821 16-Mar-2010 Douglas Gregor <dgregor@apple.com> Give SourceManager a Diagnostic object with which to report errors,
and start simplifying the interfaces in SourceManager that can fail.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98594 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
aea67dbd653a2dd6dd5cc2159279e81e855b2482 15-Mar-2010 Douglas Gregor <dgregor@apple.com> Introduce a new BufferResult class to act as the return type of
SourceManager's getBuffer() (and similar) operations. This abstract
can be used to force callers to cope with errors in getBuffer(), such
as missing files and changed files. Fix a bunch of callers to use the
new interface.

Add some very basic checks for file consistency (file size,
modification time) into ContentCache::getBuffer(), although these
checks don't help much until we've updated the main callers (e.g.,
SourceManager::getSpelling()).



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98585 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
193575455e00eca03fd7177f60e3f2e6263cb661 13-Mar-2010 Kovarththanan Rajaratnam <kovarththanan.rajaratnam@gmail.com> Use SmallString instead of SmallVector

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98436 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
65cc1e889c0c2c617a81a0ae032fa4fb24760f38 13-Mar-2010 Kovarththanan Rajaratnam <kovarththanan.rajaratnam@gmail.com> No need to call setIdentifierInfo() after LookUpIdentifierInfo() which LookUpIdentifierInfo() will automatically do

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98435 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
f5809a7b76fd5a86ae737d9b525a1eddb9339ee7 28-Dec-2009 Chris Lattner <sabre@nondot.org> The PreExpArgTokens array is indexed with an argument #,
not a token number. Fix the reserve logic to get the right
amount of space.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@92202 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
917f14387470cc98b8ff90342f5307bc921234fb 23-Dec-2009 Chris Lattner <sabre@nondot.org> comment tweak


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@92055 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
561395bead16b9dc9b0d5bfb6b257a9ed4545db6 14-Dec-2009 Chris Lattner <sabre@nondot.org> move the VarargsElided member of MacrosArgs to shrink the MacroArgs struct
on 64-bit targets. Pass Preprocessor into create/destroy methods of MacroArgs
even though it isn't used yet.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@91345 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
04f2bd859ba6a23faf98eb3cc7d40e9adf6d5eaa 04-Dec-2009 Chris Lattner <sabre@nondot.org> minor tidy.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90543 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
1eb4433ac451dc16f4133a88af2d002ac26c58ef 09-Sep-2009 Mike Stump <mrs@apple.com> Remove tabs, and whitespace cleanups.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@81346 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
3c17b8a4306b84c05454168128bc705b33a79ca3 28-May-2009 Chris Lattner <sabre@nondot.org> fix the "pasting formed 'a]', an invalid preprocessing token"
diagnostic to include the full instantiation location for the
invalid paste. For:

#define foo(a, b) a ## b
#define bar(x) foo(x, ])
bar(a)
bar(zdy)

Instead of:

t.c:3:22: error: pasting formed 'a]', an invalid preprocessing token
#define foo(a, b) a ## b
^
t.c:3:22: error: pasting formed 'zdy]', an invalid preprocessing token

we now produce:

t.c:7:1: error: pasting formed 'a]', an invalid preprocessing token
bar(a)
^
t.c:4:16: note: instantiated from:
#define bar(x) foo(x, ])
^
t.c:3:22: note: instantiated from:
#define foo(a, b) a ## b
^
t.c:8:1: error: pasting formed 'zdy]', an invalid preprocessing token
bar(zdy)
^
t.c:4:16: note: instantiated from:
#define bar(x) foo(x, ])
^
t.c:3:22: note: instantiated from:
#define foo(a, b) a ## b
^



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@72519 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
cfcceab862141b7cc3df42ced5f421d76e536a36 28-May-2009 Eli Friedman <eli.friedman@gmail.com> Make the bad paste diagnostic print the entire pasted token.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@72497 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
442a661a2b44e78e3c66e35bf842c3dc119dfe92 25-May-2009 Chris Lattner <sabre@nondot.org> improve comment, no functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@72386 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
4c3ba6c1944e6a4758f1b3a0aec8a7ff4c561051 24-May-2009 Eli Friedman <eli.friedman@gmail.com> Make sure an invalid concatentaion doesn't insert whitespace before
the RHS. Fixes assembler-with-cpp issue reported on cfe-dev.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@72370 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
6f421f91849418883186592051fce05339ee62e9 22-May-2009 Argyrios Kyrtzidis <akyrtzi@gmail.com> The TokenLexer may encounter annotations if the parser enters them using Preprocessor::EnterTokenStream.
So check for annotation before using the Token's IdentifierInfo.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@72278 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
d9b856fddd07b48388307127687a634235271849 03-May-2009 Eli Friedman <eli.friedman@gmail.com> Fix for PR4132: make sure to insert whitespace consistently before a
pasted token.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@70793 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
617bd458600c6d68ffbe1328806b58384bde848a 26-Apr-2009 Eli Friedman <eli.friedman@gmail.com> Silence gcc warnings.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@70086 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
332ee08e2fe78fde281e0868ec039b546cb6e0f4 19-Apr-2009 Chris Lattner <sabre@nondot.org> move token paste poisoning diagnostics to after the instantiation loc
for a token is set, this makes the diagnostic "expanded from stack" work
for this diagnostic. Add a testcase for PR3918.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69544 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
88e43a519e5ecc06b5e14da8ffb1e89cc15474c0 19-Apr-2009 Chris Lattner <sabre@nondot.org> Fix PR3918: Invalid use of __VA_ARGS__ not diagnosed,
by rejecting invalid poisoned tokens in the token
pasting path.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69536 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
f1538de18370c202263d7d1bc326cbc2c5434e1d 23-Mar-2009 Chris Lattner <sabre@nondot.org> The individual pieces of an invalid paste as still candidates for expansion.
This matters in assembler mode, where this is silently allowed.
This fixes rdar://6709206.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67539 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
48be3801ca729cf2afea060fdb08a479b49ab440 18-Mar-2009 Chris Lattner <sabre@nondot.org> constructs like:
#define Y X ## .
Y

are ok in .S files.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67231 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
1fa495304c81e03f07f278a47b5efe9317104aab 08-Mar-2009 Chris Lattner <sabre@nondot.org> simplify some logic by making ScratchBuffer handle the application of trailing
\0's to created tokens instead of making all clients do it. No functionality
change.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@66373 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
32fca722dd974b8202d0fb9c71b6c185c0767da6 04-Mar-2009 Chris Lattner <sabre@nondot.org> make the token lexer allocate its temporary token buffers for
preexpanded macro arguments from the preprocessor's bump pointer.
This reduces # mallocs from 12444 to 11792.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@66025 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
e7fb48466afcbf2c4ccdfa658824282fdc3c512c 15-Feb-2009 Chris Lattner <sabre@nondot.org> track "just a little more" location information for macro instantiations.
Now instead of just tracking the expansion history, also track the full
range of the macro that got replaced. For object-like macros, this doesn't
change anything. For _Pragma and function-like macros, this means we track
the locations of the ')'.

This is required for PR3579 because apparently GCC uses the line of the ')'
of a function-like macro as the location to expand __LINE__ to.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64601 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
500d3297d2a21edeac4d46cbcbe21bc2352c2a28 29-Jan-2009 Chris Lattner <sabre@nondot.org> move library-specific diagnostic headers into library private dirs. Reduce
redundant #includes. Patch by Anders Johnsen!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63271 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
20c6b3b85e186cd52d5d99489132d71d498159eb 27-Jan-2009 Chris Lattner <sabre@nondot.org> Split the single monolithic DiagnosticKinds.def file into one
.def file for each library. This means that adding a diagnostic
to sema doesn't require all the other libraries to be rebuilt.

Patch by Anders Johnsen!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63111 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
e7689886d6a68507ae90ca330bd24bf89d4d6df1 26-Jan-2009 Chris Lattner <sabre@nondot.org> remove my hacks that aggressively threw away multiple
instantiation history in an effort to speed up c99-intconst-1.c.
Now that multiple nested instantiations are allowed, we just
make them and don't pay the cost of lookups. With the other
changes that went in before this, reverting this is actually
a speedup for c99-intconst-1.c, speeding it up from 1.96s to 1.80s,
and preserves much better loc info.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63036 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
47246be8ac5b0ddde6c402b8fc6946b6135487b5 26-Jan-2009 Chris Lattner <sabre@nondot.org> This change refactors some of the low-level lexer interfaces a bit.
Token now has a class of kinds for "literals", which include
numeric constants, strings, etc. These tokens can optionally have
a pointer to the start of the token in the lexer buffer. This
makes it faster to get spelling and do other gymnastics, because we
don't have to go through source locations.

This change is performance neutral, but will make other changes
more feasible down the road.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63028 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
b99bdadf9ace3e8c793e12e9bb447501f86b4f72 26-Jan-2009 Chris Lattner <sabre@nondot.org> Only resolve a macro's instantiation loc once per macro, instead of once
per token lexed from it. This speeds up clang on c99-intconst-1.c from
the GCC testsuite from 3.64s to 2.66s (36%). This reduces the number of
binary search FileID lookups from 251570522 to 114175649 on this testcase.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62992 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
de7aeefc5573d669ed476d7bda7a8940d3bcadb7 26-Jan-2009 Chris Lattner <sabre@nondot.org> Check in the long promised SourceLocation rewrite. This lays the
ground work for implementing #line, and fixes the "out of macro ID's"
problem.

There is nothing particularly tricky about the code, other than the
very performance sensitive SourceManager::getFileID() method.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62978 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
863c486fcb6162495a94fddf7ac8409de2638995 23-Jan-2009 Chris Lattner <sabre@nondot.org> This is a follow-up to r62675:

Refactor how the preprocessor changes a token from being an tok::identifier to a
keyword (e.g. tok::kw_for). Instead of doing this in HandleIdentifier, hoist this
common case out into the caller, so that every keyword doesn't have to go through
HandleIdentifier. This drops time in HandleIdentifier from 1.25ms to .62ms, and
speeds up clang -Eonly with PTH by about 1%.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62855 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
6a170eb3ea6d6319277becabef68eb1a26bf8766 21-Jan-2009 Chris Lattner <sabre@nondot.org> Add a bit to IdentifierInfo that acts as a simple predicate which
tells us whether Preprocessor::HandleIdentifier needs to be called.
Because this method is only rarely needed, this saves a call and a
bunch of random checks. This drops the time in HandleIdentifier
from 3.52ms to .98ms on cocoa.h on my machine.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62675 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
bcc2a67e5180612417727cbdd8afd0f79fdf726d 19-Jan-2009 Chris Lattner <sabre@nondot.org> Make SourceLocation::getFileLoc private to reduce the API exposure of
SourceLocation. This requires making some cleanups to token pasting
and _Pragma expansion.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62490 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
de96c0f29c4cacabe6ea577c61db87c2a85aea6c 17-Jan-2009 Chris Lattner <sabre@nondot.org> make the verbose raw-lexer ctor fully explicit instead of having
embedded magic.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62417 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
2b2453a7d8fe732561795431f39ceb2b2a832d84 17-Jan-2009 Chris Lattner <sabre@nondot.org> this massive patch introduces a simple new abstraction: it makes
"FileID" a concept that is now enforced by the compiler's type checker
instead of yet-another-random-unsigned floating around.

This is an important distinction from the "FileID" currently tracked by
SourceLocation. *That* FileID may refer to the start of a file or to a
chunk within it. The new FileID *only* refers to the file (and its
#include stack and eventually #line data), it cannot refer to a chunk.

FileID is a completely opaque datatype to all clients, only SourceManager
is allowed to poke and prod it.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62407 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
204b2fed909b1eabea7aeb6caadd7cff718edee5 18-Nov-2008 Chris Lattner <sabre@nondot.org> Remove the last of the old-style Preprocessor::Diag methods.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59554 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
0af574270d3be2b0e73a3379dfaa633746f8fc6f 12-Oct-2008 Chris Lattner <sabre@nondot.org> Simplify raw mode lexing by treating an unterminate /**/ comment the
same we we do an unterminated string or character literal. This makes
it so we can guarantee that the lexer never calls into the
preprocessor (which would be suicide for a raw lexer).


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57395 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
590f0cc643274267d4d41125b62557e1d87886c3 12-Oct-2008 Chris Lattner <sabre@nondot.org> Change how raw lexers are handled: instead of creating them and then
using LexRawToken, create one and use LexFromRawLexer. This avoids
twiddling the RawLexer flag around and simplifies some code (even
speeding raw lexing up a tiny bit).

This change also improves the token paster to use a Lexer on the stack
instead of new/deleting it.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57393 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
48002c89578eaeb008a0bb5a10d8216e61906459 29-Sep-2008 Nico Weber <nicolasweber@gmx.de> whitespace and comment changes, to fix grammar and 80 col violations

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@56776 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp
bda0b626e74513950405c27525af87e214e605e2 16-Mar-2008 Chris Lattner <sabre@nondot.org> Make a major restructuring of the clang tree: introduce a top-level
lib dir and move all the libraries into it. This follows the main
llvm tree, and allows the libraries to be built in parallel. The
top level now enforces that all the libs are built before Driver,
but we don't care what order the libs are built in. This speeds
up parallel builds, particularly incremental ones.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48402 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/TokenLexer.cpp