History log of /external/clang/include/clang/Lex/Lexer.h
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
6bcf27bb9a4b5c3f79cb44c0e4654a6d7619ad89 29-May-2014 Stephen Hines <srhines@google.com> Update Clang for 3.5 rebase (r209713).

Change-Id: I8c9133b0f8f776dc915f270b60f94962e771bc83
/external/clang/include/clang/Lex/Lexer.h
651f13cea278ec967336033dd032faef0e9fc2ec 24-Apr-2014 Stephen Hines <srhines@google.com> Updated to Clang 3.5a.

Change-Id: I8127eb568f674c2e72635b639a3295381fe8af82
/external/clang/include/clang/Lex/Lexer.h
d2f9308220af22bfc1bcd3bc2cad118dbd8be013 19-Sep-2013 Eli Friedman <eli.friedman@gmail.com> Make Preprocessor::Lex non-recursive.

Before this patch, Lex() would recurse whenever the current lexer changed (e.g.
upon entry into a macro). This patch turns the recursion into a loop: the
various lex routines now don't return a token when the current lexer changes,
and at the top level Preprocessor::Lex() now loops until it finds a token.
Normally, the recursion wouldn't end up being very deep, but the recursion depth
can explode in edge cases like a bunch of consecutive macros which expand to
nothing (like in the testcase test/Preprocessor/macro_expand_empty.c in this
patch).

<rdar://problem/14569770>

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@190980 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
f0dd853bb1e8f3e59b169e6d34a8556c6003c47c 20-Aug-2013 Fariborz Jahanian <fjahanian@apple.com> ObjectiveC migrator: More work towards
insertion of ObjC audit pragmas.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@188733 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
4ac537b0f07f2efb9fcf081f60d54e6cfb1cf2d5 23-Jul-2013 Richard Smith <richard-llvm@metafoo.co.uk> C++1y literal suffix support:
* Allow ns, us, ms, s, min, h as numeric ud-suffixes
* Allow s as string ud-suffix


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@186933 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
49e1d9814a34e4024a347c2fce8e9567c40e6bf5 04-Jun-2013 Alexander Kornienko <alexfh@google.com> Added Lexer::getBufferEnd().

Summary:
There's Lexer::getBufferStart(), and we need getBufferEnd() to access
the whole input buffer in clang::format::reformat. We don't want to
rely on the fact that the Lexer::BufferEnd always points to '\0', as there can
be embedded '\0's as well.

Reviewers: jordan_rose

Reviewed By: jordan_rose

CC: cfe-commits

Differential Revision: http://llvm-reviews.chandlerc.com/D916

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@183236 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
fd61d6fe0aa9853f0577ca88a63901c3773e8101 25-Mar-2013 Nick Lewycky <nicholas@mxc.ca> Correct indentation. No functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@177916 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
6aad4a31b35df07fe818f193fcfd3c0197aea467 21-Feb-2013 Jordan Rose <jordan_rose@apple.com> Preprocessor: preserve whitespace in -traditional-cpp mode.

Note that unlike GNU cpp we currently do not preserve whitespace in macros
(even in -traditional-cpp mode).

<rdar://problem/12897179>

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@175778 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
c7629d941557f7179eb8fa8a2e2a74d749cbaf7c 24-Jan-2013 Jordan Rose <jordan_rose@apple.com> Handle universal character names and Unicode characters outside of literals.

This is a missing piece for C99 conformance.

This patch handles UCNs by adding a '\\' case to LexTokenInternal and
LexIdentifier -- if we see a backslash, we tentatively try to read in a UCN.
If the UCN is not syntactically well-formed, we fall back to the old
treatment: a backslash followed by an identifier beginning with 'u' (or 'U').

Because the spelling of an identifier with UCNs still has the UCN in it, we
need to convert that to UTF-8 in Preprocessor::LookUpIdentifierInfo.

Of course, valid code that does *not* use UCNs will see only a very minimal
performance hit (checks after each identifier for non-ASCII characters,
checks when converting raw_identifiers to identifiers that they do not
contain UCNs, and checks when getting the spelling of an identifier that it
does not contain a UCN).

This patch also adds basic support for actual UTF-8 in the source. This is
treated almost exactly the same as UCNs except that we consider stray
Unicode characters to be mistakes and offer a fixit to remove them.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@173369 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
d93335c43fd462145fee3ea8f4d84d430577c821 07-Jan-2013 Argyrios Kyrtzidis <akyrtzi@gmail.com> Pull the bulk of Lexer::MeasureTokenLength() out into a new function,
Lexer::getRawToken().

No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@171771 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
30a2e16f6c27f888dd11eba6bbbae1e980078fcb 04-Dec-2012 Chandler Carruth <chandlerc@gmail.com> Sort #include lines for all files under include/...

This is a simpler sort, entirely automatic with the help of
llvm/utils/sort_includes.py -- no manual edits here.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@169238 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
bb23628148f555a4cf71f98c27096a7a804c085c 11-Nov-2012 Nico Weber <nicolasweber@gmx.de> s/BCPLComment/LineComment/

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@167690 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
164fcd022185fddc0e689f61ac8ad4b3ac744c2a 06-Nov-2012 Nico Weber <nicolasweber@gmx.de> Delete an outdated comment.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@167441 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
f56faa01936b9cf909623d7f06e3c2569ca4a78e 15-Sep-2012 Dmitri Gribenko <gribozavr@gmail.com> Use LLVM_DELETED_FUNCTION in place of 'DO NOT IMPLEMENT' comments.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@163983 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
9366a5a8d0c5834cffda2c31c924605fb9dffc9b 07-Sep-2012 Abramo Bagnara <abramo.bagnara@bugseng.com> Moved back getCharAndSizeNoWarn to public area.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@163408 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
e506f8a41063410c75945ebb804758bd0202947f 31-Aug-2012 Eli Friedman <eli.friedman@gmail.com> Make a bunch of methods on Lexer private.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@162970 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
64f9b500987394c69cd9a2f5483d06b5cafb0824 28-Aug-2012 Dmitri Gribenko <gribozavr@gmail.com> Fix a -Wdocumentation warning.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@162721 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
70517ca5c07c4b41ff8662b94ee22047b0299f8c 23-Aug-2012 Dmitri Gribenko <gribozavr@gmail.com> Fix a bunch of -Wdocumentation warnings.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@162452 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
7e6f1a23fe26748a4eafc5c64ff6c319f6d932f3 17-Jun-2012 James Dennett <jdennett@google.com> Documentation cleanup: escaping #define in Doxygen comments

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158629 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
092bf67e5ca560d2fc6aa70be1f172b8b3a5ff96 09-Jun-2012 Dmitri Gribenko <gribozavr@gmail.com> Correct method name in comment: from LexRawToken to LexFromRawLexer, according
to a change done long ago in r57393.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158243 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
d880b3aa6d594d1a7f2d307c29378c6f59b216ff 07-Jun-2012 Jordan Rose <jordan_rose@apple.com> Insert a space if necessary when suggesting CFBridgingRetain/Release.

This was a problem for people who write 'return(result);'

Also fix ARCMT's corresponding code, though there's no test case for this
because implicit casts like this are rejected by the migrator for being
ambiguous, and explicit casts have no problem.

<rdar://problem/11577346>

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158130 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
3093b20d824a953d8bc7a786dd952414898f8d6d 18-May-2012 Benjamin Kramer <benny.kra@googlemail.com> Lexer::ReadToEndOfLine: Only build the string if it's actually used and do so in a less malloc-intensive way.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@157064 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
95da18142abeec5099dedc50d6f418917bd6167d 04-May-2012 Argyrios Kyrtzidis <akyrtzi@gmail.com> Add more comments for Lexer::makeFileCharRange.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@156188 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
4e4d08403ca5cfd4d558fa2936215d3a4e5a528d 11-Mar-2012 David Blaikie <dblaikie@gmail.com> Unify naming of LangOptions variable/get function across the Clang stack (Lex to AST).

The member variable is always "LangOpts" and the member function is always "getLangOpts".

Reviewed by Chris Lattner

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@152536 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
5cc2c6eb67b6e5361bbe96f79b519fd62ec666d6 05-Mar-2012 Richard Smith <richard-llvm@metafoo.co.uk> Lexing support for user-defined literals. Currently these lex as the same token
kinds as the underlying string literals, and we silently drop the ud-suffix;
those issues will be fixed by subsequent patches.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@152012 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
a83f4d2315dbeb3914868f1ccb8e74fb2ccdbb0c 03-Feb-2012 Argyrios Kyrtzidis <akyrtzi@gmail.com> Change Lexer::makeFileCharRange() to have it accept a CharSourceRange
instead of a SourceRange, and handle the case where the range is
a char (not token) range.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@149677 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
e64d9037658a1b95c79ea275af6167a110b3c563 19-Jan-2012 Argyrios Kyrtzidis <akyrtzi@gmail.com> Introduce Lexer::getSourceText() that returns a string for the source
that the given source range encompasses.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148481 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
11b652d41d0d97380ab321a1dba48ecb044f9de8 19-Jan-2012 Argyrios Kyrtzidis <akyrtzi@gmail.com> Introduce Lexer::makeFileCharRange() that accepts a token source range
and returns a character range with file locations.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148480 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
69bda4c027671df7163619f215209529eb236620 19-Jan-2012 Argyrios Kyrtzidis <akyrtzi@gmail.com> For Lexer's isAt[Start/End]OfMacroExpansion add an out parameter for the macro
start/end location.

It is commonly needed after calling the function; with this way we avoid
recalculating it.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148479 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
c2a8d6cee01fc4845f5409bf5c021a64616ac8c3 18-Jan-2012 Anna Zaks <ganna@apple.com> Refactor: Pull getImmediateMacroName() out of DiagnosticRenderer and
into Lexer and Preprocessor; making it widely available.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148410 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
99ba9e3bd70671f3441fb974895f226a83ce0e66 20-Dec-2011 David Blaikie <dblaikie@gmail.com> Unweaken vtables as per http://llvm.org/docs/CodingStandards.html#ll_virtual_anch

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@146959 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
d5e1d606f8c22ebda17c6fbf952f8c1696428758 12-Oct-2011 Richard Smith <richard-llvm@metafoo.co.uk> Handle Perforce-style conflict markers like normal conflict markers. Perforce
swaps over the <<<< and >>>> markers, and uses shorter markers than traditional
tools.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@141751 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
d6471f7c1921c7802804ce3ff6fe9768310f72b9 26-Sep-2011 David Blaikie <dblaikie@gmail.com> Rename Diagnostic to DiagnosticsEngine as per issue 5397


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@140478 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
7d100872341f233c81e1d7b72b40457e62c36862 04-Sep-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Support code-completion for C++ inline methods and ObjC buffering methods.

Previously we would cut off the source file buffer at the code-completion
point; this impeded code-completion inside C++ inline methods and,
recently, with buffering ObjC methods.

Have the code-completion inserted into the source buffer so that it can
be buffered along with a method body. When we actually hit the code-completion
point the cut-off lexing or parsing.

Fixes rdar://10056932&8319466

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139086 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
03c107a42fae79e89d0016999a1a04c07d65591a 25-Aug-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Make Lexer::ComputePreamble accept a LangOptions parameter, otherwise it may be
out-of-sync how a file is compiled. Patch by Matthias Kleine!

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138580 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
2fa4e86b4fdada3b9ecbbbd99965b83ed879f69b 11-Aug-2011 Craig Topper <craig.topper@gmail.com> Add support for C++0x raw string literals.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@137298 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
aca25bccefe56121b686706afc84c8cb5d46e65b 27-Jul-2011 Anna Zaks <ganna@apple.com> Add a utility function to the Lexer, which makes it easier to find a token after the given location. (It is a generalized version of trans::findLocationAfterSemi from ArcMigrate, which will be changed to use the Lexer utility).

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136268 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
5cee1195584fa8672253139c86e922daeda69b9e 27-Jul-2011 Douglas Gregor <dgregor@apple.com> Add support for C++0x unicode string and character literals, from Craig Topper!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136210 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
686775deca8b8685eb90801495880e3abdd844c2 20-Jul-2011 Chris Lattner <sabre@nondot.org> now that we have a centralized place to do so, add some using declarations for
some common llvm types: stringref and smallvector. This cleans up the codebase
quite a bit.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135576 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
433db06b614f26dc6829e86d6ff469e2cca7d4f9 14-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Convert terminology in the Lexer from 'instantiate' and variants to
'expand'. Also update the public API it provides to the new term, and
propagate that update to the various clients.

No functionality changed.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135138 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
7a759606d93975866051f67104ae58446e55f404 07-Jul-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Move SourceManager::isAt[Start/End]OfMacroInstantiation functions to the Lexer, since they depend on it now.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134644 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
c69a505cfa318d571ce8a0cd038c8d958585a735 23-Apr-2011 Jay Foad <jay.foad@gmail.com> Remove unused STL header includes.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@130068 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
834e3f6c77d9ac03997a3f0c56934edcf406a355 08-Mar-2011 John McCall <rjmccall@apple.com> Fix my earlier commit to work with escaped newlines and leave breadcrumbs
in case we want to make a world where we can check intermediate instantiations
for this kind of breadcrumb.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@127221 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
b0607279cb98bbf2bbfe0db170aed39ef91e86a2 17-Nov-2010 Chris Lattner <sabre@nondot.org> move getSpelling from Preprocessor to Lexer, which it is more conceptually related to.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@119479 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
7ef5c27eb6e8ebe58b52013246c06753c3613263 17-Nov-2010 Chris Lattner <sabre@nondot.org> move AdvanceToTokenCharacter and getLocForEndOfToken from
Preprocessor to Lexer where they make more sense.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@119474 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
6cf750298d3621d8a10a6dd07fcee8e274b9d94d 30-Aug-2010 Sean Hunt <scshunt@csclub.uwaterloo.ca> Revert my user-defined literal commits - r1124{58,60,67} pending
some issues being sorted out.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112493 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
0016d519b831859526b79405cdae4c64c73731c8 29-Aug-2010 Sean Hunt <scshunt@csclub.uwaterloo.ca> Implement C++0x user-defined string literals.

The extra data stored on user-defined literal Tokens is stored in extra
allocated memory, which is managed by the PreprocessorLexer because there isn't
a better place to put it that makes sure it gets deallocated, but only after
it's used up. My testing has shown no significant slowdown as a result, but
independent testing would be appreciated.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112458 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
df95a13ec73d2cdaea79555cb412d767f4963120 09-Aug-2010 Douglas Gregor <dgregor@apple.com> Use precompiled preambles for in-process code completion.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@110596 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
f4f6c9db68465b886ec2e596feaa6ecc782395a4 26-Jul-2010 Douglas Gregor <dgregor@apple.com> Introduce basic support for loading a precompiled preamble while
reparsing an ASTUnit. When saving a preamble, create a buffer larger
than the actual file we're working with but fill everything from the
end of the preamble to the end of the file with spaces (so the lexer
will quickly skip them). When we load the file, create a buffer of the
same size, filling it with the file and then spaces. Then, instruct
the lexer to start lexing after the preamble, therefore continuing the
parse from the spot where the preamble left off.

It's now possible to perform a simple preamble build + parse (+
reparse) with ASTUnit. However, one has to disable a bunch of checking
in the PCH reader to do so. That part isn't committed; it will likely
be handled with some other kind of flag (e.g., -fno-validate-pch).

As part of this, fix some issues with null termination of the memory
buffers created for the preamble; we were trying to explicitly
NULL-terminate them, even though they were also getting implicitly
NULL terminated, leading to excess warnings about NULL characters in
source files.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@109445 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
a8e5c5bdbe387b2552c1c23b828f54abcf085a40 22-Jul-2010 Douglas Gregor <dgregor@apple.com> Improve performance during cursor traversal when a region of interest
is present.

Rather than using clang_getCursorExtent(), which requires
us to lex the token at the ending position to determine its
length. Then, we'd be comparing [a, b) source ranges that cover the
characters in the range rather than the normal behavior for Clang's
source ranges, which covers the tokens in the range. However, relexing
causes us to read the source file (which may come from a precompiled
header), which is rather unfortunate and affects performance.

In the new scheme, we only use Clang-style source ranges that cover
the tokens in the range. At the entry points where this matters
(clang_annotateTokens, clang_getCursor), we make sure to move source
locations to the start of the token.

Addresses most of <rdar://problem/8049381>.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@109134 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
f033f1da4a34f8df6e95e9929dc04ff54bb8fb01 20-Jul-2010 Douglas Gregor <dgregor@apple.com> Introduce a new lexer function to compute the "preamble" of a file,
which is the part of the file that contains all of the initial
comments, includes, and preprocessor directives that occur before any
of the actual code. Added a new -print-preamble cc1 action that is
only used for testing.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@108913 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
fc8ea23eb6cbaaa5046f2abb4c033e24c8659efd 26-Jan-2010 Douglas Gregor <dgregor@apple.com> Introduce a CIndex API for lexing the raw tokens within a given source
range. The token-annotation function does nothing, yet.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@94551 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
4718e67ca9018342483dccb40d43e339101ac386 23-Dec-2009 Chris Lattner <sabre@nondot.org> update comments


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@92022 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
34f349da38a7bd99103e12d8ea6c73bc8d025193 14-Dec-2009 Chris Lattner <sabre@nondot.org> teach clang to recover gracefully from conflict markers left in source
files: PR5238.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@91270 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
2968442603b029949246467253eeac8139a5b6d8 02-Dec-2009 Douglas Gregor <dgregor@apple.com> Extend the source manager with the ability to override the contents of
files with the contents of an arbitrary memory buffer. Use this new
functionality to drastically clean up the way in which we handle file
truncation for code-completion: all of the truncation/completion logic
is now encapsulated in the preprocessor where it belongs
(<rdar://problem/7434737>).



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90300 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
6e2901407bff59aeb4cc301cc58b034723d0eb49 30-Nov-2009 Chris Lattner <sabre@nondot.org> Fix PR5633 by making the preprocessor handle the case where we can
stat a file but where mmaping it fails. In this case, we emit an
error like:
t.c:1:10: fatal error: error opening file '../../foo.h'

instead of "cannot find file".



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90110 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
0369297684c13d5495ab490d46ed9d14ef235ce3 18-Sep-2009 Chris Lattner <sabre@nondot.org> allow clearing this value.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@82271 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
81b747b7fcc91c2fba9a3183d8fac80adbfc1d3e 17-Sep-2009 Douglas Gregor <dgregor@apple.com> Initial implementation of a code-completion interface in Clang. In
essence, code completion is triggered by a magic "code completion"
token produced by the lexer [*], which the parser recognizes at
certain points in the grammar. The parser then calls into the Action
object with the appropriate CodeCompletionXXX action.

Sema implements the CodeCompletionXXX callbacks by performing minimal
translation, then forwarding them to a CodeCompletionConsumer
subclass, which uses the results of semantic analysis to provide
code-completion results. At present, only a single, "printing" code
completion consumer is available, for regression testing and
debugging. However, the design is meant to permit other
code-completion consumers.

This initial commit contains two code-completion actions: one for
member access, e.g., "x." or "p->", and one for
nested-name-specifiers, e.g., "std::". More code-completion actions
will follow, along with improved gathering of code-completion results
for the various contexts.

[*] In the current -code-completion-dump testing/debugging mode, the
file is truncated at the completion point and EOF is translated into
"code completion".


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@82166 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
1eb4433ac451dc16f4133a88af2d002ac26c58ef 09-Sep-2009 Mike Stump <mrs@apple.com> Remove tabs, and whitespace cleanups.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@81346 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
033749571f8d4c804eeb357c70b06424aa24503b 19-Apr-2009 Chris Lattner <sabre@nondot.org> add a new Lexer::SkipEscapedNewLines method.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69483 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
24f0e48c0aa62f2268e061aad70f9b19a59e7b52 19-Apr-2009 Chris Lattner <sabre@nondot.org> factor escape newline measuring out into its own helper function.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69482 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
2c78b873f4f3823ae859c15674cb3d76c8554113 15-Apr-2009 Chris Lattner <sabre@nondot.org> Change Lexer::MeasureTokenLength to take a LangOptions reference.
This allows it to accurately measure tokens, so that we get:

t.cpp:8:13: error: unknown type name 'X'
static foo::X P;
~~~~~^

instead of the woefully inferior:

t.cpp:8:13: error: unknown type name 'X'
static foo::X P;
~~~~ ^

Most of this is just plumbing to push the reference around.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69099 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
1fa495304c81e03f07f278a47b5efe9317104aab 08-Mar-2009 Chris Lattner <sabre@nondot.org> simplify some logic by making ScratchBuffer handle the application of trailing
\0's to created tokens instead of making all clients do it. No functionality
change.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@66373 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
e7fb48466afcbf2c4ccdfa658824282fdc3c512c 15-Feb-2009 Chris Lattner <sabre@nondot.org> track "just a little more" location information for macro instantiations.
Now instead of just tracking the expansion history, also track the full
range of the macro that got replaced. For object-like macros, this doesn't
change anything. For _Pragma and function-like macros, this means we track
the locations of the ')'.

This is required for PR3579 because apparently GCC uses the line of the ')'
of a function-like macro as the location to expand __LINE__ to.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64601 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
de7aeefc5573d669ed476d7bda7a8940d3bcadb7 26-Jan-2009 Chris Lattner <sabre@nondot.org> Check in the long promised SourceLocation rewrite. This lays the
ground work for implementing #line, and fixes the "out of macro ID's"
problem.

There is nothing particularly tricky about the code, other than the
very performance sensitive SourceManager::getFileID() method.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62978 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
bcc2a67e5180612417727cbdd8afd0f79fdf726d 19-Jan-2009 Chris Lattner <sabre@nondot.org> Make SourceLocation::getFileLoc private to reduce the API exposure of
SourceLocation. This requires making some cleanups to token pasting
and _Pragma expansion.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62490 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
42e00d19b0dac64732eb5449d52a076282fcbf77 17-Jan-2009 Chris Lattner <sabre@nondot.org> Change the Lexer ctor used to lex _Pragma directives into a static factory
method. This lets us clean up the interface and make it more obvious that
this method is *really really* _Pragma specific.

Note that _Pragma handling uglifies the Lexer in the critical path. It would
be very interesting to consider making _Pragma remapping be a new special
lexer class of its own.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62425 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
88d3ac1341aa016cabd966c5b113a95ac05ea43f 17-Jan-2009 Chris Lattner <sabre@nondot.org> Change the Lexer ctor used in the non _Pragma case to take a FileID instead
of a SourceLocation. This should speed it up and definitely simplifies it.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62422 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
0770dabb1ae81a2a9c2e7199262067103062a0b3 17-Jan-2009 Chris Lattner <sabre@nondot.org> More simplifications to the lexer ctors.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62419 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
de96c0f29c4cacabe6ea577c61db87c2a85aea6c 17-Jan-2009 Chris Lattner <sabre@nondot.org> make the verbose raw-lexer ctor fully explicit instead of having
embedded magic.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62417 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
025c3a66402fb713c2d9bf5dc174ff264765379a 17-Jan-2009 Chris Lattner <sabre@nondot.org> add a simplified lexer ctor that sets up the lexer to raw-lex an
entire file.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62414 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
22d91ca8d7c134eac5cc6a4869e6a84c461ad624 17-Jan-2009 Chris Lattner <sabre@nondot.org> refactor some common initialization code out of the two lexer ctors.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62411 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
bc0f6bc0391ecdff331885cdc769c20b2cb628a6 11-Dec-2008 Ted Kremenek <kremenek@apple.com> PreprocessorLexer (and subclasses):
- Added virtual method 'getSourceLocation()' (no arguments) that gets the location of the next "observable" location (e.g., next character, next token).
PPLexerChange.cpp:
- Implemented FIXME by using PreprocessorLexer::getSourceLocation() to get the location in the file we are returning to after lexing a #included file. This appears to be slightly faster than having the branch (i.e., 'if(CurLexer)'). It's also not a really hot part of the Preprocessor.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60860 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
30c6476e8ef00caeb3a10ea0228165013a4eb3f3 22-Nov-2008 Chris Lattner <sabre@nondot.org> actually, this version isn't really needed.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59859 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
306fda76b0818eb7c215094123f2e16729a44aed 22-Nov-2008 Chris Lattner <sabre@nondot.org> remove a sneaky version of Diag hiding in PreprocessorLexer.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59858 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
3cbfe2c4159e0a219ae660d50625c013aa4afbd0 22-Nov-2008 Chris Lattner <sabre@nondot.org> Split the DiagnosticInfo class into two disjoint classes:
one for building up the diagnostic that is in flight (DiagnosticBuilder)
and one for pulling structured information out of the diagnostic when
formatting and presenting it.

There is no functionality change with this patch.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59849 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
3692b09faa9fe346f39bc922db6dce48cdcc3f63 18-Nov-2008 Chris Lattner <sabre@nondot.org> Convert the lexer and start converting the PP over to using canonical Diag methods.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59511 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
4b391087f9c5582d448ab66ccbc7028f7673f380 18-Nov-2008 Ted Kremenek <kremenek@apple.com> - Add Lexer::isPragma() accessor for clients of Lexer that aren't friends.
- Add static method to test if the current lexer is a non-macro/non-pragma
lexer.
- Refactor some code in PPLexerChange to use this static method.
- No performance change.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59486 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
d6a2e7d71e9a910c45a3eb0268a5d94635003d58 13-Nov-2008 Ted Kremenek <kremenek@apple.com> Move some diagnostic handling to PreprocessorLexer.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59191 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
ab91b7086cceadce2cfd6a69cc90cebcdfcea07f 12-Nov-2008 Ted Kremenek <kremenek@apple.com> Move LexIncludeFilename from Lexer to PreprocessorLexer.
PreprocessorLexer now has a virtual method "IndirectLex" which allows it to call the lex method of its subclasses. This is not for performance intensive operations.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59185 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
0e977de1eacfbc143bdedad87c14b53814159023 12-Nov-2008 Ted Kremenek <kremenek@apple.com> Move pieces of Lexer that the Preprocessor mutates to a new base class 'PreprocessorLexer'. This will also be the base class of the new Preprocessed-Token-Header (PTH) lexer. No functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59168 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
9e6293d4dfd688429f77ee3b6edba9dfd7ada3a2 12-Oct-2008 Chris Lattner <sabre@nondot.org> Change FormTokenWithChars to take the token kind to form, since all clients
were setting a kind and then forming it. This is just a minor API cleanup,
no functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57404 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
d88dc48e33d71732708960170e57a3d1bdc8f847 12-Oct-2008 Chris Lattner <sabre@nondot.org> Add a new mode to the lexer which enables it to return all characters,
even whitespace, as tokens from the file. This is enabled with
L->SetKeepWhitespaceMode(true) on a raw lexer. In this mode, you too
can use clang as a really complex version of 'cat' with code like this:

Lexer RawLex(SourceLocation::getFileLoc(SM.getMainFileID(), 0),
PP.getLangOptions(), File.first, File.second);

RawLex.SetKeepWhitespaceMode(true);

Token RawTok;
RawLex.LexFromRawLexer(RawTok);
while (RawTok.isNot(tok::eof)) {
std::cout << PP.getSpelling(RawTok);
RawLex.LexFromRawLexer(RawTok);
}

This will emit exactly the input file, with no canonicalization or other
translation. Realistic clients actually do something with the tokens of
course :)



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57401 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
fa95a019da00b926d64ff83358ba73bbc6ae1e37 12-Oct-2008 Chris Lattner <sabre@nondot.org> add a new inKeepCommentMode() accessor to abstract the KeepCommentMode
ivar.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57397 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
0af574270d3be2b0e73a3379dfaa633746f8fc6f 12-Oct-2008 Chris Lattner <sabre@nondot.org> Simplify raw mode lexing by treating an unterminate /**/ comment the
same we we do an unterminated string or character literal. This makes
it so we can guarantee that the lexer never calls into the
preprocessor (which would be suicide for a raw lexer).


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57395 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
590f0cc643274267d4d41125b62557e1d87886c3 12-Oct-2008 Chris Lattner <sabre@nondot.org> Change how raw lexers are handled: instead of creating them and then
using LexRawToken, create one and use LexFromRawLexer. This avoids
twiddling the RawLexer flag around and simplifies some code (even
speeding raw lexing up a tiny bit).

This change also improves the token paster to use a Lexer on the stack
instead of new/deleting it.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57393 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
678c6358c8d4e368c78629099142397c63c1ee35 16-Apr-2008 Chris Lattner <sabre@nondot.org> reenable highlighting of (the first line of) comments


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@49816 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
0bc735ffcfb223c0186419547abaa5c84482663e 29-Dec-2007 Chris Lattner <sabre@nondot.org> Don't attribute in file headers anymore. See llvmdev for the
discussion of this change.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@45410 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
fd1f344f874799ce8a320d81043a431b65fd1fe2 17-Oct-2007 Chris Lattner <sabre@nondot.org> Fix assertion for raw lexer.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@43091 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
9a6119437672f42be5f50c3fe89fe843b1bfa5b5 17-Oct-2007 Chris Lattner <sabre@nondot.org> Move token length calculation out of the diagnostics machinery into
the lexer, where it can be shared.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@43090 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
168ae2d44a443da75ea85db5f3b5081eb0bce113 17-Oct-2007 Chris Lattner <sabre@nondot.org> Add a new type of lexer: a raw lexer, which does not require a preprocessor
object in order to do its thing.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@43084 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
e5956bd2730c051835f9acd9e957c5d79f99e7c3 17-Oct-2007 Chris Lattner <sabre@nondot.org> add some comments.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@43079 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
53b0dabbe52219a8057659b90539837394ef0fa1 10-Oct-2007 Chris Lattner <sabre@nondot.org> Make a significant change to invert the control flow handling
predefined macros. Previously, these were handled by the driver,
now they are handled by the preprocessor.

Some fallout of this:

1. Instead of preprocessing two buffers (the predefines, then the
main source file) we now start preprocessing the main source
file and inject the predefines as a "psuedo #include" from the
main source file.
2. #1 allows us to nuke the Lexer::IsMainFile flag and simplify
Preprocessor::isInPrimaryFile.
3. The driver doesn't have to know about standard #defines, the
preprocessor knows, which is nice for people wanting to define
their own drivers.
4. This allows us to put normal tokens in the predefine buffer,
for example a definition for __builtin_va_list that is
target-specific, and a typedef for id in objc.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@42818 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
d8e3083840fef752d11ca183f42786470ed061e3 24-Jul-2007 Chris Lattner <sabre@nondot.org> Use a smallstring instead of an std::string in FileChanged to avoid some malloc traffic.

This speeds up -E on xalancbmk by 2.4%


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@40461 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
448cec4c1c3705f6f49ffdefb58a7329942a2dd8 22-Jul-2007 Chris Lattner <sabre@nondot.org> Change hte lexer to start a start pointer to the underlying
memorybuffer instead of a pointer to the memorybuffer itself. This
reduces coupling and eliminates a pointer dereference on a hot path.
This speeds up -Eonly on 483.xalancbmk by 2.1%


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@40394 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
d217773f106856a11879ec79dc468efefaf2ee75 20-Jul-2007 Chris Lattner <sabre@nondot.org> At one point there were going to be lexer and parser tokens.
Since that point is now long gone, we should rename LexerToken to
Token, as it is the only kind of token we have.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@40105 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
25bdb51276d3bfc180a68688082071505a00ed27 20-Jul-2007 Chris Lattner <sabre@nondot.org> simplify the lexer ctor to take a SLoc instead of a sloc and a redundant buffer*.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@40104 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
9dc1f530c086d2c16f8cba758b0f59a5bf41323a 20-Jul-2007 Chris Lattner <sabre@nondot.org> Reimplement SourceLocation. Instead of having a
fileid/offset pair, it now contains a bit discriminating between
mapped locations and file locations. This separates the tables for
macros and files in SourceManager, and allows better separation of
concepts in the rest of the compiler. This allows us to have *many*
macro instantiations before running out of 'addressing space'.

This is also more efficient, because testing whether something is a
macro expansion is now a bit test instead of a table lookup (which
also used to require having a srcmgr around, now it doesn't).

This is fully functional, but there are several refinements and
optimizations left.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@40103 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
97ba77cf09bf7b83b679165ce67ad7d49ffd568c 16-Jul-2007 Chris Lattner <sabre@nondot.org> Add a new Preprocessor::AdvanceToTokenCharacter method which, given a sloc
specifying the start of a token and a logical (phase 3) character number,
returns a sloc representing the input character corresponding to it.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@39905 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
f66d3e390fac1dce6b0e3ff1a3d0180638e560f6 16-Jul-2007 Chris Lattner <sabre@nondot.org> factor a common predicate into a static method.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@39903 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
9dc62f044a6ba21f503bd56607d94b32704e7945 12-Jul-2007 Chris Lattner <sabre@nondot.org> Improve portability to compilers where <cassert> is not implicitly
included. Patch contributed by Benoit Boissinot!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@39779 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h
5f016e2cb5d11daeb237544de1c5d59f20fe1a6e 11-Jul-2007 Reid Spencer <rspencer@reidspencer.com> Stage two of getting CFE top correct.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@39734 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/include/clang/Lex/Lexer.h