History log of /external/clang/lib/Lex/PPLexerChange.cpp
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
ef8225444452a1486bd721f3285301fe84643b00 21-Jul-2014 Stephen Hines <srhines@google.com> Update Clang for rebase to r212749.

This also fixes a small issue with arm_neon.h not being generated always.

Includes a cherry-pick of:
r213450 - fixes mac-specific header issue
r213126 - removes a default -Bsymbolic on Android

Change-Id: I2a790a0f5d3b2aab11de596fc3a74e7cbc99081d
/external/clang/lib/Lex/PPLexerChange.cpp
6bcf27bb9a4b5c3f79cb44c0e4654a6d7619ad89 29-May-2014 Stephen Hines <srhines@google.com> Update Clang for 3.5 rebase (r209713).

Change-Id: I8c9133b0f8f776dc915f270b60f94962e771bc83
/external/clang/lib/Lex/PPLexerChange.cpp
651f13cea278ec967336033dd032faef0e9fc2ec 24-Apr-2014 Stephen Hines <srhines@google.com> Updated to Clang 3.5a.

Change-Id: I8127eb568f674c2e72635b639a3295381fe8af82
/external/clang/lib/Lex/PPLexerChange.cpp
223e7a8b0fadbbc9eaea1fb460cd90c8d9c8dd28 13-Oct-2013 Ismail Pazarbasi <ismail.pazarbasi@gmail.com> Relax header guard mismatch warning with edit distance heuristic.

If the edit distance between the two macros is more than 50%, DefinedMacro may not be header guard or can be header guard of another header file or it might be defining something completely different set by the build environment.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@192547 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
d2f9308220af22bfc1bcd3bc2cad118dbd8be013 19-Sep-2013 Eli Friedman <eli.friedman@gmail.com> Make Preprocessor::Lex non-recursive.

Before this patch, Lex() would recurse whenever the current lexer changed (e.g.
upon entry into a macro). This patch turns the recursion into a loop: the
various lex routines now don't return a token when the current lexer changes,
and at the top level Preprocessor::Lex() now loops until it finds a token.
Normally, the recursion wouldn't end up being very deep, but the recursion depth
can explode in edge cases like a bunch of consecutive macros which expand to
nothing (like in the testcase test/Preprocessor/macro_expand_empty.c in this
patch).

<rdar://problem/14569770>

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@190980 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
671538e8a51eab5bd65a1f9f327ba7f44f84e486 12-Jun-2013 Richard Trieu <rtrieu@google.com> Introducing -Wheader-guard, a warning that checks header guards actually work
properly. This warning checks that the #ifndef and #define directives at
the beginning of a header refer to the same macro name. Includes a fix-it
hint to correct the header guard.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@183867 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
8229d22e6449851b89361bf2f41804557328be63 12-Jun-2013 Rafael Espindola <rafael.espindola@gmail.com> Include Path.h instead of PathV2.h.

I am about to move PathV2.h to Path.h.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@183795 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
58ea48dc1dd970f131207e00ae7ece159d1a08ed 20-May-2013 Douglas Gregor <dgregor@apple.com> Add -Wincomplete-module, which detects when a header is included from a module but isn't itself part of a module.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182263 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
7247c88d1e41514a41085f83ebf03dd5220e054a 15-May-2013 David Blaikie <dblaikie@gmail.com> Use only explicit bool conversion operator

The most common (non-buggy) case are where such objects are used as
return expressions in bool-returning functions or as boolean function
arguments. In those cases I've used (& added if necessary) a named
function to provide the equivalent (or sometimes negative, depending on
convenient wording) test.

DiagnosticBuilder kept its implicit conversion operator owing to the
prevalent use of it in return statements.

One bug was found in ExprConstant.cpp involving a comparison of two
PointerUnions (PointerUnion did not previously have an operator==, so
instead both operands were converted to bool & then compared). A test
is included in test/SemaCXX/constant-expression-cxx1y.cpp for the fix
(adding operator== to PointerUnion in LLVM).

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@181869 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
c9c3907e66659e4457593fdec4a27c671c12e041 04-Jan-2013 Douglas Gregor <dgregor@apple.com> Add the module name to the 'incomplete umbrella header' warning.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@171497 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
bb06b5000b9639a92e92c368ff3e5e5f8bb1e724 22-Dec-2012 Argyrios Kyrtzidis <akyrtzi@gmail.com> [libclang] Fix crash when code-completing a macro invocation that
reached EOF and did not expand the argument into the source context.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@170980 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
55fc873017f10f6f566b182b70f6fc22aefa3464 04-Dec-2012 Chandler Carruth <chandlerc@gmail.com> Sort all of Clang's files under 'lib', and fix up the broken headers
uncovered.

This required manually correcting all of the incorrect main-module
headers I could find, and running the new llvm/utils/sort_includes.py
script over the files.

I also manually added quite a few missing headers that were uncovered by
shuffling the order or moving headers up to be main-module-headers.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@169237 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
c30981a563a8947cb26b1e308d122fa2ef90fceb 30-Aug-2012 Richard Smith <richard-llvm@metafoo.co.uk> Make preprocessor act in a GCC-compatible fashion when a macro is redefined
within its own argument list. The original definition is used for the immediate
expansion, but the new definition is used for any subsequent occurences within
the argument list or after the expansion.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@162906 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
1a5f9000b690d08e65b6b8821e4f7c57d63beb1d 22-Jun-2012 James Dennett <jdennett@google.com> Documentation cleanup: escape # characters in Doxygen comments as needed.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158970 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
e55329d6834647ba0e06f8a319e5d84c77310035 16-Mar-2012 Axel Naumann <Axel.Naumann@cern.ch> From Vassil Vassilev:
Enable incremental parsing by the Preprocessor,
where more code can be provided after an EOF.
It mainly prevents the tearing down of the topmost lexer.
To be used like this:
PP.enableIncrementalProcessing();
while (getMoreSource()) {
while (Parser.ParseTopLevelDecl(ADecl)) {...}
}
PP.enableIncrementalProcessing(false);


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@152914 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
f7ccbad5d9949e7ddd1cbef43d482553b811e026 05-Feb-2012 Dylan Noblesmith <nobled@dreamwidth.org> Basic: import SmallString<> into clang namespace

(I was going to fix the TODO about DenseMap too, but
that would break self-host right now. See PR11922.)

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@149799 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
51f564f80d9f71e175635b452ffeeeff899e9bf1 31-Dec-2011 Douglas Gregor <dgregor@apple.com> Implement support for module requirements, which indicate the language
features needed for a particular module to be available. This allows
mixed-language modules, where certain headers only work under some
language variants (e.g., in C++, std.tuple might only be available in
C++11 mode).


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@147387 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
224bfeeaf046103937a5d1c8f8a9a7b79c6c14ed 23-Dec-2011 Douglas Gregor <dgregor@apple.com> Remove spurious, but now legal, typename

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@147208 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
585ec93e1bd4fdd5a37aa19848e5f92084f0d923 23-Dec-2011 Douglas Gregor <dgregor@apple.com> When building a module with an umbrella header, warn about any headers
found within that umbrella directory that were not actually included
by the umbrella header. They should either be referenced in the module
map or included by the umbrella header.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@147207 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
d83d2e73fdbe425acc4cac7b8130c464cb98333d 12-Dec-2011 Douglas Gregor <dgregor@apple.com> Don't mark include guard macros as implicitly private. This isn't
actually a terribly good heuristic, and the world is too horrible for
it to work.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@146393 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
d80d90d65895a2004b0d01c67ac417cde6cc62fd 18-Oct-2011 John McCall <rjmccall@apple.com> Argyrios says this change is required for safety under PTH.
Me, I believe him.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@142327 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
3eff3214647fe25e80dbf1dd096cc73a11d9a77a 18-Oct-2011 John McCall <rjmccall@apple.com> Fix several bugs with #pragma clang arc_cf_code_audited and macros.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@142324 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
aa93a875605536d72a10359a0098396192b7d4ec 17-Oct-2011 Douglas Gregor <dgregor@apple.com> For modules, all macros that aren't include guards are implicitly
public. Add a __private_macro__ directive to hide a macro, similar to
the __module_private__ declaration specifier.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@142188 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
c892c5fa92db137ff68d95afb13fe969d17f4fb2 11-Oct-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> For the FileChanged Preprocessor callback, when exiting a file, pass its FileID.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@141681 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
8dfac0baaf0f81d3945bcb306480e358ba8d1f08 30-Sep-2011 John McCall <rjmccall@apple.com> Add explicit attributes to mark functions as having had their
CoreFoundation object-transfer properties audited, and add a #pragma
to cause them to be automatically applied to functions in a particular
span of code. This has to be implemented largely in the preprocessor
because of the requirement that the region be entirely contained in
a single file; that's hard to impose from the parser without registering
for a ton of callbacks.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@140846 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
a64ccefdf0ea4e03ec88805d71b0af74950c7472 19-Sep-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Rename SourceLocation::getFileLocWithOffset -> getLocWithOffset.

It already works (and is useful with) macro locs as well.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@140057 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
b8db7cd9ac05c522855631670ec2e97255384f5a 08-Sep-2011 Douglas Gregor <dgregor@apple.com> Optimize the preprocessor's handling of the __import_module__
keyword. We now handle this keyword in HandleIdentifier, making a note
for ourselves when we've seen the __import_module__ keyword so that
the next lexed token can trigger a module import (if needed). This
greatly simplifies Preprocessor::Lex(), and completely erases the 5.5%
-Eonly slowdown Argiris noted when I originally implemented
__import_module__. Big thanks to Argiris for noting that horrible
regression!



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139265 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
7d100872341f233c81e1d7b72b40457e62c36862 04-Sep-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Support code-completion for C++ inline methods and ObjC buffering methods.

Previously we would cut off the source file buffer at the code-completion
point; this impeded code-completion inside C++ inline methods and,
recently, with buffering ObjC methods.

Have the code-completion inserted into the source buffer so that it can
be buffered along with a method body. When we actually hit the code-completion
point the cut-off lexing or parsing.

Fixes rdar://10056932&8319466

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139086 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
d9d2b679d0728ea7f539f38aaea38e26b8b08043 22-Aug-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Boost the efficiency of SourceManager::getMacroArgExpandedLocation.

Currently getMacroArgExpandedLocation is very inefficient and for the case
of a location pointing at the main file it will end up checking almost all of
the SLocEntries. Make it faster:

-Use a map of macro argument chunks to their expanded source location. The map
is for a single source file, it's stored in the file's ContentCache and lazily
computed, like the source lines cache.
-In SLocEntry's FileInfo add an 'unsigned NumCreatedFIDs' field that keeps track
of the number of FileIDs (files and macros) that were created during preprocessing
of that particular file SLocEntry. This is useful when computing the macro argument
map in skipping included files while scanning for macro arg FileIDs that lexed from
a specific source file. Due to padding, the new field does not increase the size
of SLocEntry.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138225 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
5b3284a9275a27f5c4410e25eb8933be540601d6 30-Jun-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Introduce a caching mechanism for macro expanded tokens.

Previously macro expanded tokens were added to Preprocessor's bump allocator and never released,
even after the TokenLexer that were lexing them was finished, thus they were wasting memory.
A very "useful" boost library was causing clang to eat 1 GB just for the expanded macro tokens.

Introduce a special cache that works like a stack; a TokenLexer can add the macro expanded tokens
in the cache, and when it finishes, the tokens are removed from the end of the cache.

Now consumed memory by expanded tokens for that library is ~ 1.5 MB.

Part of rdar://9327049.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134105 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
84021556baceb76eedf7d44be8ba71d9b8cfacce 28-Feb-2011 Peter Collingbourne <peter@pcc.me.uk> Rename tok::eom to tok::eod.

The previous name was inaccurate as this token in fact appears at
the end of every preprocessing directive, not just macro definitions.
No functionality change, except for a diagnostic tweak.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@126631 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
0827408865e32789e0ec4b8113a302ccdc531423 15-Dec-2010 Argyrios Kyrtzidis <akyrtzi@gmail.com> Fix diagnostic pragmas.

Diagnostic pragmas are broken because we don't keep track of the diagnostic state changes and we only check the current/latest state.
Problems manifest if a diagnostic is emitted for a source line that has different diagnostic state than the current state; this can affect
a lot of places, like C++ inline methods, template instantiations, the lexer, etc.

Fix the issue by having the Diagnostic object keep track of the source location of the pragmas so that it is able to know what is the diagnostic state at any given source location.

Fixes rdar://8365684.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@121873 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
e127a0d80155b45dafe77f2b4380e5fa111a3345 20-Apr-2010 Chris Lattner <sabre@nondot.org> push some source location information down through the compiler,
into ContentCache::getBuffer. This allows it to produce
diagnostics on the broken #include line instead of without a
location.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@101939 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
dbd8209b33e6c9f151e4913a9c095d64a95439c4 23-Mar-2010 Daniel Dunbar <daniel@zuster.org> PPCallbacks: Add hook for reaching the end of the main file, and fix DependencyFile to not do work in its destructor.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@99257 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
aae58b0c3efb5fa9f97a3e4b1c1a2d31077efe5b 16-Mar-2010 Douglas Gregor <dgregor@apple.com> Audit all getBuffer() callers (for both the FullSourceLoc and
SourceManager versions), updating those callers that need to recover
gracefully from failure.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98665 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
36c35ba0aca641e60e5dbee8efbc620c08b9bd61 16-Mar-2010 Douglas Gregor <dgregor@apple.com> Use SourceManager's Diagnostic object for all file-reading errors,
simplifying the SourceManager interfaces somewhat.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98598 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
aea67dbd653a2dd6dd5cc2159279e81e855b2482 15-Mar-2010 Douglas Gregor <dgregor@apple.com> Introduce a new BufferResult class to act as the return type of
SourceManager's getBuffer() (and similar) operations. This abstract
can be used to force callers to cope with errors in getBuffer(), such
as missing files and changed files. Fix a bunch of callers to use the
new interface.

Add some very basic checks for file consistency (file size,
modification time) into ContentCache::getBuffer(), although these
checks don't help much until we've updated the main callers (e.g.,
SourceManager::getSpelling()).



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98585 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
88a35862fbe473f2a4f0c19f24dbe536937e1dc6 04-Jan-2010 Douglas Gregor <dgregor@apple.com> Teach Preprocessor::macro_begin/macro_end to lazily load all macro
definitions from a precompiled header. This ensures that
code-completion with macro names behaves the same with or without
precompiled headers.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@92497 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
2ffb14f004affc363d86d6c7c63c356190db3679 06-Dec-2009 Daniel Dunbar <daniel@zuster.org> Unbreak and add test case for r90276, a situation in which getBuffer is expected to fail.

Also, update SourceManager.h doxyments for getBuffer() to reflect reality.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90701 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
63ceaa32a2371e38d1f912080fe471285e6b6e56 06-Dec-2009 Daniel Dunbar <daniel@zuster.org> Change Preprocessor::EnterSourceFile to make ErrorStr non-optional, clients should be forced to deal with error conditions.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90700 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
39d9841ed4c0568d4b44dfbc12ac04491f60a374 01-Dec-2009 Chris Lattner <sabre@nondot.org> pass the reason for failure up from MemoryBuffer and report it
in diagnostics when we fail to open a file. This allows us to
report things like:

$ clang test.c -I.
test.c:2:10: fatal error: error opening file './foo.h': Permission denied
#include "foo.h"
^



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90276 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
6e2901407bff59aeb4cc301cc58b034723d0eb49 30-Nov-2009 Chris Lattner <sabre@nondot.org> Fix PR5633 by making the preprocessor handle the case where we can
stat a file but where mmaping it fails. In this case, we emit an
error like:
t.c:1:10: fatal error: error opening file '../../foo.h'

instead of "cannot find file".



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90110 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
e7f2cbd8c093dcaebaa04c3f7b8fb1a0b4533b15 29-Nov-2009 Nuno Lopes <nunoplopes@sapo.pt> remove stall comment

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90080 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
1eb4433ac451dc16f4133a88af2d002ac26c58ef 09-Sep-2009 Mike Stump <mrs@apple.com> Remove tabs, and whitespace cleanups.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@81346 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
83d63c78810556d26b62ac4cbae2eda6cdd2570c 24-Apr-2009 Steve Naroff <snaroff@apple.com> Add PCH support for #import.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69987 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
42aa16cccb3d6cd4195ccff9c32d8cb50a25a094 18-Mar-2009 Chris Lattner <sabre@nondot.org> when preprocessing a .S file, unknown directives should just be passed through,
and the token after the # should be expanded if it is not a valid directive.
This allows us to transform things like:

#define FOO BAR
# FOO

into # BAR, even though FOO is not normally expanded for directives.

This should fix PR3833



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67236 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
7f549dfd7b1d3b2b09e5878eeb754efae253d557 13-Mar-2009 Chris Lattner <sabre@nondot.org> use accessor instead of poking ivar directly


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@66954 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
e7fb48466afcbf2c4ccdfa658824282fdc3c512c 15-Feb-2009 Chris Lattner <sabre@nondot.org> track "just a little more" location information for macro instantiations.
Now instead of just tracking the expansion history, also track the full
range of the macro that got replaced. For object-like macros, this doesn't
change anything. For _Pragma and function-like macros, this means we track
the locations of the ')'.

This is required for PR3579 because apparently GCC uses the line of the ')'
of a function-like macro as the location to expand __LINE__ to.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64601 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
67116083e70ad44799808c7472d9cd9e7e21d4d5 14-Feb-2009 Chris Lattner <sabre@nondot.org> add an assertion from Alexei Svitkine!



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64503 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
09b5c1d08a33ecf5d9c61b922fbe679867336684 06-Feb-2009 Chris Lattner <sabre@nondot.org> add interface for walking macro table.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63925 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
500d3297d2a21edeac4d46cbcbe21bc2352c2a28 29-Jan-2009 Chris Lattner <sabre@nondot.org> move library-specific diagnostic headers into library private dirs. Reduce
redundant #includes. Patch by Anders Johnsen!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63271 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
20c6b3b85e186cd52d5d99489132d71d498159eb 27-Jan-2009 Chris Lattner <sabre@nondot.org> Split the single monolithic DiagnosticKinds.def file into one
.def file for each library. This means that adding a diagnostic
to sema doesn't require all the other libraries to be rebuilt.

Patch by Anders Johnsen!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63111 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
8c61b53fdb283dade0bbf51c3c5befbc5d0aefa7 19-Jan-2009 Chris Lattner <sabre@nondot.org> do not use SourceManager::getFileCharacteristic(FileID), it is not
safe because a #line can change the file characteristic on a per-loc
basis.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62502 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
f056d92e182cbe4f62c8d14102544dc38066dabc 17-Jan-2009 Chris Lattner <sabre@nondot.org> simplify PTHManager::CreateLexer


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62424 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
88d3ac1341aa016cabd966c5b113a95ac05ea43f 17-Jan-2009 Chris Lattner <sabre@nondot.org> Change the Lexer ctor used in the non _Pragma case to take a FileID instead
of a SourceLocation. This should speed it up and definitely simplifies it.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62422 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
2b2453a7d8fe732561795431f39ceb2b2a832d84 17-Jan-2009 Chris Lattner <sabre@nondot.org> this massive patch introduces a simple new abstraction: it makes
"FileID" a concept that is now enforced by the compiler's type checker
instead of yet-another-random-unsigned floating around.

This is an important distinction from the "FileID" currently tracked by
SourceLocation. *That* FileID may refer to the start of a file or to a
chunk within it. The new FileID *only* refers to the file (and its
#include stack and eventually #line data), it cannot refer to a chunk.

FileID is a completely opaque datatype to all clients, only SourceManager
is allowed to poke and prod it.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62407 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
6aa6ac4b394c67f3f2f6f8c88c17928370c74586 23-Dec-2008 Ted Kremenek <kremenek@apple.com> Remove old PTH token-generation test harness.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61382 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
59d08cb672136322375e5400578ee1fbd0947de2 23-Dec-2008 Ted Kremenek <kremenek@apple.com> PTH: Remove some methods and simplify some conditions in PTHLexer::Lex(). No big functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61381 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
bc0f6bc0391ecdff331885cdc769c20b2cb628a6 11-Dec-2008 Ted Kremenek <kremenek@apple.com> PreprocessorLexer (and subclasses):
- Added virtual method 'getSourceLocation()' (no arguments) that gets the location of the next "observable" location (e.g., next character, next token).
PPLexerChange.cpp:
- Implemented FIXME by using PreprocessorLexer::getSourceLocation() to get the location in the file we are returning to after lexing a #included file. This appears to be slightly faster than having the branch (i.e., 'if(CurLexer)'). It's also not a really hot part of the Preprocessor.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60860 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
6137dc99ef0c2b14050631367057758b0d596cb3 02-Dec-2008 Ted Kremenek <kremenek@apple.com> Preprocessor:
- Added method "setPTHManager" that will be called by the driver to install
a PTHManager for the Preprocessor.
- Fixed some comments.
- Added EnterSourceFileWithPTH to mirror EnterSourceFileWithLexer.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60437 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
82a500b141b9a8001dac69f047478a43e2aebdff 27-Nov-2008 Ted Kremenek <kremenek@apple.com> PTHLexer now owns the Token vector.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60136 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
b93efa3d8fbc9e1209ea6a778f7aeb6a9df5e685 21-Nov-2008 Ted Kremenek <kremenek@apple.com> Reenable the default lexer.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59843 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
d5a8f0bde67505851e0fa264e1ec4de8de8f5bc2 21-Nov-2008 Ted Kremenek <kremenek@apple.com> When creating the raw tokens for PTHLexer, make sure the token representing the file to include is checked for being an identifier.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59842 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
c840f0cbd9552a28adc548ee353303c4b838f61c 21-Nov-2008 Ted Kremenek <kremenek@apple.com> When creating raw tokens for the PTHLexer specially handle angled strings for #include directives.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59840 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
81d24e135c60dcbe421864e6a4334ba99b8fe7ec 20-Nov-2008 Ted Kremenek <kremenek@apple.com> Rename IsNonPragmaNonMacroLexer to IsFileLexer.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59731 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
15ba2af0f9ad52e67e908179b8ec2eb404e11c61 20-Nov-2008 Ted Kremenek <kremenek@apple.com> Add ugly "test harness" for PTHLexer that is not enabled by default. The
(temporary hack) to test the PTHLexer is that whenever we would create a Lexer
object we instead raw lex a memory buffer first and then use the PTHLexer. This
logic exists only to driver the PTHLexer and will be removed/changed in the
future. Note that the regular path using normal Lexer objects is what is used by
default.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59723 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
a751217e769cbe73df2d579a28153f1c947aef10 20-Nov-2008 Ted Kremenek <kremenek@apple.com> Make FIXME a hard assertion.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59695 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
68e48e4a81767997ef0231e47eca4f665102c95e 20-Nov-2008 Ted Kremenek <kremenek@apple.com> Preprocessor::getCurrentFileLexer() now returns a PreprocessorLexer* instead of
a Lexer*. This means it will either return the current (normal) file Lexer or a
PTHLexer.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59694 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
1a531570d294ffc300cce1bc5e73de847a370f4f 19-Nov-2008 Ted Kremenek <kremenek@apple.com> Move more cases of using 'CurLexer' to 'CurPPLexer'.
Use PTHLexer::isNextPPTokenLParen() when using the PTHLexer.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59671 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
41938c8493b4380df738263166b746eacb33c309 19-Nov-2008 Ted Kremenek <kremenek@apple.com> - Move static function IsNonPragmaNonMacroLexer into Preprocessor.h.
- Add variants of IsNonPragmaNonMacroLexer to accept an IncludeMacroStack entry
(simplifies some uses).
- Use IsNonPragmaNonMacroLexer in Preprocessor::LookupFile.
- Add 'FileID' to PreprocessorLexer, and have Preprocessor query this fileid
when looking up the FileEntry for a file

Performance testing of -Eonly on Cocoa.h shows no performance regression because
of this patch.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59666 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
2df37b8eaecc5382e0e511a738f88918f48d9454 19-Nov-2008 Ted Kremenek <kremenek@apple.com> Revert 59574 (caused tests to fail).


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59579 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
0a3626444039396802fa35db24733c1963aebfac 19-Nov-2008 Ted Kremenek <kremenek@apple.com> - Move static function IsNonPragmaNonMacroLexer into Preprocessor.h.
- Add variants of IsNonPragmaNonMacroLexer to accept an IncludeMacroStack entry
(simplifies some uses).
- Use IsNonPragmaNonMacroLexer in Preprocessor::LookupFile.

Performance testing of -Eonly on Cocoa.h shows no performance regression because
of this patch.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59574 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
4b391087f9c5582d448ab66ccbc7028f7673f380 18-Nov-2008 Ted Kremenek <kremenek@apple.com> - Add Lexer::isPragma() accessor for clients of Lexer that aren't friends.
- Add static method to test if the current lexer is a non-macro/non-pragma
lexer.
- Refactor some code in PPLexerChange to use this static method.
- No performance change.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59486 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
9c1b750c59d510e6c9eccb1f37bccc46ccfe6844 18-Nov-2008 Ted Kremenek <kremenek@apple.com> - Add 'CurPPLexer' to Preprocessor to keep track of the current
PreprocessorLexer, which will either be a 'Lexer' or 'PTHLexer'.
- Added stub field 'CurPTHLexer' to keep track of the current PTHLexer.
- Modified IncludeStackInfo to track both the current PTHLexer and
current PreprocessorLexer.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59472 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
caaa7df2c78bbd40197823034c0275f3dcbd63e7 13-Nov-2008 Ted Kremenek <kremenek@apple.com> Using llvm::OwningPtr<> for CurLexer and CurTokenLexer. This makes both the ownership semantics of these objects explicit within the Preprocessor and also tightens up the code (explicit deletes not needed).


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59249 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
ed04c4cdca11119cac7d2fd65685444ce25f9e37 13-Nov-2008 Ted Kremenek <kremenek@apple.com> Use PushIncludeMacroStack/PopMacroStack instead of manually pushing/popping from IncludeMacroStack. This is both cleaner and makes the include stack transparently extensible.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59248 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
9d72851fec9e9c62570a027d42701562bbf29751 27-Oct-2008 Chris Lattner <sabre@nondot.org> Rename Characteristic_t to CharacteristicKind



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@58224 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
9e6293d4dfd688429f77ee3b6edba9dfd7ada3a2 12-Oct-2008 Chris Lattner <sabre@nondot.org> Change FormTokenWithChars to take the token kind to form, since all clients
were setting a kind and then forming it. This is just a minor API cleanup,
no functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57404 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
0b9e736308af5397f558ffc8e780c438c2fdb563 26-Sep-2008 Chris Lattner <sabre@nondot.org> clean up a bunch of fixme's I added, by moving
DirectoryLookup::DirType into SourceManager.h


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@56692 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
721818304ac462d8c6ce05eecd02884033db78f1 26-Sep-2008 Chris Lattner <sabre@nondot.org> Fix the rest of rdar://6243860 hopefully. This requires changing FileIDInfo
to whether the fileid is a 'extern c system header' in addition to whether it
is a system header, most of this is spreading plumbing around. Once we have that,
PPLexerChange bases its "file enter/exit" notifications to PPCallbacks to
base the system header state on FileIDInfo instead of HeaderSearch. Finally,
in Preprocessor::HandleIncludeDirective, mirror logic in GCC: the system headerness
of a file being entered can be set due to the #includer or the #includee.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@56688 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
acc5f3e42334525bf28c86471551f83dfce222d5 11-Aug-2008 Daniel Dunbar <daniel@zuster.org> More #include cleaning
- Kill unnecessary #includes in .cpp files. This is an automatic
sweep so some things removed are actually used, but happen to be
included by a previous header. I tried to get rid of the obvious
examples and this was the easiest way to trim the #includes in one
fell swoop.
- We now return to regularly scheduled development.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@54632 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
03db1b31dd926409b7defc1c90b66549464652c0 10-Aug-2008 Argyrios Kyrtzidis <akyrtzi@gmail.com> Allow the preprocessor to cache the lexed tokens, so that we can do efficient lookahead and backtracking.

1) New public methods added:
-EnableBacktrackAtThisPos
-DisableBacktrack
-Backtrack
-isBacktrackEnabled

2) LookAhead() implementation is replaced with a more efficient one.
3) LookNext() is removed.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@54611 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
63cf923e411177f8bf626961b5fc4afa1594b1bc 12-Jul-2008 Argyrios Kyrtzidis <akyrtzi@gmail.com> Convert CRLF -> LF line endings.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@53519 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
9e0ed0bd5a3a7bac73973980ff32132a7724e674 10-Jul-2008 Argyrios Kyrtzidis <akyrtzi@gmail.com> Add Preprocessor::LookNext method, which implements an efficient way to 'take a peek' at the next token without consuming it.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@53375 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
ea301e15a65240aaf92c858d7df66169e6dce33d 24-Mar-2008 Chris Lattner <sabre@nondot.org> Avoid overflowing buffer, patch by Algeris Kirtzidis!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48741 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp
bda0b626e74513950405c27525af87e214e605e2 16-Mar-2008 Chris Lattner <sabre@nondot.org> Make a major restructuring of the clang tree: introduce a top-level
lib dir and move all the libraries into it. This follows the main
llvm tree, and allows the libraries to be built in parallel. The
top level now enforces that all the libs are built before Driver,
but we don't care what order the libs are built in. This speeds
up parallel builds, particularly incremental ones.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48402 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/PPLexerChange.cpp