History log of /external/clang/lib/Lex/Lexer.cpp
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
66d5ce11b9426f6a59f61a03cbd8dbf047cc9350 17-Oct-2011 Ted Kremenek <kremenek@apple.com> Move about 20 random diagnostics under -W flags. Patch by Ahmed Charles!

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@142284 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
661a99690bc133bbaa029da925481d4a860dec90 15-Oct-2011 Richard Smith <richard-llvm@metafoo.co.uk> -Wc++98-compat warnings for the lexer.

This also adds a -Wc++98-compat-pedantic for warning on constructs which would
be diagnosed by -std=c++98 -pedantic (that is, it warns even on C++11 features
which we enable by default, with no warning, in C++98 mode).


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@142034 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
46717308883538841da7059e72bc898abe0c8724 12-Oct-2011 Douglas Gregor <dgregor@apple.com> We do parse hexfloats in C++11; make it actually work.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@141798 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
d5e1d606f8c22ebda17c6fbf952f8c1696428758 12-Oct-2011 Richard Smith <richard-llvm@metafoo.co.uk> Handle Perforce-style conflict markers like normal conflict markers. Perforce
swaps over the <<<< and >>>> markers, and uses shorter markers than traditional
tools.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@141751 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
a08529cc3f00e0b47a3c028823634129ac46847b 03-Oct-2011 Abramo Bagnara <abramo.bagnara@gmail.com> Fixed exapnsion range for # and ##.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@141012 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
a64ccefdf0ea4e03ec88805d71b0af74950c7472 19-Sep-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Rename SourceLocation::getFileLocWithOffset -> getLocWithOffset.

It already works (and is useful with) macro locs as well.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@140057 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
62ec1f2fd7368542bb926c04797fb07023547694 17-Sep-2011 Francois Pichet <pichet2000@gmail.com> Rename LangOptions::Microsoft to LangOptions::MicrosoftExt to make it clear that this flag must be used only for Microsoft extensions and not emulation; to avoid confusion with the new LangOptions::MicrosoftMode flag.

Many of the code now under LangOptions::MicrosoftExt will eventually be moved under the LangOptions::MicrosoftMode flag.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139987 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
1daa58ea8973854b62e79f0939fa37cb6fed00e6 05-Sep-2011 Benjamin Kramer <benny.kra@googlemail.com> Speed up BCPL comment lexing by looking aggressively for newlines and then scannig backwards to see if the newline is escaped.

3% speedup in preprocessing all of clang with -Eonly. Also includes a small testcase for coverage.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139116 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
5d6ae288bc661572e52ea71fc14b249eb5e2d9e9 05-Sep-2011 Benjamin Kramer <benny.kra@googlemail.com> Use the Lexer's definition of whitespace here.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139115 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
7d100872341f233c81e1d7b72b40457e62c36862 04-Sep-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Support code-completion for C++ inline methods and ObjC buffering methods.

Previously we would cut off the source file buffer at the code-completion
point; this impeded code-completion inside C++ inline methods and,
recently, with buffering ObjC methods.

Have the code-completion inserted into the source buffer so that it can
be buffered along with a method body. When we actually hit the code-completion
point the cut-off lexing or parsing.

Fixes rdar://10056932&8319466

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139086 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
c8c97a03eb0fdeb4f5fc9c4dea308ebbf46c2c93 04-Sep-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Fix Lexer::ComputePreamble when MaxLines parameter is non-zero.

The function was only counting lines that included tokens and not empty lines,
but MaxLines (mainly initiated to the line where the code-completion point resides)
is a count of overall lines (even empty ones).

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139085 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
6aa52ec6b969faabf3764baf79d89810b8249a7e 27-Aug-2011 Douglas Gregor <dgregor@apple.com> Introduce support for a simple module import declaration, which
loads the named module. The syntax itself is intentionally hideous and
will be replaced at some later point with something more
palatable. For now, we're focusing on the semantics:
- Module imports are handled first by the preprocessor (to get macro
definitions) and then the same tokens are also handled by the parser
(to get declarations). If both happen (as in normal compilation),
the second one is redundant, because we currently have no way to
hide macros or declarations when loading a module. Chris gets credit
for this mad-but-workable scheme.
- The Preprocessor now holds on to a reference to a module loader,
which is responsible for loading named modules. CompilerInstance is
the only important module loader: it now knows how to create and
wire up an AST reader on demand to actually perform the module load.
- We search for modules in the include path, using the module name
with the suffix ".pcm" (precompiled module) for the file name. This
is a temporary hack; we hope to improve the situation in the
future.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138679 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
03c107a42fae79e89d0016999a1a04c07d65591a 25-Aug-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Make Lexer::ComputePreamble accept a LangOptions parameter, otherwise it may be
out-of-sync how a file is compiled. Patch by Matthias Kleine!

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138580 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
f8c50652f7b224e66b0b6098d1fba07e036019b4 23-Aug-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> In Lexer::isAtEndOfMacroExpansion use SourceManager::isInFileID and avoid
the extra SourceManager::getFileID call.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138376 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
0e870622e4d4b2ecb7bc6ffd2c97f74fd14220b6 17-Aug-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Make Lexer::GetBeginningOfToken able to handle macro arg expansion locations.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@137795 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
2fa4e86b4fdada3b9ecbbbd99965b83ed879f69b 11-Aug-2011 Craig Topper <craig.topper@gmail.com> Add support for C++0x raw string literals.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@137298 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
aca25bccefe56121b686706afc84c8cb5d46e65b 27-Jul-2011 Anna Zaks <ganna@apple.com> Add a utility function to the Lexer, which makes it easier to find a token after the given location. (It is a generalized version of trans::findLocationAfterSemi from ArcMigrate, which will be changed to use the Lexer utility).

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136268 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
5cee1195584fa8672253139c86e922daeda69b9e 27-Jul-2011 Douglas Gregor <dgregor@apple.com> Add support for C++0x unicode string and character literals, from Craig Topper!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136210 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
1728762d5a8cfaf8d64385f47b311e84de1ae7a2 26-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Migrate 'Instantiation' data and API bits of SLocEntry to 'Expansion'
etc. With this I think essentially all of the SourceManager APIs are
converted. Comments and random other bits of cleanup should be all thats
left.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136057 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
78df836808aee22c3157e1bc23bc4ec569b80568 26-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Convert InstantiationInfo and much of the related code to ExpansionInfo
and various other 'expansion' based terms. I've tried to reformat where
appropriate and catch as many references in comments but I'm going to do
several more passes. Also I've tried to expand parameter names to be
more clear where appropriate.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136056 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
bf340e452339e374ea6eef78c1f0a2abdd16c5a3 26-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Rename create(MacroArg)InstantiationLoc to create(MacroArg)ExpansionLoc.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136054 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
999f739404edf2078cf9f9c28b4dc45c19765842 25-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Rename SourceManager::getImmediateInstantiationRange to
getImmediateExpansionRange.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135960 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
edc3dccece244a584f8ebdb81da6c962c08e79be 25-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Rename SourceManager::getInstantiationRange to getExpansionRange.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135915 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
402785357ab053dd53f4fdd858b9630a5e0f8bad 25-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Mechanically rename SourceManager::getInstantiationLoc and
FullSourceLoc::getInstantiationLoc to ...::getExpansionLoc. This is part
of the API and documentation update from 'instantiation' as the term for
macros to 'expansion'.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135914 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
5f9e272e632e951b1efe824cd16acb4d96077930 23-Jul-2011 Chris Lattner <sabre@nondot.org> remove unneeded llvm:: namespace qualifiers on some core types now that LLVM.h imports
them into the clang namespace.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135852 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
19207f1e5f51261a33492602501fb7ada50ea546 20-Jul-2011 Joerg Sonnenberger <joerg@bec.de> Spelling


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135545 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
f62d43d2afe1960755a1b5813cae1e5983bcac1b 19-Jul-2011 Douglas Gregor <dgregor@apple.com> Revamp the SourceManager to separate the representation of parsed
source locations from source locations loaded from an AST/PCH file.

Previously, loading an AST/PCH file involved carefully pre-allocating
space at the beginning of the source manager for the source locations
and FileIDs that correspond to the prefix, and then appending the
source locations/FileIDs used for parsing the remaining translation
unit. This design forced us into loading PCH files early, as a prefix,
whic has become a rather significant limitation.

This patch splits the SourceManager space into two parts: for source
location "addresses", the lower values (growing upward) are used to
describe parsed code, while upper values (growing downward) are used
for source locations loaded from AST/PCH files. Similarly, positive
FileIDs are used to describe parsed code while negative FileIDs are
used to file/macro locations loaded from AST/PCH files. As a result,
we can load PCH/AST files even during parsing, making various
improvemnts in the future possible, e.g., teaching #include <foo.h> to
look for and load <foo.h.gch> if it happens to be already available.

This patch was originally written by Sebastian Redl, then brought
forward to the modern age by Jonathan Turner, and finally
polished/finished by me to be committed.




git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135484 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
433db06b614f26dc6829e86d6ff469e2cca7d4f9 14-Jul-2011 Chandler Carruth <chandlerc@gmail.com> Convert terminology in the Lexer from 'instantiate' and variants to
'expand'. Also update the public API it provides to the new term, and
propagate that update to the various clients.

No functionality changed.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135138 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
7a759606d93975866051f67104ae58446e55f404 07-Jul-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Move SourceManager::isAt[Start/End]OfMacroInstantiation functions to the Lexer, since they depend on it now.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134644 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
b73377eeb3eff76be134203aebb6068244b177f3 07-Jul-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Make the Preprocessor more memory efficient and improve macro instantiation diagnostics.

When a macro instantiation occurs, reserve a SLocEntry chunk with length the
full length of the macro definition source. Set the spelling location of this chunk
to point to the start of the macro definition and any tokens that are lexed directly
from the macro definition will get a location from this chunk with the appropriate offset.

For any tokens that come from argument expansion, '##' paste operator, etc. have their
instantiation location point at the appropriate place in the instantiated macro definition
(the argument identifier and the '##' token respectively).
This improves macro instantiation diagnostics:

Before:

t.c:5:9: error: invalid operands to binary expression ('struct S' and 'int')
int y = M(/);
^~~~
t.c:5:11: note: instantiated from:
int y = M(/);
^

After:

t.c:5:9: error: invalid operands to binary expression ('struct S' and 'int')
int y = M(/);
^~~~
t.c:3:20: note: instantiated from:
\#define M(op) (foo op 3);
~~~ ^ ~
t.c:5:11: note: instantiated from:
int y = M(/);
^

The memory savings for a candidate boost library that abuses the preprocessor are:

- 32% less SLocEntries (37M -> 25M)
- 30% reduction in PCH file size (900M -> 635M)
- 50% reduction in memory usage for the SLocEntry table (1.6G -> 800M)

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134587 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
7ddf6b2d77ac4cb27f78d817d7884e6ce17afd0c 24-Jun-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Allow Lexer::getLocForEndOfToken to return the location just passed the macro instantiation
if the location given points at the last token of the macro instantiation.

Fixes rdar://9045701.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@133804 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
969f9d47338fc36ebb6d24ad3a51e45eda07fd58 10-May-2011 Eli Friedman <eli.friedman@gmail.com> Don't strlen() every file before parsing it.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@131132 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
fc8f0e14ad142ed811e90fbd9a30e419e301c717 15-Apr-2011 Chris Lattner <sabre@nondot.org> fix a bunch of comment typos found by codespell. Patch by
Luis Felipe Strano Moraes!



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@129559 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
87a1e19735a27d84edc2ac1331c040e2fb4c3b1a 14-Apr-2011 Richard Smith <richard-llvm@metafoo.co.uk> Implement C++0x [lex.pptoken]p3's handling of <::.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@129525 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
156119df1d076b63609618976281961283f871db 09-Apr-2011 Eric Christopher <echristo@apple.com> Eat the UTF-8 BOM at the beginning of a file since it's ignored anyhow.

Nom Nom Nom.

Patch by Anton Korobeynikov!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@129174 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
77ebb3829f2abb9be2d40927b793b72347626aad 06-Apr-2011 John McCall <rjmccall@apple.com> Fix getLocForEndOfToken to not double-count spurious internal characters
within a token, like trigraphs and escaped newlines.
Patch by Marcin Kowalczyk!



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@128978 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
2ed42287b9b88a0bb08ab10e95e306da3f294ba9 18-Mar-2011 Daniel Dunbar <daniel@zuster.org> Lexer: Add extremely limited support for -traditional-cpp, ignoring BCPL
comments.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@127910 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
834e3f6c77d9ac03997a3f0c56934edcf406a355 08-Mar-2011 John McCall <rjmccall@apple.com> Fix my earlier commit to work with escaped newlines and leave breadcrumbs
in case we want to make a world where we can check intermediate instantiations
for this kind of breadcrumb.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@127221 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
84021556baceb76eedf7d44be8ba71d9b8cfacce 28-Feb-2011 Peter Collingbourne <peter@pcc.me.uk> Rename tok::eom to tok::eod.

The previous name was inaccurate as this token in fact appears at
the end of every preprocessing directive, not just macro definitions.
No functionality change, except for a diagnostic tweak.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@126631 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
ff1ed985ede24b9a45e52d726db95767e35fe21e 16-Feb-2011 Argyrios Kyrtzidis <akyrtzi@gmail.com> Warn for missing terminating " or ' instead of error for gcc compatibility. Fixed rdar://8914293.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@125616 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
1b791d6465d42a9763927be1dd8af229efcbbf5e 09-Feb-2011 Peter Collingbourne <peter@pcc.me.uk> Lexer: add CUDA kernel call tokens

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@125218 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
3de84241d90f3dd280126fdf2c4651667151c967 31-Jan-2011 Douglas Gregor <dgregor@apple.com> Harden Lexer::GetBeginningOfToken() against bogus source locations and
the disappearance/alteration of files.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@124616 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
c4bf2b9afb7d47445a9dc6bc848657098a4e3851 22-Dec-2010 Abramo Bagnara <abramo.bagnara@gmail.com> Introduced raw_identifier token kind.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@122394 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
b0607279cb98bbf2bbfe0db170aed39ef91e86a2 17-Nov-2010 Chris Lattner <sabre@nondot.org> move getSpelling from Preprocessor to Lexer, which it is more conceptually related to.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@119479 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
7ef5c27eb6e8ebe58b52013246c06753c3613263 17-Nov-2010 Chris Lattner <sabre@nondot.org> move AdvanceToTokenCharacter and getLocForEndOfToken from
Preprocessor to Lexer where they make more sense.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@119474 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
14bd96571ef6f0e97dc79ec4d01b547d60e8fa68 23-Oct-2010 Chandler Carruth <chandlerc@gmail.com> Update remaining attribute macros to new style.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@117204 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
c3526d89ef9c31639ec8b25180cfb22354344241 30-Sep-2010 Sebastian Redl <sebastian.redl@getdesigned.at> In MeasureTokenLength, the FileLoc supplied to the lexer must point to the start of the buffer, or we risk overflow.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@115117 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
6ab55ebab20086f725c4017b48ef8d7691ef870f 31-Aug-2010 Chris Lattner <sabre@nondot.org> improve isHexaLiteral to work with escaped newlines and trigraphs,
patch by Francois Pichet!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112602 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
2d804c4325fcf3893386e16970b82fd0f9af1d7c 31-Aug-2010 Chris Lattner <sabre@nondot.org> silence a warning


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112549 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
6cf750298d3621d8a10a6dd07fcee8e274b9d94d 30-Aug-2010 Sean Hunt <scshunt@csclub.uwaterloo.ca> Revert my user-defined literal commits - r1124{58,60,67} pending
some issues being sorted out.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112493 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
4a551000bee716ac8b1bbe16134a53f0ad221a5a 30-Aug-2010 Chris Lattner <sabre@nondot.org> add a fixme.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112491 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
b2f4a20ddc281b194caa00e850ad74759e0d50ba 30-Aug-2010 Chris Lattner <sabre@nondot.org> use 'features' instead of 'PP->getLangOptions'.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112490 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
a75ec43d625753b4439b0d6f70bd988444c74617 30-Aug-2010 Douglas Gregor <dgregor@apple.com> In Microsoft compatibility mode, don't parse the exponent as part of
the pp-number in a hexadecimal floating point literal, from Francois
Pichet! Fixes PR7968.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112481 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
0016d519b831859526b79405cdae4c64c73731c8 29-Aug-2010 Sean Hunt <scshunt@csclub.uwaterloo.ca> Implement C++0x user-defined string literals.

The extra data stored on user-defined literal Tokens is stored in extra
allocated memory, which is managed by the PreprocessorLexer because there isn't
a better place to put it that makes sure it gets deallocated, but only after
it's used up. My testing has shown no significant slowdown as a result, but
independent testing would be appreciated.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112458 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
55817afdf9d453a443262a733f6caf6692dca118 25-Aug-2010 Douglas Gregor <dgregor@apple.com> Introduce a preprocessor code-completion hook for contexts where we
expect "natural" language and should not provide any completions,
e.g., comments, string literals, #error.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112054 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
f44e854ed1e3aa86d2ed6d615ccd109d50ddcff9 24-Aug-2010 Douglas Gregor <dgregor@apple.com> Introduce basic code-completion support for preprocessor directives,
e.g., after a "#" we'll suggest #if, #ifdef, etc.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@111943 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
2d474ba9e8ae43a1a5a9f72718c0d79092b9453f 12-Aug-2010 Douglas Gregor <dgregor@apple.com> Don't emit end-of-file diagnostics like "unterminated conditional" or
"unterminated string" when we're performing code completion.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@110933 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
476d8b863cb65b2b5833235d97315cdb46e6f5aa 11-Aug-2010 Benjamin Kramer <benny.kra@googlemail.com> Random temporary string cleanup.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@110807 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
df95a13ec73d2cdaea79555cb412d767f4963120 09-Aug-2010 Douglas Gregor <dgregor@apple.com> Use precompiled preambles for in-process code completion.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@110596 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
f4f6c9db68465b886ec2e596feaa6ecc782395a4 26-Jul-2010 Douglas Gregor <dgregor@apple.com> Introduce basic support for loading a precompiled preamble while
reparsing an ASTUnit. When saving a preamble, create a buffer larger
than the actual file we're working with but fill everything from the
end of the preamble to the end of the file with spaces (so the lexer
will quickly skip them). When we load the file, create a buffer of the
same size, filling it with the file and then spaces. Then, instruct
the lexer to start lexing after the preamble, therefore continuing the
parse from the spot where the preamble left off.

It's now possible to perform a simple preamble build + parse (+
reparse) with ASTUnit. However, one has to disable a bunch of checking
in the PCH reader to do so. That part isn't committed; it will likely
be handled with some other kind of flag (e.g., -fno-validate-pch).

As part of this, fix some issues with null termination of the memory
buffers created for the preamble; we were trying to explicitly
NULL-terminate them, even though they were also getting implicitly
NULL terminated, leading to excess warnings about NULL characters in
source files.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@109445 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
a8e5c5bdbe387b2552c1c23b828f54abcf085a40 22-Jul-2010 Douglas Gregor <dgregor@apple.com> Improve performance during cursor traversal when a region of interest
is present.

Rather than using clang_getCursorExtent(), which requires
us to lex the token at the ending position to determine its
length. Then, we'd be comparing [a, b) source ranges that cover the
characters in the range rather than the normal behavior for Clang's
source ranges, which covers the tokens in the range. However, relexing
causes us to read the source file (which may come from a precompiled
header), which is rather unfortunate and affects performance.

In the new scheme, we only use Clang-style source ranges that cover
the tokens in the range. At the entry points where this matters
(clang_annotateTokens, clang_getCursor), we make sure to move source
locations to the start of the token.

Addresses most of <rdar://problem/8049381>.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@109134 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
f033f1da4a34f8df6e95e9929dc04ff54bb8fb01 20-Jul-2010 Douglas Gregor <dgregor@apple.com> Introduce a new lexer function to compute the "preamble" of a file,
which is the part of the file that contains all of the initial
comments, includes, and preprocessor directives that occur before any
of the actual code. Added a new -print-preamble cc1 action that is
only used for testing.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@108913 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
d80f786689d608e5c22d6e1045884de7aff76c40 08-Jul-2010 Chris Lattner <sabre@nondot.org> fix PR4499, patch by Kyle Dean!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@107836 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
571339c215dc3fa44acccb89c7c6b1d5b962b7f2 31-May-2010 Chris Lattner <sabre@nondot.org> simpler fix for rdar://8044135 - escaped newlines have already
been processed, so they don't have to be tip-toed around.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@105182 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
33611e0d5ab1372608a7649b1877cd4300621c71 31-May-2010 Douglas Gregor <dgregor@apple.com> Improve our handling of NULL after an escaping '\' in a string
literal. Fixes <rdar://problem/8044135>.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@105181 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
dc8453422bec3bbf70c03920e01498d75783d122 25-May-2010 Douglas Gregor <dgregor@apple.com> Improve code completion in failure cases in two ways:
1) Suppress diagnostics as soon as we form the code-completion
token, so we don't get any error/warning spew from the early
end-of-file.
2) If we consume a code-completion token when we weren't expecting
one, go into a code-completion recovery path that produces the best
results it can based on the context that the parser is in.





git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@104585 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
3d488990e13835cb35538b2f0cd53dbda92e36be 17-May-2010 Chris Lattner <sabre@nondot.org> robustify the conflict marker stuff. Don't add 7 twice, which would
make it miss (invalid) things like:
<<<<<<<
>>>>>>>

and crash if

<<<<<<<

was at the end of the line. When we find a >>>>>>> that is not at the
end of the line, make sure to reset Pos so we don't crash on something
like:
<<<<<<< >>>>>>>

This isn't worth making testcases for, since each would require a new file.

rdar://7987078 - signal 11 compiling "<<<<<<<<<<"


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@103968 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
150fcd5dcb79ad0a9cc6824c5589000e3669c7a4 16-May-2010 Chris Lattner <sabre@nondot.org> when code completing inside a C-style block comment, don't emit errors about
a missing */ since we truncated the file.

This fixes rdar://7948776


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@103913 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
515f43f9f23de50d155b481b8774ec40bdfd7ff2 13-Apr-2010 Chris Lattner <sabre@nondot.org> fix a minor bug I noticed while work with Jordy's patch for PR6101,
in an input file like this:

# 42
int x;

we were emitting:

# <something>
int x;

(with a space before the int) because we weren't clearing the leading
whitespace flag properly after the \n from the directive was handled.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@101084 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
849b243d4065f56742a4677d6dc8277609a151f8 31-Mar-2010 Douglas Gregor <dgregor@apple.com> Reinstate my CodeModificationHint -> FixItHint renaming patch, without
the C-only "optimization".


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@100022 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
275313cbb0847f1f117f60d144d113804d4fa42d 31-Mar-2010 Douglas Gregor <dgregor@apple.com> Revert r100008, which inexplicably breaks the clang-i686-darwin10 builder

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@100018 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
d0ebe080eee7c37e73754068b47fd90cc506e128 31-Mar-2010 Douglas Gregor <dgregor@apple.com> Rename CodeModificationHint to FixItHint, since we've been using the
term "fix-it" everywhere and even *I* get tired of long names
sometimes. No functionality change.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@100008 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
e5877b08a688daec2ff19648922a9cb3b61d4425 16-Mar-2010 Douglas Gregor <dgregor@apple.com> Remove unused variable

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98691 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
453091cc2082e207ea2c2dda645a9bc01b37fb0c 16-Mar-2010 Douglas Gregor <dgregor@apple.com> Audit all Preprocessor::getSpelling() callers, improving failure
recovery for those that need it.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98689 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
a543016fe07030f695d6d56fd22c8c8da617e0d7 16-Mar-2010 Douglas Gregor <dgregor@apple.com> Audit all callers of SourceManager::getCharacterData(); update some of
them to recover more gracefully on failure.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98672 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
f6ac97b101c8840efa92bf29166077ce4049e293 16-Mar-2010 Benjamin Kramer <benny.kra@googlemail.com> Let SourceManager::getBufferData return StringRef instead of a pair of two const char*.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98630 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
f715ca12bfc9fddfde75f98a197424434428b821 16-Mar-2010 Douglas Gregor <dgregor@apple.com> Give SourceManager a Diagnostic object with which to report errors,
and start simplifying the interfaces in SourceManager that can fail.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98594 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
aea67dbd653a2dd6dd5cc2159279e81e855b2482 15-Mar-2010 Douglas Gregor <dgregor@apple.com> Introduce a new BufferResult class to act as the return type of
SourceManager's getBuffer() (and similar) operations. This abstract
can be used to force callers to cope with errors in getBuffer(), such
as missing files and changed files. Fix a bunch of callers to use the
new interface.

Add some very basic checks for file consistency (file size,
modification time) into ContentCache::getBuffer(), although these
checks don't help much until we've updated the main callers (e.g.,
SourceManager::getSpelling()).



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98585 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
3d0ad58b28e0d50fca7f21c6a078b05370510288 03-Feb-2010 Chris Lattner <sabre@nondot.org> don't inform comment handlers about comments in #if 0 blocks,
doing so invalidates the file guard optimization and is not
in the spirit of "#if 0" because it is supposed to completely
skip everything, even if it isn't lexically valid. Patch by
Abramo Bagnara!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@95253 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
33e9abd21083a0191a7676a04b497006d2da184d 22-Jan-2010 Douglas Gregor <dgregor@apple.com> Teach CIndex's cursor visitor to restrict its traversal to a specific
region of interest (if provided). Implement clang_getCursor() in terms
of this traversal rather than using the Index library; the unified
cursor visitor is more complete, and will be The Way Forward.

Minor other tweaks needed to make this work:
- Extend Preprocessor::getLocForEndOfToken() to accept an offset
from the end, making it easy to move to the last character in the
token (rather than just past the end of the token).
- In Lexer::MeasureTokenLength(), the length of whitespace is zero.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@94200 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
046c2277dcbcc8eb89dbb5b1b8c5226b7cb81635 18-Jan-2010 Chris Lattner <sabre@nondot.org> allow the HandlerComment callback to push tokens into the
preprocessor. This could be used by an OpenMP implementation
or something. Patch by Abramo Bagnara!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@93795 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
cd991dbb12f24368753cef086c7ad3ec203c9ea6 11-Jan-2010 Chris Lattner <sabre@nondot.org> add a TODO for a perf improvement in LexIdentifier.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@93141 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
8c723404c34e9347fae473489194e8c200c4ac83 11-Jan-2010 Sean Hunt <rideau3@gmail.com> Do not parse hexadecimal floating point literals in C++0x mode because they are
incompatible with user-defined literals, specifically with the following form:

0x1p+1

The preprocessing-number token extends only as far as the 'p'; the '+' is not
included. Previously we could get away with this extension as p was an invalid
suffix, but now with user-defined literals, 'p' might well be a valid suffix
and we are forced to consider it as such.

This patch also adds a warning in non-0x C++ modes telling the user that
this extension is incompatible with C++0x that is enabled by default
(previously and with other languages, we warn only with a compliance
option such as -pedantic).


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@93135 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
a2bf105739cef7aea8e4d219629c627fcd11742e 17-Dec-2009 Chris Lattner <sabre@nondot.org> reimplement r90860, fixing a couple of problems:
1. Don't make a copy of LangOptions every time a lexer is created.
2. Don't make CharInfo global mutable state.
3. Fix the implementation to properly treat ^Z as EOF instead of as
horizontal whitespace, which matches the semantic implemented by VC++.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@91586 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
34f349da38a7bd99103e12d8ea6c73bc8d025193 14-Dec-2009 Chris Lattner <sabre@nondot.org> teach clang to recover gracefully from conflict markers left in source
files: PR5238.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@91270 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
7b682656310b0d7e94a98d982444bc7ddedc653f 08-Dec-2009 Steve Naroff <snaroff@apple.com> Integrate the following from the 'objective-rewrite' branch:

http://llvm.org/viewvc/llvm-project?view=rev&revision=80043



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90860 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
2968442603b029949246467253eeac8139a5b6d8 02-Dec-2009 Douglas Gregor <dgregor@apple.com> Extend the source manager with the ability to override the contents of
files with the contents of an arbitrary memory buffer. Use this new
functionality to drastically clean up the way in which we handle file
truncation for code-completion: all of the truncation/completion logic
is now encapsulated in the preprocessor where it belongs
(<rdar://problem/7434737>).



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90300 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
6e2901407bff59aeb4cc301cc58b034723d0eb49 30-Nov-2009 Chris Lattner <sabre@nondot.org> Fix PR5633 by making the preprocessor handle the case where we can
stat a file but where mmaping it fails. In this case, we emit an
error like:
t.c:1:10: fatal error: error opening file '../../foo.h'

instead of "cannot find file".



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90110 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
c997eb4ac1b332406b1f16d4eddce7988d68c56b 14-Nov-2009 Benjamin Kramer <benny.kra@googlemail.com> Move DISABLE_INLINE to the front of the decl so MSVC can parse it. Patch by Amine Khaldi!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@88797 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
39de7409bffb6b725a8aa64f0ba77ab51e8c9eb3 14-Oct-2009 Chris Lattner <sabre@nondot.org> Teach Lexer::MeasureTokenLength to be able to measure the
length of comment tokens. Patch by Abramo Bagnara!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@84100 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
b657f115c3b4e262e72906a28cbcf3eaccd9460c 22-Sep-2009 Douglas Gregor <dgregor@apple.com> Replace the -code-completion-dump option with

-code-completion-at=filename:line:column

which performs code completion at the specified location by truncating
the file at that position and enabling code completion. This approach
makes it possible to run multiple tests from a single test file, and
gives a more natural command-line interface.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@82571 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
86d9a52c24d390631a888d4ff812e1b15445e0a0 21-Sep-2009 Douglas Gregor <dgregor@apple.com> Refactor and simplify the CodeCompleteConsumer, so that all of the
real work is performed within Sema. Addresses Chris's comments, but
still retains the heavyweight list-of-multimaps data structure.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@82459 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
81b747b7fcc91c2fba9a3183d8fac80adbfc1d3e 17-Sep-2009 Douglas Gregor <dgregor@apple.com> Initial implementation of a code-completion interface in Clang. In
essence, code completion is triggered by a magic "code completion"
token produced by the lexer [*], which the parser recognizes at
certain points in the grammar. The parser then calls into the Action
object with the appropriate CodeCompletionXXX action.

Sema implements the CodeCompletionXXX callbacks by performing minimal
translation, then forwarding them to a CodeCompletionConsumer
subclass, which uses the results of semantic analysis to provide
code-completion results. At present, only a single, "printing" code
completion consumer is available, for regression testing and
debugging. However, the design is meant to permit other
code-completion consumers.

This initial commit contains two code-completion actions: one for
member access, e.g., "x." or "p->", and one for
nested-name-specifiers, e.g., "std::". More code-completion actions
will follow, along with improved gathering of code-completion results
for the various contexts.

[*] In the current -code-completion-dump testing/debugging mode, the
file is truncated at the completion point and EOF is translated into
"code completion".


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@82166 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
1eb4433ac451dc16f4133a88af2d002ac26c58ef 09-Sep-2009 Mike Stump <mrs@apple.com> Remove tabs, and whitespace cleanups.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@81346 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
03b986660cc9ae7a08588f99913f2d91e359a65f 07-Jul-2009 Chris Lattner <sabre@nondot.org> Convert the CharInfo table to be statically initialized, instead of dynamically initialized. Patch by Ryan Flynn!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@74919 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
efb173ddd95325c7fd3c15070392b27c07a49a85 07-Jul-2009 Chris Lattner <sabre@nondot.org> fix an out-of-date comment.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@74894 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
2e22253e03e175144aeb9d13350a12fd83f858be 02-Jul-2009 Douglas Gregor <dgregor@apple.com> Add support for retrieving the Doxygen comment associated with a given
declaration in the AST.

The new ASTContext::getCommentForDecl function searches for a comment
that is attached to the given declaration, and returns that comment,
which may be composed of several comment blocks.

Comments are always available in an AST. However, to avoid harming
performance, we don't actually parse the comments. Rather, we keep the
source ranges of all of the comments within a large, sorted vector,
then lazily extract comments via a binary search in that vector only
when needed (which never occurs in a "normal" compile).

Comments are written to a precompiled header/AST file as a blob of
source ranges. That blob is only lazily loaded when one requests a
comment for a declaration (this never occurs in a "normal" compile).

The indexer testbed now supports comment extraction. When the
-point-at location points to a declaration with a Doxygen-style
comment, the indexer testbed prints the associated comment
block(s). See test/Index/comments.c for an example.

Some notes:
- We don't actually attempt to parse the comment blocks themselves,
beyond identifying them as Doxygen comment blocks to associate them
with a declaration.
- We won't find comment blocks that aren't adjacent to the
declaration, because we start our search based on the location of
the declaration.
- We don't go through the necessary hops to find, for example,
whether some redeclaration of a declaration has comments when our
current declaration does not. Similarly, we don't attempt to
associate a \param Foo marker in a function body comment with the
parameter named Foo (although that is certainly possible).
- Verification of my "no performance impact" claims is still "to be
done".



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@74704 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
5636a3b6ece2c1f413464b72545e08eb0b7f06e4 23-Jun-2009 Chris Lattner <sabre@nondot.org> Fix our check for "random whitespace between a \ and newline" to work
with dos style newlines. I have a trivial test for this:

// RUN: clang-cc %s -verify
#define test(x, y) \
x ## y

but I don't know how to get svn to not change newlines and testrunner
doesn't work with dos style newlines either, so "not worth it". :)

rdar://6994000



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@73945 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
766703b76dc1eb0c49d11cafc2a17c7407876ebc 13-May-2009 Chris Lattner <sabre@nondot.org> Fix rdar://6880630 - # in _Pragma does not start a preprocessor directive.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@71643 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
f01fdff97b245caac98100d232c760b4d0531411 28-Apr-2009 Eli Friedman <eli.friedman@gmail.com> Get rid of some useless uses of NoExtensions. The philosophy here is
that if we're going to print an extension warning anyway,
there's no point to changing behavior based on NoExtensions: it will
only make error recovery worse.

Note that this doesn't cause any behavior change because NoExtensions
isn't used by the current front-end. I'm still considering what to do about
the remaining use of NoExtensions in IdentifierTable.cpp.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@70273 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
a864cf7c1d774a0f790bfc46befc87d9dbf1f65c 24-Apr-2009 Chris Lattner <sabre@nondot.org> fix rdar://6816766 - Crash with function-like macro test at end of directive.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69964 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
033749571f8d4c804eeb357c70b06424aa24503b 19-Apr-2009 Chris Lattner <sabre@nondot.org> add a new Lexer::SkipEscapedNewLines method.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69483 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
24f0e48c0aa62f2268e061aad70f9b19a59e7b52 19-Apr-2009 Chris Lattner <sabre@nondot.org> factor escape newline measuring out into its own helper function.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69482 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
0edfab68d463201ac2933cf88243701dbc3ba5ab 18-Apr-2009 Chris Lattner <sabre@nondot.org> remove unneeded scopes.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69481 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
9cb51ce03caf0755759f7e67aea7d4be684ff738 18-Apr-2009 Chris Lattner <sabre@nondot.org> Fix two problems from PR3916, and one problem I noticed while hacking
on the code.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69404 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
2c78b873f4f3823ae859c15674cb3d76c8554113 15-Apr-2009 Chris Lattner <sabre@nondot.org> Change Lexer::MeasureTokenLength to take a LangOptions reference.
This allows it to accurately measure tokens, so that we get:

t.cpp:8:13: error: unknown type name 'X'
static foo::X P;
~~~~~^

instead of the woefully inferior:

t.cpp:8:13: error: unknown type name 'X'
static foo::X P;
~~~~ ^

Most of this is just plumbing to push the reference around.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69099 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
ead616c5d8681a41b11273327813e61bda01907a 05-Apr-2009 Chris Lattner <sabre@nondot.org> fix rdar://6757323, where an escaped newline in a // comment
was causing the char after the newline to get eaten.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@68430 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
20d0ee5d23c0f0e879ce1711d11df3118b3fb033 02-Apr-2009 Mike Stump <mrs@apple.com> A code modification hint for files that don't end in a newline.
Eventually, would be nice to be able to run these modifications even
when we don't want the warning or errors for the actual diagnostic.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@68272 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
33ab3f6c7e3065550de202088f8f335ecfa16ae1 18-Mar-2009 Chris Lattner <sabre@nondot.org> silence some errors that should not apply to .S files on code like:

''
'
'



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67237 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
e91e93225db2e66906878513c6ef4dd6a7ee2b6a 18-Mar-2009 Chris Lattner <sabre@nondot.org> properly form a full token for # before calling HandleDirective.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67235 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
1fa495304c81e03f07f278a47b5efe9317104aab 08-Mar-2009 Chris Lattner <sabre@nondot.org> simplify some logic by making ScratchBuffer handle the application of trailing
\0's to created tokens instead of making all clients do it. No functionality
change.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@66373 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
b66158c299786042cd6277b163a2c20fd3d11da5 19-Feb-2009 Chris Lattner <sabre@nondot.org> fix PR3609, emit:

t.c:1:10: error: missing terminating '>' character
#include <stdio.h
^

instead of:

t.c:1:10: error: missing terminating " character
#include <stdio.h
^



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@65052 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
e7fb48466afcbf2c4ccdfa658824282fdc3c512c 15-Feb-2009 Chris Lattner <sabre@nondot.org> track "just a little more" location information for macro instantiations.
Now instead of just tracking the expansion history, also track the full
range of the macro that got replaced. For object-like macros, this doesn't
change anything. For _Pragma and function-like macros, this means we track
the locations of the ')'.

This is required for PR3579 because apparently GCC uses the line of the ')'
of a function-like macro as the location to expand __LINE__ to.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64601 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
500d3297d2a21edeac4d46cbcbe21bc2352c2a28 29-Jan-2009 Chris Lattner <sabre@nondot.org> move library-specific diagnostic headers into library private dirs. Reduce
redundant #includes. Patch by Anders Johnsen!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63271 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
20c6b3b85e186cd52d5d99489132d71d498159eb 27-Jan-2009 Chris Lattner <sabre@nondot.org> Split the single monolithic DiagnosticKinds.def file into one
.def file for each library. This means that adding a diagnostic
to sema doesn't require all the other libraries to be rebuilt.

Patch by Anders Johnsen!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63111 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
363fdc29656cc03c1817268888f95e6343470aa8 26-Jan-2009 Chris Lattner <sabre@nondot.org> Fix a bug I introduced in my changes, which caused MeasureTokenLength
to crash when given an instantiation location. Thanks to Fariborz for
the testcase.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63057 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
071831dcc7fb502d8957535366baa6ac7f72e70d 26-Jan-2009 Chris Lattner <sabre@nondot.org> allow _Pragmas formed from #defines to keep their full instantiation
history



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63035 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
47246be8ac5b0ddde6c402b8fc6946b6135487b5 26-Jan-2009 Chris Lattner <sabre@nondot.org> This change refactors some of the low-level lexer interfaces a bit.
Token now has a class of kinds for "literals", which include
numeric constants, strings, etc. These tokens can optionally have
a pointer to the start of the token in the lexer buffer. This
makes it faster to get spelling and do other gymnastics, because we
don't have to go through source locations.

This change is performance neutral, but will make other changes
more feasible down the road.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63028 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
de7aeefc5573d669ed476d7bda7a8940d3bcadb7 26-Jan-2009 Chris Lattner <sabre@nondot.org> Check in the long promised SourceLocation rewrite. This lays the
ground work for implementing #line, and fixes the "out of macro ID's"
problem.

There is nothing particularly tricky about the code, other than the
very performance sensitive SourceManager::getFileID() method.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62978 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
863c486fcb6162495a94fddf7ac8409de2638995 23-Jan-2009 Chris Lattner <sabre@nondot.org> This is a follow-up to r62675:

Refactor how the preprocessor changes a token from being an tok::identifier to a
keyword (e.g. tok::kw_for). Instead of doing this in HandleIdentifier, hoist this
common case out into the caller, so that every keyword doesn't have to go through
HandleIdentifier. This drops time in HandleIdentifier from 1.25ms to .62ms, and
speeds up clang -Eonly with PTH by about 1%.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62855 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
d1186fa38166a581b51975f0382a45fc3a0733d0 21-Jan-2009 Chris Lattner <sabre@nondot.org> a trivial micro optimization to save a load.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62676 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
6a170eb3ea6d6319277becabef68eb1a26bf8766 21-Jan-2009 Chris Lattner <sabre@nondot.org> Add a bit to IdentifierInfo that acts as a simple predicate which
tells us whether Preprocessor::HandleIdentifier needs to be called.
Because this method is only rarely needed, this saves a call and a
bunch of random checks. This drops the time in HandleIdentifier
from 3.52ms to .98ms on cocoa.h on my machine.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62675 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
a11d61793341fea195c29a0dab3fbd74f2b39a8c 19-Jan-2009 Chris Lattner <sabre@nondot.org> Rename SourceManager::getCanonicalFileID -> getFileID. There is
no longer such thing as a non-canonical FileID.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62499 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
bcc2a67e5180612417727cbdd8afd0f79fdf726d 19-Jan-2009 Chris Lattner <sabre@nondot.org> Make SourceLocation::getFileLoc private to reduce the API exposure of
SourceLocation. This requires making some cleanups to token pasting
and _Pragma expansion.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62490 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
3b4d5e955e819dd3a4bed37ea2e47d6e4cb05274 17-Jan-2009 Chris Lattner <sabre@nondot.org> Rename SourceLocation::getFileID to getChunkID, because it returns
the chunk ID not the file ID. This exposes problems in
TextDiagnosticPrinter where it should have been using the canonical
file ID but wasn't. Fix these along the way.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62427 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
8350394c65b81bba3986dfe44ae17423873741de 17-Jan-2009 Chris Lattner <sabre@nondot.org> simplify some lookups.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62426 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
42e00d19b0dac64732eb5449d52a076282fcbf77 17-Jan-2009 Chris Lattner <sabre@nondot.org> Change the Lexer ctor used to lex _Pragma directives into a static factory
method. This lets us clean up the interface and make it more obvious that
this method is *really really* _Pragma specific.

Note that _Pragma handling uglifies the Lexer in the critical path. It would
be very interesting to consider making _Pragma remapping be a new special
lexer class of its own.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62425 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
88d3ac1341aa016cabd966c5b113a95ac05ea43f 17-Jan-2009 Chris Lattner <sabre@nondot.org> Change the Lexer ctor used in the non _Pragma case to take a FileID instead
of a SourceLocation. This should speed it up and definitely simplifies it.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62422 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
0770dabb1ae81a2a9c2e7199262067103062a0b3 17-Jan-2009 Chris Lattner <sabre@nondot.org> More simplifications to the lexer ctors.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62419 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
de96c0f29c4cacabe6ea577c61db87c2a85aea6c 17-Jan-2009 Chris Lattner <sabre@nondot.org> make the verbose raw-lexer ctor fully explicit instead of having
embedded magic.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62417 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
025c3a66402fb713c2d9bf5dc174ff264765379a 17-Jan-2009 Chris Lattner <sabre@nondot.org> add a simplified lexer ctor that sets up the lexer to raw-lex an
entire file.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62414 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
22d91ca8d7c134eac5cc6a4869e6a84c461ad624 17-Jan-2009 Chris Lattner <sabre@nondot.org> refactor some common initialization code out of the two lexer ctors.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62411 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
2b2453a7d8fe732561795431f39ceb2b2a832d84 17-Jan-2009 Chris Lattner <sabre@nondot.org> this massive patch introduces a simple new abstraction: it makes
"FileID" a concept that is now enforced by the compiler's type checker
instead of yet-another-random-unsigned floating around.

This is an important distinction from the "FileID" currently tracked by
SourceLocation. *That* FileID may refer to the start of a file or to a
chunk within it. The new FileID *only* refers to the file (and its
#include stack and eventually #line data), it cannot refer to a chunk.

FileID is a completely opaque datatype to all clients, only SourceManager
is allowed to poke and prod it.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62407 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
c6fe32a91c7372caf09152ee31a24c4b5d24deed 17-Jan-2009 Chris Lattner <sabre@nondot.org> Instead of iterating over FileID's, have PTH generation iterate over the
content cache directly. Content cache has a 1-1 mapping with fileentries,
whereas multiple FileIDs can be the same FileEntry.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62401 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
8402c73dd880e8af46c826d873681820aebe32eb 16-Jan-2009 Chris Lattner <sabre@nondot.org> Fix PR2477 - clang misparses "//*" in C89 mode



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62368 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
f7cf85b330bedd2877e1371fb0a83e99751ae162 16-Jan-2009 Chris Lattner <sabre@nondot.org> more SourceLocation lexicon change: instead of referring to the
"logical" location, refer to the "instantiation" location.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62316 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
df7c17a8d02fe09a3466786bae3e40fc3252687a 16-Jan-2009 Chris Lattner <sabre@nondot.org> Change some terminology in SourceLocation: instead of referring to
the "physical" location of tokens, refer to the "spelling" location.
This is more concrete and useful, tokens aren't really physical objects!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62309 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
bc3e9842212b78bcf49682641307e4f3419ecc5e 12-Dec-2008 Chris Lattner <sabre@nondot.org> rdar://6060752 - don't warn about trigraphs in bcpl-style comments


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60942 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
47a2b40b7a3d9c89d6c3d12e5a81e2498c207750 12-Dec-2008 Chris Lattner <sabre@nondot.org> fix thought-o


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60937 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
bec1c9d6f3feeec112cc8eeae90c1be29c6aaf13 01-Dec-2008 Douglas Gregor <dgregor@apple.com> Objective-C keywords are not always identifiers. Some are also C++ keywords

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60373 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
ddd3e8b90a92c39f620bda7a0945320c0b9e60db 25-Nov-2008 Daniel Dunbar <daniel@zuster.org> Comment fix.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59997 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
4984212fced6d9c6538ebb7319839105e3880b45 22-Nov-2008 Chris Lattner <sabre@nondot.org> Fix a weird inconsistency with hex floats. Previously the lexer
would not eat the "-1" in "0x0p-1", but LiteralSupport would accept
it when extensions are on. This caused strangeness and failures
when hexfloats were properly treated as an extension (not error)
in LiteralSupport.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59865 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
30c6476e8ef00caeb3a10ea0228165013a4eb3f3 22-Nov-2008 Chris Lattner <sabre@nondot.org> actually, this version isn't really needed.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59859 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
306fda76b0818eb7c215094123f2e16729a44aed 22-Nov-2008 Chris Lattner <sabre@nondot.org> remove a sneaky version of Diag hiding in PreprocessorLexer.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59858 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
74d15dfd183b2082e8a5d4dfbf66bd861b220901 22-Nov-2008 Chris Lattner <sabre@nondot.org> Change the Lexer::Diag method to not magically silence warnings,
force the caller to check instead. This eliminates the need (and the
risk!) of weird null DiagnosticBuilder's floating around.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59856 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
3cbfe2c4159e0a219ae660d50625c013aa4afbd0 22-Nov-2008 Chris Lattner <sabre@nondot.org> Split the DiagnosticInfo class into two disjoint classes:
one for building up the diagnostic that is in flight (DiagnosticBuilder)
and one for pulling structured information out of the diagnostic when
formatting and presenting it.

There is no functionality change with this patch.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59849 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
41938c8493b4380df738263166b746eacb33c309 19-Nov-2008 Ted Kremenek <kremenek@apple.com> - Move static function IsNonPragmaNonMacroLexer into Preprocessor.h.
- Add variants of IsNonPragmaNonMacroLexer to accept an IncludeMacroStack entry
(simplifies some uses).
- Use IsNonPragmaNonMacroLexer in Preprocessor::LookupFile.
- Add 'FileID' to PreprocessorLexer, and have Preprocessor query this fileid
when looking up the FileEntry for a file

Performance testing of -Eonly on Cocoa.h shows no performance regression because
of this patch.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59666 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
3692b09faa9fe346f39bc922db6dce48cdcc3f63 18-Nov-2008 Chris Lattner <sabre@nondot.org> Convert the lexer and start converting the PP over to using canonical Diag methods.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59511 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
d6a2e7d71e9a910c45a3eb0268a5d94635003d58 13-Nov-2008 Ted Kremenek <kremenek@apple.com> Move some diagnostic handling to PreprocessorLexer.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59191 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
6db72c092d6af5f36d15ee5ce9b9fee736cdf41c 12-Nov-2008 Ted Kremenek <kremenek@apple.com> Remove Lexer::LexIncludeFilename.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59186 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
9e6293d4dfd688429f77ee3b6edba9dfd7ada3a2 12-Oct-2008 Chris Lattner <sabre@nondot.org> Change FormTokenWithChars to take the token kind to form, since all clients
were setting a kind and then forming it. This is just a minor API cleanup,
no functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57404 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
31f0ecae4b2d4d268d04c562a43d9fbfee96f189 12-Oct-2008 Chris Lattner <sabre@nondot.org> When in keep whitespace mode, make sure to return block comments that are
unterminated.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57403 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
2d3818959df7b71bfcf8c2f2b573856bfc4c3a21 12-Oct-2008 Chris Lattner <sabre@nondot.org> Change SkipBlockComment and SkipBCPLComment to return true when in
keep comment mode, instead of returning false. This matches SkipWhitespace.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57402 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
d88dc48e33d71732708960170e57a3d1bdc8f847 12-Oct-2008 Chris Lattner <sabre@nondot.org> Add a new mode to the lexer which enables it to return all characters,
even whitespace, as tokens from the file. This is enabled with
L->SetKeepWhitespaceMode(true) on a raw lexer. In this mode, you too
can use clang as a really complex version of 'cat' with code like this:

Lexer RawLex(SourceLocation::getFileLoc(SM.getMainFileID(), 0),
PP.getLangOptions(), File.first, File.second);

RawLex.SetKeepWhitespaceMode(true);

Token RawTok;
RawLex.LexFromRawLexer(RawTok);
while (RawTok.isNot(tok::eof)) {
std::cout << PP.getSpelling(RawTok);
RawLex.LexFromRawLexer(RawTok);
}

This will emit exactly the input file, with no canonicalization or other
translation. Realistic clients actually do something with the tokens of
course :)



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57401 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
f744d133e8d36eec65828d046ee695bda238125c 12-Oct-2008 Chris Lattner <sabre@nondot.org> Fix a couple more places that poke KeepCommentMode unnecesarily.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57398 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
fa95a019da00b926d64ff83358ba73bbc6ae1e37 12-Oct-2008 Chris Lattner <sabre@nondot.org> add a new inKeepCommentMode() accessor to abstract the KeepCommentMode
ivar.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57397 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
a2c7ad9c7312df162300ba95828ed3b6c3b38131 12-Oct-2008 Chris Lattner <sabre@nondot.org> fix misleading comment.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57396 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
0af574270d3be2b0e73a3379dfaa633746f8fc6f 12-Oct-2008 Chris Lattner <sabre@nondot.org> Simplify raw mode lexing by treating an unterminate /**/ comment the
same we we do an unterminated string or character literal. This makes
it so we can guarantee that the lexer never calls into the
preprocessor (which would be suicide for a raw lexer).


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57395 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
8527b71b1944b155a2bd60ec364d700299bc3ff7 12-Oct-2008 Chris Lattner <sabre@nondot.org> add a comment.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57394 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
590f0cc643274267d4d41125b62557e1d87886c3 12-Oct-2008 Chris Lattner <sabre@nondot.org> Change how raw lexers are handled: instead of creating them and then
using LexRawToken, create one and use LexFromRawLexer. This avoids
twiddling the RawLexer flag around and simplifies some code (even
speeding raw lexing up a tiny bit).

This change also improves the token paster to use a Lexer on the stack
instead of new/deleting it.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57393 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
33d34a6f7a65be1821ab31c7d08b8e9f36d8759c 12-Oct-2008 Chris Lattner <sabre@nondot.org> silence some release-assert warnings.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57391 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
7a4f004a77421af876152281b92c4f8ea24afb2d 12-Oct-2008 Chris Lattner <sabre@nondot.org> improve a comment.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57389 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
662e8b5647adbb1bc9eeceece7b64600cfa87471 15-Aug-2008 Daniel Dunbar <daniel@zuster.org> Change Parser & Sema to use interned "super" for comparions.
- Added as private members for each because it is not clear where to
put the common definition. Perhaps the IdentifierInfos all of these
"pseudo-keywords" should be collected into one place (this would
KnownFunctionIDs and Objective-C property IDs, for example).

Remove Token::isNamedIdentifier.
- There isn't a good reason to use strcmp when we have interned
strings, and there isn't a good reason to encourage clients to do
so.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@54794 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
5253c7ff266ae79308050c9f43d60dd1a67c5fb9 14-Apr-2008 Nate Begeman <natebegeman@mac.com> Fix typo


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@49632 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
b25e5d79d9d1967df058a242e96a62d0d0ace074 12-Apr-2008 Chris Lattner <sabre@nondot.org> don't diagnose empty source files, thanks Neil!


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@49575 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
8fa3078cfe8b25fe8450e7d31dc5500b5888b95e 11-Apr-2008 Chris Lattner <sabre@nondot.org> don't read off the front of the buffer. Thanks to Sam for pointing this out.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@49535 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp
bda0b626e74513950405c27525af87e214e605e2 16-Mar-2008 Chris Lattner <sabre@nondot.org> Make a major restructuring of the clang tree: introduce a top-level
lib dir and move all the libraries into it. This follows the main
llvm tree, and allows the libraries to be built in parallel. The
top level now enforces that all the libs are built before Driver,
but we don't care what order the libs are built in. This speeds
up parallel builds, particularly incremental ones.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48402 91177308-0d34-0410-b5e6-96231b3b80d8
/external/clang/lib/Lex/Lexer.cpp