History log of /external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
6d297cbec48e9026ddf31d2ff95df7c182572dc9 20-Mar-2016 Serhiy Storchaka <storchaka@gmail.com> Issue #26581: Use the first coding cookie on a line, not the last one.
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
3eb554fc828c812a31c1a3cd9f619eacbb708010 05-Sep-2014 Serhiy Storchaka <storchaka@gmail.com> Issue #22221: Backported fixes from Python 3 (issue #18960).

* Now the source encoding declaration on the second line isn't effective if
the first line contains anything except a comment. This affects compile(),
eval() and exec() too.

* IDLE now ignores the source encoding declaration on the second line if the
first line contains anything except a comment.

* 2to3 and the findnocoding.py script now ignore the source encoding
declaration on the second line if the first line contains anything except
a comment.
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
da952f3ff43aad1a4602b1bf6e93fd24f0a159f6 10-Apr-2014 Benjamin Peterson <benjamin@python.org> add matrix multiplication operator support to 2to3
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
2612679ddc202ff684ea525142d2802c962b4f64 25-Nov-2013 Ezio Melotti <ezio.melotti@gmail.com> #19620: Fix typo in docstring (noticed by Christopher Welborn).
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
e4818f693749af358b1a2b3f96853c756f80856e 17-Sep-2013 Serhiy Storchaka <storchaka@gmail.com> Remove the use of non-existing re.ASCII.
(fixes a regression in 3d46ef0c62c5, issue #18873)
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
e787bce79c016cf7b9e743a6230ceb8e97b4a938 16-Sep-2013 Serhiy Storchaka <storchaka@gmail.com> Issue #18873: IDLE, 2to3, and the findnocoding.py script now detect Python
source code encoding only in comment lines.
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
5dfad9dc9a93ad78f56de4042ac2f3b45dd51a7b 07-May-2010 Benjamin Peterson <benjamin@python.org> Merged revisions 79911,79916-79917,80018,80418,80572-80573,80635-80639,80668,80922 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3

........
r79911 | benjamin.peterson | 2010-04-09 15:38:53 -0500 (Fri, 09 Apr 2010) | 1 line

use absolute import
........
r79916 | benjamin.peterson | 2010-04-09 16:05:21 -0500 (Fri, 09 Apr 2010) | 1 line

generalize detection of __future__ imports and attach them to the tree
........
r79917 | benjamin.peterson | 2010-04-09 16:11:44 -0500 (Fri, 09 Apr 2010) | 1 line

don't try to 'fix' relative imports when absolute_import is enabled #8858
........
r80018 | benjamin.peterson | 2010-04-12 16:12:12 -0500 (Mon, 12 Apr 2010) | 4 lines

prevent diffs from being mangled is multiprocess mode #6409

Patch by George Boutsioukis.
........
r80418 | benjamin.peterson | 2010-04-23 16:00:03 -0500 (Fri, 23 Apr 2010) | 1 line

remove unhelpful description
........
r80572 | benjamin.peterson | 2010-04-27 20:33:54 -0500 (Tue, 27 Apr 2010) | 1 line

use unicode literals
........
r80573 | jeffrey.yasskin | 2010-04-27 23:08:27 -0500 (Tue, 27 Apr 2010) | 6 lines

Don't transform imports that are already relative. 2to3 turned
from . import refactor
into
from .. import refactor
which broke the transformation of 2to3 itself.
........
r80635 | benjamin.peterson | 2010-04-29 16:02:23 -0500 (Thu, 29 Apr 2010) | 1 line

remove imports
........
r80636 | benjamin.peterson | 2010-04-29 16:02:41 -0500 (Thu, 29 Apr 2010) | 1 line

unicode literal
........
r80637 | benjamin.peterson | 2010-04-29 16:03:42 -0500 (Thu, 29 Apr 2010) | 1 line

must pass a string to Number
........
r80638 | benjamin.peterson | 2010-04-29 16:05:34 -0500 (Thu, 29 Apr 2010) | 1 line

unicode literals
........
r80639 | benjamin.peterson | 2010-04-29 16:06:09 -0500 (Thu, 29 Apr 2010) | 1 line

pass string to Number
........
r80668 | jeffrey.yasskin | 2010-04-30 18:02:47 -0500 (Fri, 30 Apr 2010) | 4 lines

Make 2to3 run under Python 2.5 so that the benchmark suite at
http://hg.python.org/benchmarks/ can use it and still run on implementations
that haven't gotten to 2.6 yet. Fixes issue 8566.
........
r80922 | benjamin.peterson | 2010-05-07 11:06:25 -0500 (Fri, 07 May 2010) | 1 line

prevent xrange transformation from wrapping range calls it produces in list
........
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
798e54004f47b8173ef70e6df0132bedceb13311 22-Mar-2010 Benjamin Peterson <benjamin@python.org> Merged revisions 79077,79137,79304-79305 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3

........
r79077 | benjamin.peterson | 2010-03-18 18:05:29 -0500 (Thu, 18 Mar 2010) | 1 line

port detect_encoding improvements from py3k
........
r79137 | benjamin.peterson | 2010-03-20 11:12:53 -0500 (Sat, 20 Mar 2010) | 1 line

add a fixer for setting sys.exitfunc #2356
........
r79304 | benjamin.peterson | 2010-03-22 17:20:22 -0500 (Mon, 22 Mar 2010) | 1 line

fix test_parser when it's run in a path with spaces #7666
........
r79305 | benjamin.peterson | 2010-03-22 17:27:07 -0500 (Mon, 22 Mar 2010) | 1 line

normalize whitespace
........
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
b9d4963a989accce30234b7b74bce874c0142209 05-Jan-2010 Antoine Pitrou <solipsis@pitrou.net> Issue #7092: Fix the DeprecationWarnings emitted by the standard library
when using the -3 flag. Patch by Florent Xicluna.
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
42d26d94cc5fe633613c853c1e1b4b51f0855df4 25-Nov-2009 Benjamin Peterson <benjamin@python.org> Merged revisions 76160-76161,76250,76252,76447,76506 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3

........
r76160 | benjamin.peterson | 2009-11-08 18:53:48 -0600 (Sun, 08 Nov 2009) | 1 line

undeprecate the -p option; it's useful for converting python3 sources
........
r76161 | benjamin.peterson | 2009-11-08 19:05:37 -0600 (Sun, 08 Nov 2009) | 1 line

simplify condition
........
r76250 | benjamin.peterson | 2009-11-13 16:56:48 -0600 (Fri, 13 Nov 2009) | 1 line

fix handling of a utf-8 bom #7313
........
r76252 | benjamin.peterson | 2009-11-13 16:58:36 -0600 (Fri, 13 Nov 2009) | 1 line

remove pdb turd
........
r76447 | benjamin.peterson | 2009-11-22 18:17:40 -0600 (Sun, 22 Nov 2009) | 1 line

#7375 fix nested transformations in fix_urllib
........
r76506 | benjamin.peterson | 2009-11-24 18:34:31 -0600 (Tue, 24 Nov 2009) | 1 line

use generator expressions in any()
........
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
f9e7d5421a0c3af271443350cd64a749c39c6ce3 02-Nov-2009 Benjamin Peterson <benjamin@python.org> Merged revisions 74359,75081,75088,75213,75278,75303,75427-75428,75734-75736,75865,76059-76061 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3

........
r74359 | benjamin.peterson | 2009-08-12 17:23:13 -0500 (Wed, 12 Aug 2009) | 1 line

don't pass the deprecated print_function option
........
r75081 | benjamin.peterson | 2009-09-26 22:02:57 -0500 (Sat, 26 Sep 2009) | 1 line

let 2to3 work with extended iterable unpacking
........
r75088 | benjamin.peterson | 2009-09-27 11:25:21 -0500 (Sun, 27 Sep 2009) | 1 line

look on the type only for __call__
........
r75213 | benjamin.peterson | 2009-10-03 10:09:46 -0500 (Sat, 03 Oct 2009) | 5 lines

revert 75212; it's not correct

People can use isinstance(x, collections.Callable) if they expect objects with
__call__ in their instance dictionaries.
........
r75278 | benjamin.peterson | 2009-10-07 16:25:56 -0500 (Wed, 07 Oct 2009) | 4 lines

fix whitespace problems with fix_idioms #3563

Patch by Joe Amenta.
........
r75303 | benjamin.peterson | 2009-10-09 16:59:11 -0500 (Fri, 09 Oct 2009) | 1 line

port latin-1 and utf-8 cookie improvements
........
r75427 | benjamin.peterson | 2009-10-14 20:35:57 -0500 (Wed, 14 Oct 2009) | 1 line

force floor division
........
r75428 | benjamin.peterson | 2009-10-14 20:39:21 -0500 (Wed, 14 Oct 2009) | 1 line

silence -3 warnings about __hash__
........
r75734 | benjamin.peterson | 2009-10-26 16:25:53 -0500 (Mon, 26 Oct 2009) | 2 lines

warn on map(None, ...) with more than 2 arguments #7203
........
r75735 | benjamin.peterson | 2009-10-26 16:28:25 -0500 (Mon, 26 Oct 2009) | 1 line

remove unused result
........
r75736 | benjamin.peterson | 2009-10-26 16:29:02 -0500 (Mon, 26 Oct 2009) | 1 line

using get() here is a bit pointless
........
r75865 | benjamin.peterson | 2009-10-27 15:49:00 -0500 (Tue, 27 Oct 2009) | 1 line

explain reason for warning
........
r76059 | benjamin.peterson | 2009-11-02 11:43:47 -0600 (Mon, 02 Nov 2009) | 1 line

tuples are no longer used for children
........
r76060 | benjamin.peterson | 2009-11-02 11:55:40 -0600 (Mon, 02 Nov 2009) | 1 line

revert r76059; apparently some fixers rely on Leaf no () for children
........
r76061 | benjamin.peterson | 2009-11-02 12:06:17 -0600 (Mon, 02 Nov 2009) | 1 line

make fix_tuple_params keep the tree valid #7253
........
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
84ad84e0bb15e7c64109e88060afdcb60ae7b740 09-May-2009 Benjamin Peterson <benjamin@python.org> Merged revisions 72491-72493 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3

........
r72491 | benjamin.peterson | 2009-05-08 19:33:27 -0500 (Fri, 08 May 2009) | 7 lines

make 2to3 use unicode internally on 2.x

This started out as a fix for #2660, but became this large refactoring
when I realized the dire state this was in. 2to3 now uses
tokenize.detect_encoding to decode the files correctly into unicode.
........
r72492 | benjamin.peterson | 2009-05-08 19:35:38 -0500 (Fri, 08 May 2009) | 1 line

remove compat code
........
r72493 | benjamin.peterson | 2009-05-08 19:54:15 -0500 (Fri, 08 May 2009) | 1 line

add a test for \r\n newlines
........
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py
5e37baea8007cb64b65a180e4d6c80de292a8a4a 19-Mar-2008 Martin v. Löwis <martin@v.loewis.de> Import lib2to3.
/external/python/cpython2/Lib/lib2to3/pgen2/tokenize.py