1 [The "BSD license"]
2 Copyright (c) 2010 Terence Parr
3 Maven Plugin - Copyright (c) 2009      Jim Idle
4
5 All rights reserved.
6
7 Redistribution and use in source and binary forms, with or without
8 modification, are permitted provided that the following conditions
9 are met:
10 1. Redistributions of source code must retain the above copyright
11    notice, this list of conditions and the following disclaimer.
12 2. Redistributions in binary form must reproduce the above copyright
13    notice, this list of conditions and the following disclaimer in the
14    documentation and/or other materials provided with the distribution.
15 3. The name of the author may not be used to endorse or promote products
16    derived from this software without specific prior written permission.
17
18 THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
19 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
20 OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
21 IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
22 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
23 NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
24 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
25 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
26 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
27 THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
28
29============================================================================
30
31This file contains the build instructions for the ANTLR toolset as
32of version 3.1.3 and beyond.
33
34The ANTLR toolset must be built using the Maven build system as
35this build system updates the version numbers and controls the
36whole build process. However, if you just want the latest build
37and do not care to learn anything about Maven, then visit the 'target'
38directories (for jars) under the depot mirror root here:
39
40   http://antlr.org/depot
41
42If you are looking for the latest released version of ANTLR, then
43visit the downloads page on the main antlr.org website.
44
45These instructions are mainly for the ANTLR development team,
46though you are free to build ANTLR yourself of course.
47
48Source code Structure
49-----------------------
50
51The main development branch of ANTLR is stored within the Perforce SCM at:
52
53   //depot/code/antlr/main/...
54
55release branches are stored in Perforce like so:
56
57   //depot/code/antlr/release-3.1.3/...
58
59In this top level directory, you will find a master build file for
60Maven called pom.xml and you will also note that there are a number of
61subdirectories:
62
63 tool                  - The ANTLR tool itself
64 runtime/Java          - The ANTLR Java runtime
65 runtime/X             - The runtime for language target X
66 gunit                 - The grammar test tool
67 antlr3-maven-plugin   - The plugin tool for Maven allowing Maven
68 		          projects to process ANTLR grammars.
69
70Each of these sub-directories also contains a file pom.xml that
71controls the build of each sub-component (or module in Maven
72parlance).
73
74Build Parameters
75-----------------
76
77Alongside each pom.xml (other than for the antlr3-maven-plugin), you
78will see that there is a file called antlr.config. This file is called
79a filter and should contain a set of key/value pairs in the same
80manner as Java properties files:
81
82antlr.something="Some config thang!"
83
84When the build of any component happens, any values in the
85antlr.config for the master build file and any values in the
86antlr.config file for each component are made available to the
87build. This is mainly used by the resource processor, which will
88filter any file it finds under: src/main/resources/** and replace any
89references such as ${antlr.something} with the actual value at the
90time of the build.
91
92Building
93--------
94
95Building ANTLR is trivial, assuming that you have loaded Maven version
963.0.3 or better on to your build system and installed it as explained
97here:
98
99http://maven.apache.org/download.html
100
101Note that the ANTLR toolset will ONLY build with version 3.0.3 of Maven
102as of release 3.4.
103
104If you are unfamiliar with Maven (and even if you are), the best
105resource for learning about it is The Definitive Guide:
106
107http://www.sonatype.com/books/maven-book/reference/public-book.html
108
109The instructions here assume that Maven is installed and working correctly.
110
111If this is the first time you have built the ANTLR toolset, you will
112possibly need to install the master pom in your local repository
113(however the build may be able to locate this in the ANTLR snapshot or
114release repository). If you try to build sub-modules on their own (as
115in run the mvn command in the sub directory for that tool, such as
116runtime/Java), and you receive a message that maven cannot find the
117master pom, then execute this in the main (or release) directory:
118
119mvn -N install
120
121This command will install the master build pom in your local maven
122repository (it's ~/.m2 on UNIX) and individual builds of sub-modules
123will now work correctly.
124
125To build then, simply cd into the master build directory
126(e.g. $P4ROOT//code/antlr/main) and type:
127
128mvn -Dmaven.test.skip=true
129
130Assuming that everything is correctly installed and synchronized, then
131ANTLR will build and skip any unit tests in the modules (the ANTLR
132tool tests can take a long time).
133
134This command will build each of the tools in the correct order and
135will create the jar artifacts of all the components in your local
136development Maven repository (which takes precedence over remote
137repositories by default). At the end of the build you should see:
138
139[INFO] ------------------------------------------------------------------------
140[INFO] Reactor Summary:
141[INFO] ------------------------------------------------------------------------
142[INFO] ANTLR Master build control POM ........................ SUCCESS [1.373s]
143[INFO] Antlr 3 Runtime ....................................... SUCCESS [0.879s]
144[INFO] ANTLR Grammar Tool .................................... SUCCESS [5.431s]
145[INFO] Maven plugin for ANTLR V3 ............................. SUCCESS [1.277s]
146[INFO] ANTLR gUnit ........................................... SUCCESS [1.566s]
147[INFO] Maven plugin for gUnit ANTLR V3 ....................... SUCCESS [0.079s]
148[INFO] ------------------------------------------------------------------------
149[INFO] ------------------------------------------------------------------------
150[INFO] BUILD SUCCESSFUL
151[INFO] ------------------------------------------------------------------------
152[INFO] Total time: 11 seconds
153
154However, unless you are using Maven exclusively in your projects, you
155will most likely want to build the ANTLR Uber Jar, which is an
156executable jar containing all the components that ANTLR needs to build
157and run parsers (note that at runtime, you need only the runtime
158components you use, such as the Java runtime and say stringtemplate).
159
160Because the Uber jar is not something we want to deploy to Maven
161repositories it is built with a special invocation of Maven:
162
163mvn -Dmaven.test.skip=true package assembly:assembly
164
165Note that Maven will appear to build everything twice, which is a
166quirk of how it calculates the dependencies and makes sure it has
167everything packaged up so it can build the uber-jar assembly.
168
169Somewhere in the build output (towards the end), you will find a line
170like this:
171
172[INFO] Building jar: /home/jimi/antlrsrc/code/antlr/main/target/antlr-master-3.4-SNAPSHOT-completejar.jar
173
174This is the executable jar that you need and you can either copy it
175somewhere or, like me, you can create this script (assuming UNIX)
176somewhere in your PATH:
177
178#! /bin/bash
179java -jar ~/antlrsrc/code/antlr/main/target/antlr-master-3.4-SNAPSHOT-completejar.jar $*
180
181Version Numbering
182-------------------
183
184The first and Golden rule is that any pom files stored under the main
185branch of the toolset should never be modified to contain a release
186version number. They should always contain a.b.c-SNAPSHOT
187(e.g. 3.1.3-SNAPSHOT). Only release branches should have their pom
188version numbers set to a release version. You can release as many
189SNAPSHOTS as you like, but only one release version. However, release
190versions may be updated with a patch level: 3.1.3-1, 3.1.3-2 and so
191on.
192
193Fortunately, Maven helps us with the version numbering in a number of
194ways. Firstly, the pom.xml files for the various modules do not
195specify a version of the artifacts themselves. They pick up their
196version number from the master build pom.  However, there is a catch,
197because they need to know what version of the parent pom they inherit
198from and so they DO mention the version number. However, this does
199prevent accidentally releasing different versions of sub-modules than
200the master pom describes.
201
202Fortunately once again, Maven has a neat way of helping us change the
203version.  All you need do is check out all the pom.xml files from
204perforce, then modify the <version>a.b.c-SNAPSHOT</version> in the
205master pom. When the version number is correct in the master pom, you
206make sure your working directory is the location of the master pom and
207type:
208
209mvn versions:update-child-modules
210
211This command will then update the child pom.xml files to reflect the
212version number defined in the master pom.xml.
213
214There is unfortunately one last catch here though and that is that the
215antlr3-maven-plugin and the gunit-maven-plugin are not able to use the
216parent pom. The reason for this is subtle but makes sense as doing so
217would create a circular dependency between the ANTLR tool (which uses
218the plugin to build its own grammar files), and the plugins (which
219uses the tool to build grammar files and gunit to test).
220
221This catch-22 situation means that the pom.xml file in the
222antlr3-maven-plugin directory and the one in the gunit-maven-plugin
223directory MUST be updated manually (or we must write a script to do
224this).
225
226Finally, we need to remember that because the tool is dependent on the
227antlr3-maven-plugin and the plugin is itself dependent on the
228tool, that we must manually update the versions of each that they
229reference. So, when we bump the version of the toolset to say
2303.1.4-SNAPSHOT, we need to change the antlr3-maven-plugin pom.xml and
231the gunit-maven-plugin pom.xml to reference that version of the antlr
232tool. The tool itself is always built with the prior released version
233of the plugin, so when we release we must change the main branch of
234the plugin to use the newly released version of the plugin. This is
235covered in the release checklist.
236
237Deploying
238----------
239
240Deploying the tools at the current version is relatively easy, but to
241deploy to the ANTLR repositories (snapshot or release) you must have
242been granted access to the Sonatype OSS repositories' ANTLR login. 
243Few people will have this access of course.
244
245Next, because we do not publish access information for antlr.org, you
246will need to configure the repository server names locally. You do
247this by creating (or adding to) the file:
248
249~/.m2/settings.xml
250
251Which should look like this:
252
253<?xml version="1.0" encoding="UTF-8"?>
254<settings>
255  <servers>
256    <server>
257      <id>sonatype-nexus-snapshots</id>
258      <username>xxxxxxx</username>
259      <password>xxxxxxx</password>
260    </server>
261    <server>
262      <id>sonatype-nexus-staging</id>
263      <username>xxxxxxx</username>
264      <password>xxxxxxx</password>
265    </server>
266  </servers>
267</settings>
268
269When this configuration is in place, you will be able to deploy the components,
270either individually or from the master directory:
271
272mvn -Dmaven.test.skip=true -Ddeplot deploy
273
274You will then see lots of information about checking existing version
275information and so on, and the components will be deployed once you
276supply the ANTLR public key passphrase to sign the jars.
277
278Note that so long as the artifacts are versioned with a.b.c-SNAPSHOT
279then deployment will always be to the development snapshot
280directory. When the artifacts are versioned with a release version
281then deployment will be to the release stahinh repository, which
282will then be mirrored around the world if closed and release.
283The sonatype documentation should be consulted.
284
285Release Checklist
286------------------
287
288Here is the procedure to use to make a release of ANTLR. Note that we
289should really use the mvn release:release command, but the perforce
290plugin for Maven is not commercial quality and I want to rewrite it.
291
292For this checklist, let's assume that the current development version
293of ANTLR is 3.1.3-SNAPSHOT. This means that it will probably (but not
294necessarily) become release version 3.1.3 and that the development
295version will bump to 3.1.4-SNAPSHOT.
296
2970) Run a build of the main branch and check that it is builds and
298   passes as many tests as you want it to.
299
3001) First make a branch from main into the target release
301   directory. Then submit this to perforce. You could change versions
302   numbers before submitting, but doing that in separate stages will
303   keep things sane;
304
305--- Use main development branch from here ---
306
3072) Before we deploy the release, we want to update the versions of the
308   development branch, so we don't deploy what is now the new release
309   as an older snapshot (this is not super important, but procedure is
310   good right?).
311
312   Check out all the pom.xml files (and if you are using any
313   antlr.config parameters that must change, then do that too).
314
3153) Edit the master pom.xml in the main directory and change the version from
316   3.1.3-SNAPSHOT to 3.1.4-SNAPSHOT.
317
3184) Edit the pom.xml file for antlr3-maven-plugin under the main
319   directory and change the version from 3.1.3-SNAPSHOT to
320   3.1.4-SNAPSHOT. Do the same for the pom.xml in the
321   gunit-maven-plugin directory.
322
323   Update the pom.xml for the archetype manually too.
324
3255) Now (from the main directory), run the command:
326
327         mvn versions:update-child-modules
328
329      You should see:
330
331         [INFO] [versions:update-child-modules]
332         [INFO] Module: gunit
333         [INFO]   Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT
334         [INFO] Module: runtime/Java
335         [INFO]   Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT
336         [INFO] Module: tool
337         [INFO]   Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT
338
3396) Run a build of the main branch:
340
341         mvn -Dmaven.test.skip=true
342
343       All should be good.
344
3457) Submit the pom changes of the main branch to perforce.
346
3478) Deploy the new snapshot as a placeholder for the next release. It
348   will go to the snapshot repository of course:
349
350	  mvn -N deploy
351          mvn -Dmaven.test.skip=true deploy
352
3539) You are now finished with the main development branch and should change
354   working directories to the release branch you made earlier.
355
356--- Use release branch from here ---
357
35810) Check out all the pom.xml files in the release branch (and if you are
359    using any antlr.config parameters that must change, then do that too).
360
36111) Edit the master pom.xml in the release-3.1.3 directory and change
362    the version from 3.1.3-SNAPSHOT to 3.1.3.
363
36412) Edit the pom.xml file for antlr3-maven-plugin under the
365    release-3.1.3 directory and change the version from 3.1.3-SNAPSHOT
366    to 3.1.3. Also change the version of the tool that the this
367    pom.xml references from 3.1.3-SNAPSHOT to 3.1.3 as we are now
368    releasing the plugin of course and it needs to reference the
369    version we are about to release. You will find this reference in
370    the dependencies section of the antlr3-maven-plugin pom.xml. Also
371    change the version references in the pom for gunit-maven-plugin.
372
37313)  Now (from the release-3.1.3 directory), run the command:
374
375           mvn versions:update-child-modules
376
377        You should see:
378
379	[INFO] [versions:update-child-modules]
380	[INFO] Module: gunit
381	[INFO]   Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT,
382	       now org.antlr:antlr-master:3.1.3
383	[INFO] Module: runtime/Java
384	[INFO]   Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT,
385	       now org.antlr:antlr-master:3.1.3
386	[INFO] Module: tool
387	[INFO]   Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT,
388	       now org.antlr:antlr-master:3.1.3
389
39014)  Run a build of the release-3.1.3 branch:
391
392           mvn   # Note I am letting unit tests run here!
393
394        All should be good, or as good as it gets ;-)
395
39615)  Submit the pom changes of the release-3.1.3 branch to perforce.
397
39816)  Deploy the new release (this is it guys, make sure you are happy):
399
400	  mvn -N deploy
401          mvn -Dmaven.test.skip=true deploy
402
403        Note that we must skip the tests as Maven will not let you
404        deploy releases that fail any junit tests.
405
40617) The final step is that we must update the main branch pom.xml for
407     the tool to reference the newly release version of the
408     antlr3-maven-plugin. This is because each release of ANTLR is
409     built with the prior release of ANTLR, and we have just released
410     a new version. Edit the pom.xml for the tool (main/tool/pom.xml)
411     under the main (that's the MAIN branch, not the release branch)
412     and find the dependency reference to the antlr plugin. If you
413     just released say 3.1.3, then the tool should now reference
414     version 3.1.3 of the plugin. Having done this, you should
415     probably rebuild the main branch and let it run the junit
416     tests. Later, I will automate this dependency update as mvn can
417     do this for us.
418
41918)  Having deployed the release to maven, you will want to create the
420     uber jar for the new release, to make it downloadable from the
421     antlr.org website. This is a repeat of the earlier described step
422     to build the uber jar:
423
424       mvn -Dmaven.test.skip=true package assembly:assembly
425
426     MAven will produce the uber jar in the target directory:
427
428	antlr-master-3.1.3-completejar.jar
429
430     And this is the complete jar that can be downloaded from the web site. You
431     may wish to produce an md5 checksum to go with the jar:
432
433     md5sum target/antlr-master-3.1.3-completejar.jar
434     xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  target/antlr-master-3.1.4-SNAPSHOT-completejar.jar
435
436     The command you just ran will also produce a second jar:
437
438        antlr-master-3.1.3-src.jar
439
440     This is the source code for everythign you just deployed and can
441     be unjarred and built from scratch using the very procedures
442     described here, which means you will now be reading this
443     BUILD.txt file for ever.
444
44519)  Reward anyone around you with good beer.
446
447
448Miscellany
449-----------
450
451It was a little tricky to get all the interdependencies correct
452because ANTLR builds itself using itself and the maven plugin
453references the ANTLR Tool as well. Hence the maven tool is not a child
454project of the master pom.xml file, even though it is built by it.
455
456An observant person will not that when the assembly:assembly phase is
457run, that it invokes the build of the ANTLR tool using the version of
458the Maven plugin that it has just built, and this results in the
459plugin using the version of ANTLR tool that it has just built. This is
460safe because everything will already be up to date and so we package
461up the version of the tool that we expect, but the Maven plugin we
462deploy will use the correct version of ANTLR, even though there is
463technically a circular dependency.
464
465The master pom.xml does give us a way to cause the build of the ANTLR
466tool to use itself to build itself. This is because in
467dependencyManagement in the master pom.xml, we can reference the
468current version of the Tool and the Maven plugin, even though in the
469pom.xml for the tool itself refers to the previous version of the
470plugin.
471
472What happens is that if we first cd into the tool and maven
473directories and build ANTLR, it will build itself with the prior
474version and this will deploy locally (.m2). We can then clean build
475from the master pom and when ANTLR asks for the prior version of the
476tool, the master pom.xml will override it and build with the interim
477versions we just built manually.
478
479However, strictly speaking, we need a third build where we rebuild the
480tool again with the version of the tool that was built with itself and
481not deploy the version that was built by the version of itself that
482was built by a prior version of itself. I decided that this was not
483particularly useful and complicates things too much. Building with a
484prior version of the tool is fine and if there was ever a need to, we
485could release twice in quick succession.
486
487I have occasionally seen the MAven reactor screw up (or perhaps it is
488the ANTLR tool) when building. If this happens you will see an ANTLR
489Panic - cannot find en.stg message. If this happens to you, then just
490rerun the build and it will eventually work.
491
492Jim Idle - March 2009
493
494