Summary

• Summary
• Introduction
• Install Xtext
• Test GAMA Xtext project
• Logic of our language
• Highlighting
• Label provider
• Dynamic keywords
• Validation
• Proposal provider
• Scope provider
• Outline
• Value converter
• Formater
• Project wizard
• Launching Framework Eclipse (not Xtext)
• Dynamic Languages Toolkit (DLTK)

Introduction

I will try to explain how to test the Xtext project here.

First, you should have a look at the Xtext on eclipse.org

"Xtext is a framework for development of programming languages"

Install Xtext

See: http://www.eclipse.org/Xtext/download

if you want a ready-to-use Xtext packaged in Eclipse, see xtext.itemis.com

(I'm using eclipse-SDK-3.6-xtext-1.0.0-win32.zip + XML Editor (WST) for GAMA)

Test GAMA Xtext project

So, Xtext let you define a grammar in an ANTLR style (not fully ANTLR, but close to it)

Model :
  (elements += Entity)+;
Entity:
  'entity' name=ID value=INT;

for example here, the += let you define a list (org.eclipse.emf.common.util.EList)

(see Gaml.xtext)

You should checkout the SVN now to get the two projects msi.gama.lang.gaml and msi.gama.lang.gaml.ui

Once you defined a valid grammar, you generate some code through the GenerateGaml.mwe2 file

This step will generate (in src-gen folders)

• a set of Java class representing the meta-model of your Xtext grammar,
• the ANTLR grammar,
• the ANTLR parser
• and last but not least, the code of a ready-to-use IDE (as an Eclipse plugin).

Then you simply launch a new Eclipse Application from your Xtext project, and you'll be abble to play with your new language ! easy

And you'll get something like:

If you enjoyed, you can have a look at the feature of the next release

Logic of our language

Here we define a very simple grammar in order to let us update the language dynamically.

// a model is made of multiple lines of code
model:
  (modeldef+=syntaxbase)*;

syntaxbase:
  (kw=keyword (expr=Expression (facets+=facet)*)?) ((b=block)|';');

facet:
  kw=facetkw expr=Expression;

block:
  '{' (body+=syntaxbase)* '}';

// different class here for highlighting, warning and proposal
keyword:
  value=ID;
facetkw:
  value=ID;

And for the definition of Expression, we used JavaFX ANTLR definition that we adapted for Xtext.

• http://openjfx.java.sun.com/current-build/doc/reference/ch06s03.html
• http://kenai.com/projects/openjfx-compiler/sources/soma-patch/show/doc/reference
• http://kenai.com/projects/openjfx-compiler/sources/soma-patch/show/src/share/classes/com/sun/tools/javafx/antlr

Have a look at the grammar here (msi.gama.lang.gaml/**/Gaml.xtext)

Highlighting

• Xtext documentation
• msi.gama.lang.gaml.ui/**/highlight/GamlSemanticHighlightingCalculator.java
• msi.gama.lang.gaml.ui/**/GamlUiModule.java

The AbstractGamlUiModule class is used to register components to be used within the IDE, here the Highlighter. In the GamlSemanticHighlightingCalculator we implement provideHighlightingFor from ISemanticHighlightingCalculator, which receive XtextResource where we'll find a reference to all nodes of the model. Then for each node, we generate a specific style via acceptor.addPosition regarding on the node's type (instanceof).

// for each node, if this node is a keyword (ID) apply bold font
if (abstractNode.getElement() instanceof keyword) {
	acceptor.addPosition(abstractNode.getOffset(), abstractNode.getLength(), 
			DefaultLexicalHighlightingConfiguration.KEYWORD_ID);
}

Label provider

• Xtext documentation
• msi.gama.lang.gaml.ui/**/labeling/GamlLabelProvider.java

Here we can easily associate a type of the meta-model to the Outline UI (associated text and icons).

// syntaxbase : keyword.value
String text(syntaxbase ele) {
	return ele.getKw().getValue();
}
// keyword : value
String image(keyword ele) {
	return "_"+ele.getValue()+".png";
}

Icons should be in the msi.gama.lang.gaml.ui/icons folder.

Dynamic keywords

• msi.gama.lang.gaml/**/GamaKeywords.java

In order to give us the possibility to upgrade the language (by adding methods and data-models), we want to differentiate language's keywords from the grammar, so for all the user interface development tools we'll need to link both... This class aim to manage this link in a proper way. Somehow is a part of the MVC pattern, making link between the Model and the View.

Validation

• Xtext documentation
• msi.gama.lang.gaml/**/validation/GamlJavaValidator.java

Here, we can define different Check in order to validate the code (syntax analysis).

@Check
public void checkKeywordExists(keyword kw) {
	// kw: keyword, kw.eContainer(): syntaxbase
	if (!GamaKeywords.getInstance().isKeyword(kw.getValue(), getContextList(kw.eContainer()))) {
		warning("unknown keyword '"+kw.getValue()+"'", GamlPackage.KEYWORD);
	}
}

We check if we know the keyword, and if not, we produce a warning (which could as well be an error())

We call the singleton GamaKeywords for testing this, passing the context list to it.

The context-list is made like:

private List<String> getContextList(EObject container) {
	List<String> lcontext = new ArrayList<String>();
	while (container instanceof syntaxbase) {
		syntaxbase s = (syntaxbase) container;
		lcontext.add(s.getKw().getValue());
		// get the container of the parent of this syntax
		container = container.eContainer().eContainer();
	}
	return lcontext;
}

As we work with a graph, each node know his children and his parents, which is quite convenient to navigate through the model.

Proposal provider

• Xtext documentation
• msi.gama.lang.gaml.ui/**/contentassist/GamlProposalProvider.java

Here we add proposals for specific types depending on where we are in the code (the context).

TODO list

Scope provider

• Xtext documentation
• msi.gama.lang.gaml/**/scoping/GamlScopeProvider.java

You can read on this blog post more about scoping.

"So where other frameworks create a tree, Xtext also takes care of the cross-links, hence creates a graph (a.k.a model)." ^1^

"Xtext can read and write textual models into Ecore meta models be they dynamic or static." ^2^

TermLiteral
	: {VarRef} value=[VarRef|ID] // here we should specify that we can get either a ref or a new var
	| ...
	;

OR

TermExpression
	: VarRef
	| ...
	;

VarRef:
	value=[syntaxbase|ID];

warning(200): ../msi.gama.lang.gaml/src-gen/msi/gama/lang/gaml/parser/antlr/internal/InternalGaml.g:211:3: Decision can match input such as "{RULE_STRING..RULE_ID, RULE_INT..RULE_BOOLEAN, '(', '{{', '[', 'not'..'null'}" using multiple alternatives: 1, 2

As a result, alternative(s) 2 were disabled for that input

see Xtext FAQ

TODO Work on this

http://www.eclipse.org/forums/index.php?t=msg&th=167921&start=0&

Outline

• Xtext documentation
• msi.gama.lang.gaml.ui/**/outline/GamlTransformer.java

TODO Work on this

Value converter

• Xtext documentation
• msi.gama.lang.gaml/**/convert/GamlValueConverters.java
• msi.gama.lang.gaml/**/GamlRuntimeModule.java

terminal COLOR returns java.awt.Color we should specify that this returns something else than String or we'll get cast exception in ColorLiteralImpl.eSet(int, Object) : 124 "setColor((String)newValue);"

TODO find a way to put Color in Ecore, or find another class to fit the cast cf "terminal COLOR returns java.awt.Color" Gaml.xtext:164 ^{[http://code.google.com/p/gama-platform/source/browse/branches/msi.gama.lang.gaml/src/msi/gama/lang/gaml/Gaml.xtext#169 Gaml.xtext:169]}

Formater

• Xtext documentation
• msi.gama.lang.gaml/**/formatting/GamlFormatter.java

TODO See why it's not working / Debug (CTRL+SHIFT+F)

Project wizard

• Xtext documentation

TODO Work on this / read more / code it

Launching Framework Eclipse (not Xtext)

• Launching Framework at Eclipse.org
• msi.gama.lang.gaml.ui/**/launch/GamaLaunchConfigurationDelegate.java
• org.eclipse.debug.core.model.LaunchConfigurationDelegate

Maybe interesting to reuse the Eclipse launching / debug framework...

TODO Work on this / read more / code it

Dynamic Languages Toolkit (DLTK)

• http://www.eclipse.org/dltk/
• http://wiki.eclipse.org/A_guide_to_building_a_DLTK-based_language_IDE

TODO Read about it

http://www.eclipse.org/forums/?t=msg&th=168162

• GIS & Agent based Modelling Architecture
• Eclipse TMF Xtext

http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dltk/?root=Technology_Project http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/?root=Modeling_Project

see ANTLR predicate https://wincent.com/wiki/ANTLR_predicates

see Xtext ANTLR backtracking http://wiki.eclipse.org/Xtext/FAQ#OK.2C_but_I_didn.27t_get_these_warnings_in_oAW_Xtext.C2.A0.21

http://www.eclipse.org/forums/?t=msg&th=168162

http://wiki.eclipse.org/index.php?title=Xtext/planning_0.8.0&oldid=166900#Xtext_Grammar_2

• add support for semantic predicates
• add support for syntactic predicates

TODO: http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.mwe/plugins/org.eclipse.emf.mwe2.language/src/org/eclipse/emf/mwe2/language/Mwe2.xtext?root=Modeling_Project&view=markup

DeclaredProperty:
  'var' (type=[types::JvmType|FQN])? name=FQN ('=' default=Value)?;

voir si [types::JvmType|FQN] pourrait etre une liste dynamic et pas poser de pb pour la gen Xtext.... http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/org.eclipse.xtext/plugins/org.eclipse.xtext.common.types/model/JavaVMTypes.ecore?revision=1.7&root=Modeling_Project&view=markup

http://www.antlr.org/grammar/1153358328744/C.g

type_id
    :   {isTypeName(input.LT(1).getText())}? IDENTIFIER
//    	{System.out.println($IDENTIFIER.text+" is a type");}
    ;
[...]
@members {
	boolean isTypeName(String name) {
		for (int i = Symbols_stack.size()-1; i>=0; i--) {
			Symbols_scope scope = (Symbols_scope)Symbols_stack.get(i);
			if ( scope.types.contains(name) ) {
				return true;
			}
		}
		return false;
	}
}

backtrack Xtext 1.0 : http://www.eclipse.org/forums/?t=msg&th=164984

import de.itemis.xtext.antlr.*

...

			fragment = XtextAntlrGeneratorFragment {
				options = auto-inject {
					backtrack = true
				}
			}

...

			fragment = XtextAntlrUiGeneratorFragment {
				options = auto-inject {
					backtrack = true
				}
			}