Question: why not use Qt's built-in Javascript support? This would minimize the number of dependencies. Also, I 
understand that Qt makes it possible to expose any object to the scripting engine, as well as easily create GUI 
dialogs, etc. 

The attached patch implements (rudimentary, but working) LuaScripting.

The details are documented in the attached doc file (it's in the wiki format, so it
should be possible to put it onto this page if Jonathan likes this patch). Currently,
only the AfterTypeset hook is available (I hope someone with more knowledge about Lua
than me will code a log parser with it ;)) and only the current selection in the TW
window can be changed by scripts.

This obviously introduces new dependencies. On my Ubuntu Linux this could be
satisfied with
{{{
sudo apt-get install liblua-5.1-dev
}}}
Not sure about MacOS and Windows (here my guess is that you need to download
lua5_1_4_Win32_dll8_lib.zip from http://luabinaries.luaforge.net/download.html,
unpack it, put the files from the "include" directory into your <TW>/src folder to
compile and the dlls into the folder of your executable to run; if someone can
confirm this I'll update the BuildingOnWindows docs).

For the record, the following files were added and should hence get an "svn add":
{{{
  res/resfiles/scripts/
  res/resfiles/scripts/parseLaTeXLog.lua
  res/resfiles/scripts/toggleBold.lua
  src/LuaInterface.cpp
  src/LuaInterface.h
  src/LuaScript.cpp
  src/LuaScript.h
  src/ManageScriptsDialog.cpp
  src/ManageScriptsDialog.h
  src/ManageScriptsDialog.ui
}}}

issue27-lua-scripting.diff 44.0 KB   Download

Lua Documentation.txt 4.1 KB   Download

@Windows: Sorry, I forgot one detail. The zip file I mentioned for downloading
includes two .dll and two .lib files. We'll use the lua5.1.dll and lua5.1.lib. For
compiling and linking the application, you'll need the .lib file. Put it in a place
where you find it again (e.g. C:\texworks) and add the following line to TeXworks.pro
to the windows section (where you've made changes before if you followed the
BuildingOnWindows guide):
LIBS += C:/texworks/lua5.1.lib
(change the paths according to your setup).

Then you need to remove your C++ project files and rerun qmake

Scripting looks very promising!

Just a thought: in order to minimize dependencies, instead of actually embedding a lua interpreter, how about 
running luatex with the --luaonly option? There are two advantages.

* First, no additional dependencies. Compiling TeXworks on non-Linux platforms is a chore as it is
* Second, luatex is installed by recent versions of MikTeX and TeXLive, so it's basically available everywhere, 
"for free". 

The idea would be to run lua scripts much like we now invoke tex & friends, with some conventions to trasnfer 
data from/to TeXworks. Here, we can follow what TextMate does on OSX:

* it sets a bunch of environment variables, e.g. the current file name, the working directory, the currently 
selected word, the current line, the position of the caret, etc.
* it provides a way to send input (e.g. the currently selected word) to stdin and specify what to do with stdout 
(e.g. show it in a separate file, insert it at the current position, display as a HTML page).
* finally, it uses a standard way (CocoaDialog) to put up a simple GUI (e.g. a dialog box or drop-down list) if 
necessary.

This is extremely effective. TextMate is, IMHO, very effective with Latex editing. Also, I suspect providing the 
above "glue" requires much less pervasive changes to the TeXWorks code.

Any thoughts?

As pointed out by Alain Delmotte, the patch I posted accidentally contained some Qt
4.5 features, making compilation impossible on many systems. This patch should also
work with Qt 4.3

issue27-lua-scripting-qt43.diff 43.7 KB   Download

(No comment was entered for this change.)

Due to some requests to support Lua, I've resurrected my original Lua implementation
to work with the new scripting framework (see attached patch). The result is shinier,
better, and (almost) as powerful as the QtScript scripting.

To use the new lua plugin, start by applying the patch. Then build Tw normally. Next run
qmake -t subdirs
make
qmake
to build the lua plugin. Finally copy the .dll/.so/.dylib file from
plugins/TwLuaPlugin to somewhere where Tw can find it (in my case, this was the
directory containing the executable/the working directory).

The good news: many more properties and methods are available now. If I understand it
correctly, all scriptable objects should be available in Lua now (subject to the
limitations mentioned below).

Any testing (in particular on non-Linux machines) is greatly appreciated!

Here a few technical/implementation-specific notes:
* The same things rearding installing the lua libs as mentioned above apply to this
patch.
* As mentioned, the lua code is contained in a plugin. This way, it doesn't have to
be loaded for people who don't want it/need it/have doubts about lua. This required
moving some classes from TWScriptable.* to TWScriptableInterface.* so the main app
and the plugin can share definitions
* Note that plugins must be built for the system they are run on (i.e., they have to
built separately for Windows, Linux, Mac, just as the Tw app itself). Also note that
plugins won't run with applications built with a different (lower) Qt version. This
shouldn't be a problem normally, though.
* The new files (to be svn add'ed in case this is comitted) are:
plugins/TwLuaPlugin/TwLuaPlugin.cpp, plugins/TwLuaPlugin/TwLuaPlugin.h,
plugins/TwLuaPlugin/TwLuaPlugin.pro, src/TWScriptableInterface.cpp,
src/TWScriptableInterface.h
* For the plugin to find some functions in the Tw executable (mostly related to meta
methods of the shared TWScript) it was necessary to add the new linker flag
QMAKE_LFLAGS += -Wl,--export-dynamic. This should work for all gnu-based linkers, but
I'm not sure how to acchieve the same thing for MSVC (I didn't test it yet). In case
linking fails because of this flag, simple comment it - everything should work fine,
but the lua plugin won't load.
* I had to make the TWScript::run method non-constant. This is because the lua
scripts modify the lua state at run-time.
* Lua scripts can access Tw via the global "TW" variable. It is the same as the
"target" variable in QtScript scripts.
* Lua scripts can return a value through the global "result" variable.
* Currently, only basic types (bool, int, double) and strings (QString) are
supported. Support for arbitrary QObjects should be added in the future, but ATM the
basic types should be enough for many nice applications
* Return values from methods are currently not supported
* Only properties/methods available through the meta object system are available in
Lua. I think these are all properties defined by Q_PROPERTY and all public slots, but
I'm not sure yet.

issue-27_lua-plugin.diff 26.8 KB   Download

Wow - that sounds very cool. I haven't looked at the implementation yet, but if it's able to access exposed 
properties and methods via the Qt meta objects, I think that's the right way to go. That should allow the bulk 
of the application to be scriptable via either language, for no extra effort or maintenance burden.

There are a few build issues I'll want to look into before committing this to the main repository; I have not 
tried building Lua on the Mac, for example (with luck, it'll be trivial!), and on Windows there'll be the issue 
that I have been static-linking Qt for my "release" builds, but that prevents dynamic plugin loading.

Overall, though, I think this sounds awesome, and I'll try to look at it more closely over the Christmas period. 
Thanks!

Attached is an updated version of the lua plugin patch. All comments from above still
apply. I improved the conversion of data between C and lua format, though. It is now
possible to pass arrays and hashes between C and lua. This is necessary e.g. for the
AfterTypeset hook, which requires (nested) lists. This patch replaces the previous one.

A few definitions:
*) array
On the C side of the code, this is a QVariantList with zero-based indexes
On the lua side, this is a table with the indices 1..n for n entries
*) hash
On the C side, this is a QVariantHash
On the lua side, this is a table with string keys and simple values (i.e., strings,
numbers, arrays, hashes; no functions, threads, and similar stuff that can't be
translated to C)

issue-27_lua-plugin.diff 29.6 KB   Download

And one more gimmick to play with. A plugin for Python (my personal favorite
scripting language).

BTW: I also attached a sample version of Jonathan's "Make Bold" script in lua and
python, respectively, for easy testing.

It works similar to the lua plugin. All the interfaces, files, directories,
principles etc. are the same. Both the 2.x and the 3.x series of Python should be
supported.

Notes:
* There are currently only build flags for *nix in the .pro file. The flags for
Mac/Windows will be similar, but as I don't know how the files are named exactly I
didn't include any statements. They should be fairly simple to find out, though.
* If you want/must use Python 2.x, be sure to define the PYTHON2 directive (by
QMAKE_CXXFLAGS += -DPYTHON2). Otherwise you will get some errors with int types
(which were superseded by long in 3.x)
* If you want/must use Python 3.x, the plugin compiles correctly, but Tw crashes
complaining about unresolved symbols PyUnicode_* (or if you run into similar problems
with 2.x), add QMAKE_CXXFLAGS += -DPy_UNICODE_WIDE to your plugin's .pro file. This
defines unicode characters (the predominant form for strings in 3.x) to be wide (i.e.
4 bytes). Whether this is necessary or not depends on how your python dll/so/dylib
was compiled (it is necessary on Ubuntu).

issue-27_python-plugin.diff 33.3 KB   Download

makeBold.lua 292 bytes   Download

makeBold.py 286 bytes   Download

Even more awesome - I know Python is pretty popular, and this is a great demonstration of how readily other 
scripting languages can be connected to the same base functionality.

One thing I've wondered about - I have not yet looked closely enough at either the Lua or the new Python 
support, so maybe I'm missing something - is the "TW" object you're exposing, corresponding to the QtScript 
"target". Was there a strong reason to use this name? Or do you think we should call it "TW" in QtScript as 
well? It would seem preferable to keep the naming consistent across languages if possible.

Note that in QtScript, there are two significant objects exposed to the script: "target", which is the active (TeX 
or PDF) document at the time the script is called, and "app", which is the texworks application object, for 
access to various "global" properties and methods that are not associated with the specific document. In the 
Lua/Python interfaces, I've only noticed the equivalent of "target".

Readily? Well, that depends on the POV ;). No, in principle you're right. The main
problem is to dig into the C API of the respective scripting language (which I did
not achieve so "readily" for Python ;))

Yeah, "TW" corresponds to "target". There is no particular reason to call it that.
IIRC, I originally called it that under the assumption that I'd actually implement
and interface to the Tw application in general. And I missed the "app" object
altogether, though it should be fairly simple to expose that, too.

So, in general, I'm totally in favor of having the same name in all scripting
languages. However, I'd prefer to somehow make a distinction in nomenclature between
"normal" variables/code in the respective scripting language and variables/objects
that relate directly to Tw. Apart from making target and app properties of some
module/object (which would require a little more work, and maybe doesn't work so
easily in QtScript), I'd suggest something like TwTarget (or TwContext) and TwApp.

And yet another new version of the two plugins (lua+python). The attached patches
should be used instead of the previous ones.
Changes:
 * renamed target to TWTarget
 * added TWApp
 * moved moc-specific code to TWScript base class. This should make both maintenance
and adding new features/languages easier.
 * implemented proper handling of return values (of calls to TWTarget/TWApp methods)
 * added support for passing other QObject derived objects to the script (not the
other way round, though)
 * added new doParseHeader() methods to TWScript to simplify parsing of (text-based)
script files. This could be used for JSScript, too.
 * documentation (Doxygen compatible)
 * updated the sample scripts
 
BTW: I found out that methods don't need to be public symbols to be known to the
moc-system. Q_INVOKABLE should also do the trick (though I didn't try).

issue-27_lua-plugin.diff 42.6 KB   Download

issue-27_python-plugin.diff 45.7 KB   Download

makeBold.py 310 bytes   Download

makeBold.lua 317 bytes   Download

I have committed both the Lua and Python plugins to the main repository, with slightly adapted interfaces; they 
seem to work as expected for me in simple tests here, at least. Have not yet tackled the full range of build 
issues, so I'm not posting new binaries just yet, but this is looking very promising.

Stefan, please take a look at the adaptations I made, and let me know if I've broken anything, or if you think they 
were unhelpful; by committing the patches in this form, I don't mean that this is the final word, it's just to give 
us something we can continue to refine.

(Note that I have changed the "target" and "app" objects in the QtScript implementation to be TWTarget and 
TWApp respectively, so existing scripts will need to be updated when a new build appears.)

Looks fine at a quick glance.
A few things I noticed:
 * The python header should be included first (according to the python docs). So the
inclusion of "TWScript.h" should be moved down after it (this removes several
warnings from the build process; I haven't checked if it would have worked otherwise).
 * I think we should stick to Python 2.x for the time being (it's the one recommended
on the python page). And Python 3.x is not available from the repositories on older
*nix systems, AFAIK. This basically means commenting out the other lines in
TWPythonPlugin.pro for the time being. (we could also think about replacing the
preprocessor definition PYTHON2 by one for PYTHON3 (it's only used in the code once)).
 * It would be nice to have a preprocessor definition to hard-code the plugin path at
compile time, which is necessary for packaging (on *nix, the plugins should go in
/usr/lib/texworks/ instead of /usr/bin/plugins)

Ok, I moved the TWScript.h header in the python plugin (requires using a different preprocessor symbol to 
detect the Mac environment, where the Python include path is different).

I had already eliminated the use of the PYTHON2 define; using Python's predefined PY_MAJOR_VERSION instead.

I've provided TW_PLUGINPATH as a definition that can be set in the .pro file (default is /usr/local/lib/texworks); 
also, you can now use an env var to override this at runtime.

Hi, this is great!

I started packaging the plugins for Ubuntu. During the process, I found that my
plugin code is not compatible with Qt < 4.5 The attached patch should partially fix
and partially work around this. It also adds some comments to TWPythonPlugin.pro
(explanations re. why pkg-config is not used, and an example of how one could use
Python 3.x, which will probably become the standard eventually).

issue-27_qt43-compatibility.diff 4.8 KB   Download

I just noticed while updating the building instructions that it would be nice to add
install information to the plugins' .pro files. Something like
	target.path = /usr/local/lib/texworks
	INSTALLS += target
would enable the user to simply say `sudo make install` when finished building to
place the plugin where it's supposed to be. This would spare him the need to manually
create folders and link libraries.

Of course this is only useful on *nix, so it should go in a conditional clause.

Looks fine to me, committed as r.528.

And another compatibility issue. For whatever reason, in Qt 4.3 a QMetaMethod can't
be invoked directly. Instead, QMetaObject::invokeMethod has to be used.

The attached patch should fix this issue and also implement the suggestion above.

issue-27_qt43-compatibility2.diff 1.4 KB   Download

Thanks, committed this also.