Currently (as of version 2.13.7), the only way to use another language than the default for the notenames is:

\include "italiano.ly"

This has two downsides: it's not user-friendly at all, and cannot be used in safe mode.

It has been proposed (you may want to read the thread starting on http://lists.gnu.org/archive/html/lilypond-devel/2009-05/msg00254.html for instance) that a more convenient shortcut could be used.

Several shortcuts have been suggested:

\notelanguage "italiano" { ... } (only applies to one block)

\lang "italiano"

or even: \italiano

I am ready to work on a patch as soon as we define what the policy would be: shall we simply add the new shortcuts on top of the existing syntax or shall we deprecate the \include-based syntax, shall we wait for a major version bump, shall we write a convert-ly rule or not, etc.

The fact that it cannot be used in safe mode is only one manifestation of a more fundamental issue. \include "italiano.ly" is semantically poor, because it uses a catch-all construct that does not express what the user intention is. In other words, it's a hack that should be promoted to a first-class construct.

I definitely support this and I'm happy to help with the implementation, although I would not imagine it taking long to develop even by one person.

My take on design:

1. I would favour \lang "italiano", but I am not too fussed. Adding one command for each language that we support now and in the future seems excessive and more difficult to maintain.

2. I would prefer a block-scoped command, but we can start with a non-scoped command if it's easier, and we can have both variations in the final version.

3. The current syntax is already self-deprecating. If your question is whether we should explicitly error on \include "italiano" going forward, I think we should not. It would be both cumbersome to implement and maintain, and it would break backward-compatibility with precious little upside. Of course the manual(s) will make it clear that going forward the new construct should be used, and only mention the old construct as a historical note.

4. I think this change is simple, small and safe enough to start coding ASAP. Waiting for a major release is not justified in my opinion.

Thanks

In keeping with LilyPond style we should use

\language "italiano" or \noteLanguage italiano

We should not use

\lang

because we don't use abbreviations, contractions, or other shortened words.

Of course we will need a convert-ly rule, even if it's a NotSmart rule (don't know if my capitalization is correct).

Block scoping would probably be nice, but I expect it to require a major change, because right now LilyPond does not have block scoping, IIUC.

If block scoping is implemented, there should probably be both block scoping constructs and state changing constructs, IMO.

e.g. \setNoteLanguage "italiano" would change the state to italian for the parser, without any block limitations, while \useNoteLanguage "italiano" {block} would parse block with italian as the input note language.

But I think the best way to approach this (like all enhancements) is to work on a patch, and post it for review, and then get feedback.

Thanks,

Carl

It doesn't need a new command, only a bit of reorganization. The command is there:

(ly:parser-set-note-names parser pitchnamesItaliano)

Only pitchnamesItaliano should be defined upon startup. For example if lilypond had the switch to parse certain file before running, it would make perfect in performance sense as well: lilypond -DincludeFiles=italiano.ly,deutsch.ly

Then you can use the above command to set the note names anywhere.

Thanks fodber,

I think the ability to include files from the command line at startup may have its own merits, but in this case my feeling is that it would make the existing hack a worse one.

Although your proposal would fix the important point of usability in safe mode, I don't see a great syntax usability improvement from

\include "italiano.ly"

to

#(ly:parser-set-note-names parser pitchnamesItaliano)

As for performance, I don't think your proposal would make such a massive change, and in fact if we are worried about performance we should probably encode the note names in a static C++ language-to-note-names map, but is there any point?

I broadly agree with Carl's comments.

I'm not fussed about having block scope, since it's easy enough to implement this as a music function, thus allowing it to be set at top-level and in music.

It is however, a bit more complicated than just doing

(ly:parser-set-note-names parser dutchPitchnames)

since the identifier `pitchnames' must also be set to the pitchnames alist each time (it's reserved by the parser for use with \notemode).

Cheers, Neil

As for the syntax, my choice would be

\language "italiano"

because it's simpler, quicker and we don't need to change it in case we introduce localisations other than the note names. But I don't really care, and it can always be easily changed around once we have a patch to submit for review.

Ready to go then?

My only caution is that we might want to save this change until 3.0. That said, it should be relatively easy to do this in convert-ly, so I guess it's not a big deal.

Oh, various IDE authors might complain. I'm happy to ignore them and just point them at you. If you're prepared for that, then there's no problem. :)

I really cannot see any risks involved to warrant waiting for a major release. If you disagree, please come forward with specific concerns.

Client applications have nothing to complain about, because this change will be completely backward-compatible. (And sure if they do complain, feel free to send them to me.)

Thanks

A major change in general input syntax, even optional, should be accompanied by a noticeable version number change.

I don't mean it in a confrontational way, but may I ask what the rationale is please?

"Tool x can process Lilypond files of version y or later" - that's sensible for an announcement.

convert-ly also has to work depending on version numbers. A syntax change that may make a whole file (rather than just some small parts) utterly unparseable to an external program is not a small change.

Would the language feature be feasible to implement as a music function with no arguments?

(the function implementation has changed since I last looked at it, so I am not sure.)

Thanks. Again, I am most likely wrong because I'm new here, but I'm trying to understand. What do you mean by "noiceable version number change"? 2.14? 3.0?

"Tool x can process Lilypond files of version y or later"

This can never be strictly true, else it means that all changes in lilypond syntax must be forward-compatible, which according to the first line of the convert-ly documentation, historically they have not been.

Indeed, convert-ly --show-rules seems to confirm this, since it outputs a number of minor-release transformations that are not forward-compatible, or at least that's my interpretation, and incidentally I wonder if they are even backward-compatible. One of them chosen at random is

2.7.10: \applyxxx -> \applyXxx

(I am assuming that \applyXxx did not exist prior to 2.7.10.)

If on the other hand there is an explicit agreement between lilypond and dependent tools that non-forward-compatible changes to the syntax must occur in a major lilypond release, then fair enough. If not, we may want to play nice and give the various dependent tools some notice, or negotiate a deadline for the inclusion of this syntax change. But again I would ask, what has been done in the past?

Just to be clear, since the proposed change is backwards-compatible (lilypond will always allow \include), convert-ly for this command will have to do nothing to convert from an old version to a new one.

By the way, does convert-ly also do downgrades? It's not too clear from the documentation. If so, in order to convert a new version to a previous one it needs to basically substitute \language "xxx" with \include "xxx.ly", after checking that xxx is in the list of supported languages.

Thank you

Historically, we have changed the syntax whenever the whim struck us. This policy is changing; there will be no syntax changes within stable numbers (i.e. 2.12.x, 2.14.y). As for "client programs", I'm thinking of things like lilywizardtemplate (or whatever it's called). If we change the format of \include "language.ly", then this tool will break. That's not a huge problem, but we shouldn't do it randomly.

In addition, lilypond 3.0 (estimated 12-18 months from now) will have a stable syntax for normal, non-tweaked music. (quite what that means will be determined later)
Users and IDE writers will need to change their scores and programs when this comes out; I'm not certain that it's worth changing the language command in the near future, and then change it again a year later.

there will be no syntax changes within stable numbers (i.e. 2.12.x, 2.14.y).

Sorry, what do you mean? According to the download page, currently 2.12.2 is stable, and 2.13.x is not. Does it mean that we can have a syntax change in 2.13.x+1, but not in a potential 2.12.3? I just think that postponing this change for 12-18 months is not backed up by our risk analysis so far. Also, not knowing anything about 3.0, I don't understand why the \language syntax would need to change again in that version.

As for "client programs", I'm thinking of things like lilywizardtemplate (or whatever it's called). If we change the format of \include "language.ly", then this tool will break.

OK, I need to understand this better. Could you please give me a link that explains what this tool does? How is it going to break exactly? Again, sorry to insist on this point, but please note that we are not really changing the format of \include "language.ly", we are just extending the syntax a new, better way of doing it. The worst that I can imagine happen so far is that an IDE fails to highlight the new syntax if the user upgrades the back-end but not the front-end; or if the front-end development is not quick enough to pick up the back-end change. I think that's a risk worth taking, since the upside is to have a \language construct that works in safe mode, apart from other cosmetic considerations.

Thanks.

Oooh, I just re-read this bit. Before it gets misinterpreted:

not knowing anything about 3.0

I was referring to myself, not you guys of course.

Cheers

LilyPondTool parses the input file to do syntax checking. If you change the way languages are defined, it will be broken and will have to be fixed.

Also it has a wizard, that generate \include "language.ly" at the beginning. It also should be changed.

In general, actually you are trying to introduce a new syntax instead something that is very well known, and working almost perfectly well. Safe mode note name changing can be achieved by other means.

Thanks, Bertalan.

veryfurryfur: yes, there can be syntax changes in 2.13.9, but not in 2.12.3. This is no negotiable. There can be syntax additions, but I don't like the idea of having a \language command and the old \include method. If we want to move to \language (which is a good idea), then we'd remove the language definitions from the ly/ dir.

The only two questions are: 1) What will be the actual name? (i.e. \noteLanguage vs. \language) 2) When should this change occur?

1 could be debated right now, but I think that GLISS is the best format to have such

debates. #2 should occur when we have major other changes, namely lilypond 3.0.

LilyPondTool parses the input file to do syntax checking.

I see, ok this makes sense. But still, is it really going to take 18 months for dependencies to pick up a change in syntax? How did LilyPondTool cope with previous changes? Is there a protocol in place? (E.g. we notify them in advance, they build the necessary changes in preparation for our release, etc.) It seems like we are going from one extreme (whenever) to the other (18 months)...

Safe mode note name changing can be achieved by other means.

I'm interested (as a work-around). Could you please let me know, perhaps in private communication (just add @gmail.com)? Thanks.

I don't like the idea of having a \language command and the old \include method

Can you elaborate on your preference please? To me it seems like throwing away backwards compatibility for no particular benefit. I agree we should deprecate, but still support the old syntax, and get rid of it in the next major version. Seems standard procedure for this kind of things...

Thanks.

We could do it immediately and have it reflected in 2.14 in a few months, but since this will require an amount of rewriting for other tools, I figured it would be polite to wait. 3.0 will definitely require tool modification; as it stands right now, I don't think that 2.14 will require any tool modification. Why not make all the changes at once?

Why not make all the changes at once?

If it is true that it's possible to change note names in safe mode via some sort of work-around (please let me know), then I agree it's not so urgent and it falls into the "pure cosmetic" bucket that can wait for 3.0.

Otherwise, there is a functional/usability element to it that should be addressed as soon as reasonable.

I don't think that 2.14 will require any tool modification.

If this is the only planned syntax change that is not purely cosmetic, then it would be one argument in favour of waiting for 3.0.

veryfurryfur: yes, there can be syntax changes in 2.13.9, but not in 2.12.3. This is no negotiable. There can be syntax additions, but I don't like the idea of having a \language command and the old \include method. If we want to move to \language (which is a good idea), then we'd remove the language definitions from the ly/ dir.

IIRC, we added the functionality to support the new crescendo commands, thus making it possible to use them, without eliminating the old crescendo commands.

Why could we not do the same with language selection, particularly since there is an immediate need to have language selection in safe mode?

Thanks,

Carl

Carl: I don't see why not, since it would be easy to deprecate the \include syntax while keeping the language files until such time as we decide to remove them completely; all that's required is to clear out the contents of each file and replace them with the new command(s).

Attached is a patch that suggests a much simpler solution than what has been discussed above.

This commit adds a new file, lang-init.ly, that introduces new shortcut commands for notenames language selection: instead of using \include "italiano.ly", just use \langItaliano, and so on.

Standalone shortcuts have been defined for all available languages. A different syntax has been considered, such as

\lang "italiano"

However, since the only way to pass arguments is currently through a music-functions, and since you can't have toplevel LilyPond definitions inside a music function, this would require to modify either the parser (a rather heavy change) either the italiano.ly&co files, which would break compatibility with existing code. The way I chose was the lightest, safest and the most straightforward.

For now, full compatibility with the old syntax is preserved. Once people get used to the new syntax, we may consider a slightly different implementation (or not). For instance, if we were to implement the \lang "italiano" solution, then introducing it would only be a matter of changing lang-init accordingly: langItaliano = \lang "italiano"

If this proposal is accepted, of course I'll take care of updating the docs, and I'll open a new issue to gracefully deprecate the old syntax (writing convert-ly rules, etc.) -- for which the GLISS process would be an appropriate time.

Also on Rietveld: http://codereview.appspot.com/2606042

The \language command has been implemented with commit 02fe038744e634b42f1a3377c4f0dc3d25e80344.

Simply brilliant.

Patch has been reverted: it caused a memory leak, and was indeed hackish anyway.

New patch. http://codereview.appspot.com/2699041/

(Yes, I know, the suspense is unbearable... ;-)

This was pushed as 5e25fb3bf792636e329bc1a063ef82f67d181a63

Oh, indeed.