It needs to support extra params like:
--plugins
(like --plugin=protoc-gen-java_rpc=LOC)
and open ended parameters like:
--java_rpc_out=nano=false:${project.build.directory}/generated-sources/protoc

that seem to be plugin specific

Who has the permission to push an update to Maven Central?

more active fork is at github.com/dtrott/maven-protoc-plugin

Maybe the best is to ask the 3 or 4 developers there if they would agree to a contributor agreement and we merge back the changes in this repo (or we start from a clean moved repo to github Google account?
This way, we can have open source contributions, while pushing the latest bits to Maven Central in the Google namespace?

Filed dtrott/maven-protoc-plugin#27

The maintainers of protobufs have permissions for com.google groupId in Maven repository. I will start a branch and pull the code of existing work. Thanks for starting a thread.

Great! I'm happy to help, we need to make sure Googlers are backing and supporting long term this, i.e for any protoc changes, make sure the Maven plugin is up to date.
We also need to see how the compiler can be easily accessible from the plugin (i.e automatic download or a new install goal, on demand)

Unfortunately, there's no good way to distribute the protoc binaries through normal Maven distribution channels, for a simple reason: it is a native application, which has to be built for the target system and architecture. There's no pure java version of protoc available, and to the best of my knowledge, it has never been considered a possibility (as far as I remember, the protobuf team promptly declined the idea because of an overwhelming complexity and an massive overhead associated with maintaining two parallel java and c++ implementations). We worked around that limitation in the maven plugin by utilising Maven toolchains mechanism, which still requires a manual protoc installation and a manual configuration of toolchains.xml, but the benefit of toolchains is that they decouple tools configuration from your project configuration, and you can use versions or version ranges to pick up the desired version of the tools (e.g. protoc). I.e. one of your projects may declare that they need protoc and protobuf library version 2.4.1, and another one may request any version from the [2.6,2.7) range, and provided that there are matching toolchains for both, it will use Maven version resolution magic to resolve both the dependencies and tools in a consistent way.
Please correct me if my knowledge is outdated.
BTW, is grpc protoc plugin a native or a java application?

The gprc Java protoc plugin is also a native binary.

On Wed, Mar 11, 2015 at 7:49 AM Sergei Ivanov notifications@github.com
wrote:

Unfortunately, there's no good way to distribute the protoc binaries
through normal Maven distribution channels, for a simple reason: it is a
native application, which has to be built for the target system and
architecture. There's no pure java version of protoc available, and to
the best of my knowledge, it has never been considered a possibility (as
far as I remember, the protobuf team promptly declined the idea because of
an overwhelming complexity and an massive overhead associated with
maintaining two parallel java and c++ implementations). We worked around
that limitation in the maven plugin by utilising Maven toolchains
mechanism, which still requires a manual protoc installation and a manual
configuration of toolchains.xml, but the benefit of toolchains is that
they decouple tools configuration from your project configuration, and you
can use versions or version ranges to pick up the desired version of the
tools ( e.g. protoc). I.e. one of your projects may declare that they
need protoc and protobuf library version 2.4.1, and another one may
request any version from the [2.6,2.7) range, and provided that there are
matching toolchains for both, it will use Maven version resolution magic to
resolve both the dependencies and tools in a consistent way.
Please correct me if my knowledge is outdated.
BTW, is grpc protoc plugin a native or a java application?

—
Reply to this email directly or view it on GitHub
#230 (comment).

Indeed it is, now that I've browsed the grpc codebase. Going forward, this is going to be another major PITA from the Maven integration point of view. Sure, one can put the binary on the %PATH% or $LD_LIBRARY_PATH, but that completely deprives Maven build of any control over selection of versions of the binaries. Unfortunately, binary dependencies as a class do not fit into Maven dependency management and resolution mechanism. A limited degree of control is provided through toolchains, and that is probably the best one can get with Maven toolset. I reckon grpc people may need to consider creating a Maven toolchain implementation for grpc protoc plugins, and Maven plugin will need a way to resolve the locations of protoc plugin binaries through the toolchain mechanism. Our version of the maven plugin allows to run java-based protoc plugins by resolving them as Maven dependencies and automatically wrapping them in winrun4j binary or a shell script, but I am afraid there's currently no generic way to deal with native applications in Maven plugins.

Also, could anybody provide me a link to protobuf documentation, where it explains in greater detail how the protoc plugins are invoked and configured on the command line. I remember that I came across it before, but I cannot find it anymore. Any useful pointers will be greatly appreciated.

Is the colon-separated list of parameters (as in the grpc example above) a generic way of passing parameters to protoc plugins, or is it something specific to grpc:

--java_rpc_out=nano=false:${project.build.directory}/generated-sources/protoc

On Wed, Mar 11, 2015 at 12:46 PM, Sergei Ivanov notifications@github.com
wrote:

Also, could anybody provide me a link to protobuf documentation, where it
explains in greater detail how the protoc plugins are invoked and
configured on the command line. I remember that I came across it before,
but I cannot find it anymore. Any useful pointers will be greatly
appreciated.

It can be found in the API doc generated from the source code:
https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.compiler.plugin

Is the colon-separated list of parameters (as in the grpc example above) a
generic way of passing parameters to protoc plugins, or is it something
specific to grpc:

This works for any protoc plugins.

--java_rpc_out=nano=false:${project.build.directory}/generated-sources/protoc

—
Reply to this email directly or view it on GitHub
#230 (comment).

Thanks, I did not realise it was buried inside C++ API documentation. Has the optional parameter always been there, even in protobuf v2? I.e. are v2 and v3 fully compatible in terms of command line interface? Is there some character escaping supported for the parameter?

On Wed, Mar 11, 2015 at 4:21 PM, Sergei Ivanov notifications@github.com
wrote:

Thanks, I did not realise it was buried inside C++ API documentation. Has
the optional parameter always been there, even in protobuf v2? I.e. are v2
and v3 fully compatible in terms of command line interface?

Yes. This feature is available in v2 and it's not changed in v3.

Is there some character escaping supported for the parameter?

No escaping is supported. protoc passes exactly the same char sequence it
gets to the plugin.

—
Reply to this email directly or view it on GitHub
#230 (comment).

@xiaofeng-gg Thanks a lot for your help. Now I have enough information to proceed with an implementation in my Maven plugin fork, and I hope I'll be able to conjure something up in the next few days.

@ludoch I hold you to your promise of organising a proper takeover on Google side. We need to make sure it is done properly and that there will be a guaranteed continuity in development. Please contact me at sergei_ivanov@mail.ru if you'd like to discuss the detailed arrangements.

@sergei-ivanov I talked with the gprc team about the idea to distribute protoc and grpc plugin binaries for multiple platforms for each release. Do you think this would make it easier for the maven plugin? e.g. pick up the binary for the specific platform. We can fall back to use an installed version if no pre-built binary is found for the target platform.

On the one hand, protoc (as well as grpc plugin) are self-sufficient standalone executables that do not require any other .dll or .so to run, and they can be placed anywhere in the file system.

On the other hand, Maven is still pretty rubbish in dealing with binary dependencies. Solving the problem purely by Maven means, one will probably need to create a bunch of profiles that are activated under different OSes, use those profiles to pull down the correct binary from the repository, then pass the path to that binary to maven protoc plugin. Alternatively, maven protoc plugin may attempt to detect the version of protobuf library in the dependencies, then detect the operating environment, then download the correct protoc native binary and use it. However, that does not solve the problem with other native binary protoc plugins, such as grpc. It is probably going to be unwieldy. Honestly, despite my extensive Maven experience, I do not know a good Maven way of dealing with versioned native binary dependencies that are part of the build process. Toolchains proved to have been the best (or rather, "the least worst") option for us so far.

And then there is a harsh reality of working in large corporate environments (especially in the sectors like finance, where IT is not the core activity, and security and regulation reaches insane levels). Downloading of binaries is typically thwarted by corporate firewalls, and there is typically a blanket ban on accessing services like Bintray. Even worse, procuring gcc on a linux server may be an issue. I really wish there were a pure java version of protoc: that would have made life so much easier.

IMHO, there may still be some benefit in deploying protoc binaries into Maven central (using classifiers to denote OS/architecture), because it will save the effort of building them from the sources, and Maven central is well indexed, which makes finding the right version easy. You'll need to make sure that the most popular platforms and architectures are covered.

@pherl Actually, can you please ask grpc team to upload their latest protoc plugin binaries for win32 somewhere. I do not have MSVC to build them, but I have got a new snapshot version of maven plugin that should be able to integrate grpc into the build. I'd like to give it a whirl locally before I push the changes out to github. Thanks.

@zhangkun83

@ejona86 and I have been working on a solution for grpc plugin (grpc/grpc-java#71). Our plan is to upload the native plugins to Maven Central with an auto-detected classifier denoting the OS/arch, and make the protobuf plugins for Maven and Gradle download and use the proper grpc plugin. I have made a proof-of-concept change on gradle-plugin-protobuf and it works very well.

This solution would also work for protoc binaries. The protobuf plugins for Maven and Gradle may also deduce the protoc version from the protobuf dependency to make sure the protoc is compatible with the protobuf library.

@sergei-ivanov Here is the freshly compiled grpc plugin for win32.

Thanks, got it. I also took a look at the linked issue, and I think I'll be able to incorporate some ideas from there into the plugin. Unfortunately, with the way the plugin is structured now, introducing any new goals or settings makes the configuration even more unwieldy and confusing. I have a feeling that a major rewrite is imminent, but I guess we can still have a short term stop-gap solution.

I actually managed to get it all to work with grpc. I still need to clean it up and document before pushing, but I'll try to get it done as soon as I can. @zhangkun83 Was the binary you gave me a debug version? Asking that because it's quite a large file at 500k. I am curious if you could do me a favour and produce a minimal optimised win32 binary plugin that would output an empty java class with the name matching the proto. If the exe file ends up being reasonably small (<100k) then I'll be able to add it to the integration test suite.

No it's a release version. Unfortunately I don't have a windows machine. That one I gave you was built by my colleague.

Related: os72/protoc-jar-maven-plugin#4

The plugin has a new home, and it is finally available in Maven Central.
https://www.xolstice.org/protobuf-maven-plugin-0.5.0-released/

This is great news! Thanks a lot.