This page describes exactly what Python definitions the protocol buffer compiler generates for any given protocol definition. You should read the language guide before reading this document.
The Python Protocol Buffers implementation is a little different from C++ and Java. In Python, the compiler only outputs code to build descriptors for the generated classes, and a Python metaclass does the real work. This document describes what you get after the metaclass has been applied.
The protocol buffer compiler produces Python output when invoked with the --python_out= command-line flag. The parameter to the --python_out= option is the directory where you want the compiler to write your Python output. The compiler creates a .py file for each .proto file input. The names of the output files are computed by taking the name of the .proto file and making two changes:
|
.proto) is replaced with _pb2.py.
--proto_path= or -I command-line flag) is replaced with the output path (specified with the --python_out= flag).
So, for example, let's say you invoke the compiler as follows:
protoc --proto_path=src --python_out=build/gen src/foo.proto src/bar/baz.proto
The compiler will read the files src/foo.proto and src/bar/baz.proto and produce two output files: build/gen/foo_pb2.py and build/gen/bar/baz_pb2.py. The compiler will automatically create the directory build/gen/bar if necessary, but it will not create build or build/gen; they must already exist.
Note that if the .proto file or its path contains any characters which cannot be used in Python module names (for example, hyphens), they will be replaced with underscores. So, the file foo-bar.proto becomes the Python file foo_bar_pb2.py.
The number 2 in the extension _pb2.py designates version 2 of Protocol Buffers. Version 1 was used primarily inside Google, though you might be able to find parts of it included in other Python code that was released before Protocol Buffers. Since version 2 of Python Protocol Buffers has a completely different interface, and since Python does not have compile-time type checking to catch mistakes, we chose to make the version number be a prominent part of generated Python file names.
The Python code generated by the protocol buffer compiler is completely unaffected by the package name defined in the .proto file. Instead, Python packages are identified by directory structure.
Given a simple message declaration:
message Foo {}
The protocol buffer compiler generates a class called Foo, which subclasses google.protobuf.Message. The class is a concrete class; no abstract methods are left unimplemented. Unlike C++ and Java, Python generated code is unaffected by the optimize_for option in the .proto file; in effect, all Python code is optimized for code size.
You should not create your own Foo subclasses. Generated classes are not designed for subclassing and may lead to "fragile base class" problems. Besides, implementation inheritance is bad design.
Python message classes have no particular public members other than those defined by the Message interface and those generated for nested fields, messages, and enum types (described below).
A message can be declared inside another message. For example:
message Foo {
message Bar {
}
}
In this case, the Bar class is declared as a static member of Foo, so you can refer to it as Foo.Bar.
For each field in a message type, the corresponding class has a member with the same name as the field. How you can manipulate the member depends on its type.
As well as accessor methods, the compiler generates an integer constant for each field containing its field number. The constant name is the field name converted to upper-case followed by _FIELD_NUMBER. For example, given the field optional int32 foo_bar = 5;, the compiler will generate the constant FOO_BAR_FIELD_NUMBER = 5.
If you have a singular (optional or repeated) field foo of any non-message type, you can manipulate the field foo as if it were a regular field. For example, if foo's type is int32, you can say:
message.foo = 123 print message.foo
Note that setting foo to a value of the wrong type will raise a TypeError.
If foo is read when it is not set, its value is the default value for that field. To check if foo is set, or to clear the value of foo, you must call the HasField() or ClearField() methods of the Message interface. For example:
assert not message.HasField("foo")
message.foo = 123
assert message.HasField("foo")
message.ClearField("foo")
assert not message.HasField("foo")
Message types work slightly differently. You cannot assign a value to an embedded message field. Instead, assigning a value to any field within the child message implies setting the message field in the parent. So, for example, let's say you have the following .proto definition:
message Foo {
optional Bar bar = 1;
}
message Bar {
optional int32 i = 1;
}
You cannot do the following:
foo = Foo() foo.bar = Bar() # WRONG!
Instead, to set bar, you simply assign a value directly to a field within bar, and - presto! - foo has a bar field:
foo = Foo()
assert not foo.HasField("bar")
foo.bar.i = 1
assert foo.HasField("bar")
Note that simply reading a field inside bar does not set the field:
foo = Foo()
assert not foo.HasField("bar")
print foo.bar.i # Print i's default value
assert not foo.HasField("bar")
Repeated fields are represented as an object that acts like a Python sequence. As with embedded messages, you cannot assign the field directly, but you can manipulate it. For example, given this message definition:
message Foo {
repeated int32 nums = 1;
}
You can do the following:
foo = Foo() foo.nums.append(15) foo.nums.append(32) assert len(foo.nums) == 2 assert foo.nums[0] == 15 assert foo.nums[1] == 32 for i in foo.nums: print i foo.nums[1] = 56 assert foo.nums[1] == 56
As with singular fields, to clear the field you must call the ClearField() method of the Message interface.
Repeated messages work like other repeated fields except that the add() method returns the new object rather than taking a value as a parameter. For example, given this message definition:
message Foo {
repeated Bar bars = 1;
}
message Bar {
optional int32 i = 1;
}
You can do the following:
foo = Foo() bar = foo.bars.add() bar.i = 15 bar = foo.bars.add() bar.i = 32 assert len(foo.bars) = 2 assert foo.bars[0].i == 15 assert foo.bars[1].i == 32 for bar in foo.bars: print bar.i foo.bars[1].i = 56 assert foo.bars[1].i == 56
In Python, enums are just integers. A set of integral constants are defined corresponding to the enum's defined values. For example, given:
enum Foo {
VALUE_A = 1;
VALUE_B = 5;
VALUE_C = 1234;
}
The constants VALUE_A, VALUE_B, andVALUE_C are defined with values 1, 5, and 1234, respectively. No type corresponding to Foo is defined.
An enum can be defined within a message type. In this case, the enum value constants are defined as static members of the message.
Note that in C++ and Java, an enum field cannot contain a numeric value other than those defined for the enum type. If an unknown enum value is encountered while parsing, the field will be treated as if its tag number were unknown. Therefore, you should never assign an enum field to an undefined value in Python, either. A future version of the library may explicitly disallow this.
Given a message with an extension range:
message Foo {
extensions 100 to 199;
}
The Python class corresponding to Foo will have a member called Extensions, which is a dictionary mapping extension identifiers to their current values.
Given an extension definition:
extend Foo {
optional int32 bar = 123;
}
The protocol buffer compiler generates an "extension identifier" called bar. The identifier acts as a key to the Extensions dictionary. The result of looking up a value in this dictionary is exactly the same as if you accessed a normal field of the same type. So, given the above example, you could do:
foo = Foo() foo.Extensions[bar] = 2 assert foo.Extensions[bar] == 2
Analogous to normal fields, Extensions[...] returns a message object for singular messages and a sequence for repeated fields.
Note that the Message interface's HasField() and ClearField() methods do not work with extensions; you must use HasExtension() and ClearExtension() instead.
Given a service definition:
service Foo {
rpc Bar(FooRequest) returns(FooResponse);
}
The protocol buffer compiler will generate a class Foo to represent this service. Foo will have a method for each method defined in the service definition. In this case, the method Bar is defined as:
def Bar(self, rpc_controller, request, done)
The parameters are equivalent to the parameters of Service.CallMethod(), except that the method_descriptor argument is implied.
These generated methods are intended to be overridden by subclasses. The default implementations simply call controller.SetFailed() with an error message indicating that the method is unimplemented, then invoke the done callback. When implementing your own service, you must subclass this generated service and implement its methods as appropriate.
Foo subclasses the Service interface. The protocol buffer compiler automatically generates implementations of the methods of Service as follows:
GetDescriptor: Returns the service's ServiceDescriptor.
CallMethod: Determines which method is being called based on the provided method descriptor and calls it directly.
GetRequestClass and GetResponseClass: Returns the class of the request or response of the correct type for the given method.
The protocol buffer compiler also generates a "stub" implementation of every service interface, which is used by clients wishing to send requests to servers implementing the service. For the Foo service (above), the stub implementation Foo_Stub will be defined.
Foo_Stub is a subclass of Foo. Its constructor takes an RpcChannel as a parameter. The stub then implements each of the service's methods by calling the channel's CallMethod() method.
The Protocol Buffer library does not include an RPC implementation. However, it includes all of the tools you need to hook up a generated service class to any arbitrary RPC implementation of your choice. You need only provide implementations of RpcChannel and RpcController.