Getting Started
protobuf-net is a contract based serializer for .NET code, that happens to write data in the "protocol buffers" serialization format engineered by Google. The API, however, is very different to Google's, and follows typical .NET patterns (it is broadly comparable, in usage, to XmlSerializer, DataContractSerializer, etc). It should work for most .NET languages that write standard types and can use attributes.
Hello World
The simplest way to get started is simply to write your data; for example (I'll use C# for most examples; sorry about that):
class Person {
public int Id {get;set;}
public string Name {get;set:}
public Address Address {get;set;}
}
class Address {
public string Line1 {get;set;}
public string Line2 {get;set;}
}That is a good start, but by itself is not enough for protobuf-net. Unlike XmlSerializer, the member-names are not encoded in the data - instead, you must pick an integer to identify each member. Additionally, to show intent it is necessary to show that we intend this type to be serialized (i.e. that it is a data contract):
[ProtoContract]
class Person {
[ProtoMember(1)]
public int Id {get;set;}
[ProtoMember(2)]
public string Name {get;set:}
[ProtoMember(3)]
public Address Address {get;set;}
}
[ProtoContract]
class Address {
[ProtoMember(1)]
public string Line1 {get;set;}
[ProtoMember(2)]
public string Line2 {get;set;}
}Notes for Identifiers
- they must be positive integers
- they must be unique within a single type
- but the same numbers can be re-used in sub-types if inheritance is enabled
- the identifiers must not conflict with any inheritance identifiers (discussed later)
- lower numbers take less space - don't start 100,000,000
- the identifier is important; you can change the member-name, or shift it between a property and a field, but changing the identifier changes the data
Notes on types
supported:
- custom classes that:
- are marked as data-contract
- have a parameterless constructor
- for Silverlight: are public
- many common primitives etc
- single dimension arrays: T[]
- List<T> / IList<T>
- Dictionary<TKey,TValue> / IDictionary<TKey,TValue>
- any type which implements IEnumerable<T> and has an Add(T) method
The code assumes that types will be mutable around the elected members. Accordingly, custom structs are not supported, since they should be immutable.
Serializing Data
Since "protocol buffers" is a binary format, protobuf-net is based heavily around the Stream class; this makes it possible to use with a wide variety of implementations simply. For example, to write to a file:
var person = new Person {
Id = 12345, Name = "Fred",
Address = new Address {
Line1 = "Flat 1",
Line2 = "The Meadows"
}
};
using (var file = File.Create("person.bin")) {
Serializer.Serialize(file, person);
}This writes a 32 byte file to "person.bin". It might not be obvious in the above, but Serialize is a generic method - the line could also be:
using (var file = File.Create("person.bin")) {
Serializer.Serialize<Person>(file, person);
}But most of the time we can let the compiler's generic type inference do the work for us.
Deserializing Data
We also need to get out data back!
Person newPerson;
using (var file = File.OpenRead("person.bin")) {
newPerson = Serializer.Deserialize<Person>(file);
}This reads the data back from "person.bin". Note we need to tell it the type this time (the <Person>), but otherwise the code is very similar.
Deployment
You must deploy the protobuf-net dll alongside your project. Make sure you deploy the version appropriate to your environment.
I really like what you guys have done. Any chance that there might be optional support for unmarked custom classes in one way or another? e.g. via an external mapping file / mapping logic? Just seems like there's use cases for wanting to serialize objects that you don't own the source code for. Granted, one could write a wrapper, but if it's in the framework, so much the better.
can you give little example of using it with linq DBML?
Need's an example of class with enums
I second paul.vencill's comment. To me, this is a big deficiency of the declarative approach to serialization in general. In my own code, I can control scenarios like deserializing read-only properties by adding declarative attributes. But with entities from external libraries, this is excessively difficult.
@paul.vencill - this is all implemented in "v2".
@duc - you should just be able to set the DBML's serialization mode in the designer; that should be it.
in line "public string Name {get;set:}" type ";" instead of ":"
when I tried to prepare .cs file by created binary file person.proto (I chanhed extension) file with help of protogen.exe
I received next file
Discussion of that here
Hi there, just tried to serialize an ArrayList? via Serializer.SerializeWithLengthPrefix?(stream, myArrayList, PrefixStyle?.Base128); and got an InvalidOperationException?: Message="No suitable Default ArrayList? encoding found." Did I do something wrong or is the serialization of arraylists not supported?
@useless.peon sorry for delay - I don't get notifications on these! ArrayList? is supported in v2
Some more details / examples on inheritance and the use of multiple instances within a given stream would be good, as this is the sort of thing you would use over a TCP network stream. (perhaps an example of a basic client / server setup using TCPListener)
e.g. a single base class for a message, then several child class's for different types of messages / payloads, with the use of TypeOf? at the receiver end to work out the kind of message being dealt with / class type
I think I've managed to figure this out okay for myself, but there are just a couple of small gotcha's to do with inheritance
In the case of sending multiple messages / class instances over a single network stream From what I understand the normal Deserialize / Serialize assumes there's only 1 class instance for the whole stream
But for DeserializeWithLengthPrefix? / SerializeWithLengthPrefix? with PrefixStyle?.Base128, this automagically prefix's the data with the length of the instance so that it can read off multiple class instances / messages from a given stream
For inheritance, in the child class this can have tag / ProtoMember? numbers that start from 1 (there's no clash with the tags in the base class) However in the base class you need to tell it about any child class's that exist / could be used for serialization using the Protoinclude tag, e.g.
<ProtoInclude?(5, GetType?(child1class))> where 5 is a unique tag / ProtoMember? number (needs to be different from any of the numbers already used by the ProtoMember? attributes in the same base class)
great work BTW
I have a question . Does this project rewrite the protopuf from google ,or a wrapper ? Does it support deserialize/serialize the file or object which use google protopuf processed ? Thanks.
Re the above, it is a completely separate implementation following the same encoding spec. I do, however, make use of the "protoc.exe" tool when parsing ".proto" schema definition files as part of code-generation. So: if you are using a .proto there is a little bit of google code running when you pre-process the .proto, but otherwise: not.
To support portability, would i need to manually create a .proto file defining the messages for communicating between this c# library and other (i.e jave, cpp) implementation? Is there a way to auto-generate the .proto file from parsed class?
FYI, there is a syntax error in the example code: "public string Name {get;set:}" has colon instead of semicolon after set, in both code blocks.
I am using protobuf-net-VS10.msi with C++ protobuf-2.3.0.zip. I see they moved to protobuf-2.4.1.zip. Anyone using protobuf-net with later C++ 2.4.1. Any problems?
Reminder: I DON'T get notification on comments you leave here. If you have a question / blockage / suggestion, please contact me directly or log an issue.
@Amit in v1, GetProto?(). This method is not yet implemented in v2.
I am having problems with de-serializing a collection of objects with ProtoBuf?.net. It correctly serializes it in the byte array and I am passing that byte array from a WCF Ria service to Silverlight which is trying to deserialize the results.
The code I am using is:
When trying to deserialize it gives the error:
Invalid callback signature Parameter name: callback (User)
I implemented this framework in WCF and everything serializes/deserializes just fine. Now I'm at the stage to make SOAPUI to work with my services, but nothing is getting serialize/deserialized. When I'm sending the request from SOAPUI its arriving as null to the service method, and if I return anything in my response (for testing purposes) I get the below encoded string back.
Is there certain settings to do to make SOAPUI work, or is this something with SOAPUI itself?
Thanks.
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
</s:Envelope>How can i store Serializing Data on RAM? and in case,if RAM is opted out this Serializing data then how do i change from RAM to file?
Serialization is meant to pack data together to store it to a file or sent it over the network. While you could probably create a virtual file and serialize to there, it makes no sense serializing your data to the RAM... :)
I'm working on e-commerce site.I'm using ASP.NET MVC3.I've used protoBuf in product listing page.when number of user increase,it is taking 50 to 80% of cpu usage.it is working fine when count of user is 15 to 16. So what could be the reason?
I have run the protobuf-net-VS10.msi, but the protobuf-net-VS09 not VS10 is installed. I have both VS 2008 and VS 2010 installed. I wonder why?
@justin there's a typo in the installer; that installer should work for VS2010
Still needing an example of enums. Would like also to see 'optional' in action. Does the VS10 tooling support auto-generation of code from .proto files? Could this be shown in a short example or series of screenshots.
A follow-up to my own previous comment - I figured out how to do enums and optional types. The latter is done via nullable types, I never saw that documented here.
If anyone knows how to encode default values, I would love to hear about it.
Meanwhile, here's a demo proto definition and its corresponding proto.net implemenation. Hope that helpful to someone:
// ----------------------------------------------------------------------------- // DemoMessage.proto // ----------------------------------------------------------------------------- message DemoMessage { message WildThing { enum Coolness { COOLNESS_NOT_SO_MUCH = 0; COOLNESS_EXTREMELY_SO = 1; } optional Coolness coolness = 1 [default = COOLNESS_NOT_SO_MUCH]; optional float toxicity_PPM = 2 [default = 0.12]; } optional InnerThing wildThing = 1; optional uint64 posixTime_Msec = 2; optional float volume_dBm = 3 [default = -112.0]; } // ----------------------------------------------------------------------------- // Proto.DemoMessage.cs // ----------------------------------------------------------------------------- using System; using System.IO; using ProtoBuf; // ----------------------------------------------------------------------------- namespace Whatever { // ------------------------------------------------------------------------- [ProtoContract] public sealed class Proto_DemoMessage { [ProtoContract] public sealed class WildThingMessage { public enum CoolnessSetting { NotSoMuch = 0, ExtremelySo = 1 } [ProtoMember(1, IsRequired = false)] public CoolnessSetting? Coolness { get; set; } [ProtoMember(2, IsRequired = false)] public float? Toxicity_PPM { get; set; } } [ProtoMember(1, IsRequired = false)] public WildThingMessage WildThing { get; set; } [ProtoMember(2, IsRequired = false)] public UInt64? PosixTime_Msec { get; set; } [ProtoMember(3, IsRequired = false)] public float? Volume_dBm { get; set; } // --------------------------------------------------------------------- // --------------------------------------------------------------------- public byte[] Serialize() { byte[] b = null; using (var ms = new MemoryStream()) { Serializer.Serialize<Proto_DemoMessage>(ms, this); b = new byte[ms.Position]; var fullB = ms.GetBuffer(); Array.Copy(fullB, b, b.Length); } return b; } // --------------------------------------------------------------------- public static Proto_DemoMessage Deserialize(byte[] serializationBytes) { Proto_DemoMessage m = null; using (var ms = new MemoryStream(serializationBytes)) { m = Serializer.Deserialize<Proto_DemoMessage>(ms); } return m; } } }