zero_copy_stream.h

#include <google/protobuf/io/zero_copy_stream.h>
namespace google::protobuf::io

This file contains the ZeroCopyInputStream and ZeroCopyOutputStream interfaces, which represent abstract I/O streams to and from which protocol buffers can be read and written.

For a few simple implementations of these interfaces, see zero_copy_stream_impl.h.

These interfaces are different from classic I/O streams in that they try to minimize the amount of data copying that needs to be done. To accomplish this, responsibility for allocating buffers is moved to the stream object, rather than being the responsibility of the caller. So, the stream can return a buffer which actually points directly into the final data structure where the bytes are to be stored, and the caller can interact directly with that buffer, eliminating an intermediate copy operation.

As an example, consider the common case in which you are reading bytes from an array that is already in memory (or perhaps an mmap()ed file). With classic I/O streams, you would do something like:

char buffer[BUFFER_SIZE];
input->Read(buffer, BUFFER_SIZE);
DoSomething(buffer, BUFFER_SIZE);

Then, the stream basically just calls memcpy() to copy the data from the array into your buffer. With a ZeroCopyInputStream, you would do this instead:

const void* buffer;
int size;
input->Next(&buffer, &size);
DoSomething(buffer, size);

Here, no copy is performed. The input stream returns a pointer directly into the backing array, and the caller ends up reading directly from it.

If you want to be able to read the old-fashion way, you can create a CodedInputStream or CodedOutputStream wrapping these objects and use their ReadRaw()/WriteRaw() methods. These will, of course, add a copy step, but Coded*Stream will handle buffering so at least it will be reasonably efficient.

ZeroCopyInputStream example:

 Read in a file and print its contents to stdout.
int fd = open("myfile", O_RDONLY);
ZeroCopyInputStream* input = new FileInputStream(fd);

const void* buffer;
int size;
while (input->Next(&buffer, &size)) {
  cout.write(buffer, size);
}

delete input;
close(fd);

ZeroCopyOutputStream example:

 Copy the contents of "infile" to "outfile", using plain read() for
 "infile" but a ZeroCopyOutputStream for "outfile".
int infd = open("infile", O_RDONLY);
int outfd = open("outfile", O_WRONLY);
ZeroCopyOutputStream* output = new FileOutputStream(outfd);

void* buffer;
int size;
while (output->Next(&buffer, &size)) {
  int bytes = read(infd, buffer, size);
  if (bytes < size) {
 Reached EOF.
    output->BackUp(size - bytes);
    break;
  }
}

delete output;
close(infd);
close(outfd);

class ZeroCopyInputStream

#include <google/protobuf/io/zero_copy_stream.h>
namespace google::protobuf::io

Abstract interface similar to an input stream but designed to minimize copying.

Known subclasses:

• ArrayInputStream
• ConcatenatingInputStream
• CopyingInputStreamAdaptor
• FileInputStream
• GzipInputStream
• IstreamInputStream
• LimitingInputStream

virtual bool ZeroCopyInputStream::Next(
        const void ** data,
        int * size) = 0

virtual void ZeroCopyInputStream::BackUp(
        int count) = 0

virtual bool ZeroCopyInputStream::Skip(
        int count) = 0

class ZeroCopyOutputStream

#include <google/protobuf/io/zero_copy_stream.h>
namespace google::protobuf::io

Abstract interface similar to an output stream but designed to minimize copying.

Known subclasses:

• ArrayOutputStream
• CopyingOutputStreamAdaptor
• FileOutputStream
• GzipOutputStream
• OstreamOutputStream
• StringOutputStream

virtual bool ZeroCopyOutputStream::Next(
        void ** data,
        int * size) = 0

virtual void ZeroCopyOutputStream::BackUp(
        int count) = 0