ch08.xml - svnbook - Project Hosting on Google Code

‹r1121

r3654

Source path: svn/ tags/ en-1.0-final/ src/ book/ book/ ch08.xml

<chapter id="svn-ch-8">
  <title>Developer Information</title>
  
  <simplesect id="svn-ch-8-sect-0">
    <para>Subversion is an open-source software project developed
      under an Apache-style software license.  The project is
      financially backed by CollabNet, Inc., a California-based
      software development company.  The community that has formed
      around the development of Subversion always welcomes new members
      who can donate their time and attention to the project.
      Volunteers are encouraged to assist in any way they can, whether
      that means finding and diagnosing bugs, refining existing source
      code, or fleshing out whole new features.</para>
    
    <para>This chapter is for those who wish to assist in the
      continued evolution of Subversion by actually getting their
      hands dirty with the source code.  We will cover some of the
      software's more intimate details, the kind of technical
      nitty-gritty that those developing Subversion itself&mdash;or
      writing entirely new tools based on the Subversion
      libraries&mdash;should be aware of.  If you don't foresee
      yourself participating with the software at such a level, feel
      free to skip this chapter with confidence that your experience
      as a Subversion user will not be affected.</para>

  </simplesect>

  <!-- ******************************************************************* -->
  <!-- *** SECTION 1:  LAYERED LIBRARY DESIGN                          *** -->
  <!-- ******************************************************************* -->
  <sect1 id="svn-ch-8-sect-1">
    <title>Layered Library Design</title>

    <para>Subversion has a modular design, implemented as a collection
      of C libraries.  Each library has a well-defined purpose and
      interface, and most modules are said to exist in one of three
      main layers&mdash;the Repository Layer, the Repository Access
      (RA) Layer, or the Client Layer.  We will examine these layers
      shortly, but first, see our brief inventory of Subversion's
      libraries in <xref linkend="svn-ch-8-table-1"/>.  For the sake
      of consistency, we will refer to the libraries by their
      extensionless Unix library names (e.g.: libsvn_fs, libsvn_wc,
      mod_dav_svn).</para>

    <table id="svn-ch-8-table-1">
      <title>A Brief Inventory of the Subversion Libraries</title>
      <tgroup cols="2">
        <thead>
          <row>
            <entry>Library</entry>
            <entry>Description</entry>
          </row>
        </thead>
        <tbody>
          <row>
            <entry>libsvn_client</entry> 
            <entry>Primary interface for client programs</entry>
          </row>
          <row>
            <entry>libsvn_delta</entry>
            <entry>Tree and text differencing routines</entry>
          </row>
          <row>
            <entry>libsvn_fs</entry>
            <entry>The Subversion filesystem library</entry>
          </row>
          <row>
            <entry>libsvn_ra</entry>
            <entry>Repository Access commons and module loader</entry>
          </row>
          <row>
            <entry>libsvn_ra_dav</entry>
            <entry>The WebDAV Repository Access module</entry>
          </row>
          <row>
            <entry>libsvn_ra_local</entry>
            <entry>The local Repository Access module</entry>
          </row>
          <row>
            <entry>libsvn_ra_svn</entry>
            <entry>A custom protocol Repository Access module</entry>
          </row>
          <row>
            <entry>libsvn_repos</entry>
            <entry>Repository interface</entry>
          </row>
          <row>
            <entry>libsvn_subr</entry>
            <entry>Miscellaneous helpful subroutines</entry>
          </row>
          <row>
            <entry>libsvn_wc</entry>
            <entry>The working copy management library</entry>
          </row>
          <row>
            <entry>mod_authz_svn</entry>
            <entry>Apache authorization module for Subversion
            repositories access via WebDAV.</entry>
          </row>
          <row>
            <entry>mod_dav_svn</entry>
            <entry>Apache module for mapping WebDAV operations to
            Subversion ones</entry>
          </row>
        </tbody>
      </tgroup>
    </table>

    <para>The fact that the word <quote>miscellaneous</quote> only
      appears once in <xref linkend="svn-ch-8-table-1"/> is a good
      sign.  The Subversion development team is serious about making
      sure that functionality lives in the right layer and libraries.
      Perhaps the greatest advantage of the modular design is its lack
      of complexity from a developer's point of view.  As a developer,
      you can quickly formulate that kind of <quote>big
      picture</quote> that allows you to pinpoint the location of
      certain pieces of functionality with relative ease.</para>

    <para>Another benefit of modularity is the ability to replace a
      given module with a whole new library that implements the same
      API without affecting the rest of the code base.  In some sense,
      this happens within Subversion already.  The libsvn_ra_dav,
      libsvn_ra_local, and libsvn_ra_svn all implement the same
      interface.  And all three communicate with the Repository
      Layer&mdash;libsvn_ra_dav and libsvn_ra_svn do so across a
      network, and libsvn_ra_local connects to it directly.</para>

    <para>The client itself also highlights modularity in the
      Subversion design.  While Subversion currently comes with only a
      command-line client program, there are already a few other
      programs being developed by third parties to act as GUIs for
      Subversion.  Again, these GUIs use the same APIs that the stock
      command-line client does.  Subversion's libsvn_client library is
      the one-stop shop for most of the functionality necessary for
      designing a working Subversion client (see <xref
      linkend="svn-ch-8-sect-1.3"/>).</para>

    <!-- ***************************************************************** -->
    <sect2 id="svn-ch-8-sect-1.1">
      <title>Repository Layer</title>

      <para>When referring to Subversion's Repository Layer, we're
        generally talking about two libraries&mdash;the repository
        library, and the filesystem library.  These libraries provide
        the storage and reporting mechanisms for the various revisions
        of your version-controlled data.  This layer is connected to
        the Client Layer via the Repository Access Layer, and is, from
        the perspective of the Subversion user, the stuff at the
        <quote>other end of the line.</quote></para>

      <para>The Subversion Filesystem is accessed via the libsvn_fs
        API, and is not a kernel-level filesystem that one would
        install in an operating system (like the Linux ext2 or NTFS),
        but a virtual filesystem.  Rather than storing
        <quote>files</quote> and <quote>directories</quote> as real
        files and directories (as in, the kind you can navigate
        through using your favorite shell program), it uses a database
        system for its back-end storage mechanism.  Currently, the
        database system in use is Berkeley DB.
        <footnote>
          <para>The choice of Berkeley DB brought several automatic
            features that Subversion needed, such as data integrity,
            atomic writes, recoverability, and hot backups.</para>
        </footnote>
        However, there has been considerable interest by the
        development community in giving future releases of Subversion
        the ability to use other back-end database systems, perhaps
        through a mechanism such as Open Database Connectivity
        (ODBC).</para>

      <para>The filesystem API exported by libsvn_fs contains the
        kinds of functionality you would expect from any other
        filesystem API: you can create and remove files and
        directories, copy and move them around, modify file contents,
        and so on.  It also has features that are not quite as common,
        such as the ability to add, modify, and remove metadata
        (<quote>properties</quote>) on each file or directory.
        Furthermore, the Subversion Filesystem is a versioning
        filesystem, which means that as you make changes to your
        directory tree, Subversion remembers what your tree looked
        like before those changes.  And before the previous changes.
        And the previous ones.  And so on, all the way back through
        versioning time to (and just beyond) the moment you first
        started adding things to the filesystem.</para>

      <para>All the modifications you make to your tree are done
        within the context of a Subversion transaction.  The following
        is a simplified general routine for modifying your
        filesystem:</para>

      <orderedlist>
        <listitem>
          <para>Begin a Subversion transaction.</para>
        </listitem>
        <listitem>
          <para>Make your changes (adds, deletes, property
            modifications, etc.).</para>
        </listitem>
        <listitem>
          <para>Commit your transaction.</para>
        </listitem>
      </orderedlist>


      <para>Once you have committed your transaction, your filesystem
        modifications are permanently stored as historical artifacts.
        Each of these cycles generates a single new revision of your
        tree, and each revision is forever accessible as an immutable
        snapshot of <quote>the way things were.</quote></para>

      <sidebar>
        <title>The Transaction Distraction</title>

        <para>The notion of a Subversion transaction, especially given
          its close proximity to the database code in libsvn_fs, can
          become easily confused with the transaction support provided
          by the underlying database itself.  Both types of
          transaction exist to provide atomicity and isolation.  In
          other words, transactions give you the ability to perform a
          set of actions in an <quote>all or nothing</quote>
          fashion&mdash;either all the actions in the set complete
          with success, or they all get treated as if
          <emphasis>none</emphasis> of them ever happened&mdash;and in
          a way that does not interfere with other processes acting on
          the data.</para>

        <para>Database transactions generally encompass small
          operations related specifically to the modification of data
          in the database itself (such as changing the contents of a
          table row).  Subversion transactions are larger in scope,
          encompassing higher-level operations like making
          modifications to a set of files and directories which are
          intended to be stored as the next revision of the filesystem
          tree.  If that isn't confusing enough, consider this:
          Subversion uses a database transaction during the creation
          of a Subversion transaction (so that if the creation of
          Subversion transaction fails, the database will look as if
          we had never attempted that creation in the first
          place)!</para>

        <para>Fortunately for users of the filesystem API, the
          transaction support provided by the database system itself
          is hidden almost entirely from view (as should be expected
          from a properly modularized library scheme).  It is only
          when you start digging into the implementation of the
          filesystem itself that such things become visible (or
          interesting).</para>

      </sidebar>

      <para>Most of the functionality provided by the filesystem
        interface comes as an action that occurs on a filesystem path.
        That is, from outside of the filesystem, the primary mechanism
        for describing and accessing the individual revisions of files
        and directories comes through the use of path strings like
        <filename>/foo/bar</filename>, just as if you were addressing
        files and directories through your favorite shell program.
        You add new files and directories by passing their paths-to-be
        to the right API functions.  You query for information about
        them by the same mechanism.</para>

      <para>Unlike most filesystems, though, a path alone is not
        enough information to identify a file or directory in
        Subversion.  Think of a directory tree as a two-dimensional
        system, where a node's siblings represent a sort of
        left-and-right motion, and descending into subdirectories a
        downward motion.  <xref linkend="svn-ch-8-dia-1"/> shows
        a typical representation of a tree as exactly that.</para>

      <figure id="svn-ch-8-dia-1">
        <title>Files and directories in two dimensions</title>
        <graphic fileref="images/ch08dia1.png"/>
      </figure>

      <para>Of course, the Subversion filesystem has a nifty third
        dimension that most filesystems do not have&mdash;Time!
        <footnote>
          <para>We understand that this may come as a shock to sci-fi
            fans who have long been under the impression that Time was
            actually the <emphasis>fourth</emphasis> dimension, and we
            apologize for any emotional trauma induced by our
            assertion of a different theory.</para>
        </footnote>
        In the filesystem interface, nearly every function that has a
        <parameter>path</parameter> argument also expects a
        <parameter>root</parameter> argument.  This
        <structname>svn_fs_root_t</structname> argument describes
        either a revision or a Subversion transaction (which is
        usually just a revision-to-be), and provides that
        third-dimensional context needed to understand the difference
        between <filename>/foo/bar</filename> in revision 32, and the
        same path as it exists in revision 98.  <xref
        linkend="svn-ch-8-dia-2"/> shows revision history as an
        added dimension to the Subversion filesystem universe.</para>

      <figure id="svn-ch-8-dia-2">
        <title>Versioning time&mdash;the third dimension!</title>
        <graphic fileref="images/ch08dia2.png"/>
      </figure>

      <!-- Perhaps dig into the DAG/tree layers a bit here, talking
           about the hard-link design and how that affords such
           pleasures as cheap copies.  If "bubble-up" isn't covered
           twelve other times in the book, maybe give it a go here. -->

      <para>As we mentioned earlier, the libsvn_fs API looks and feels
        like any other filesystem, except that it has this wonderful
        versioning capability.  It was designed to be usable by any
        program interested in a versioning filesystem.  Not
        coincidentally, Subversion itself is interested in that
        functionality.  But while the filesystem API should be
        sufficient for basic file and directory versioning support,
        Subversion wants more&mdash;and that is where libsvn_repos
        comes in.</para>

      <para>The Subversion repository library (libsvn_repos) is
        basically a wrapper library around the filesystem
        functionality.  This library is responsible for creating the
        repository layout, making sure that the underlying filesystem
        is initialized, and so on.  Libsvn_repos also implements a set
        of hooks&mdash;scripts that are executed by the repository
        code when certain actions take place.  These scripts are
        useful for notification, authorization, or whatever purposes
        the repository administrator desires.  This type of
        functionality, and other utilities provided by the repository
        library, are not strictly related to implementing a versioning
        filesystem, which is why it was placed into its own
        library.</para>
      
      <para>Developers who wish to use the libsvn_repos API will find
        that it is not a complete wrapper around the filesystem
        interface.  That is, only certain major events in the general
        cycle of filesystem activity are wrapped by the repository
        interface.  Some of these include the creation and commit of
        Subversion transactions, and the modification of revision
        properties.  These particular events are wrapped by the
        repository layer because they have hooks associated with them.
        In the future, other events may be wrapped by the repository
        API.  All of the remaining filesystem interaction will
        continue to occur directly via the libsvn_fs API, though.</para>

      <para>For example, here is a code segment that illustrates the
        use of both the repository and filesystem interfaces to create
        a new revision of the filesystem in which a directory is
        added.  Note that in this example (and all others throughout
        this book), the <function>SVN_ERR</function> macro simply
        checks for a non-successful error return from the function it
        wraps, and returns that error if it exists.</para>

      <example id="svn-ch-8-sect-1.1-ex-1">
        <title>Using the Repository Layer</title>

        <programlisting>
/* Create a new directory at the path NEW_DIRECTORY in the Subversion
   repository located at REPOS_PATH.  Perform all memory allocation in
   POOL.  This function will create a new revision for the addition of
   NEW_DIRECTORY.  */
static svn_error_t *
make_new_directory (const char *repos_path,
                    const char *new_directory,
                    apr_pool_t *pool)
{
  svn_error_t *err;
  svn_repos_t *repos;
  svn_fs_t *fs;
  svn_revnum_t youngest_rev;
  svn_fs_txn_t *txn;
  svn_fs_root_t *txn_root;
  const char *conflict_str;

  /* Open the repository located at REPOS_PATH.  */
  SVN_ERR (svn_repos_open (&amp;repos, repos_path, pool));

  /* Get a pointer to the filesystem object that is stored in
     REPOS.  */
  fs = svn_repos_fs (repos);

  /* Ask the filesystem to tell us the youngest revision that
     currently exists.  */
  SVN_ERR (svn_fs_youngest_rev (&amp;youngest_rev, fs, pool));

  /* Begin a new transaction that is based on YOUNGEST_REV.  We are
     less likely to have our later commit rejected as conflicting if we
     always try to make our changes against a copy of the latest snapshot
     of the filesystem tree.  */
  SVN_ERR (svn_fs_begin_txn (&amp;txn, fs, youngest_rev, pool));

  /* Now that we have started a new Subversion transaction, get a root
     object that represents that transaction.  */
  SVN_ERR (svn_fs_txn_root (&amp;txn_root, txn, pool));
  
  /* Create our new directory under the transaction root, at the path
     NEW_DIRECTORY.  */
  SVN_ERR (svn_fs_make_dir (txn_root, new_directory, pool));

  /* Commit the transaction, creating a new revision of the filesystem
     which includes our added directory path.  */
  err = svn_repos_fs_commit_txn (&amp;conflict_str, repos, 
                                 &amp;youngest_rev, txn, pool);
  if (! err)
    {
      /* No error?  Excellent!  Print a brief report of our success.  */
      printf ("Directory '%s' was successfully added as new revision "
              "'%" SVN_REVNUM_T_FMT "'.\n", new_directory, youngest_rev);
    }
  else if (err->apr_err == SVN_ERR_FS_CONFLICT)
    {
      /* Uh-oh.  Our commit failed as the result of a conflict
         (someone else seems to have made changes to the same area 
         of the filesystem that we tried to modify).  Print an error
         message.  */
      printf ("A conflict occurred at path '%s' while attempting "
              "to add directory '%s' to the repository at '%s'.\n", 
              conflict_str, new_directory, repos_path);
    }
  else
    {
      /* Some other error has occurred.  Print an error message.  */
      printf ("An error occurred while attempting to add directory '%s' "
              "to the repository at '%s'.\n", 
              new_directory, repos_path);
    }


  /* Return the result of the attempted commit to our caller.  */
  return err;
} 
</programlisting>
      </example>

      <para>In the previous code segment, calls were made to both the
        repository and filesystem interfaces.  We could just as easily
        have committed the transaction using
        <function>svn_fs_commit_txn</function>.  But the filesystem
        API knows nothing about the repository library's hook
        mechanism.  If you want your Subversion repository to
        automatically perform some set of non-Subversion tasks every
        time you commit a transaction (like, for example, sending an
        email that describes all the changes made in that transaction
        to your developer mailing list), you need to use the
        libsvn_repos-wrapped version of that
        function&mdash;<function>svn_repos_fs_commit_txn</function>.
        This function will actually first run the
        <literal>pre-commit</literal> hook script if one exists, then
        commit the transaction, and finally will run a
        <literal>post-commit</literal> hook script.  The hooks provide
        a special kind of reporting mechanism that does not really
        belong in the core filesystem library itself.  (For more
        information regarding Subversion's repository hooks, see <xref
        linkend="svn-ch-5-sect-2.1" />.)</para>
      
      <para>The hook mechanism requirement is but one of the reasons
        for the abstraction of a separate repository library from the
        rest of the filesystem code.  The libsvn_repos API provides
        several other important utilities to Subversion.  These
        include the abilities to:</para>

      <orderedlist>
        <listitem>
          <para>create, open, destroy, and perform recovery steps on a
            Subversion repository and the filesystem included in that
            repository.</para>
        </listitem>
        <listitem>
          <para>describe the differences between two filesystem
            trees.</para>
        </listitem>
        <listitem>
          <para>query for the commit log messages
            associated with all (or some) of the revisions in which a
            set of files was modified in the filesystem.</para>
        </listitem>
        <listitem>
          <para>generate a human-readable <quote>dump</quote> of the
            filesystem, a complete representation of the revisions in
            the filesystem.</para>
        </listitem>
        <listitem>
          <para>parse that dump format, loading the dumped revisions
            into a different Subversion repository.</para>
        </listitem>
      </orderedlist>

      <para>As Subversion continues to evolve, the repository library
        will grow with the filesystem library to offer increased
        functionality and configurable option support.</para>

    </sect2>

    <!-- ****************************************************************** -->
    <sect2 id="svn-ch-8-sect-1.2">
      <title>Repository Access Layer</title>

      <para>If the Subversion Repository Layer is at <quote>the other
        end of the line</quote>, the Repository Access Layer is the
        line itself.  Charged with marshalling data between the client
        libraries and the repository, this layer includes the
        libsvn_ra module loader library, the RA modules themselves
        (which currently includes libsvn_ra_dav, libsvn_ra_local, and
        libsvn_ra_svn), and any additional libraries needed by one or
        more of those RA modules, such as the mod_dav_svn Apache
        module with which libsvn_ra_dav communicates or
        libsvn_ra_svn's server, <command>svnserve</command>.</para>

      <para>Since Subversion uses URLs to identify its repository
        resources, the protocol portion of the URL schema (usually
        <literal>file:</literal>, <literal>http:</literal>,
        <literal>https:</literal>, or <literal>svn:</literal>) is used
        to determine which RA module will handle the communications.
        Each module registers a list of the protocols it knows how to
        <quote>speak</quote> so that the RA loader can, at runtime,
        determine which module to use for the task at hand.  You can
        determine which RA modules are available to the Subversion
        command-line client, and what protocols they claim to support,
        by running <command>svn --version</command>:</para>
     
      <screen>
$ svn --version
svn, version 1.0.1 (r9023)
   compiled Mar 17 2004, 09:31:13

Copyright (C) 2000-2004 CollabNet.
Subversion is open source software, see http://subversion.tigris.org/
This product includes software developed by CollabNet (http://www.Collab.Net/).

The following repository access (RA) modules are available:

* ra_dav : Module for accessing a repository via WebDAV (DeltaV) protocol.
  - handles 'http' schema
  - handles 'https' schema
* ra_local : Module for accessing a repository on local disk.
  - handles 'file' schema
* ra_svn : Module for accessing a repository using the svn network protocol.
  - handles 'svn' schema
</screen>
    
      <sect3 id="svn-ch-8-sect-1.2.1">
        <title>RA-DAV (Repository Access Using HTTP/DAV)</title>

        <para>The libsvn_ra_dav library is designed for use by clients
          that are being run on different machines than the servers
          with which they communicating, specifically servers reached
          using URLs that contain the <literal>http:</literal> or
          <literal>https:</literal> protocol portions.  To understand
          how this module works, we should first mention a couple of
          other key components in this particular configuration of the
          Repository Access Layer&mdash;the powerful Apache HTTP
          Server, and the Neon HTTP/WebDAV client library.</para>
  
        <para>Subversion's primary network server is the Apache HTTP
          Server.  Apache is a time-tested, extensible open-source
          server process that is ready for serious use.  It can
          sustain a high network load and runs on many platforms.  The
          Apache server supports a number of different standard
          authentication protocols, and can be extended through the
          use of modules to support many others.  It also supports
          optimizations like network pipelining and caching.  By using
          Apache as a server, Subversion gets all of these features
          for free.  And since most firewalls already allow HTTP
          traffic to pass through, sysadmins typically don't even have
          to change their firewall configurations to allow Subversion
          to work.</para>
  
        <para>Subversion uses HTTP and WebDAV (with DeltaV) to
          communicate with an Apache server.  You can read more about
          this in the WebDAV section of this chapter, but in short,
          WebDAV and DeltaV are extensions to the standard HTTP 1.1
          protocol that enable sharing and versioning of files over
          the web.  Apache 2.0 comes with mod_dav, an Apache module
          that understands the DAV extensions to HTTP.  Subversion
          itself supplies mod_dav_svn, though, which is another Apache
          module that works in conjunction with (really, as a back-end
          to) mod_dav to provide Subversion's specific implementations
          of WebDAV and DeltaV.</para>

        <para>When communicating with a repository over HTTP, the RA
          loader library chooses libsvn_ra_dav as the proper access
          module.  The Subversion client makes calls into the generic
          RA interface, and libsvn_ra_dav maps those calls (which
          embody rather large-scale Subversion actions) to a set of
          HTTP/WebDAV requests.  Using the Neon library, libsvn_ra_dav
          transmits those requests to the Apache server.  Apache
          receives these requests (exactly as it does generic HTTP
          requests that your web browser might make), notices that the
          requests are directed at a URL that is configured as a DAV
          location (using the <sgmltag>Location</sgmltag> directive in
          <filename>httpd.conf</filename>), and hands the request off
          to its own mod_dav module.  When properly configured,
          mod_dav knows to use Subversion's mod_dav_svn for any
          filesystem-related needs, as opposed to the generic
          mod_dav_fs that comes with Apache.  So ultimately, the
          client is communicating with mod_dav_svn, which binds
          directly to the Subversion Repository Layer.</para>
  
        <para>That was a simplified description of the actual
          exchanges taking place, though.  For example, the Subversion
          repository might be protected by Apache's authorization
          directives.  This could result in initial attempts to
          communicate with the repository being rejected by Apache on
          authorization grounds.  At this point, libsvn_ra_dav gets
          back the notice from Apache that insufficient identification
          was supplied, and calls back into the Client Layer to get
          some updated authentication data.  If the data is supplied
          correctly, and the user has the permissions that Apache
          seeks, libsvn_ra_dav's next automatic attempt at performing
          the original operation will be granted, and all will be
          well.  If sufficient authentication information cannot be
          supplied, the request will ultimately fail, and the client
          will report the failure to the user.</para>
  
        <!-- A diagram here? -->
  
        <para>By using Neon and Apache, Subversion gets free
          functionality in several other complex areas, too.  For
          example, if Neon finds the OpenSSL libraries, it allows the
          Subversion client to attempt to use SSL-encrypted
          communications with the Apache server (whose own mod_ssl can
          <quote>speak the language</quote>).  Also, both Neon itself
          and Apache's mod_deflate can understand the
          <quote>deflate</quote> algorithm (the same one used by the
          PKZIP and gzip programs), so requests can be sent in smaller,
          compressed chunks across the wire.  Other complex features
          that Subversion hopes to support in the future include the
          ability to automatically handle server-specified redirects
          (for example, when a repository has been moved to a new
          canonical URL) and taking advantage of HTTP
          pipelining.</para>
  
        <!-- Talk about another difference between CVS and Subversion.
             CVS users had to specify which auth mechanism to use
             (with :ext: vs. :pserver:) and whether or not to use
             compressed communications (with the -z option).  In
             Subversion, Apache takes some of that responsibility.
             The server will tell the client whether it can understand
             compression, and ... hmm.  Is this really true? -->


      </sect3>

      <sect3 id="svn-ch-8-sect-1.2.2">
        <title>RA-SVN (Custom Protocol Repository Access)</title>

        <para>In addition to the standard HTTP/WebDAV protocol,
          Subversion also provides an RA implementation that uses a
          custom protocol.  The libsvn_ra_svn module implements
          its own network socket connectivity, and communicates with a
          stand-alone server&mdash;the <filename>svnserve</filename>
          program&mdash;on the machine that hosts the
          repository.  Clients access the repository using the
          <literal>svn://</literal> schema.</para>

        <para>This RA implementation lacks most of the advantages of
          Apache mentioned in the previous section; however, it may be
          appealing to some sysadmins nonetheless.  It is dramatically
          easier to configure and run; setting up an
          <filename>svnserve</filename> process is nearly
          instantaneous.  It is also much smaller (in terms of lines
          of code) than Apache, making it much easier to audit, for
          security reasons or otherwise.  Furthermore, some sysadmins
          may already have an SSH security infrastructure in place,
          and want Subversion to use it.  Clients using ra_svn can
          easily tunnel the protocol over SSH.</para>

      </sect3>

      <sect3 id="svn-ch-8-sect-1.2.3">
        <title>RA-Local (Direct Repository Access)</title>

        <para>Not all communications with a Subversion repository
          require a powerhouse server process and a network layer.
          For users who simply wish to access the repositories on
          their local disk, they may do so using
          <literal>file:</literal> URLs and the functionality provided
          by libsvn_ra_local.  This RA module binds directly with the
          repository and filesystem libraries, so no network
          communication is required at all.</para>

        <para>Subversion requires that the server name included as part
          of the <literal>file:</literal> URL be either
          <literal>localhost</literal> or empty, and that there be no
          port specification.  In other words, your URLs should look
          like either
          <literal>file://localhost/path/to/repos</literal> or
          <literal>file:///path/to/repos</literal>.</para>

        <para>Also, be aware that Subversion's
          <literal>file:</literal> URLs cannot be used in a regular
          web browser the way typical <literal>file:</literal> URLs
          can.  When you attempt to view a <literal>file:</literal>
          URL in a regular web browser, it reads and displays the
          contents of the file at that location by examining the
          filesystem directly.  However, Subversion's resources exist
          in a virtual filesystem (see <xref
          linkend="svn-ch-8-sect-1.1" />), and your browser will not
          understand how to read that filesystem.</para>

      </sect3>

      <sect3 id="svn-ch-8-sect-1.2.4">
        <title>Your RA Library Here</title>

        <para>For those who wish to access a Subversion repository
          using still another protocol, that is precisely why the
          Repository Access Layer is modularized!  Developers can
          simply write a new library that implements the RA interface
          on one side and communicates with the repository on the
          other.  Your new library can use existing network protocols,
          or you can invent your own.  You could use inter-process
          communication (IPC) calls, or&mdash;let's get crazy, shall
          we?&mdash;you could even implement an email-based protocol.
          Subversion supplies the APIs; you supply the creativity.</para>

      </sect3>
    </sect2>

    <!-- ***************************************************************** -->
    <sect2 id="svn-ch-8-sect-1.3">
      <title>Client Layer</title>
      
      <para>On the client side, the Subversion working copy is where
        all the action takes place.  The bulk of functionality
        implemented by the client-side libraries exists for the sole
        purpose of managing working copies&mdash;directories full of
        files and other subdirectories which serve as a sort of local,
        editable <quote>reflection</quote> of one or more repository
        locations&mdash;and propagating changes to and from the
        Repository Access layer.</para>

      <para>Subversion's working copy library, libsvn_wc, is directly
        responsible for managing the data in the working copies.  To
        accomplish this, the library stores administrative information
        about each working copy directory within a special
        subdirectory.  This subdirectory, named
        <filename>.svn</filename>, is present in each working copy
        directory and contains various other files and directories
        which record state and provide a private workspace for
        administrative action.  For those familiar with CVS, this
        <filename>.svn</filename> subdirectory is similar in purpose
        to the <filename>CVS</filename> administrative directories
        found in CVS working copies.  For more information about the
        <filename>.svn</filename> administrative area, see <xref
        linkend="svn-ch-8-sect-3"/>in this chapter.</para>

      <para>The Subversion client library, libsvn_client, has the
        broadest responsibility; its job is to mingle the
        functionality of the working copy library with that of the
        Repository Access Layer, and then to provide the highest-level
        API to any application that wishes to perform general revision
        control actions.  For example, the function
        <function>svn_client_checkout</function> takes a URL as an
        argument.  It passes this URL to the RA layer and opens an
        authenticated session with a particular repository.  It then
        asks the repository for a certain tree, and sends this tree
        into the working copy library, which then writes a full
        working copy to disk (<filename>.svn</filename> directories
        and all).</para>

      <para>The client library is designed to be used by any
        application.  While the Subversion source code includes a
        standard command-line client, it should be very easy to write
        any number of GUI clients on top of the client library.  New
        GUIs (or any new client, really) for Subversion need not be
        clunky wrappers around the included command-line
        client&mdash;they have full access via the libsvn_client API
        to same functionality, data, and callback mechanisms that the
        command-line client uses.</para>

      <sidebar>
        <title>Binding Directly&mdash;A Word About Correctness</title>

        <para>Why should your GUI program bind directly with a
          libsvn_client instead of acting as a wrapper around a
          command-line program?  Besides simply being more efficient,
          this can address potential correctness issues as well.  A
          command-line program (like the one supplied with Subversion)
          that binds to the client library needs to effectively
          translate feedback and requested data bits from C types to
          some form of human-readable output.  This type of
          translation can be lossy.  That is, the program may not
          display all of the information harvested from the API, or
          may combine bits of information for compact representation.</para>

        <para>If you wrap such a command-line program with yet another
          program, the second program has access only to
          already-interpreted (and as we mentioned, likely incomplete)
          information, which it must <emphasis>again</emphasis>
          translate into <emphasis>its</emphasis> representation
          format.  With each layer of wrapping, the integrity of the
          original data is potentially tainted more and more, much
          like the result of making a copy of a copy (of a copy &hellip;)
          of a favorite audio or video cassette.</para>

      </sidebar>

    </sect2>
  </sect1>

  <!-- ******************************************************************* -->
  <!-- *** SECTION 2:  USING THE APIS                                  *** -->
  <!-- ******************************************************************* -->
  <sect1 id="svn-ch-8-sect-2">
    <title>Using the APIs</title>

    <para>Developing applications against the Subversion library APIs
      is fairly straightforward.  All of the public header files live
      in the <filename>subversion/include</filename> directory of the
      source tree.  These headers are copied into your system
      locations when you build and install Subversion itself from
      source.  These headers represent the entirety of the functions
      and types meant to be accessible by users of the Subversion
      libraries.</para>

    <para>The first thing you might notice is that Subversion's
      datatypes and functions are namespace protected.  Every public
      Subversion symbol name begins with <literal>svn_</literal>,
      followed by a short code for the library in which the symbol is
      defined (such as <literal>wc</literal>,
      <literal>client</literal>, <literal>fs</literal>, etc.),
      followed by a single underscore (<literal>_</literal>) and
      then the rest of the symbol name.  Semi-public functions (used
      among source files of a given library but not by code outside
      that library, and found inside the library directories
      themselves) differ from this naming scheme in that instead of a
      single underscore after the library code, they use a double
      underscore (<literal>__</literal>).  Functions that are private
      to a given source file have no special prefixing, and are declared
      <literal>static</literal>.  Of course, a compiler isn't
      interested in these naming conventions, but they help to clarify
      the scope of a given function or datatype.</para>

    <!-- ***************************************************************** -->
    <sect2 id="svn-ch-8-sect-2.1">
      <title>The Apache Portable Runtime Library</title>

      <para>Along with Subversion's own datatype, you will see many
        references to datatypes that begin with
        <literal>apr_</literal>&mdash;symbols from the Apache
        Portable Runtime (APR) library.  APR is Apache's portability
        library, originally carved out of its server code as an
        attempt to separate the OS-specific bits from the
        OS-independent portions of the code.  The result was a library
        that provides a generic API for performing operations that
        differ mildly&mdash;or wildly&mdash;from OS to OS.  While the
        Apache HTTP Server was obviously the first user of the APR
        library, the Subversion developers immediately recognized the
        value of using APR as well.  This means that there are
        practically no OS-specific code portions in Subversion itself.
        Also, it means that the Subversion client compiles and runs
        anywhere that the server does.  Currently this list includes
        all flavors of Unix, Win32, BeOS, OS/2, and Mac OS X.</para>


      <para>In addition to providing consistent implementations of
        system calls that differ across operating systems,
        <footnote>
          <para>Subversion uses ANSI system calls and datatypes as much
            as possible.</para>
        </footnote>
        APR gives Subversion immediate access to many custom
        datatypes, such as dynamic arrays and hash tables.  Subversion
        uses these types extensively throughout the codebase.  But
        perhaps the most pervasive APR datatype, found in nearly every
        Subversion API prototype, is the
        <literal>apr_pool_t</literal>&mdash;the APR memory pool.
        Subversion uses pools internally for all its memory allocation
        needs (unless an external library requires a different memory
        management schema for data passed through its API),
        <footnote>
          <para>Neon and Berkeley DB are examples of such libraries.</para>
        </footnote>
        and while a person coding against the Subversion APIs is
        not required to do the same, they are required to provide
        pools to the API functions that need them.  This means that
        users of the Subversion API must also link against APR, must
        call <function>apr_initialize()</function> to initialize the
        APR subsystem, and then must acquire a pool for use with
        Subversion API calls.  See <xref linkend="svn-ch-8-sect-5"/>
        for more information.</para>

    </sect2>

    <!-- ***************************************************************** -->
    <sect2 id="svn-ch-8-sect-2.2">
      <title>URL and Path Requirements</title>

      <para>With remote version control operation as the whole point
        of Subversion's existence, it makes sense that some attention
        has been paid to internationalization (i18n) support.  After
        all, while <quote>remote</quote> might mean <quote>across the
        office</quote>, it could just as well mean <quote>across the
        globe.</quote> To facilitate this, all of Subversion's public
        interfaces that accept path arguments expect those paths to be
        canonicalized, and encoded in UTF-8.  This means, for example,
        that any new client binary that drives the libsvn_client
        interface needs to first convert paths from the
        locale-specific encoding to UTF-8 before passing those paths
        to the Subversion libraries, and then re-convert any resultant
        output paths from Subversion back into the locale's encoding
        before using those paths for non-Subversion purposes.
        Fortunately, Subversion provides a suite of functions (see
        <filename>subversion/include/svn_utf.h</filename>) that can be
        used by any program to do these conversions.</para>

      <para>Also, Subversion APIs require all URL parameters to be
        properly URI-encoded.  So, instead of passing <systemitem
        class="url">file:///home/username/My File.txt</systemitem> as
        the URL of a file named <literal>My File.txt</literal>, you
        need to pass <systemitem
        class="url">file:///home/username/My%20File.txt</systemitem>.
        Again, Subversion supplies helper functions that your
        application can
        use&mdash;<function>svn_path_uri_encode</function> and
        <function>svn_path_uri_decode</function>, for URI encoding and
        decoding, respectively.</para>
    </sect2>

    <!-- ***************************************************************** -->
    <sect2 id="svn-ch-8-sect-2.3">
      <title>Using Languages Other than C and C++</title> 

      <para>If you are interested in using the Subversion libraries in
        conjunction with something other than a C program&mdash;say a
        Python script or Java application&mdash;Subversion has some
        initial support for this via the Simplified Wrapper and
        Interface Generator (SWIG).  The SWIG bindings for Subversion
        are located in <filename>subversion/bindings/swig</filename>
        and are slowly maturing into a usable state.  These bindings
        allow you to call Subversion API functions indirectly, using
        wrappers that translate the datatypes native to your
        scripting language into the datatypes needed by Subversion's
        C libraries.</para>

      <para>There is an obvious benefit to accessing the Subversion
        APIs via a language binding&mdash;simplicity.  Generally
        speaking, languages such as Python and Perl are much more
        flexible and easy to use than C or C++.  The sort of
        high-level datatypes and context-driven type checking provided
        by these languages are often better at handling information
        that comes from users.  As you know, humans are proficient at
        botching up input to a program, and scripting languages tend
        to handle that misinformation more gracefully.  Of course,
        often that flexibility comes at the cost of performance.  That
        is why using a tightly-optimized, C-based interface and
        library suite, combined with a powerful, flexible binding
        language, is so appealing.</para>

      <para>Let's look at an example that uses Subversion's Python
        SWIG bindings.  Our example will do the same thing as our last
        example.  Note the difference in size and complexity of the
        function this time!</para>

      <example id="svn-ch-8-sect-2.3-ex-1">
        <title>Using the Repository Layer with Python</title>

        <programlisting>
from svn import fs
import os.path

def crawl_filesystem_dir (root, directory, pool):
  """Recursively crawl DIRECTORY under ROOT in the filesystem, and return
  a list of all the paths at or below DIRECTORY.  Use POOL for all 
  allocations."""

  # Get the directory entries for DIRECTORY.
  entries = fs.dir_entries(root, directory, pool)

  # Initialize our returned list with the directory path itself.
  paths = [directory]

  # Loop over the entries
  names = entries.keys()
  for name in names:
    # Calculate the entry's full path.
    full_path = os.path.join(basepath, name)

    # If the entry is a directory, recurse.  The recursion will return
    # a list with the entry and all its children, which we will add to
    # our running list of paths.
    if fs.is_dir(fsroot, full_path, pool):
      subpaths = crawl_filesystem_dir(root, full_path, pool)
      paths.extend(subpaths)

    # Else, it is a file, so add the entry's full path to the FILES list.
    else:
      paths.append(full_path)

  return paths
</programlisting>
      </example>

      <para>An implementation in C of the previous example would
        stretch on quite a bit longer.  The same routine in C would
        need to pay close attention to memory usage, and need to use
        custom datatypes for representing the hash of entries and the
        list of paths.  Python has hashes (called
        <quote>dictionaries</quote>) and lists as built-in datatypes,
        and provides a wonderful selection of methods for operating on
        those types.  And since Python uses reference counting and
        garbage collection, users of the language don't have to bother
        themselves with allocating and freeing memory.</para>

      <para>In the previous section of this chapter, we mentioned the
        <filename>libsvn_client</filename> interface, and how it
        exists for the sole purpose of simplifying the process of
        writing a Subversion client.  The following is a brief example
        of how that library can be accessed via the SWIG bindings.  In
        just a few lines of Python, you can check out a fully
        functional Subversion working copy!</para>

      <example id="svn-ch-8-sect-2.3-ex-2">
        <title>A Simple Script to Check Out a Working Copy.</title>

        <programlisting>
#!/usr/bin/env python
import sys
from svn import util, _util, _client

def usage():
  print "Usage: " + sys.argv[0] + " URL PATH\n"
  sys.exit(0)

def run(url, path):
  # Initialize APR and get a POOL.
  _util.apr_initialize()
  pool = util.svn_pool_create(None)

  # Checkout the HEAD of URL into PATH (silently)
  _client.svn_client_checkout(None, None, url, path, -1, 1, None, pool)

  # Cleanup our POOL, and shut down APR.
  util.svn_pool_destroy(pool)
  _util.apr_terminate()

if __name__ == '__main__':
  if len(sys.argv) != 3:
    usage()
  run(sys.argv[1], sys.argv[2])
</programlisting>
      </example>

      <para>Subversion's language bindings unfortunately tend to lack
        the level of attention given to the core Subversion modules.
        However, there have been significant efforts towards creating
        functional bindings for Python, Perl, and Java.  Once you have
        the SWIG interface files properly configured, generation of
        the specific wrappers for all the supported SWIG languages
        (which currently includes versions of C#, Guile, Java,
        MzScheme, OCaml, Perl, PHP, Python, Ruby, and Tcl) should
        theoretically be trivial.  Still, some extra programming is
        required to compensate for complex APIs that SWIG needs some
        help generalizing.  For more information on SWIG itself, see
        the project's website at <systemitem
        class="url">http://www.swig.org/</systemitem>.</para>


    </sect2>
  </sect1>

  <!-- ******************************************************************* -->
  <!-- *** SECTION 3:  INSIDE THE WORKING COPY ADMINISTRATION AREA     *** -->
  <!-- ******************************************************************* -->
  <sect1 id="svn-ch-8-sect-3">
    <title>Inside the Working Copy Administration Area</title>
    
    <para>As we mentioned earlier, each directory of a Subversion
      working copy contains a special subdirectory called
      <filename>.svn</filename> which houses administrative data about
      that working copy directory.  Subversion uses the information in
      <filename>.svn</filename> to keep track of things like:</para>

    <itemizedlist>
      <listitem>
        <para>Which repository location(s) are represented by the
          files and subdirectories in the working copy
          directory.</para>
      </listitem>
      <listitem>
        <para>What revision of each of those files and directories are
          currently present in the working copy.</para>
      </listitem>
      <listitem>
        <para>Any user-defined properties that might be attached
          to those files and directories.</para>
      </listitem>
      <listitem>
        <para>Pristine (un-edited) copies of the working copy
        files.</para>
      </listitem>
    </itemizedlist>

    <para>While there are several other bits of data stored in the
      <filename>.svn</filename> directory, we will examine only a
      couple of the most important items.</para>

    <!-- ***************************************************************** -->
    <sect2 id="svn-ch-8-sect-3.1">
      <title>The Entries File</title>

      <para>Perhaps the single most important file in the
        <filename>.svn</filename> directory is the
        <filename>entries</filename> file.  The entries file is an XML
        document which contains the bulk of the administrative
        information about a versioned resource in a working copy
        directory.  It is this one file which tracks the repository
        URLs, pristine revision, file checksums, pristine text and
        property timestamps, scheduling and conflict state
        information, last-known commit information (author, revision,
        timestamp), local copy history&mdash;practically everything
        that a Subversion client is interested in knowing about a
        versioned (or to-be-versioned) resource!</para>

      <sidebar>
        <title>Comparing the Administrative Areas of Subversion and
          CVS</title>

        <para>A glance inside the typical <filename>.svn</filename>
          directory turns up a bit more than what CVS maintains in its
          <filename>CVS</filename> administrative directories.  The
          <filename>entries</filename> file contains XML which
          describes the current state of the working copy directory,
          and basically serves the purposes of CVS's
          <filename>Entries</filename>, <filename>Root</filename>, and
          <filename>Repository</filename> files combined.</para>

      </sidebar>

      <para>The following is an example of an actual entries
        file:</para>

      <example id="svn-ch-8-sect-3-ex-1">
        <title>Contents of a Typical <filename>.svn/entries</filename>
          File</title>
        <programlisting>
&lt;?xml version="1.0" encoding="utf-8"?&gt;
&lt;wc-entries
   xmlns="svn:"&gt;
&lt;entry
   committed-rev="1"
   name="svn:this_dir"
   committed-date="2002-09-24T17:12:44.064475Z"
   url="http://svn.red-bean.com/tests/.greek-repo/A/D"
   kind="dir"
   revision="1"/&gt;
&lt;entry
   committed-rev="1"
   name="gamma"
   text-time="2002-09-26T21:09:02.000000Z"
   committed-date="2002-09-24T17:12:44.064475Z"
   checksum="QSE4vWd9ZM0cMvr7/+YkXQ=="
   kind="file"
   prop-time="2002-09-26T21:09:02.000000Z"/&gt;
&lt;entry
   name="zeta"
   kind="file"
   schedule="add"
   revision="0"/&gt;
&lt;entry
   url="http://svn.red-bean.com/tests/.greek-repo/A/B/delta"
   name="delta"
   kind="file"
   schedule="add"
   revision="0"/&gt;
&lt;entry
   name="G"
   kind="dir"/&gt;
&lt;entry
   name="H"
   kind="dir"
   schedule="delete"/&gt;
&lt;/wc-entries&gt;
</programlisting>
      </example>

      <para>As you can see, the entries file is essentially a list of
        entries.  Each <sgmltag>entry</sgmltag> tag represents one of
        three things: the working copy directory itself (called the
        <quote>this directory</quote> entry, and noted as having an
        empty value for its <structfield>name</structfield>
        attribute), a file in that working copy directory (noted by
        having its <structfield>kind</structfield> attribute set to
        <literal>"file"</literal>), or a subdirectory in that working
        copy (<structfield>kind</structfield> here is set to
        <literal>"dir"</literal>).  The files and subdirectories whose
        entries are stored in this file are either already under
        version control, or (as in the case of the file named
        <filename>zeta</filename> above) are scheduled to be added to
        version control when the user next commits this working copy
        directory's changes.  Each entry has a unique name, and each
        entry has a node kind.</para>

      <para>Developers should be aware of some special rules that
        Subversion uses when reading and writing its
        <filename>entries</filename> files.  While each entry has a
        revision and URL associated with it, note that not every
        <sgmltag>entry</sgmltag> tag in the sample file has explicit
        <structfield>revision</structfield> or
        <structfield>url</structfield> attributes attached to it.
        Subversion allows entries to not explicitly store those two
        attributes when their values are the same as (in the
        <structfield>revision</structfield> case) or trivially
        calculable from
        <footnote>
          <para>That is, the URL for the entry is the same as the
            concatenation of the parent directory's URL and the
            entry's name.</para>
        </footnote>
        (in the <structfield>url</structfield> case) the data stored
        in the <quote>this directory</quote> entry.  Note also that
        for subdirectory entries, Subversion stores only the crucial
        attributes&mdash;name, kind, url, revision, and schedule.  In
        an effort to reduce duplicated information, Subversion
        dictates that the method for determining the full set of
        information about a subdirectory is to traverse down into that
        subdirectory, and read the <quote>this directory</quote> entry
        from its own <filename>.svn/entries</filename> file.  However,
        a reference to the subdirectory is kept in its parent's
        <filename>entries</filename> file, with enough information to
        permit basic versioning operations in the event that the
        subdirectory itself is actually missing from disk.</para>

    </sect2>

    <!-- ***************************************************************** -->
    <sect2 id="svn-ch-8-sect-3.2">
      <title>Pristine Copies and Property Files</title>

      <para>As mentioned before, the <filename>.svn</filename>
        directory also holds the pristine <quote>text-base</quote>
        versions of files.  Those can be found in
        <filename>.svn/text-base</filename>.  The benefits of these
        pristine copies are multiple&mdash;network-free checks for
        local modifications and difference reporting, network-free
        reversion of modified or missing files, smaller transmission
        of changes to the server&mdash;but comes at the cost of having
        each versioned file stored at least twice on disk.  These
        days, this seems to be a negligible penalty for most files.
        However, the situation gets uglier as the size of your
        versioned files grows.  Some attention is being given to
        making the presence of the <quote>text-base</quote> an option.
        Ironically though, it is as your versioned files' sizes get
        larger that the existence of the <quote>text-base</quote>
        becomes more crucial&mdash;who wants to transmit a huge file
        across a network just because they want to commit a tiny
        change to it?</para>

      <para>Similar in purpose to the <quote>text-base</quote> files
        are the property files and their pristine
        <quote>prop-base</quote> copies, located in
        <filename>.svn/props</filename> and
        <filename>.svn/prop-base</filename> respectively.  Since
        directories can have properties, too, there are also
        <filename>.svn/dir-props</filename> and
        <filename>.svn/dir-prop-base</filename> files.  Each of these
        property files (<quote>working</quote> and <quote>base</quote>
        versions) uses a simple <quote>hash-on-disk</quote> file
        format for storing the property names and values.</para>


    </sect2>
  </sect1>

  <!-- ******************************************************************* -->
  <!-- *** SECTION 4:  WEBDAV                                          *** -->
  <!-- ******************************************************************* -->
  <sect1 id="svn-ch-8-sect-4">
    <title>WebDAV</title>

    <para>WebDAV (shorthand for <quote>Web-based Distributed Authoring
      and Versioning</quote>) is an extension of the standard HTTP
      protocol designed to make the web into a read/write medium,
      instead of the basically read-only medium that exists today.
      The theory is that directories and files can be shared&mdash;as
      both readable and writable objects&mdash;over the web.  RFCs
      2518 and 3253 describe the WebDAV/DeltaV extensions to HTTP, and
      are available (along with a lot of other useful information) at
      <systemitem
      class="url">http://www.webdav.org/</systemitem>.</para>

    <para>A number of operating system file browsers are already able
      to mount networked directories using WebDAV.  On Win32, the
      Windows Explorer can browse what it calls WebFolders (which are
      just WebDAV-ready network locations) as if they were regular
      shared folders.  Mac OS X also has this capability, as do the
      Nautilus and Konqueror browsers (under GNOME and KDE,
      respectively).</para>

    <para>How does all of this apply to Subversion?  The mod_dav_svn
      Apache module uses HTTP, extended by WebDAV and DeltaV, as one
      of its network protocols.  Subversion uses mod_dav_svn to map
      between Subversion's versioning concepts and those of RFCs 2518
      and 3253.
    </para>

    <para>For a more thorough discussion of WebDAV, how it works, and
      how Subversion uses it, see <xref linkend="svn-ap-c"/>.  Among
      other things, that appendix discusses the degree to which
      Subversion adheres to the generic WebDAV specification, and how
      that affects interoperability with generic WebDAV
      clients.</para>
  </sect1>

  <!-- ******************************************************************* -->
  <!-- *** SECTION 5:  PROGRAMMING WITH MEMORY POOLS                   *** -->
  <!-- ******************************************************************* -->
  <sect1 id="svn-ch-8-sect-5">
    <title>Programming with Memory Pools</title>

    <para>Almost every developer who has used the C programming
      language has at some point sighed at the daunting task of
      managing memory usage.  Allocating enough memory to use, keeping
      track of those allocations, freeing the memory when you no
      longer need it&mdash;these tasks can be quite complex.  And of
      course, failure to do those things properly can result in a
      program that crashes itself, or worse, crashes the computer.
      Fortunately, the APR library that Subversion depends on for
      portability provides the <structname>apr_pool_t</structname>
      type, which represents a pool from which the application may
      allocate memory.</para>

    <para>A memory pool is an abstract representation of a chunk of
      memory allocated for use by a program.  Rather than requesting
      memory directly from the OS using the standard
      <function>malloc()</function> and friends, programs that link
      against APR can simply request that a pool of memory be created
      (using the <function>apr_pool_create()</function> function).
      APR will allocate a moderately sized chunk of memory from the
      OS, and that memory will be instantly available for use by the
      program.  Any time the program needs some of the pool memory, it
      uses one of the APR pool API functions, like
      <function>apr_palloc()</function>, which returns a generic
      memory location from the pool.  The program can keep requesting
      bits and pieces of memory from the pool, and APR will keep
      granting the requests.  Pools will automatically grow in size to
      accommodate programs that request more memory than the original
      pool contained, until of course there is no more memory
      available on the system.</para>

    <para>Now, if this were the end of the pool story, it would hardly
      have merited special attention.  Fortunately, that's not the
      case.  Pools can not only be created; they can also be cleared
      and destroyed, using <function>apr_pool_clear()</function> and
      <function>apr_pool_destroy()</function> respectively.  This
      gives developers the flexibility to allocate several&mdash;or
      several thousand&mdash;things from the pool, and then clean up
      all of that memory with a single function call!  Further, pools
      have hierarchy.  You can make <quote>subpools</quote> of any
      previously created pool.  When you clear a pool, all of its
      subpools are destroyed; if you destroy a pool, it and its
      subpools are destroyed.</para>

    <para>Before we go further, developers should be aware that they
      probably will not find many calls to the APR pool functions we
      just mentioned in the Subversion source code.  APR pools offer
      some extensibility mechanisms, like the ability to have custom
      <quote>user data</quote> attached to the pool, and mechanisms
      for registering cleanup functions that get called when the pool
      is destroyed.  Subversion makes use of these extensions in a
      somewhat non-trivial way.  So, Subversion supplies (and most of
      its code uses) the wrapper functions
      <function>svn_pool_create()</function>,
      <function>svn_pool_clear()</function>, and
      <function>svn_pool_destroy()</function>.</para>

    <para>While pools are helpful for basic memory management, the
      pool construct really shines in looping and recursive scenarios.
      Since loops are often unbounded in their iterations, and
      recursions in their depth, memory consumption in these areas of
      the code can become unpredictable.  Fortunately, using nested
      memory pools can be a great way to easily manage these
      potentially hairy situations.  The following example
      demonstrates the basic use of nested pools in a situation that
      is fairly common&mdash;recursively crawling a directory tree,
      doing some task to each thing in the tree.</para>

    <example id="svn-ch-8-sect-5-ex-1">
      <title>Effective Pool Usage</title>
      <programlisting>
/* Recursively crawl over DIRECTORY, adding the paths of all its file
   children to the FILES array, and doing some task to each path
   encountered.  Use POOL for the all temporary allocations, and store
   the hash paths in the same pool as the hash itself is allocated in.  */
static apr_status_t 
crawl_dir (apr_array_header_t *files,
           const char *directory,
           apr_pool_t *pool)
{
  apr_pool_t *hash_pool = files-&gt;pool;  /* array pool */
  apr_pool_t *subpool = svn_pool_create (pool);  /* iteration pool */
  apr_dir_t *dir;
  apr_finfo_t finfo;
  apr_status_t apr_err;
  apr_int32_t flags = APR_FINFO_TYPE | APR_FINFO_NAME;

  apr_err = apr_dir_open (&amp;dir, directory, pool);
  if (apr_err)
    return apr_err;

  /* Loop over the directory entries, clearing the subpool at the top of
     each iteration.  */
  for (apr_err = apr_dir_read (&amp;finfo, flags, dir);
       apr_err == APR_SUCCESS;
       apr_err = apr_dir_read (&amp;finfo, flags, dir))
    {
      const char *child_path;

      /* Clear the per-iteration SUBPOOL.  */
      svn_pool_clear (subpool);

      /* Skip entries for "this dir" ('.') and its parent ('..').  */
      if (finfo.filetype == APR_DIR)
        {
          if (finfo.name[0] == '.'
              &amp;&amp; (finfo.name[1] == '\0'
                  || (finfo.name[1] == '.' &amp;&amp; finfo.name[2] == '\0')))
            continue;
        }

      /* Build CHILD_PATH from DIRECTORY and FINFO.name.  */
      child_path = svn_path_join (directory, finfo.name, subpool);

      /* Do some task to this encountered path. */
      do_some_task (child_path, subpool);

      /* Handle subdirectories by recursing into them, passing SUBPOOL
         as the pool for temporary allocations.  */
      if (finfo.filetype == APR_DIR)
        {
          apr_err = crawl_dir (files, child_path, subpool);
          if (apr_err)
            return apr_err;
        }

      /* Handle files by adding their paths to the FILES array.  */
      else if (finfo.filetype == APR_REG)
        {
          /* Copy the file's path into the FILES array's pool.  */
          child_path = apr_pstrdup (hash_pool, child_path);

          /* Add the path to the array.  */
          (*((const char **) apr_array_push (files))) = child_path;
        }
    }

  /* Destroy SUBPOOL.  */
  svn_pool_destroy (subpool);

  /* Check that the loop exited cleanly. */
  if (apr_err)
    return apr_err;

  /* Yes, it exited cleanly, so close the dir. */
  apr_err = apr_dir_close (dir);
  if (apr_err)
    return apr_err;

  return APR_SUCCESS;
}
</programlisting>
    </example>


    <para>The previous example demonstrates effective pool usage in
      <emphasis>both</emphasis> looping and recursive situations.
      Each recursion begins by making a subpool of the pool passed to
      the function.  This subpool is used for the looping region, and
      cleared with each iteration.  The result is memory usage is
      roughly proportional to the depth of the recursion, not to total
      number of file and directories present as children of the
      top-level directory.  When the first call to this recursive
      function finally finishes, there is actually very little data
      stored in the pool that was passed to it.  Now imagine the extra
      complexity that would be present if this function had to
      <function>alloc()</function> and <function>free()</function>
      every single piece of data used!</para>
    
    <para>Pools might not be ideal for every application, but they are
      extremely useful in Subversion.  As a Subversion developer,
      you'll need to grow comfortable with pools and how to wield them
      correctly.  Memory usage bugs and bloating can be difficult to
      diagnose and fix regardless of the API, but the pool construct
      provided by APR has proven a tremendously convenient,
      time-saving bit of functionality.</para>

  </sect1>

  <!-- ******************************************************************* -->
  <!-- *** SECTION 6:  CONTRIBUTING TO SUBVERSION                      *** -->
  <!-- ******************************************************************* -->
  <sect1 id="svn-ch-8-sect-6">
    <title>Contributing to Subversion</title>
    
    <para>The official source of information about the Subversion
      project is, of course, the project's website at <systemitem
      class="url">http://subversion.tigris.org/</systemitem>.  There
      you can find information about getting access to the source code
      and participating on the discussion lists.  The Subversion
      community always welcomes new members.  If you are
      interested in participating in this community by contributing
      changes to the source code, here are some hints on how to get
      started.</para>

    <!-- ***************************************************************** -->
    <sect2 id="svn-ch-8-sect-6.1">
      <title>Join the Community</title>
      
      <para>The first step in community participation is to find a way
        to stay on top of the latest happenings.  To do this most
        effectively, you will want to subscribe to the main developer
        discussion list (<email>dev@subversion.tigris.org</email>) and
        commit mail list (<email>svn@subversion.tigris.org</email>).
        By following these lists even loosely, you will have access
        to important design discussions, be able to see actual changes
        to Subversion source code as they occur, and be able to
        witness peer reviews of those changes and proposed changes.
        These email based discussion lists are the primary
        communication media for Subversion development.  See the
        Mailing Lists section of the website for other
        Subversion-related lists you might be interested in.</para>

      <para>But how do you know what needs to be done?  It is quite
        common for a programmer to have the greatest intentions of
        helping out with the development, yet be unable to find a good
        starting point.  After all, not many folks come to the
        community having already decided on a particular itch they
        would like to scratch.  But by watching the developer
        discussion lists, you might see mentions of existing bugs or
        feature requests fly by that particularly interest you.  Also,
        a great place to look for outstanding, unclaimed tasks is the
        Issue Tracking database on the Subversion website.  There you
        will find the current list of known bugs and feature requests.
        If you want to start with something small, look for issues
        marked as <quote>bite-sized</quote>.</para>
      
    </sect2>
    
    <!-- ***************************************************************** -->
    <sect2 id="svn-ch-8-sect-6.2">
      <title>Get the Source Code</title>
      
      <para>To edit the code, you need to have the code.  This means
        you need to check out a working copy from the public
        Subversion source repository.  As straightforward as that
        might sound, the task can be slightly tricky.  Because
        Subversion's source code is versioned using Subversion itself,
        you actually need to <quote>bootstrap</quote> by getting a
        working Subversion client via some other method.  The most
        common methods include downloading the latest binary
        distribution (if such is available for your platform), or
        downloading the latest source tarball and building your own
        Subversion client.  If you build from source, make sure to
        read the <filename>INSTALL</filename> file in the top level of
        the source tree for instructions.</para>

      <para>After you have a working Subversion client, you are now
        poised to checkout a working copy of the Subversion source
        repository from <systemitem
        class="url">http://svn.collab.net/repos/svn/trunk/</systemitem>:
        <footnote>
          <para>Note that the URL checked out in the example above
            ends not with <literal>svn</literal>, but with a
            subdirectory thereof called <literal>trunk</literal>.  See
            our discussion of Subversion's branching and tagging model
            for the reasoning behind this.</para>
        </footnote></para>

      <screen>
$ svn checkout http://svn.collab.net/repos/svn/trunk subversion
A  HACKING
A  INSTALL
A  README
A  autogen.sh
A  build.conf
&hellip;
</screen>

      <para>The above command will checkout the bleeding-edge, latest
        version of the Subversion source code into a subdirectory
        named <filename>subversion</filename> in your current working
        directory.  Obviously, you can adjust that last argument as
        you see fit.  Regardless of what you call the new working copy
        directory, though, after this operation completes, you will
        now have the Subversion source code.  Of course, you will
        still need to fetch a few helper libraries (apr, apr-util,
        etc.)&mdash;see the <filename>INSTALL</filename> file in the
        top level of the working copy for details.</para>

    </sect2>

    <!-- ***************************************************************** -->
    <sect2 id="svn-ch-8-sect-6.3">
      <title>Become Familiar with Community Policies</title>
      
      <para>Now that you have a working copy containing the latest
        Subversion source code, you will most certainly want to take a
        cruise through the <filename>HACKING</filename> file in that
        working copy's top-level directory.  The
        <filename>HACKING</filename> file contains general
        instructions for contributing to Subversion, including how to
        properly format your source code for consistency with the rest
        of the codebase, how to describe your proposed changes with an
        effective change log message, how to test your changes, and so
        on.  Commit privileges on the Subversion source repository are
        earned&mdash;a government by meritocracy.
        <footnote>
          <para>While this may superficially appear as some sort of
            elitism, this <quote>earn your commit privileges</quote>
            notion is about efficiency&mdash;whether it costs more in
            time and effort to review and apply someone else's changes
            that are likely to be safe and useful, versus the
            potential costs of undoing changes that are
            dangerous.</para>
        </footnote>
        The HACKING file is an invaluable resource when it comes to 
        making sure that your proposed changes earn the praises they
        deserve without being rejected on technicalities.</para>

    </sect2>

    <!-- ***************************************************************** -->
    <sect2 id="svn-ch-8-sect-6.4">
      <title>Make and Test Your Changes</title>
      
      <para>With the code and community policy understanding in hand,
        you are ready to make your changes.  It is best to try to make
        smaller but related sets of changes, even tackling larger
        tasks in stages, instead of making huge, sweeping
        modifications.  Your proposed changes will be easier to
        understand (and therefore easier to review) if you disturb
        the fewest lines of code possible to accomplish your task
        properly.  After making each set of proposed changes, your
        Subversion tree should be in a state in which the software
        compiles with no warnings.</para>

      <para>Subversion has a fairly thorough
        <footnote>
          <para>You might want to grab some popcorn.
            <quote>Thorough</quote>, in this instance, translates to
            somewhere in the neighborhood of thirty minutes of
            non-interactive machine churn.</para>
        </footnote>
        regression test suite, and your proposed changes are expected
        to not cause any of those tests to fail.  By running
        <command>make check</command> (in Unix) from the top of the
        source tree, you can sanity-check your changes.  The fastest
        way to get your code contributions rejected (other than
        failing to supply a good log message) is to submit changes
        that cause failure in the test suite.</para>

      <!-- ### TODO: Describe building and testing on Windows. -->

      <para>In the best-case scenario, you will have actually added
        appropriate tests to that test suite which verify that your
        proposed changes work as expected.  In fact,
        sometimes the best contribution a person can make is solely
        the addition of new tests.  You can write regression tests for
        functionality that currently works in Subversion as a way to
        protect against future changes that might trigger failure in
        those areas.  Also, you can write new tests that demonstrate
        known failures.  For this purpose, the Subversion test suite
        allows you to specify that a given test is expected to fail
        (called an <literal>XFAIL</literal>), and so long as
        Subversion fails in the way that was expected, a test result
        of <literal>XFAIL</literal> itself is considered a success.
        Ultimately, the better the test suite, the less time wasted on
        diagnosing potentially obscure regression bugs.</para>


    </sect2>

    <!-- ***************************************************************** -->
    <sect2 id="svn-ch-8-sect-6.5">
      <title>Donate Your Changes</title>
      
      <para>After making your modifications to the source code,
        compose a clear and concise log message to describe those
        changes and the reasons for them.  Then, send an email to the
        developers list containing your log message and the output of
        <command>svn diff</command> (from the top of your Subversion
        working copy).  If the community members consider your changes
        acceptable, someone who has commit privileges (permission to
        make new revisions in the Subversion source repository) will
        add your changes to the public source code tree.  Recall that
        permission to directly commit changes to the repository is
        granted on merit&mdash;if you demonstrate comprehension of
        Subversion, programming competency, and a <quote>team
        spirit</quote>, you will likely be awarded that
        permission.</para>

    </sect2>
  </sect1>
</chapter>

<!--
local variables: 
sgml-parent-document: ("book.xml" "chapter")
end:
-->

Show details Hide details

Change log

r1122 by cmpilato on Feb 25, 2005 Diff

Rename tag to associate with English
version only.

Go to:

Older revisions

r1107 by sussman on Feb 23, 2005 Diff

Create 'book-1.0-final' tag.  This is
a snapshot of the last version
of the book's trunk before we started
documenting 1.1 features.

r653 by maxb on Sep 05, 2004 Diff

Misc. book fixes: grammar and
reference, and also modified some text
in the
incremental vs. full backup section of
chapter 5 to avoid confusion.
...

r651 by maxb on Sep 04, 2004 Diff

A bunch of minor typographical
corrections from issue 1216.
Thanks to Gareth McCaughan
<Gareth.McCaughan_at_pobox.com>.

...

All revisions of this file

File info

Size: 82141 bytes, 1689 lines

View raw file

File properties

svn:mime-type: text/xml
svn:eol-style: native