ResourceOracle - google-web-toolkit - This document describes how GWT chooses resources.

Updated Apr 15, 2009 by amitman...@google.com

ResourceOracle

This document describes how GWT chooses resources.

Introduction

GWT models pretty much everything as a resource. A resource might be a java source file, a class file, an html file, or a jar file. This design document describes how GWT determines which resources to include in a module.

Glossary

Determining the resources a GWT module includes

Conceptually, this determination is a three step process.

Step 1: Construct the list of `ClassPathEntry` and the `PathPrefixSet` for a module

The resources included by a GWT module depends mainly on two things:

ClassPathEntry: The order in which the ClassPathEntry are specified is important. This order is solely determined by the order in which entries are specified on the java classpath.
PathPrefixSet: The order in which various PathPrefix were added to the PathPrefixSet is important. This order is determined by traversing the module inheritance tree rooted at the entry-point module in the lexical order. Plus, whether or not a PathPrefix causes its resources to be rerooted is important. Note that a PathPrefix with a source tag is not re-rooted but PathPrefix with public and super-src tags are re-rooted. For example, if the modules are as follows:

 Module MyApp.gwt.xml:
  <module>
    <inherits name="User.gwt.xml"/>
    <inherits name="Dev.gwt.xml"/>
    <source path="a/b" includes="*.java" />
    <entry-point class="..." />
  </module>

 Module User.gwt.xml: 
   <module>
     <source path="" />
   </module>

 Module Dev.gwt.xml: 
   <module>
     <super-source path="a" />
   </module>

the PathPrefixSet constructed is as follows. Think of the modules being inlined to determine the lexical ordering of the PathPrefix.

  PathPrefixSet pps = new PathPrefixSet();
  p1 = new PathPrefix("");  // From module User.gwt.xml
  p2 = new PathPrefix("a/", WITH_REROOTING); // From module Dev.gwt.xml
  p3 = new PathPrefix("a/b/", filter allows only *.java);  // From module MyApp.gwt.xml

  pps.add(p1);
  pps.add(p2);
  pps.add(p3);

Step 2: Determine the bag of resources for the list of `ClassPathEntry` and `PathPrefixSet`

For each ClassPathEntry, there is a deterministic collection of resources that is allowed by a PathPrefixSet. Specifically, each Resource matches a unique PathPrefix in a PathPrefixSet. The resource is allowed if the filter of the matching PathPrefix allows the resource. Lastly, the path of each allowed resource is computed. A resource is either a rerooted resource or a normal resource, depending on whether the PathPrefix that allows it requires rerooting or not. A rerooted resource's path is its path name following the ClassPathEntry and the PathPrefix. A normal resource's path is its name following the ClassPathEntry. In the example below, resources r2 and r5 match PathPrefix p2. Therefore, their (rerooted) path is "Test.java" and "a/b/Test.java" respectively.

      Initial path     Matching PathPrefix           allowed        Path
r1    Test.java               p1                       yes          Test.java
r2    a/Test.java             p2                       yes          Test.java (rerooted)
r3    a/b/Test.java           p3                       yes          a/b/Test.java
r4    a/b/gwt.gif             p3                       No           n/a
r5    a/a/b/Test.java         p2                       yes          a/b/Test.java (rerooted)

Step 3: Determine which resource to include for each unique path

From this collection of resources, where each resource has an associated PathPrefix, ClassPathEntry, and a path, one resource is selected for each unique path. If there are multiple resources for a path, the following tie-breaker rules are used:

A rerooted resource is preferred over a non-rerooted resource.
A resource with a matching PathPrefix that was added to the PathPrefixSet later is preferred over a resource with a matching PathPrefix that was added earlier to the PathPrefixSet.
A resource with earlier ClassPathEntry is preferred over a resource with a later ClassPathEntry.

These rules are ordered. Thus, a rerooted resource is always preferred over a non-rerooted resource, no matter how their matching PathPrefix and ClassPathEntry compare. To continue with the above example, the final resource map is:

Test.java => r2
a/b/Test.java => r5

because resource r2 shadows r1 and resource r5 shadows r3. The examples above did not involve a tie-breaker rule involving ClassPathEntry, but it is easy to see how it would be used.

Steps taken during a refresh

A ResourceOracleImpl refresh maintains the following invariants:

If no resources change during the refresh, the identities of all the Collections exposed by the ResourceOracleImpl remains the same.
If any of the previous resource does not change during the refresh, its identity remains the same.

During a refresh, as a first step, the resource map is recomputed. If none of the resources in the map have changed, the previous collections are kept as is. This guarantees invariant 1. Furthermore, if a resource has not changed, the old resource is used instead of the new resource, thus guaranteeing invariant 2.

When multiple `PathPrefix`es have the same path

(Note: As of Gwt 1.6, this section has not been implemented)

As of Gwt 1.6, if multiple PathPrefixes have the same path attribute but different ResourceFilters, only the last one is kept. This is an undesirable outcome in most cases. For example, if there are two PathPrefixes p4 and p5:

p4: <source path="client" includes="*.java" />

p5: <source path="client" includes="Foo.java" />

only the second path-prefix (p5) is kept. In particular, a path like client/Bar.java is excluded, as a result of being excluded by the second path-prefix. This outcome is probably different from what the user expects. In this scenario, any path-prefixes defined by a user, however specific, could end up clobbering any path-prefixes that the user's modules inherit. The primary reason this clobbering happens is because Gwt 1.6 internally does not distinguish between the inclusion (exclusion) being specifically mentioned by the PathPrefix and the inclusion (exclusion) due to defaults. When a resource with a specified path is processed by a Gwt 1.6 PathPrefix, the output is either include or exclude. The resource is included if the output is include. Else, the output is exclude and the resource is excluded. To handle multiple PathPrefixes meaningfully, we generalize the output to be:

include: if the include filter of the PathPrefix specifically includes the resource and its exclude filter does not excludes the resource.
exclude: if the exclude filter of the PathPrefix specifically excludes the resource.
unspecified_include: if the resource is included just by default. For example, a path-prefix <source path="client" /> includes all java files in the client directory by default.
unspecified_exclude: if none of the above three outputs apply. For example, a path-prefix <source path="client" includes="Foo.java" /> excludes a file with path as client/Bar.java by default.

For each path-prefix path, an ordered list of PathPrefixes is maintained, with the lexically last PathPrefix at the head of the list. If a resource matches a PathPrefix path, path-prefixes are applied from the ordered list starting from the head. The next PathPrefix in the list is applied only if the previous path-prefix filter's output is either unspecified_include or unspecified_exclude. If the output remains unspecified_include (unspecified_exclude) at the end, it is treated as an include (exclude). As with Gwt 1.6, the resource is included if the output is include at the end. Else, the output is exclude and it is excluded.

This algorithm is just a generalization of Gwt 1.6's resource filtering algorithm. It should not be a breaking change except in a few edge cases relying on inclusion or exclusion by default.