Our Mashing it up with Google Mashup Editor tutorial showed how to plot Google Calendar meeting locations on an interactive map by invoking the Google Maps API from a Google Mashup Editor (GME) application. When the user selects an entry from the list of calendar events, that GME application invokes a small JavaScript function to convert the calendar feed's textual location to longitude and latitude via the Google Map API's GClientGeocoder.getLatLng() method. This time we examine an alternative solution that eliminates the need for JavaScript altogether.
Interestingly, the solution lies in bringing Yahoo! Pipes into the mix. The resulting implementation of the same functionality does not require a single line of code, at least as long as you don't count XML as code. A Yahoo! Pipe enriches the Google calendar feed with geo data, i.e. longitude and latitude. Pipes makes the enriched feed available as an RSS feed, allowing GME to pick it up and render the list and map of calendar events (see Figure 1):
Though at first glance Yahoo! Pipes and Google Mashup Editor may appear to provide similar functionality, they actually complement each another very nicely. Yahoo! Pipes allows users to design a data processing pipeline (hence the name), which combines, transforms, or enriches incoming data feeds. Pipes makes the results available in a list browser, on a map, or as an RSS feed. The last option is particularly enticing as it allows Pipes to be combined with other tools that can process RSS feeds, which includes most mashup tools, such as Google Mashup Editor. GME also allows you to manipulate feeds (including creating, reading, updating, and deleting items), but its emphasis is on rich presentation in a browser. The GME workspace allows you to embed special GME modules into an HTML layout of your choosing, giving you fine control over the visual appearance of the application. GME templates format the dynamic parts of your the application, i.e., data feed items, to match the look and feel of the application. Let's build a simple example that combines Yahoo! Pipes' data manipulation abilities with GME's rich user interactivity.
Let's start at the beginning, i.e. the calendar. We'll use the same public Google Calendar feed as in the previous tutorial. This feed, which contains my conference speaking engagements, is available at: http://www.google.com/calendar/feeds/k6bnejmmglue0r1tsg6bfhnku8@group.calendar.google.com/public/full. To access your own calendar via an Atom feed, simply replace the ID portion of this URL with the ID of your calendar. You can find this ID under Calendar Settings as described in the Mashing It Up With Google Mashup Editor tutorial.
The calendar feed follows the Atom format and looks like this (some XML elements omitted for readability):
|
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:gd="http://schemas.google.com/g/2005">
<entry>
<id>http://www.google.com/calendar/feeds/… </id>
<published>2007-05-31T22:56:41.000Z</published>
<updated>2007-06-16T17:15:30.000Z</updated>
<title type="text">DEBS - Distributed Event-based Systems</title>
<content type="text">http://www.debs.msrg.utoronto.ca/</content>
<gd:when startTime="2007-06-19" endTime="2007-06-23"/>
<gd:where valueString="Toronto, Canada"/>
</entry>
…
</feed>
The feed contains elements for location and date of the event along with the name and the URL of the conference's Web site. It does not, however, contain location data, i.e. longitude and latitude. This is where Yahoo! Pipes comes in. To build a pipe, visit pipes.yahoo.com and sign in with your Yahoo! ID. The Pipes editor provides a drag-and-drop environment that allows you to compose a data pipeline from a set of predefined modules (see Figure 2).
Naturally our pipe begins with the construction of the calendar feed URL. Because the URL is a bit lengthy it is a good idea to use Yahoo! Pipes' URL Builder module. We compose the URL from the base (http://www.google.com/calendar/feeds), the unique calendar ID, and the remainder of the path (public/full), which specifies the type of feed. Using the URL Builder also makes it easier to replace the calendar ID with your own without running into copy-paste errors. The combined URL feeds into the Fetch Feed module, which makes the feed data available to the pipe.
The main purpose of the pipe is to add location data to the feed stream. The Location Extractor module makes this trivial. Simply place a Location Extractor module below the Fetch Feed module and connect the two modules with a data pipe. To examine the behavior of the pipe so far you can select the Location Extractor module and open debugger pane below the editor pane (the panes can be resized by dragging the divider). The debugger shows that the item data now includes a y:location element, which contains the following data:
y:location
country Canada
lat 43.648565
state ON
city Toronto
lon -79.385329
quality 40
How come we didn't have to configure a thing? Yahoo! Pipes examines the incoming stream and figures out which field contains location information (gd:where in our case). No need for XPath or the like.
We are almost done, but we have to overcome one more (small) hurdle. Yahoo! Pipes publishes the pipe output as a standard RSS feed, which does not accommodate the special Google Google Data fields, such as gd:where or gd:when. Therefore, we copy these fields into standard RSS fields using the Rename module. We store the conference URL in the RSS link field and copy the event date into the description field. The complete pipe looks as follows:
You can save and run the pipe from the Yahoo! Pipes editor. From there, you can also find out how to access the pipe output as an RSS feed (see Figure 3).
Copy and save the link to the RSS feed so you can use it later in the GME application. If you open the pipe's RSS feed in a browser, you'll see the following feed data:
<rss version="2.0"
xmlns:geo=http://www.w3.org/2003/01/geo/wgs84_pos# ...>
<channel>
<title>ConferenceTravel</title>
<description>Pipes Output</description>
<link>http://pipes.yahoo.com/pipes/pipe.info?_...</link>
<pubDate>Thu, 05 Jul 2007 04:44:33 PDT</pubDate>
<generator>http://pipes.yahoo.com/pipes/</generator>
<item>
<title>DEBS - Distributed Event-based Systems</title>
<link>http://www.debs.msrg.utoronto.ca/</link>
<description>2007-06-19</description>
<guid isPermaLink="false">...</guid>
<pubDate>Thu, 31 May 2007 22:56:41 PDT</pubDate>
<geo:lat>43.648565</geo:lat>
<geo:long>-79.385329</geo:long>
</item>
</channel>
</rss>
You notice the longitude and latitude in the geo:long and geo:lat fields, which follow the W3C geo vocabulary. Each item also contains the conference site URL in the link field (our original calendar feed (ab)used the content field for this purpose). The start date of the event lives in the description field as instructed by the pipe's Rename module.
We are now ready to implement the second half of our mashup using GME. GME converts external RSS feeds into the Atom format for internal processing. To see the result of this transformation select External Feed in GME's Feed Browser tab and paste the pipe's feed URL into the URL field. The feed browser shows the following format:
<atom:entry>
<id>...</id>
<published>2007-06-01T05:56:41.000Z</published>
<source>
<id>tag:google.com,2005:reader/feed/http://pipes.yahoo.com/pipes/
pipe.run?_id=2hmIdiYT3BG_E4m5y6ky6g&_render=rss</id>
<title type="html">ConferenceTravel</title>
<link href=http://pipes.yahoo.com/pipes/pipe.info?_id=2hmIdiYT3BG_E4m5y6ky6g
rel="alternate" type="text/html"></link>
</source>
<title type="html">DEBS - Distributed Event-based Systems</title>
<summary type="html">2007-06-19</summary>
<author>
<name>(author unknown)</name>
</author>
<geo:lat>43.648565</geo:lat>
<geo:long>-79.385329</geo:long>
<updated>2007-06-01T05:56:41.000Z</updated>
<link href="http://www.debs.msrg.utoronto.ca/" rel="alternate" type="text/html"></link>
</atom:entry>
You notice that the geo tags are present in the internal representation of the feed. The start date moved into the summary element whereas the link element still contains the URL to the conference web site. When inspecting feeds from the Feed Browser, be aware that GME caches external feeds. Accordingly, changes to your pipe (or another external feed) might not be reflected immediately.
Displaying the feed information in a list and a map within GME is now rather trivial. As before, we use a list, an item (formatted by a template) and a map module. The complete GME application is surprisingly short and looks as follows:
<gm:page title="GregorConferenceMap2">
<h1>Gregor's Conference Travel (with Pipes)</h1>
<table width="900">
<tr valign="top">
<td width="300">
<gm:list id="myList" pagesize="10"
data="http://pipes.yahoo.com/pipes/pipe.run?
_id=2hmIdiYT3BG_E4m5y6ky6g&_render=rss">
<gm:handleEvent src="map"/>
</gm:list>
<gm:item id="item" template="itemTemplate">
<gm:handleEvent src="myList"/>
</gm:item>
</td>
<td class="mainPanel">
<gm:map id="map" data="${myList}" latref="geo:lat" lngref="geo:long">
<gm:handleEvent src="myList"/>
</gm:map>
</td>
</tr>
</table>
<gm:template id="itemTemplate">
<div>
<b><gm:link ref="link/@href" labelref="title"/></b><br/>
<gm:text ref="summary"/><br/>
</div>
</gm:template>
</gm:page>
All modules share the same data source, specified in the list module. The map module's latref and lngref attributes instruct the map to extract location data from the corresponding fields in the feed. The item module uses the itemTemplate template to render a link to the conference web site using data from the title, link, and summary fields. handleevent elements connect the list, item, and map modules via events, making the application fully interactive. Selecting an entry in the list updates the item and re-centers the map. Selecting a marker on the map selects the corresponding entry in the list and also updates the item detail—all without a single line of code. The final application looks as shown in Figure 4.
You can run the application from the GME editor or publish it for other users to enjoy. The running application is also available at http://confmappipes.googlemashups.com.
Mashups are about data consolidation and transformation. The availability of standard feed formats like RSS and Atom enable us to combine different mashup tools to create sophisticated solutions. This example barely scratched the surface of Yahoo! Pipes' or GME's potential but instead focused on the interaction between the two. Many more exciting mashups are waiting to be built from these tools. Happy mashing!
Google is a trademark of Google Inc. All other company and product names may be trademarks of the respective companies with which they are associated.