My favorites | Sign in
Project Home Wiki Issues Source
READ-ONLY: This project has been archived. For more information see this post.
Search
for
RelationToMarineMap  
Purpose of happy-kml-tool in relation to MarineMap and primary data sources
Updated Dec 23, 2010 by underbluewaters

Background

The MarineMap Decision Support Tool relies on data from a variety of primary sources. Many of these primary sources such as the Multipurpose Marine Cadastre and West Coast Atlas Network provide data as web services in open formats, but these cannot be directly consumed by MarineMap for several important reasons:

  1. Data used in MarineMap projects may require pre-processing and cartographic styling to address regional process needs not considered by primary sources.
  2. Most primary data sources don't have the capacity to serve data to thousands of geographically distributed MarineMap users.
  3. While available web services represent an important advance in data sharing, different providers use an assortment of different format standards and technology. It's unreasonable (and undesirable from an innovation standpoint) to expect them all to use the same formats. MarineMap requires that data be in KML (keyhole markup language) format.

Historically the MarineMap Consortium has gathered, preprocessed, styled, and published KML data layers based on primary sources largely by hand. This manual process requires careful coordination and a large variety of desktop tools. As MarineMap's use expands this approach is unlikely to scale effectively. We've also found that KML generation tools can be limited when handling large, complex datasets often used in decision making. From our past experience we believe that better tools can be developed.

What we propose is a hosted web application that encompasses and streamlines the entire workflow required to take data from a variety of different platforms with primary data, and make it useful within MarineMap. MarineMap's cartographer and partners will be able to login and define cartographic stylesheets for each data layer. These stylesheets will identify data sources, any pre-processing steps to perform on the data, and all cartographic styling. This hosted application will then be responsible for generating from this information a final KML layer. These layers will be distributed on a content delivery network for fast access by thousands of users, and can be utilized directly by MarineMap or any other KML client like Google Maps and Earth. This tool will also be responsible for detecting data updates in primary sources, and will automatically regenerate KML layers when appropriate.

PDF Version

Components

This system can likely be broken down into two separate components, an open-source KML generation tool and it's hosted, web-based interface. The KML generation tool (happy-kml-tool) handles the pre-processing and styling of data into KML outputs based on a custom stylesheet language. It can be downloaded and run on a desktop computer by anyone, and would represent a significant contribution to the open source geospatial community. happy-kml-tool's hosted, web application counterpart would only be available to the MarineMap consortium and it's partners, and would allow us to collaborate with primary data providers directly and automate even more of the layer publishing workflow.

KML Cartographic Workflow (happy-kml-tool)

happy-kml-tool streamlines the KML generation process. Unlike many existing KML tools that repurpose existing 2d cartography tools, it's designed from the ground up for the KML format and provides the following features:

  • Data sources, pre-processing steps, and styling can all be defined using a simple but powerful stylesheet language. It can be edited with any text editor and leverages the successful model of CSS.
  • Once stylesheets are defined, the entire KML layer generation workflow can be run in a completely automated fashion. This makes handling updates to data or incremental style changes easy.
  • Data sources can be specified from a variety of file formats, hosted locally or accessed via web services.
  • Advanced KML features such as NetworkLinks and Regions are fully supported, enabling much higher-density and efficient data displays than are possible with existing tools.

More technical documentation on a preliminary design can be found here and here

Hosted Web Application

happy-kml-tool provides a set of core tools needed to efficiently define the cartography of KML layers, but must be installed and setup on a local computer. The hosted version of the tool would be made available to the MarineMap Consortium and partners to use happy-kml-tool's features entirely through a web-based interface. In addition, the hosted version of happy-kml-tool would:

  • Manage and organize cartographic stylesheets, allowing partners to collaborate on layers and track changes.
  • Continually and automatically update KML layers when it detects updates in primary source data or on a defined schedule.
  • Push data layers out to a content delivery network (CDN) for efficient hosting to thousands of users.

Development Phases

Phase 1 - Pilot

The pilot phase of this project involves getting the desktop version of happy-kml-tool bootstrapped. At the end of it's first milestone, it would be capable of:

  • reading source data in a single local file format (likely shapefile)
  • creating a kml document that included spatial features as Placemarks with basic attributes and info windows
  • styling placemarks via a css-like stylesheet

Work items for this phase can be found here

Phase 2 - Expanded Data Sources, Styling Options, and Hosted App

This phase would add additional data sources support to happy-kml-tool, such as WMS, WFS and the ESRI REST API. The tool would also gain more advanced styling capabilities. This phase would introduce the hosted version of happy-kml-tool for partners to collaboratively develop KML layers. The hosted version would allow users to:

  • Login and upload cartographic stylesheets that can be processed using the computing resources of the server
  • Share these stylesheets with other users for collaborative development
  • Schedule cartographic workflows to be processed at certain intervals, or when specified data sources are update
The hosted version would make the output kml layers available on a high performance content delivery network that could immediately scale to thousands of users.

Work items for this phase can be found here

Phase 3 - Data Optimization and Regionation Support

This phase would introduce tools to help optimize complex KML layers. These tools would improve performance when downloading these layers as well as improve client performance when rendering. It should be stressed that without these features, the work from previous milestones is likely to be of limited utility since most data used in CMSP would be considered "complex". Using the products of this phase, users can:

  • Optimize very large datasets for delivery over the web
  • Troubleshoot performance problems in data layers using helpful statistics generated by the tool
  • Implement scale-dependent renderings of data to not only improve performance but also readability of layers at different scales

Work items for this phase can be found here

Powered by Google Project Hosting