My favorites | Sign in
Project Home Downloads Wiki Issues Source
READ-ONLY: This project has been archived. For more information see this post.
Search
for
OPEN_HSM  

Open – OHSM for ext2/ext3

Abstract

Significant amount of work has been done in the area of storage but all the related work in this area as of now is completely restricted by licensing, availability and platform specific framework designs. Online Hierarchal Storage is a method to reduce the cost of online storage of files in a multi tier storage environment. Here online means that placement and relocation of files is done in parallel with the I/O operation being performed on the filer. Hierarchal storage signifies multi level storage which uses virtual or physical storage devices with different I/O performance, data availability, and relative cost characteristics to provide differentiated online storage for enterprise level storage servers. Multi grade storage facility helps a lot because of the expected access patterns, load balancing and availability. Different types of data have different I/O performance needs like high data transfer is important for streams but moderate I/O request are acceptable. The key to deriving value from multiple grades of storage is to place data on the appropriate type of storage devices. Most critical files should be placed on high performing, more reliable (and therefore more expensive) devices, less critical data can be placed on less costly. Whenever, a particular pattern of data is to be placed on disk, placement policies decide, on which type of storage the data should be stored.

A file placement policy consists of rules that govern the initial location and subsequent relocation of designated sets of files. A rule may designate the files to which it applies by name, by directory, by ownership, or by combination of the three. The problem of matching a file to the ‘right’ types of storage device is not technically challenging. Administrators can cause files to be created on the right type of storage, for example by assigning users or applications to specific devices. The challenge lies in the numbers, however. Getting millions of files placed on the right devices is far too time-consuming to do effectively without some form of automation.

So, after having a close look into any of the commodity filers, we couldn’t find any significant work done in this area. Though enterprise level solutions are present but covered with the clouds of licensing and proprietary issues. Not much has been done in this regard in the open source front. Therefore our design overrules the restrictions imposed by specific file systems and we are looking forward to a generic solution in this regard. Which would mostly be a proof of concept and to unfold that immense research is possible is this area.

Powered by Google Project Hosting