• Introduction
• Installation
• Tomcat
• Configuration files
• Custom classpath
• Deploy war
• Configuration
• Property Names
• Source
• Basic Usage
• Create a bucket
• Add an object
• Retrieve an object
• Delete an object
• Delete a bucket
• List buckets

Introduction

LittleS3 is distributed as a WAR file designed to be installed and run from a Java-based application server. This documentation uses Tomcat as an example. The goal is to install littleS3 and be able to manually store objects within the application.

Installation

Tomcat

To start, download and install Tomcat as appropriate for your environment.

Configuration files

LittleS3 uses two configuration files by default: StorageEngine.properties and StorageEngine-servlet.xml. These configuration files are loaded automatically from the classpath.

Custom classpath

You can modify the Tomcat start up script to modify the default. For example...

In the tomcat home directory, C:\apps\apache-tomcat-5.5.23 for instance, create a new directory named "settings".

Edit the catalina.bat file, found in the "tomcat home directory/bin" directory, adding:

rem Extra classpath environment stuff
set CLASSPATH=%CLASSPATH%;C:\apps\apache-tomcat-5.5.23\settings

to this section:

:noJsse
set CLASSPATH=%CLASSPATH%;%CATALINA_HOME%\bin\bootstrap.jar

rem Extra classpath environment stuff
set CLASSPATH=%CLASSPATH%;C:\apps\apache-tomcat-5.5.23\settings

if not "%CATALINA_BASE%" == "" goto gotBase
set CATALINA_BASE=%CATALINA_HOME%
:gotBase

The two configuration files StorageEngine.properties and StorageEngine-servlet.xml can now be placed in the new directory "settings".

Deploy war

The littleS3 application is distributed as a WAR file. The WAR packaging follows the pattern

littleS3-[version].war: littleS3-2.1.0.war for instance.

A WAR file can be deployed to a default Tomcat instance by placing the WAR file in the "tomcat home directory/webapps" directory. Tomcat will automatically unpack and deploy the WAR. This will make the littleS3 application available at the URL:

http://localhost:8080/littleS3-2.1.0/

Where "localhost:8080" is the host and default port that Tomcat uses.

There are more advanced Tomcat deployment options that allow you to control the "context path", the "/littleS3-2.1.0" portion of the URL. You could deploy to a context path of "/littleS3", or even to the Tomcat root, so the context path would be "/". The context path is important when crafting the URL to access littlS3, as the S3 protocol uses URL components to create buckets.

Configuration

There are two configuration files for littleS3: StorageEngine-servlet.xml and StorageEngine.properties.

• StorageEngine-servlet.xml is a Spring Framework bean wiring configuration file. This controls the component makeup of the application. You typically don't have to change the configuration values in this file.
• StorageEngine.properties is a configuration file that is consumed by components configured in the StorageEngine-servlet.xml configuration file.

Property Names

host

May include a "token" of $resolvedLocalHost$ to have the system "resolve" the local host value. For instance, $resolvedLocalHost$:8080. This would use the Java methods InetAddress.getLocalHost().getCanonicalHostName() to determine what the local value of the hostname is. Used by com.jpeterson.littles3.StorageEngine. Example: www.myserver.com:8080

storageLocation

Identifies the local file system storage location that will subsequently be used for objects and metadata. This directory should already exist. Used by com.jpeterson.littles3.dao.je.JeCentral, com.jpeterson.littles3.dao.filesystem.FileBucketDao, com.jpeterson.littles3.dao.filesystem.FileS3ObjectDao. Example: C:/temp/StorageEngine

dir.buckets

Identifies the directory under storageLocation/dir.meta that will be used to store bucket information. Used by com.jpeterson.littles3.dao.filesystem.FileBucketDao. Default value: buckets. Example: buckets

dir.db

Directory under storageLocation that will be used for to store the Oracle Berkeley DB Java Edition (JE) database based data. Used by com.jpeterson.littles3.dao.je.JeCentral. Example: db

dir.meta

Identifies the directory under storageLocation that will be used to store meta information. Used by com.jpeterson.littles3.dao.filesystem.FileBucketDao, com.jpeterson.littles3.dao.filesystem.FileS3ObjectDao. Default value: meta. Example: meta

dir.objects

Identifies the directory under storageLocation/dir.meta that will be used to store object information. Used by com.jpeterson.littles3.dao.filesystem.FileS3ObjectDao. Default value: objects. Example: objects

db.object

Directory under storageLocation/dir.db where the object information is stored in an Oracle Berkeley DB Java Edition (JE) database. Used by com.jpeterson.littles3.dao.je.JeCentral. Example: objectDatabase

db.bucket

Directory under storageLocation/dir.db where the bucket information is stored in an Oracle Berkeley DB Java Edition (JE) database. Used by com.jpeterson.littles3.dao.je.JeCentral. Example: bucketDatabase

user.file

Sample configuration file: users.config

Source

The source is divided into 4 different components: api, webapp, filesystem module, and je module. The api component provides the common objects used by the system. The webapp is the collection of the other components into a runnable system. The filesystem and je modules provide different persistent storage options. The filesystem module stores the persistent data using plain files in the file system. The je module store stores the persistent data using the Oracle Berkeley DB Java Edition (JE) database.

Basic Usage

In these samples, I have used a base URL of http://localhost:8080/littleS3-2.1.0/. The StorageEngine.properties configuration parameter host is set to the value localhost:8080:

host=localhost:8080

This will allow the littleS3 system to properly parse the URL request and determine what function to perform.

Create a bucket

The following REST request will create the bucket firstBucket.

curl --request PUT "http://localhost:8080/littleS3-2.1.0/firstBucket"

Based on the other configuration parameters and using the filesystem module, this command will:

• Create the directory C:\temp\StorageEngine.
• Create the directory C:\temp\StorageEngine\buckets.
• Create the directory C:\temp\StorageEngine\buckets\firstBucket.
• Create the directory C:\temp\StorageEngine\meta.
• Create the directory C:\temp\StorageEngine\meta\buckets.
• Create the directory C:\temp\StorageEngine\meta\buckets\firstBucket.
• Create the file C:\temp\StorageEngine\meta\buckets\firstBucket\firstBucket.ser.

Add an object

The following REST request will create the object foo.html.

curl --data "@foo.html" --request PUT --header "Content-Type: text/html" "http://localhost:8080/littleS3-2.1.0/firstBucket/foo.html"

This command will:

• Create this directory C:\temp\StorageEngine\buckets\firstBucket\9b. You will most likely have a different directory name; it will be two characters, but two different characters.
• Create the file C:\temp\StorageEngine\buckets\firstBucket\9b\9b8295ad7ad382ef47becac31f548902. This file contains the actual object data.
• Create the directory C:\temp\StorageEngine\meta\objects.
• Create the directory C:\temp\StorageEngine\meta\objects\firstBucket.
• Create the file C:\temp\StorageEngine\meta\objects\firstBucket\keys.ser.
• Create the directory C:\temp\StorageEngine\meta\objects\firstBucket\ad. You will most likely have a different directory name; it will be two characters, but two different characters.
• Create the file C:\temp\StorageEngine\meta\objects\firstBucket\ad\ad9f205559bcb9eb394e457d5b003b2a.ser. This file stores metadata about the object.

Retrieve an object

For this, you can use your browser. Open the following URL in your browser:

http://localhost:8080/littleS3-2.1.0/firstBucket/foo.html

Delete an object

The following REST request will delete the object foo.html.

curl --request DELETE "http://localhost:8080/littleS3-2.1.0/firstBucket/foo.html"

• Delete the directory C:\temp\StorageEngine\buckets\firstBucket\9b and any files in the directory. The object contents are deleted. Since this was the only object, all of the empty directories are also removed.
• Delete the directory C:\temp\StorageEngine\meta\objects and any child folders and files. Since the object contents were deleted, the object metadata is removed and all of the empty directories are also removed.

Delete a bucket

The following REST request will delete the object firstBucket.

curl --request DELETE "http://localhost:8080/littleS3-2.1.0/firstBucket"

• Delete the directory C:\temp\StorageEngine\buckets\firstBucket.
• Delete the directory C:\temp\StorageEngine\meta\buckets. The firstBucket directory is no longer needed and being the only bucket, the buckets directory is removed too.

List buckets

For this, you can use your browser. Open the following URL in your browser:

http://localhost:8080/littleS3-2.1.0/