#Identifiers for large data objects

Introduction

DSP tasks involve the management of arbitrary large data objects such as the PCM of an audio signal channel decoded from an audio file or the chromagram bins of an audio frame.

These data objects are too difficult to handle using prolog lists which may even crash trying to capture a whole audio signal for example. Swi-Prolog provides special terms to capture binary representation of data in memory, the so-call BLOBS (PL_blob_t). These Blobs succeed in wrapping large data but they are really cumbersome to deal with them and give rise to some compatibility problems with other modules.

We develop within this project a mechanism to manage data objects under atomic identifiers. These identifiers are generated each time a vector of data is exported to prolog and stored in a runtime database. This mechanism is defined in swilib.

The automatic and preferred format for identifiers is data_x which varies incrementally. This format is encouraged to be suitable to represent a bnode in semantic web expressions. Other ids can be assigned by the user though

SWIDATA MODULE

Thus, the logic interface to this mechanism allow us to treat each data object as an atom which is a name of the data object at least during the session in execution. Identifiers can be related to the data object along session by keeping a correspondence with the external file with the dumped data.

Module predicates specification can be found at data.pl. Some examples of this mechanism are:

data_list('data_4', L). L = 6.45, 9.87, 3.56, ...

data_out('data_4', 'myfile').

concat_blobs('data_6', 'data3434', Concat). Concat = data_342143

blob_frame('data_423', 20, 60, Frame). Frame = data_3424

LIMITATIONS

1. Each module MUST BE implemented to read these atoms and get the data from the runtime database

2. The database my collapse if the session involves too many operations during a long time

3. Data lists and data identifiers should be treated in the same way and be transparent for the case. It is not completely true in practise.

4. The binary files in disk where data is dumped are not written in any standard, surely portable format.