My favorites | Sign in
Project Logo
Project Home
  
Downloads
  
Wiki
  
Issues
  
Source
    
Search
for
Updated Feb 18, 2009 by daniel.l...@gmail.com
Labels: Featured
NDEFormat  
Information about the format used in NDE (Nullsoft Database Engine) files.

Introduction

This page describes the Nullsoft Database Engine format.

Tables are saved using two different files. filename.dat contains the actual data, and filename.idx contains an index used to access this data.

Index format

The index file has a header as follows:

Offset Data Type Size Field
0 String 8 "NDEINDEX"
8 Integer 4 Number of records
12 Integer 4 Unknown (always seems to be 0xFF 0x00 0x00 0x00)

The rest of the file is the index itself. Each record consists of two integers:

Offset Data Type Size Field
0 Integer 4 Offset of this record in the data file
4 Integer 4 Index of this record

Data format

The data file simply begins with "NDETABLE", and the rest of the file is data. Before we continue, we need to know some terminology. One data file has many records (a record is much like a row in a database). A record has many fields (a field is much like a column in a database).

Records

A record basically consists of a linked list of fields. This means that a record starts with one field, which contains the offset to the next field. To read the whole record, you keep following these offsets, until you get one that is equal to 0 (once you reach this, you've read the whole record).

There are a few "special" types of records in the index file:

Fields

Fields have a header like the following:

Offset Data Type Size Field
0 Unsigned char 1 Column ID
1 Unsigned char 1 Field type
2 Integer 4 Size of field data
6 Integer 4 Offset to next field in this record
10 Integer 4 Offset to previous field ?? (I don't use this)

Most of these fields should be self-explanatory. The next "size" bytes after this header is the actual field data (eg. if "size" is 20, then the next 20 bytes is the actual data). If the field is a "column" type, the Column ID tells you what ID this column is (first column is 1, second column is 2, etc). Otherwise, the Column ID tells you what column this data is in.

Field types

Fields may be any of the following types (types marked with a * are not implemented in NDEPHP yet):

ID Type name
0 Column
1* Index
2* Redirector
3 String
4 Integer
5* Boolean
6* Binary
7* GUID
8* Unknown
9* Float
10 Date and time
11 Length

The field type are explained in more detail below. Offsets are relative to the start of the field.

Column

Offset Data Type Size Field
0 Unsigned char 1 Column field type (see list above)
1 Unsigned char 1 Index unique values ?? (unknown what this does)
2 Unsigned char 1 Size of column name
3 String Size (offset 2, above) Name of the column

Back to field types listing

String

Offset Data Type Size Field
0 Unsigned short 2 Size of the string
2 String Size String value

Back to field types listing

Integer

The integer format is also used for the Date and time, and Length types. A length is simply an integer that represents the length of something, and a Date and time is simply a UNIX timestamp.

Offset Data Type Size Field
0 Integer 4 Value

Back to field types listing

How to read the files

Below is some psuedocode showing how to read these files: 

 Open index file
 Open data file
 Read header of index file
 Read header of data file
 
For each record in the index file
 	Start with a blank record
 	Set offset variable to offset specified in index file
 	Loop
 		Get the data from the data file at the specified offset
 		Add this data to the record
 		Set offset variable to "next field" in the field data
 	While offset is not 0
 
	// "Record" now contains the record you just read in.
 
	If record is a "column" type
 		Store the column names in a columns variable
 	Else
 		Store the data based on column names in columns variable
 	End if
 End For each
 
// You've now read the whole file and have all the data

References

The following references were used for obtaining this data:


Comment by Buttink, Apr 06, 2009

Data format

Records

When moving from an offset you want to move from the begining of the file.

Example

Set Position in File Stream to your indexes[WhereYouAre].
do
   Read in the initial field data IE Column ID, Field Type, Size, OffsetToNext, OffsetToPrev.
   Read the data as whatever Field Type it is and store it based on the Column ID.
   Set Position in File Stream to Offset.
while ( Check if Offset is 0 to exit loop (done with record) )

Field Header

Column IDs are a way to tie data to what it is. When you got all columns (fieldType == column AKA first record) the column id said that this column is this id. So when you get a record of an actual song, you will get a column id for each field which says "Column Name == Column ID of Column == Column ID of Field == What Data This Is". So if Column ID 1 is title, then when you get a string data with Column ID of 1 its a title.

Field Types

It appears that there is a new field type of 12 that represents the file name. I believe that this can be read in exactly like string but I have not done enough testing. Of course this always could just be a way to saying you are at the end of the link list.

Strings

Strings with the NDE format can be of a 8-bit ASCII (8th bit isnt used just there for spacing) OR Unicode format.

Example

Given the data

0A-00-FF-FE-52-00-6F-00-63-00-6B-00

0A-00 would be the size of the data aka 10. FF-FE is basically there to say HEY this is unicode. The rest is simply your unicode that does not have a null terminator. I believe these are supposed to be double null terminated (according to Winamp API) but I never found this to be true.

This is exactly the same thing as this data ---- 04-00-52-6F-63-6B ---- However this data is just straight 8-bit ASCII. Like before the 04-00 is your size aka 4. The rest is just 8-bit ASCII.


Sign in to add a comment
Hosted by Google Code