My favorites
▼
|
Sign in
xtreemfs
a cloud file system
Project Home
Downloads
Wiki
Issues
Source
Checkout
Browse
Changes
Source path:
svn
/
trunk
/
doc
/
BabuDBMetadataMapping.txt
r3117
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
This document describes how XtreemFS metadata in the MRC is mapped to BabuDB
database indices.
1 VERSIONING
==============================================================================
The current version of the MRC is stored in a database named 'V'. There is
only one index containing a single key-value pair (numbers indicate entry
sizes in bytes):
VERSION (BabuDB index #0)
The version index only contains one key-value pair that indicates the current
version of the database.
|----------------------------------------------------------------------------|
|key | version |
| | 1 ('v') |
|----------------------------------------------------------------------------|
|value | currentVersionNumber |
| | 4 |
|----------------------------------------------------------------------------|
2 FILE SYSTEM METADATA
==============================================================================
File system metadata of a volume is stored in a BabuDB database with the name
of the volume ID. The following indices exist, with the following types (the
numbers define different value types):
FILE_INDEX XATTR_INDEX FILE_ID_INDEX
-------------- ------------- -------------------
atime 0 XAttrs parentID + fileName
ctime 0
mtime 0
size 0
type 1
id 1
permissions 1 ACL_INDEX LAST_ID_INDEX
linkCount 1 ---------- ------------------
epoch 1 ACLEntries lastAssignedFileID
issuedEpoch 1
readOnlyFlag 1
fileName 1
owner 1
group 1
xLocList 1
Indices are structured as follows (numbers indicate entry sizes in bytes):
FILE_INDEX (BabuDB index #0)
The file index stores the primary metadata of a file or directory. This
encapsulates most of the metadata returned by the 'stat' system call. To
reduce the size of log entries, metadata has been grouped according to their
types: type-0 metadata reflects frequently-changed metadata like time stamps
and the file size, while less frequently-changed metadata like access
permissions and ownership are defined as type-1 metadata. X-Locations lists
which are frequently accessed w/ I/O operations but rarely changed are
defined as type-3 metadata. Each such type has its own key in the index.
|----------------------------------------------------------------------------|
|key | parentID filename type |
| | 8 var 1 |
|----------------------------------------------------------------------------|
|value | frequentlyChangedMetadata |
|type=0| 20 (files), 12 (directories) |
|----------------------------------------------------------------------------|
|value | rarelyChangedMetadata |
|type=1| var (files & directories), 9 (link targets) |
|----------------------------------------------------------------------------|
XATTR_INDEX (BabuDB index #1)
The extended attribute index contains any user-defined extended attributes
assigned to files or directories. Softlink targets and default striping
policies are mapped to extended attributes
|----------------------------------------------------------------------------|
|key | fileID hash(owner) hash(attrKey) collisionCount |
| | 8 4 4 2 |
|----------------------------------------------------------------------------|
|value | keyOffset valueOffset attrOwner attrKey attrValue |
| | 2 2 var var var |
|----------------------------------------------------------------------------|
ACL_INDEX (BabuDB index #2)
The ACL index stores optional access control lists for files.
|----------------------------------------------------------------------------|
|key | fileID entityName |
| | 8 var |
|----------------------------------------------------------------------------|
|value | rights |
| | 2 |
|----------------------------------------------------------------------------|
FILE_ID_INDEX (BabuDB index #3)
The file ID index is used to retrieve file metadata by means of its ID. It
may either map a file ID to a key in the file index, or directly to the three
different types of primary file metadata (i.e. types 0-1). The former kind of
mapping is used to retrieve file metadata by means of file IDs, which is
necessary with some operations that provide a file ID instead of a path. The
latter sort of mapping is used to retrieve a file's metadata if the file is
referenced by multiple hard links (type 2). In this case, the file's entries
in the file index map to the corresponding prefix key in the file ID index.
|----------------------------------------------------------------------------|
|key | fileID type |
| | 8 1 |
|----------------------------------------------------------------------------|
|value | frequentlyChangedMetadata |
|type=0| 20 (files), 12 (directories) |
|----------------------------------------------------------------------------|
|value | rarelyChangedMetadata |
|type=1| var |
|----------------------------------------------------------------------------|
|value | parentID fileName |
|type=2| 8 var |
|----------------------------------------------------------------------------|
VOLUME_INDEX (BabuDB index #4)
The 'VOLUME_INDEX' index stores single key-value pairs that contain volume-
related information, including the last ID assigned to a file, the current
number of files and directories in the volume, as well as the current size of
the volume in bytes.
|----------------------------------------------------------------------------|
|key | type |
| | 1 (one of {'i', 's', 'f', 'd'}) |
|----------------------------------------------------------------------------|
|value | longValue |
| | 8 |
|----------------------------------------------------------------------------|
Complex data types are mapped as follows:
frequentlyChangedMetadata - comprises file/directory metadata that is
frequently changed
files:
|----------------------------------------------------------------------------|
|atime ctime mtime size |
|4 4 4 8 |
|----------------------------------------------------------------------------|
directories:
|----------------------------------------------------------------------------|
|atime ctime mtime |
|4 4 4 |
|----------------------------------------------------------------------------|
rarelyChangedMetadata - comprises file/directory metadata that is rarely
changed
files:
|----------------------------------------------------------------------------|
|type id permissions linkCount w32attrs epoch issuedEpoch readOnlyFlag|
|1 8 4 2 8 4 4 1 |
| |
|groupOffset xLocOffset owner group xLocList |
|2 2 var var var |
|----------------------------------------------------------------------------|
directories:
|----------------------------------------------------------------------------|
|type id permissions linkCount w32attrs groupOffset owner group |
|1 8 4 2 8 2 var var |
|----------------------------------------------------------------------------|
link targets:
|----------------------------------------------------------------------------|
|type id |
|1 8 |
|----------------------------------------------------------------------------|
xLocList - represents an X-Locations list of a file
|----------------------------------------------------------------------------|
|version numReplicas replUpdPolOffs repl1Offs ... replNOffs |
|4 4 4 4 4 |
| |
|xLoc1 ... xLocN replUpdPol |
|var var var |
|----------------------------------------------------------------------------|
xLoc - represents a single replica in an X-Locations list
|----------------------------------------------------------------------------|
|osdListOffset replFlags stripingPolicy numOSDs osd1ofs ... osdnofs |
|2 4 var 2 2 2 |
| |
|osd1 ... osdn |
|var var |
|----------------------------------------------------------------------------|
stripingPolicy - represents a striping policy
|----------------------------------------------------------------------------|
|stripeSize width pattern |
|4 4 var |
|----------------------------------------------------------------------------|
Show details
Hide details
Change log
r2466
by stenjan on Jul 28, 2011
Diff
trunk: copied all data from pb_experimental branch (
r2465
)
Go to:
/trunk/.classpath
/trunk/.project
/trunk/AUTHORS
/trunk/ChangeLog
/trunk/LICENSE
/trunk/Makefile
/trunk/bin
/trunk/bin/cpplint.py
/trunk/bin/protoc-gen-pbrpc
/trunk/bin/protoc-gen-pbrpccpp
/trunk/bin/umount.xtreemfs
/trunk/bin/xtfs_chstatus
/trunk/bin/xtfs_cleanup
/trunk/bin/xtfs_mrcdbtool
/trunk/bin/xtfs_scrub
/trunk/bin/xtfs_snap
/trunk/contrib
/trunk/contrib/console
/trunk/contrib/console/README.txt
...trib/console/XtreemFSConsole.war
/trunk/contrib/hadoop
/trunk/contrib/hadoop/build.xml
/trunk/contrib/hadoop/manifest.mf
/trunk/contrib/hadoop/nbproject
.../hadoop/nbproject/build-impl.xml
...op/nbproject/genfiles.properties
...oop/nbproject/project.properties
...rib/hadoop/nbproject/project.xml
/trunk/contrib/hadoop/src
/trunk/contrib/hadoop/src/org
.../contrib/hadoop/src/org/xtreemfs
...b/hadoop/src/org/xtreemfs/common
.../src/org/xtreemfs/common/clients
...g/xtreemfs/common/clients/hadoop
...s/hadoop/XtreemFSFileSystem.java
/trunk/contrib/hadoop/test
/trunk/contrib/server-repl-plugin
...ntrib/server-repl-plugin/LICENSE
...ontrib/server-repl-plugin/README
...ontrib/server-repl-plugin/config
...pl-plugin/config/dir1.properties
...pl-plugin/config/dir2.properties
...pl-plugin/config/mrc1.properties
...pl-plugin/config/mrc2.properties
...k/contrib/server-repl-plugin/lib
...erver-repl-plugin/lib/Flease.jar
...server-repl-plugin/lib/PBRPC.jar
...ugin/lib/protobuf-java-2.3.0.jar
...rver-repl-plugin/replication.jar
/trunk/cpp
Project members,
sign in
to write a code review
Older revisions
All revisions of this file
File info
Size: 11071 bytes, 207 lines
View raw file
Powered by
Google Project Hosting