| Issue 65: | Directory Based Allocation. | |
| 1 person starred this issue and may be notified of changes. | Back to list |
Directory based allocation is to be included in the design. 1. Include absolute paths in the policy data structures. 2. During allocation, do a path look up on the path specified in the policy, get its dentry and inode, pass it to ext2_new_inode() and ext2_add_nondir(). 3. Change the home TID of newly created inode to that of the parent directory's home tid.
Feb 28, 2009
Project Member
#1
sandeepksinha
Mar 2, 2009
Hello Sir, I have read DST, file placement using namespace. What i feel is that allocation and relocation is not recursive or non recursive. Instead relocation is recursive and non recursive and its justified. What it says is that we allocate file of new project that is still in testing phase in a cold tier under a specified name space. When the project is ready and tested it will be relocated to a production directories. And the selection criteria can be recursive and non recursive.
Mar 2, 2009
Sorry there, the second line is that "allocation is not recursive or non recursive" and not "allocation and relocation is not recursive or non recursive."
Status:
In-progress
Mar 3, 2009
Hi all, Requirements: 1. Directory based allocation. 2. Relocation of directory sub tree 3. Relocation of specific files. Implementation: We have no special handling of directories. Directory will reside in a particular tier, here we have to handle directories separately. Allocation of directories itself has to be implemented. Now regular file: Directory base allocation of files will not require tier to be specified instead it will inherit tier from parent directory. So in the policy file the path will be specified instead of tier in this case. All the module owners kindly identify the changes in you module with respect to this implementation and see the complexity involved.
Mar 3, 2009
A couple of questions. What do you mean by Relocation of specific files ?? Its relocation of specific set of files. Which also means a directory, because directory holds together a specfic set of files. Again "Allocation of directories itself has to be implemented" makes some other sense as compared to our intention. It should rather be "OHSM's specific changes in the directory allocation needs to be done." Now once again, "So in the policy file the path will be specified instead of tier in this case.", the path and its home tier would be specified. So, when you enable OHSM the directory's home tier ID will be set to the specified one in the policy file. Later for any file which is created inside this directory, it will inherit it from the parent directory. Kindly reframe everything and better come up with doc which discusses all this.
Mar 4, 2009
Relocation of specific set of files is different from relocation of directory subtree. If i specify relocate all .c files and all .xml files, it is relocation of specific files doesnt means a directory.They can be anywhere in FS. And when i specify relocate directory /mnt/Project from tier 1 to tier 2 it means relocation of directory sub tree. Another thing,"the path and its home tier would be specified. So, when you enable OHSM the directory's home tier ID will be set to the specified one in the policy file." So if you will just set home tid of the specified directory, initially data blocks of the directory will be somewhere else, and when you set the home tid of the directory, new data blocks will be in the specified tier. This will cause the data blocks of the directory to scatter in FS, every time you change the policy. And reading this directory will be very slow seeking different BGs and if you relocate the files in this directory then reading files for relocation will be slower. So directory itself will have to be relocated to the specified tier and then the allocation of files within this directory.
Mar 4, 2009
NO. See, the idea is still the same that we had earlier. ======== METHOD 1 ======== 1. Directory non recursive. We will ask user to keep this file empty at the time of enabling OHSM. 2. Directory recusrsive. We will ask user to keep it empty at the time of enabling OHSM. When you enable OHSM, --------------------- Take lock on directory inode. Go and set the home tier id of the directory. Free the inod lock on directory. When You create a file ---------------------- Refer to the home tier id of its parent. If its not zero, set the home tier id of the new file as teh home tier id of the parent directory. If its zero, the creation of new file will follow the file system allocation. ======== METHOD 2 ======== If the directory is non-empty at the time of enabling OHSM, We can just leave the home tier id of directory as 0. And let it go by the normal allocation. At the time of relocation, we will relocate everything inside the directory and will set the home tier as the new destination, where it is relocated. Now henceforth, everything that will be allocated inside this directory will be on the new destination tier. I like both the approaches. But which one do you think makes more sense? I think the code complexity will be same in both the cases, just that in the first case we ask the user to have empty directory initially. Going by the first approach is more flexible as we can ask the user to play a trick. Suppose the user wants to allocate /mnt/mp3 on tier 1. And it is not empty. We can ask him to set a policy for /mnt/mp3files/ And after enabling OHSM, he can just copy the stuffs from /mnt/mp3 to /mnt/mp3files. This will be better IMHO.
Mar 4, 2009
I think the first approach is simpler and makes more sense, less complex and more flexible.
Mar 19, 2009
Is this done? The solution was quite trivial. I remember RKS doing some tests for this.
Labels:
Target-Release1.1 Component-Logic
Apr 24, 2009
This issue will be dealt with only when coding on OHSM resumes, putting on hold
Status:
On-Hold
Jul 13, 2009
Raising the priority. As far as I remember a initial prototype has been implemented and tested for sure. Rohit can I have a update on this? And when we have this completely functional. From code perspective, looks pretty straight forward to me.
Jul 14, 2009
Directory based allocation has not coded yet, we were more concentrated on testing relocation that time, idea is straight forward so it should not take much time. Changes must be required in policy files, dtd, parser and admin. And file system related changes will be few.
Jul 15, 2009
I am more concerned about the system code changes. Can you go ahead and provide me with the code in yours ../users/ directory of svn. Also, start with just supporting relocation based on directory name. Greg, does it make sense to have allocation policies based on directory names? IMO, the code complexity is not high, but want more insights on the use cases?
Jul 15, 2009
The suggestion seems nice to me, any files created under a parent directory whose home_tid is set will inherit the home_tid from parent.
Jul 15, 2009
This seems to be implemented, i have written a bit of code for it. For test purpose is set the home_tid in ext2_mkdir() function as 2. now when i do mkdir on mount point its sets its home tid as 2. When i create the files inside this directory i simply inherit the TID of parent if its set. So this works fine.
Jul 15, 2009
No the idea here is to simply, use the same code as we have today because the code for regular file creation and a directory is typically same. So, a single call to ohsm_get_tier, which we already have, should serve our purpose. Just that we need to make sure that allocpol contains policies for directory as well. And also, there should be a check that this file's parent directory;s tier is set or not. Makes sense?
Jul 15, 2009
Do you wish to mark this as work in progress? Or you want to do it sometimes later?
Jul 15, 2009
Let it stay for a while, as per inheriting home_tid from parent directory, its done, and for ohsm_get_tier for checking directory based policy it needs overall modification.
Jul 17, 2009
Targeting this to v1.2 And this issue is no more oh hold. I think Rohit should start working on this.
Labels:
-Target-Release1.1 Target-Release1.2
Jul 18, 2009
Sir , i have started working on it,
as far is inheriting parent directory's its done,
if(EXT2_I(dir)->home_tid)
TID = EXT2_I(dir)->home_tid;
else
TID = ohsm_get_tier_for_inode();
this is the code that i added
more over i would say we concentrate more
on recursive relocation, thats a bit of code required in admin.
And for allocation based on 'dir' the check for regular files in ext_creat()
should be removed, that would do policy checking for directory also,
Only policy check for directories is required in admin.
Status:
In-progress
|