My favorites | Sign in
Project Home Downloads Wiki Issues Source
READ-ONLY: This project has been archived. For more information see this post.
Search
for
  Advanced search   Search tips   Subscriptions
Issue 69: Tricky copy and swap Optimization
1 person starred this issue and may be notified of changes. Back to list
 
Project Member Reported by imreckless@gmail.com, Mar 3, 2009
Hello everyone,

We are reviving Tricky Copy and Swap algorithm once again.
The point here is to increase the performance of the algorithm.
It must be comparable to cp and dd atleast.

Possible Cause of low performance:

1. Block allocation optimization:
   Our implementation of block allocation lacks some of the existing
optimization. Inodes and data blocks are in different block groups.
This increases disk seeks and its a huge bottleneck.If you take the case of
inode in BG 1 and data blocks in BG 10 there is a seek of 10*128*1024*1024
bytes for a 4k block file system.

2. Lacks usage of read ahead.
   Whenever file is read sequentially read ahead optimization is used.
But our algorithm reads one block at a time. 

3. Writing to a file.
   We are allocating one block at a time while the existing write
implementation has block preallocation. Our algorithm allocates one block
at a time ext2_get_block does allocate reservation window but i am doubtful
it is used in our case,

So avoiding the above bottle necks can improve the performance.

VA can i have your insights on this.
Mar 3, 2009
Project Member #1 checkout...@gmail.com
Hello Rohit,

In my regards, allocation of blocks is similar to ext2 .

For the case you have mentioned that inode in BG 1 and Data blocks in BG 10,
seek time would be same for cp if it were to allocate blocks in Similar fashion and i
don't think we can do anything about it.

Well u might be thinking allocating the inode in the same blockgroup but it would
again cause a performance drawback for allocation of new inode. As we have to remove
various optimization associated with it.

So, decreasing the cost of relocation on the cost of allocation would be a costly
affair.As relocation is triggered comparatively less than allocation.

Yes with read ahead we can have a performance boost.

yeah we are allocating one block at a time and ext2_get_block do use reservation
window and i also think it is not used in our case. 



Mar 3, 2009
Project Member #2 imreckless@gmail.com
Hi Vineet,

Allocation of blocks differs, ext2 tries to allocate data blocks in the same BG
as that of inode and OHSM block allocation lacks this optimization.
We have no restriction on inode allocation, so inode is allocated
in BG 0 initially, and data blocks are allocated in the specified tier,can be any BG.
So seek time has to increase in such cases.
And the concept of BG itself says that they 
were intended to cluster inodes and related data blocks together.

But seek time may not be a point of concern as you said,
when block are allocated through cp it will go through OHSM block allocation only.

No VA i am not thinking of implementing inode allocation
as it will create several other problems of hardlinks and symlinks. 

I think we should go with read ahead and have some performance gains.


Apr 24, 2009
#3 rishi.b....@gmail.com
This issue will be dealt with only when coding on OHSM resumes,

putting on hold
Status: On-Hold
Labels: -Type-Defect Type-Enhancement
Jul 14, 2009
#4 rishi.b....@gmail.com
RKS or VA can you comment on this.

Opening the issue again
Status: In-progress
Jul 14, 2009
Project Member #5 imreckless@gmail.com
Relocation code is robust and it guarantees data consistency.
Foreseeable optimization can be working with chunk of blocks 
rather than block by block.

Jul 15, 2009
Project Member #6 sandeepksinha
Working on a chunk of blocks will only provide you with the ability to reduce the
lock contention and you would rather like to give other processes a chance to acquire
the lock alternatively.
How do you think its going to make things faster.

I think here what you need to focus in on the read aheads and all those stuffs.
See the implementation of how cp and dd works for a file.
Both goes via the file system.

If you want the trace for a cp. I will tell you how to do that.

Just hard code allocpol = NULL in ohsm_get_tier; and try the following.

0. Compile this modified ohsm module. (with allocpol = NULL)
1. Load ext2
2. Mount a FS
3. Insert ohsm.ko
4. Do a ohsm -enable ....
5. cp file <mountpoint> // eg cp ~/*.c /mnt

It would panic and kill the process. In dmesg you can simply get the trace.
Its simple. I have seen some readaheads and you call also find the trace in some
conversations between me and vineet some time back. Check your mailbox.


Powered by Google Project Hosting