Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set bigger buffer sizes to speedup data transfers #94

Closed
giampaolo opened this issue May 28, 2014 · 4 comments
Closed

Set bigger buffer sizes to speedup data transfers #94

giampaolo opened this issue May 28, 2014 · 4 comments
Labels
Component-Library enhancement imported imported from old googlecode site and very likely outdated Performance Priority-High

Comments

@giampaolo
Copy link
Owner

From billiej...@gmail.com on November 25, 2008 14:45:56

It seems that the current buffer size of 8k [1] is not optimal as came out 
in a recent discussion on the mailing list [2].
Using a different buffer size (e.g. 32k or 64k) could result in a 
drastical speed improvement for both incoming and outgoing data transfers.

[1] https://code.google.com/p/pyftpdlib/source/browse/tags/release-0.5.0/ 
pyftpdlib/ftpserver.py#790

[2] http://groups.google.com/group/pyftpdlib/browse_thread/thread/ 
f13a82b2f41764be

Original issue: http://code.google.com/p/pyftpdlib/issues/detail?id=94
@giampaolo
Copy link
Owner Author

From billiej...@gmail.com on November 25, 2008 09:55:28

I made a benchmark between two boxes: a Windows XP SP3 and a Linux Debian Etch, both 
placed in a 10 Mb LAN connected through a swtich.
The benchamrk consisted in moving a 20 Mb binary file from a machine to another.
The results (floats are times expressed in seconds):

Linux:

buf 4096    ; upload recv(): 2.671 download send(): 2.453
buf 8192    ; upload recv(): 2.016 download send(): 1.562
buf 16384   ; upload recv(): 2.046 download send(): 1.405
buf 32768   ; upload recv(): 2.032 download send(): 1.140
buf 65536   ; upload recv(): 2.171 download send(): 1.046
buf 131072  ; upload recv(): 2.141 download send(): 1.092
buf 262144  ; upload recv(): 2.235 download send(): 1.077


Windows:

buf 4096    ; upload recv(): 2.801 download send(): 4.236
buf 8192    ; upload recv(): 1.821 download send(): 3.730
buf 16384   ; upload recv(): 2.066 download send(): 3.775
buf 32768   ; upload recv(): 1.992 download send(): 2.487
buf 65536   ; upload recv(): 1.317 download send(): 2.221
buf 131072  ; upload recv(): 2.029 download send(): 2.310
buf 262144  ; upload recv(): 1.280 download send(): 2.165


It seems that using 64k for both send() and recv() is a good compromise.

Attachment: bench.py

@giampaolo
Copy link
Owner Author

From billiej...@gmail.com on November 26, 2008 05:11:16

Fixed in r450 .
Set both buffers at 64k.

Summary: Set bigger buffer sizes to speedup data transfers
Status: Finished

@giampaolo
Copy link
Owner Author

From billiej...@gmail.com on January 21, 2009 10:05:51

Fixed in version 0.5.1 released on 2009-01-21.

Status: Fixed

@giampaolo
Copy link
Owner Author

From billiej...@gmail.com on April 02, 2009 08:53:55

Labels: -OpSys-All

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Component-Library enhancement imported imported from old googlecode site and very likely outdated Performance Priority-High
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@giampaolo