Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Patch for pipe draining and closing #125

Closed
GoogleCodeExporter opened this issue Mar 24, 2015 · 11 comments
Closed

Patch for pipe draining and closing #125

GoogleCodeExporter opened this issue Mar 24, 2015 · 11 comments
Labels
auto-migrated

Comments

@GoogleCodeExporter
Copy link 

What steps will reproduce the problem?
Only one step:
xdelta3 -e -9 -v -B 536870912 -I 0 -s file1.iso file2.iso iso.xdelta 

file1.iso = 8 503 033 856 bites long
file2.iso = 4 494 721 024 bites long (seems like this one isn't a problem

-B specified because I want to use large buffer for source file. Actually I've 
tried to use ~4Gb but seems like it isn't possible with win32 version of 
xdelta3 under wine which I've used to be able to compare these two iso-files at 
all.

What is the expected output?
It works and generates iso.xdelta file.

What do you see instead?
Program crashes immediatelly on file1 open. Does nothing except this.

What version of the product are you using?
xdelta3.0y from repository and xdelta3.0z compiled from sources

On what operating system?
Ubuntu 10.10
Linux * 2.6.35-28-generic-pae #49-Ubuntu SMP Tue Mar 1 14:58:06 UTC 2011 i686 
GNU/Linux

Please provide any additional information below.
(stacktrace from version 3.0y)
*** buffer overflow detected ***: xdelta3 terminated
======= Backtrace: =========
/lib/libc.so.6(__fortify_fail+0x50)[0xb7807970]
/lib/libc.so.6(+0xe486a)[0xb780686a]
/lib/libc.so.6(+0xe3fa8)[0xb7805fa8]
/lib/libc.so.6(_IO_default_xsputn+0x9e)[0xb778ca2e]
/lib/libc.so.6(_IO_vfprintf+0xe34)[0xb7760044]
/lib/libc.so.6(__vsprintf_chk+0xad)[0xb780605d]
/lib/libc.so.6(__sprintf_chk+0x2d)[0xb7805f9d]
xdelta3[0x805c78f]
xdelta3[0x806a455]
xdelta3[0x806de93]
/lib/libc.so.6(__libc_start_main+0xe7)[0xb7738ce7]
xdelta3[0x8048ef1]
======= Memory map: ========
08048000-08076000 r-xp 00000000 08:01 2900047    /usr/bin/xdelta3
08076000-08077000 r--p 0002d000 08:01 2900047    /usr/bin/xdelta3
08077000-08078000 rw-p 0002e000 08:01 2900047    /usr/bin/xdelta3
08078000-08082000 rw-p 00000000 00:00 0 
09485000-094a6000 rw-p 00000000 00:00 0          [heap]
96ee3000-96efd000 r-xp 00000000 08:01 2097213    /lib/libgcc_s.so.1
96efd000-96efe000 r--p 00019000 08:01 2097213    /lib/libgcc_s.so.1
96efe000-96eff000 rw-p 0001a000 08:01 2097213    /lib/libgcc_s.so.1
96eff000-b7722000 rw-p 00000000 00:00 0 
b7722000-b7879000 r-xp 00000000 08:01 2098758    /lib/libc-2.12.1.so
b7879000-b787b000 r--p 00157000 08:01 2098758    /lib/libc-2.12.1.so
b787b000-b787c000 rw-p 00159000 08:01 2098758    /lib/libc-2.12.1.so
b787c000-b787f000 rw-p 00000000 00:00 0 
b787f000-b78a3000 r-xp 00000000 08:01 2098781    /lib/libm-2.12.1.so
b78a3000-b78a4000 r--p 00023000 08:01 2098781    /lib/libm-2.12.1.so
b78a4000-b78a5000 rw-p 00024000 08:01 2098781    /lib/libm-2.12.1.so
b78bd000-b78bf000 rw-p 00000000 00:00 0 
b78bf000-b78c0000 r-xp 00000000 00:00 0          [vdso]
b78c0000-b78dc000 r-xp 00000000 08:01 2098755    /lib/ld-2.12.1.so
b78dc000-b78dd000 r--p 0001b000 08:01 2098755    /lib/ld-2.12.1.so
b78dd000-b78de000 rw-p 0001c000 08:01 2098755    /lib/ld-2.12.1.so
bff9d000-bffbe000 rw-p 00000000 00:00 0          [stack]
Aborted

Please tell me if you need any additional information.

Original issue reported on code.google.com by lain.halfbit@gmail.com on 14 Mar 2011 at 9:15
@GoogleCodeExporter
Copy link
Author 

I am experiencing the same problem with version Xdelta version 3.0.0 on Ubuntu 
10.04.1 LTS;  the patch file was created on Windows Xdelta version 3.0.0; Here 
is output on ubuntu

*** buffer overflow detected ***: /usr/local/bin/xdelta3 terminated
======= Backtrace: =========
/lib/libc.so.6(__fortify_fail+0x37)[0x7f242fcf3217]
/lib/libc.so.6(+0xfe0d0)[0x7f242fcf20d0]
/lib/libc.so.6(+0xfd539)[0x7f242fcf1539]
/lib/libc.so.6(_IO_default_xsputn+0xcc)[0x7f242fc69d1c]
/lib/libc.so.6(_IO_vfprintf+0x628)[0x7f242fc399c8]
/lib/libc.so.6(__vsprintf_chk+0x99)[0x7f242fcf15d9]
/lib/libc.so.6(__sprintf_chk+0x7f)[0x7f242fcf151f]
/usr/local/bin/xdelta3[0x413296]
/usr/local/bin/xdelta3[0x41be3a]
/usr/local/bin/xdelta3[0x426c46]
/lib/libc.so.6(__libc_start_main+0xfd)[0x7f242fc12c4d]
/usr/local/bin/xdelta3[0x401539]
======= Memory map: ========
00400000-0042e000 r-xp 00000000 08:01 536274                             
/usr/local/bin/xdelta3
0062d000-0062e000 r--p 0002d000 08:01 536274                             
/usr/local/bin/xdelta3
0062e000-0062f000 rw-p 0002e000 08:01 536274                             
/usr/local/bin/xdelta3
0062f000-00639000 rw-p 00000000 00:00 0
022b4000-022d5000 rw-p 00000000 00:00 0                                  [heap]
7f242b1d4000-7f242b1ea000 r-xp 00000000 08:01 82109                      
/lib/libgcc_s.so.1
7f242b1ea000-7f242b3e9000 ---p 00016000 08:01 82109                      
/lib/libgcc_s.so.1
7f242b3e9000-7f242b3ea000 r--p 00015000 08:01 82109                      
/lib/libgcc_s.so.1
7f242b3ea000-7f242b3eb000 rw-p 00016000 08:01 82109                      
/lib/libgcc_s.so.1
7f242b3f2000-7f242fbf4000 rw-p 00000000 00:00 0
7f242fbf4000-7f242fd6e000 r-xp 00000000 08:01 82377                      
/lib/libc-2.11.1.so
7f242fd6e000-7f242ff6d000 ---p 0017a000 08:01 82377                      
/lib/libc-2.11.1.so
7f242ff6d000-7f242ff71000 r--p 00179000 08:01 82377                      
/lib/libc-2.11.1.so
7f242ff71000-7f242ff72000 rw-p 0017d000 08:01 82377                      
/lib/libc-2.11.1.so
7f242ff72000-7f242ff77000 rw-p 00000000 00:00 0
7f242ff77000-7f242fff9000 r-xp 00000000 08:01 82096                      
/lib/libm-2.11.1.so
7f242fff9000-7f24301f8000 ---p 00082000 08:01 82096                      
/lib/libm-2.11.1.so
7f24301f8000-7f24301f9000 r--p 00081000 08:01 82096                      
/lib/libm-2.11.1.so
7f24301f9000-7f24301fa000 rw-p 00082000 08:01 82096                      
/lib/libm-2.11.1.so
7f24301fa000-7f243021a000 r-xp 00000000 08:01 82362                      
/lib/ld-2.11.1.so
7f243040d000-7f2430410000 rw-p 00000000 00:00 0
7f2430417000-7f2430419000 rw-p 00000000 00:00 0
7f2430419000-7f243041a000 r--p 0001f000 08:01 82362                      
/lib/ld-2.11.1.so
7f243041a000-7f243041b000 rw-p 00020000 08:01 82362                      
/lib/ld-2.11.1.so
7f243041b000-7f243041c000 rw-p 00000000 00:00 0
7fff57a04000-7fff57a19000 rw-p 00000000 00:00 0                          [stack]
7fff57b0c000-7fff57b0d000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  
[vsyscall]
Aborted

Original comment by sja...@divinsa.com on 4 May 2011 at 8:07

@GoogleCodeExporter
Copy link
Author 

FYI, the command used on ubuntu was simply:
xdelta3 -d -v -f -s old  old.patch result 
so no special values were used; same holds true on Windows when old.patch was 
created (i.e. everything set to default) and the actual file sizes are not very 
large.

Here is config output on ubuntu:


xdelta3 config
Xdelta version 3.0.0, Copyright (C) 2007, 2008, 2009, 2010, Joshua 
MacDonaldnXdelta comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; see "COPYING" for details.
EXTERNAL_COMPRESSION=1
GENERIC_ENCODE_TABLES=0
GENERIC_ENCODE_TABLES_COMPUTE=0
REGRESSION_TEST=1
SECONDARY_DJW=1
SECONDARY_FGK=1
UNALIGNED_OK=1
VCDIFF_TOOLS=1
XD3_ALLOCSIZE=16384
XD3_DEBUG=0
XD3_ENCODER=1
XD3_POSIX=1
XD3_STDIO=0
XD3_WIN32=0
XD3_USE_LARGEFILE64=1
XD3_DEFAULT_LEVEL=3
XD3_DEFAULT_IOPT_SIZE=32768
XD3_DEFAULT_SPREVSZ=262144
XD3_DEFAULT_SRCWINSZ=67108864
XD3_DEFAULT_WINSIZE=8388608
XD3_HARDMAXWINSIZE=16777216
sizeof(void*)=8
sizeof(int)=4
sizeof(uint32_t)=4
sizeof(uint64_t)=8
sizeof(usize_t)=4
sizeof(xoff_t)=8


And here is config output on Windows:


C:\Program Files\xdelta3>xdelta3 config
Xdelta version 3.0.0, Copyright (C) 2007, 2008, 2009, 2010, Joshua MacDonaldnXde
lta comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; see "COPYING" for details.
EXTERNAL_COMPRESSION=0
GENERIC_ENCODE_TABLES=0
GENERIC_ENCODE_TABLES_COMPUTE=0
REGRESSION_TEST=1
SECONDARY_DJW=1
SECONDARY_FGK=1
UNALIGNED_OK=1
VCDIFF_TOOLS=1
XD3_ALLOCSIZE=16384
XD3_DEBUG=0
XD3_ENCODER=1
XD3_POSIX=0
XD3_STDIO=0
XD3_WIN32=1
XD3_USE_LARGEFILE64=1
XD3_DEFAULT_LEVEL=3
XD3_DEFAULT_IOPT_SIZE=32768
XD3_DEFAULT_SPREVSZ=262144
XD3_DEFAULT_SRCWINSZ=67108864
XD3_DEFAULT_WINSIZE=8388608
XD3_HARDMAXWINSIZE=16777216
sizeof(void*)=4
sizeof(int)=4
sizeof(uint32_t)=4
sizeof(uint64_t)=8
sizeof(usize_t)=4
sizeof(xoff_t)=8

Original comment by sja...@divinsa.com on 4 May 2011 at 8:28

@GoogleCodeExporter
Copy link
Author 

The problem seems to be that xdelta3 (version 3.0.0) has numerous sprintf and 
strcpy calls that write into 32 byte char buffers that are on the stack or are 
static allocations.  With sufficiently large files, these strings can overflow 
the 32 char buffers, and some recent gnu libc versions will detect this and 
immediately abort with the error "*** buffer overflow detected ***".  For 
example, I am seeing this abort on Kubuntu 11.04.

I fixed it on my local copy of the source with these changes to xdelta3-main.h 
and xdelta3-blkcache.h:
  changing the sprintf and strcpy calls to snprintf and strncpy calls
  passing in the buffer length to the various main_format_*() routines
  changing all the char [32] buffers to be char [48] buffers.

I presume that just changing all "[32]" to "[48]" would workaround the problem, 
but the snprintf and strncpy calls were the "correct" solution to apply as well.

Original comment by ThePytho...@gmail.com on 18 Aug 2011 at 6:05

@GoogleCodeExporter
Copy link
Author 

#3

When can we get that patches for fixing the issue? Thanks a lot

Original comment by pachora...@gmail.com on 3 Mar 2012 at 12:18

@GoogleCodeExporter
Copy link
Author 

I created three patches, with the three fixes (two related patches for the 
buffer overflow, and another unrelated patch for a hang due to improper pipe 
closing and
flushing.

These three patches can be found at:

http://thepythoniccow.us/xdelta3.0.0_patches/bigger_print_buffers
http://thepythoniccow.us/xdelta3.0.0_patches/replace_sprintf_with_snprintf
http://thepythoniccow.us/xdelta3.0.0_patches/fix_pipe_draining_and_closing

These same three patches are also attached to this comment.

Original comment by ThePytho...@gmail.com on 27 Mar 2012 at 5:26

Attachments:

@GoogleCodeExporter
Copy link
Author

Original comment by josh.mac...@gmail.com on 17 Jun 2012 at 12:25

  • Changed state: Started

@GoogleCodeExporter
Copy link
Author 

(Soon -- I'll get these fixes into 3.0.2)

Original comment by josh.mac...@gmail.com on 26 Jun 2012 at 9:45

@GoogleCodeExporter
Copy link
Author 

SVN 358 has the buffer overflow fixes. I'll get the pipe drain issue next, for 
3.0.2.

Original comment by josh.mac...@gmail.com on 27 Jun 2012 at 9:48

1 similar comment
@GoogleCodeExporter
Copy link
Author 

SVN 358 has the buffer overflow fixes. I'll get the pipe drain issue next, for 
3.0.2.

Original comment by josh.mac...@gmail.com on 27 Jun 2012 at 9:48

@GoogleCodeExporter
Copy link
Author 

The buffer overflow fixes are released in 3.0.2.

I'll keep this open and rename the Summary to keep track of the 
fix_pipe_draining_and_closing issue, which will go into the next release.

Original comment by josh.mac...@gmail.com on 30 Jun 2012 at 7:30

  • Changed title: Patch for pipe draining and closing

@GoogleCodeExporter
Copy link
Author

Original comment by josh.mac...@gmail.com on 30 Jun 2012 at 7:33

  • Changed state: Duplicate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
auto-migrated
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@GoogleCodeExporter