Issue 233: Text field longer than 3071 characters won't fetch
Status:  Complete
Owner: ----
Closed:  Sep 2012
Reported by victor.o...@gmail.com, Jan 19, 2012
What steps will reproduce the problem?

import pyodbc

conn = pyodbc.connect('DSN=xxx;UID=uuu;PWD=ppp')
cur = conn.cursor()
cur.execute("SELECT replicate('A',3071)")
cur.fetchall()
print "3071 ok"
cur.execute("SELECT replicate('A',3072)")
cur.fetchall()
print "3072 ok"

What is the expected output? What do you see instead?
3071 ok
*** glibc detected *** python: realloc(): invalid pointer: 0x089624b4 ***
======= Backtrace: =========
/lib/i686/nosegneg/libc.so.6(+0x6f8b1)[0x19f8b1]
/lib/i686/nosegneg/libc.so.6(realloc+0x2c4)[0x1a4a44]
/www/env/lib/python2.6/site-packages/pyodbc.so(+0xcab2)[0x11cab2]
/www/env/lib/python2.6/site-packages/pyodbc.so(_Z7GetDataP6Cursori+0x4f4)[0x11d29f]
/www/env/lib/python2.6/site-packages/pyodbc.so(+0x7390)[0x117390]
/www/env/lib/python2.6/site-packages/pyodbc.so(+0x7475)[0x117475]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x4790)[0x852ed7]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x83b)[0x854c69]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalCode+0x63)[0x854d7f]
/usr/lib/libpython2.6.so.1.0(+0xd5857)[0x86e857]
/usr/lib/libpython2.6.so.1.0(PyRun_FileExFlags+0x8c)[0x86e90b]
/usr/lib/libpython2.6.so.1.0(PyRun_SimpleFileExFlags+0xe2)[0x86fe03]
/usr/lib/libpython2.6.so.1.0(PyRun_AnyFileExFlags+0x75)[0x8705f9]
/usr/lib/libpython2.6.so.1.0(Py_Main+0xb95)[0x87c084]
python(main+0x27)[0x804859f]
/lib/i686/nosegneg/libc.so.6(__libc_start_main+0xe6)[0x146ce6]
python[0x8048501]
======= Memory map: ========
00110000-00123000 r-xp 00000000 ca:01 133541     /www/env/lib/python2.6/site-packages/pyodbc.so
00123000-00129000 rw-p 00012000 ca:01 133541     /www/env/lib/python2.6/site-packages/pyodbc.so
0012b000-0012e000 r-xp 00000000 ca:01 7629       /lib/libdl-2.12.so
0012e000-0012f000 r--p 00002000 ca:01 7629       /lib/libdl-2.12.so
0012f000-00130000 rw-p 00003000 ca:01 7629       /lib/libdl-2.12.so
00130000-002bd000 r-xp 00000000 ca:01 7606       /lib/i686/nosegneg/libc-2.12.so
002bd000-002be000 ---p 0018d000 ca:01 7606       /lib/i686/nosegneg/libc-2.12.so
002be000-002c0000 r--p 0018d000 ca:01 7606       /lib/i686/nosegneg/libc-2.12.so
002c0000-002c1000 rw-p 0018f000 ca:01 7606       /lib/i686/nosegneg/libc-2.12.so
002c1000-002c4000 rw-p 00000000 00:00 0 
002c4000-0039a000 r-xp 00000000 ca:01 8616       /usr/lib/libstdc++.so.6.0.13
0039a000-0039e000 r--p 000d6000 ca:01 8616       /usr/lib/libstdc++.so.6.0.13
0039e000-0039f000 rw-p 000da000 ca:01 8616       /usr/lib/libstdc++.so.6.0.13
0039f000-003a6000 rw-p 00000000 00:00 0 
003a6000-003f6000 r-xp 00000000 ca:01 14131      /usr/lib/libssl.so.1.0.0e
003f6000-003fa000 rw-p 00050000 ca:01 14131      /usr/lib/libssl.so.1.0.0e
003fa000-00434000 r-xp 00000000 ca:01 9530       /lib/libgssapi_krb5.so.2.2
00434000-00436000 rw-p 0003a000 ca:01 9530       /lib/libgssapi_krb5.so.2.2
00436000-00438000 r-xp 00000000 ca:01 8063       /lib/libcom_err.so.2.1
00438000-00439000 rw-p 00002000 ca:01 8063       /lib/libcom_err.so.2.1
00439000-0043b000 r-xp 00000000 ca:01 9526       /lib/libkeyutils.so.1.3
0043b000-0043c000 rw-p 00001000 ca:01 9526       /lib/libkeyutils.so.1.3
0043d000-004a5000 r-xp 00000000 ca:01 31339      /usr/local/lib/libodbc.so.1.0.0
004a5000-004aa000 rw-p 00067000 ca:01 31339      /usr/local/lib/libodbc.so.1.0.0
004aa000-0060e000 r-xp 00000000 ca:01 14126      /lib/libcrypto.so.1.0.0e
0060e000-00622000 rw-p 00164000 ca:01 14126      /lib/libcrypto.so.1.0.0e
00622000-00625000 rw-p 00000000 00:00 0 
00627000-00635000 r-xp 00000000 ca:01 18962      /usr/lib/python2.6/lib-dynload/datetime.so
00635000-00638000 rw-p 0000e000 ca:01 18962      /usr/lib/python2.6/lib-dynload/datetime.so
00638000-00700000 r-xp 00000000 ca:01 9536       /lib/libkrb5.so.3.3
00700000-00706000 rw-p 000c8000 ca:01 9536       /lib/libkrb5.so.3.3
00706000-0072f000 r-xp 00000000 ca:01 9534       /lib/libk5crypto.so.3.1
0072f000-00730000 rw-p 00028000 ca:01 9534       /lib/libk5crypto.so.3.1
00730000-00742000 r-xp 00000000 ca:01 7935       /lib/libz.so.1.2.3
00742000-00743000 rw-p 00011000 ca:01 7935       /lib/libz.so.1.2.3
00743000-00748000 r-xp 00000000 ca:01 18928      /usr/lib/python2.6/lib-dynload/_collectionsmodule.so
00748000-00749000 rw-p 00005000 ca:01 18928      /usr/lib/python2.6/lib-dynload/_collectionsmodule.so
00749000-0074f000 r-xp 00000000 ca:01 18975      /usr/lib/python2.6/lib-dynload/operator.so
0074f000-00751000 rw-p 00005000 ca:01 18975      /usr/lib/python2.6/lib-dynload/operator.so
00751000-00754000 r-xp 00000000 ca:01 18986      /usr/lib/python2.6/lib-dynload/timemodule.so
00754000-00756000 rw-p 00002000 ca:01 18986      /usr/lib/python2.6/lib-dynload/timemodule.so
00756000-00762000 r-xp 00000000 ca:01 7639       /lib/libnss_files-2.12.so
00762000-00763000 r--p 0000b000 ca:01 7639       /lib/libnss_files-2.12.so
00763000-00764000 rw-p 0000c000 ca:01 7639       /lib/libnss_files-2.12.so
00764000-00765000 r-xp 00000000 ca:01 7853       /usr/lib/gconv/ISO8859-1.so
00765000-00766000 r--p 00000000 ca:01 7853       /usr/lib/gconv/ISO8859-1.so
00766000-00767000 rw-p 00001000 ca:01 7853       /usr/lib/gconv/ISO8859-1.so
00767000-0077a000 r-xp 00000000 ca:01 31129      /usr/local/lib/libodbcinst.so.1.0.0
0077a000-0077b000 rw-p 00012000 ca:01 31129      /usr/local/lib/libodbcinst.so.1.0.0
0077b000-00782000 r-xp 00000000 ca:01 7651       /lib/librt-2.12.so
00782000-00783000 r--p 00006000 ca:01 7651       /lib/librt-2.12.so
00783000-00784000 rw-p 00007000 ca:01 7651       /lib/librt-2.12.so
00784000-00786000 r-xp 00000000 ca:01 7677       /usr/lib/gconv/CP1252.so
00786000-00787000 r--p 00001000 ca:01 7677       /usr/lib/gconv/CP1252.so
00787000-00788000 rw-p 00002000 ca:01 7677       /usr/lib/gconv/CP1252.so
00799000-008e6000 r-xp 00000000 ca:01 19890      /usr/lib/libpython2.6.so.1.0
008e6000-00912000 rw-p 0014d000 ca:01 19890      /usr/lib/libpython2.6.so.1.0
00912000-0091b000 rw-p 00000000 00:00 0 
0096c000-0096f000 r-xp 00000000 ca:01 18941      /usr/lib/python2.6/lib-dynload/_localemodule.so
0096f000-00970000 rw-p 00003000 ca:01 18941      /usr/lib/python2.6/lib-dynload/_localemodule.so
00970000-009c9000 r-xp 00000000 ca:01 31383      /usr/local/lib/libtdsodbc.so.0.0.0
009c9000-009cb000 rw-p 00059000 ca:01 31383      /usr/local/lib/libtdsodbc.so.0.0.0
009d3000-009d5000 r-xp 00000000 ca:01 18936      /usr/lib/python2.6/lib-dynload/_functoolsmodule.so
009d5000-009d6000 rw-p 00001000 ca:01 18936      /usr/lib/python2.6/lib-dynload/_functoolsmodule.so
00a36000-00a53000 r-xp 00000000 ca:01 15         /lib/libgcc_s-4.4.5-20110214.so.1
00a53000-00a54000 rw-p 0001c000 ca:01 15         /lib/libgcc_s-4.4.5-20110214.so.1
00a89000-00a8a000 r-xp 00000000 00:00 0          [vdso]
00ac7000-00adc000 r-xp 00000000 ca:01 7649       /lib/libresolv-2.12.so
00adc000-00add000 r--p 00014000 ca:01 7649       /lib/libresolv-2.12.so
00add000-00ade000 rw-p 00015000 ca:01 7649       /lib/libresolv-2.12.so
00ade000-00ae0000 rw-p 00000000 00:00 0 
00c3c000-00c3e000 r-xp 00000000 ca:01 7655       /lib/libutil-2.12.so
00c3e000-00c3f000 r--p 00001000 ca:01 7655       /lib/libutil-2.12.so
00c3f000-00c40000 rw-p 00002000 ca:01 7655       /lib/libutil-2.12.so
00c7c000-00c9a000 r-xp 00000000 ca:01 7616       /lib/ld-2.12.so
00c9a000-00c9b000 r--p 0001d000 ca:01 7616       /lib/ld-2.12.so
00c9b000-00c9c000 rw-p 0001e000 ca:01 7616       /lib/ld-2.12.so
00d47000-00d5e000 r-xp 00000000 ca:01 7610       /lib/i686/nosegneg/libpthread-2.12.so
00d5e000-00d5f000 r--p 00016000 ca:01 7610       /lib/i686/nosegneg/libpthread-2.12.so
00d5f000-00d60000 rw-p 00017000 ca:01 7610       /lib/i686/nosegneg/libpthread-2.12.so
00d60000-00d62000 rw-p 00000000 00:00 0 
00d8f000-00dab000 r-xp 00000000 ca:01 8179       /lib/libselinux.so.1
00dab000-00dac000 r--p 0001b000 ca:01 8179       /lib/libselinux.so.1
00dac000-00dad000 rw-p 0001c000 ca:01 8179       /lib/libselinux.so.1
00dfd000-00e25000 r-xp 00000000 ca:01 7631       /lib/libm-2.12.so
00e25000-00e26000 r--p 00027000 ca:01 7631       /lib/libm-2.12.soAborted

What version of the product are you using? On what operating system?
PyODBC ver. 3.0.3, Linux 32bit, unixODBC, FreeTDS, SQL Server Express 2008 R2

Please provide any additional information below.
The same query works correctly in FreeTDS's tsql tool. In unixODBC's isql the result is inconlusive. There was no error but for lenghts greater than 300 characters the program would output blanks:

echo "select replicate('a',301)" | /usr/local/bin/isql dsn user pass -w

Jan 19, 2012
#1 victor.o...@gmail.com
unixODBC config

odbc_config --version --libs --static-libs --libtool-libs --longodbcversion --header --ulen
2.3.0
-L/usr/local/lib -lodbc
/usr/local/lib/libodbc.a
/usr/local/lib/libodbc.la
3.52
#ifndef HAVE_UNISTD_H
 #define HAVE_UNISTD_H
#endif
#ifndef HAVE_PWD_H
 #define HAVE_PWD_H
#endif
#ifndef HAVE_SYS_TYPES_H
 #define HAVE_SYS_TYPES_H
#endif
#ifndef HAVE_LONG_LONG
 #define HAVE_LONG_LONG
#endif
#ifndef ODBCINT64
 #define ODBCINT64 long long
#endif
#ifndef UODBCINT64
 #define UODBCINT64 unsigned long long
#endif
#ifndef SIZEOF_LONG_INT
 #define SIZEOF_LONG_INT 4
#endif
-DSIZEOF_SQLULEN=4


Jan 19, 2012
#2 victor.o...@gmail.com
In the fisql tool the query definitely works correctly even on 3072 characters.
Apr 11, 2012
#3 wpu.cl...@gmail.com
Has anyone figured out a way around this problem until it is fixed?
May 13, 2012
#4 AlwaysCa...@googlemail.com
Is there still no fix or workaround?
May 14, 2012
#5 Ponimas...@gmail.com
Casting to ntext is possible workaround.
It solves problem for me.
May 14, 2012
#6 AlwaysCa...@googlemail.com
Worked for me too, thank you. A fix still would be good, I barely can use this Workaround in production.

Anyway, thanks a lot.
Jun 1, 2012
#7 katti.sh...@gmail.com
We are fetching a text field from a system table in Vertica and so there is no way we can alter that table. Can anyone please provide a solution/fix to this issue?
Jun 13, 2012
#8 br...@huron-data.com
I have also been having this same 'python: realloc(): invalid pointer' problem with pyodbc 3.0.3 and lengthy text fields.  I reverted to pyodbc 2.1.11 and everything seems to be working fine.
Jun 23, 2012
Project Member #9 mkleehammer
This may have been fixed in 3.0.6-beta07 (66c51365b2a2283a0742ca9cab771f3362a9fbb8).

Reading ASCII fields somewhere around 4K and up were failing with a realloc error.

Can you retest with this version?

Status: Investigating
Jul 13, 2012
#10 wpu.cl...@gmail.com
Finally got around to looking at this, and I just upgraded my version to 3.0.6 (not beta), and it seems to have fixed the issue. Just ran a bunch of tests and nothing else appears to have been affected.

Thanks!
Jul 13, 2012
#11 katti.sh...@gmail.com
Great. Looking at the source code, it seems hard for me to tell exactly what fixed it. Also, the release note on the home page doesn't mention this issue being fixed. It would be great if the author of the this release can point out the issue was fixed and exactly what fixed it!

Thanks pyodbc developers
Sep 27, 2012
Project Member #12 mkleehammer
(No comment was entered for this change.)
Status: Complete
Sep 2, 2013
#13 superstr...@gmail.com
Today, I find same problem. GetData, the sql column buffer length > 12000 char,
SQLGetData func, return cbData is -4...

I modify getdata.cpp:311, char tempBuffer[1026] 
to:
char tempBuffer[40260]; 

one fetch ok...

build and install .