| Issue 222: | FreeTDS->unixODBC->pyodbc character problem on linux | |
| 3 people starred this issue and may be notified of changes. | Back to list |
What steps will reproduce the problem? 1.I have a simple script that connect to a MSSQLSERVER 2008 db and extract the value of a VARCHAR field. The db is encoded in SQL_Latin1_General_CP1_CL_AS. My system is Ubuntu 11.04 and the python version: Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24) [GCC 4.5.2] on linux2 pyodbc version 2.1.11 The value contain a 'ì' character. The goal is to export the value in an excel file with the xlwt package. I run the same script on Windows too and the result is very different. When I run on Windows (XP with python 2.5.4, pyodbc 2.1.6 driver={SQL Server}) I obtain a <type str> and to write in the file correctly I must use the unicode function: val = unicode(val,'Latin_1') If I don't encode the string I obtain an error. On linux with the same script I obtain always a '?' character, encoding or not. 2. To go in deep I did try the same extraction with the unixODBC directly using the 'isql' tool of the unixODBC distribution (I stress because I did find also a java tool with the same name...). With isql I obtain the 'ì' character correctly. So it seems the problem is between pyodbc and unixODBC... 3. I did try to analyse the str value extracted character by character. The ord() function on windows return 236 for the 'ì' but on linux it return 63 (I try to uderstand if the output '?' is a content or a rendering problem...) So the character extracted is a '?'. What is the expected output? What do you see instead? 'ì' -> 'ì' on Windows 'ì' -> '?' on Linux What version of the product are you using? On what operating system? WindowsXP with python 2.5.4, pyodbc 2.1.6 driver={SQL Server} is OK Ubuntu 11.04 with python 2.7.1 pyodbc 2.1.11 driver={FreeTDS} NOT OK Please provide any additional information below. I did try also to cast the field at db level CAST(field, NVARCHAR) to estract directly a unicode value but it run ok on Windows (I don't have the necessity to encode the value with the unicode function) and nothing change on Linux.
Dec 18, 2011
This appears to be fixed in 3.0.x. If you run the tests2/freetdstests.py unit tests, you'll see one named test_varchar_upperlatin that is equivalent. (I also changed the character to 236 to match your test exactly and that worked also.)
Status:
Complete
Oct 17, 2012
Hi, I am facing a similar problem, using pyodbc '3.0.6' the database is reporting character iso_1 and collation Modern_Spanish_CI_AS using isql I get the correct results, using pyodbc, I get ? where I sould be gettin ñ or accentuated vowels, best regards, Alberto
Dec 10, 2012
python: 2.6.5 (r265:79063, Oct 1 2012, 22:04:36) [GCC 4.4.3] pyodbc: 3.0.7-beta10 /usr/src/pyodbc/build/lib.linux-x86_64-2.6/pyodbc.so odbc: 03.52 driver: libtdsodbc.so 0.82 supports ODBC version 03.00 os: Linux unicode: Py_Unicode=4 SQLWCHAR=2 server: MSSQL-2000, Cyrillic_General_BIN tests2/freetdstests.py fails 36 tests with unicode, text, varchar, etc
Dec 11, 2012
SQL_WCHAR is not supported by 0.82 http://lists.ibiblio.org/pipermail/freetds/2010q1/025572.html |
Labels: FreeTDS