My favorites | Sign in
Project Home Downloads Wiki
READ-ONLY: This project has been archived. For more information see this post.
Search
for
  Advanced search   Search tips   Subscriptions
Issue 185: pyodbc won't import within an embedded python interpreter
4 people starred this issue and may be notified of changes. Back to list
Status:  Investigating
Owner:  ----


 
Reported by wsad...@gmail.com, Jun 30, 2011
I have run into a problem I can't figure out how to work around:

I can import pyodbc without error if I do it within a command line python interpreter.  However, I need to import and use it from an embedded python interpreter within my C++ program, and this fails.
(See attached screenshot.)

Attached is a simple zipped up VS2010 C++ program that embeds a python interpreter into some C++ code and demonstrates the problem.  (Included is a class that abstracts the nasty details of the embedding the interpreter, to provide a relatively clean interface for running python and extracting results from C++ code.)

It executes two simple python commands, one to import the pyexpat module, (also a  module implemented with a .pyd), which fails, and one to import pyodbc, which fails, claiming that the module could not be found.  (The pyexpat import is included merely to show that the embedded interpreter is working; if removed, the pyodbc import still fails.)

Python version: 2.7.2
pyodbc version: 2.1.8.win32-py2.7
Embedding language: (VS2010) C++
OS: Windows Vista or 7 (fails the same way on both)

pyodbcBugScreenshot.jpg
68.8 KB   View   Download
PyOdbcTest.zip
1.4 MB   Download
Jul 8, 2011
#1 wsad...@gmail.com
I just noticed that I had a typo in my description; the import of pyexpat *succeeds*; it's just pyodbc that fails to import.  

I am eagerly awaiting some investigation of this, as I can't proceed without a fix for this problem.  (priority=high?)

Thanks,
/Will Sadkin
Jul 8, 2011
Project Member #2 mkleehammer
I am almost certain the issue is because you are using Visual Studio 2010.  Python 2.7 was built with 2008 and Microsoft put some boneheaded code into the 2005 & 2008 CRTs making it very difficult to mix and match DLLs.  They have corrected their (completely inexcusable) error in 2010, BTW.

I created a simple Win32 command line project with only the following code:
int _tmain(int argc, TCHAR* argv[], TCHAR* envp[])
{
	Py_Initialize();
	if (!Py_IsInitialized())
	{
		printf("init fail\n");
		return 1;
	}
	if (PyRun_SimpleString("import pyodbc"))
	{
		PyErr_Print();
		return 1;
	}
	return 0;
}

This works fine with 2008 but fails with the same error with 2010.

(Also note that I built these in release mode since building a debug version of Python is so much trouble on Windows.)

I'm going to close this for now, but please reopen if you 2008 tests do not work.
Status: NoFix
Jul 8, 2011
#3 wsad...@gmail.com
<jaw on floor>

Just to be clear:  Is the incompatibilty because pyodbc.pyd was generated with VS2008, or because python2.7 (which I've linked into my VS2010 code with no reported problems, by the way) is compiled with VS2008, or both?

Is this "python is only compiled for compatibility with VS2008" restriction documented anywhere?

And, if your recommendation therefore to use the older compiler, for how long should I expect this to persist?
(related) Any idea when will Python 2.7 be compiled with the latest MS compiler, and will pyodbc then follow suit? 

<jaw on floor/>

Sorry, but I'm speechless.  I spent a long time porting my code to the latest MS compiler (because that's what you *do* in the microsoft world.)  And of course, I also upgraded python at the same time, because its supposed to be the stable release... @#$=&%^!

Respectfully,
/Will Sadkin
Jul 8, 2011
Project Member #4 mkleehammer
The issue is that Python 2.5 - 2.7 are all built with VS2008.  That means Microsoft's XML manifests are embedded in the executable and the msvcrt DLL will check it and fail to load it if doesn't match exactly.  I believe what is happening is that the pyodbc manifest requires the 2008 CRT.

That means it might be possible to make something work by building without a manifest (I think would require a little distutils hacking?), modifying or removing it.  When I get some free time I'll try that, but you might look at it too.

There are no plans to ever change 2.7 compilers since it would break existing extensions.  pyodbc will always have to be built with the version that Python is built with.  I know 2010 is being looked at for some 3.x versions, but I don't remember the details.

Caveat: I haven't embedded Python in years and am mostly familiar with building extensions for python.exe.  I could be wrong, but I do know my tests worked w/ 2008 and did not work w/ 2010.  I also know I have to update the manifests from SP1 down to the original python version of some other extensions I build for work.
Status: Investigating
Jul 8, 2011
#5 wsad...@gmail.com
Another question then:  If python *were* compiled with 2010, and if, as you say, "They have corrected their (completely inexcusable) error in 2010,", then wouldn't such a version of python be able to load older (ie. 2008-compiled) extensions?
Aug 29, 2011
#7 alex.r...@gmail.com
I have a similar problem using pyodbc from an embedded interpreter compiled with VS2005 and python 2.7. While pyodbc is imported correctly from the standard interpreter, the embedded one returns a dll loading error.
Aug 29, 2011
#8 wsad...@gmail.com
I believe you will have a harder time fixing yours than I did mine.  I had to explicitly add a manifest dependency on the VS2008 CRT, using an additional manifest dependency of 
"type='Win32' name='Microsoft.VC90.CRT' version='9.0.21022.8' processorArchitecture='X86' publicKeyToken='1fc8b3b9a1e18e3b' " 

in the linker settings for the executable in question.  Once I did this, the interpreter would load properly.  However, that was compiling with VS2010; I don't know how "forward-compatible" VS2005 will be in that regard...

Aug 29, 2011
#9 alex.r...@gmail.com
Thank you very much. The strange thing is that using python 2.6 and the 2.1.7 version of pyodbc all was working.
Aug 30, 2011
#10 alex.r...@gmail.com
I temporarily solved the problem by recompiling pyodbc with VS2005, now the pyodbc.pyd module is successfully loaded from the embedded interpreter.
Oct 6, 2011
#11 alfr...@gmail.com
Please see issue 214.
Nov 8, 2011
#12 rich.ma...@gmail.com
I am running into the same issue where I cannot import pyodbc into an embedded Python interpretor, but this is on Linux (CentOS 5.5). The following error is given:

ImportError: /usr/local/lib/python2.6/site-packages/pyodbc.so: undefined symbol: _Py_ZeroStruct

Does anyone know what what could be causing this?

Powered by Google Project Hosting