Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GC3Utils should handle custom Application classes #95

Closed
GoogleCodeExporter opened this issue Aug 6, 2015 · 13 comments
Closed

GC3Utils should handle custom Application classes #95

GoogleCodeExporter opened this issue Aug 6, 2015 · 13 comments

Comments

@GoogleCodeExporter
Copy link

Commands in the GC3Utils should be able to manipulate jobs created
with other applications as well. Actually, `grosetta`, `grundb`,
etc. rely on this feature: we don't want to re-implement every job
control action in every piece of software.

However, the current `gc3libs.persistence` module saves Python objects
using the `pickle` framework: thus, the `Application` subclass
corresponding to a job must be already loaded (or at least
`import`-able) in the Python interpreter for `pickle` to be able to
"undump" the object from its on-disk representation.

In other words, if you create a custom `Application` subclass in you
client code, GC3Utils won't be able to read job files created by this
code, because the class definition is not available in GC3Libs.

Fix (suggested by Sergio): have GC3Libs/GC3Utils use a
"plugins" subdirectory: client code wishing to make use of custom
`Application` subclasses should drop a `.py` file in the "plugins"
directory, with the class definition.  There should be system- and
user-level plugins directories, which are added to GC3Libs' import
path.

Original issue reported on code.google.com by riccardo.murri@gmail.com on 10 Jan 2011 at 1:45

@GoogleCodeExporter
Copy link
Author

Temporary workaround for `grosetta`, `ggamess`, etc.: since they only
use a custom subclass to add a logging facility through the use of a
`set_info()` method, port this into a functionality of the main
GC3Libs `Application` class, and have `grosetta` etc. use the main
GC3Libs classes.

Implmented in SVN r1039.

Original comment by riccardo.murri@gmail.com on 10 Jan 2011 at 1:48

@GoogleCodeExporter
Copy link
Author

Note the workaround doesn't apply to `gdocking`, as it uses a nontrivial 
`postprocess` method.

Original comment by riccardo.murri@gmail.com on 10 Jan 2011 at 1:53

@GoogleCodeExporter
Copy link
Author

Original comment by riccardo.murri@gmail.com on 26 Jan 2011 at 1:26

  • Added labels: Milestone-Release1.0

@GoogleCodeExporter
Copy link
Author

For the record: the same issue with "pickle" is discussed here: 
http://stefaanlippens.net/pickleproblem

However, the solution suggested (adding "__module__ = ..." to classes) does not 
work.  Python's pickle module dies with an exception "Class 
gdocking.GDockingApplication is not the same as gdocking.GDockingApplication".  
The message is nonsensical, but the idea should be that the same class is 
detected as gdocking.GDockingApplication in one case but insterted as 
__main__.GDockingApplication (or the reverse)

Needs further investigation.

Original comment by riccardo.murri@gmail.com on 23 Mar 2011 at 3:25

@GoogleCodeExporter
Copy link
Author

Here's a workaround, first tried in `gdocking.py` as of SVN r1439:

1. Have the script import itself before defining the `Application`
   subclass::

        if __name__ == "__main__":
          import gdocking

2. Always call the `Application` subclass by fully-qualified name::

        def __init__(self):
            SessionBasedScript.__init__(
                # ...
                application = gdocking.GDockingApplication,
                # ...
                )

3. Now any other Python command can `pickle.load` a
   `GDockingApplication`, provided it can import the `gdocking`
   module.  (This might involve some `sys.path` or PYTHONPATH
   munging.)

Original comment by riccardo.murri@gmail.com on 6 May 2011 at 2:43

  • Changed state: Started

@GoogleCodeExporter
Copy link
Author

Actually, there is a subtle point in the previous workaround: the
fully-qualified name must be used only in code that does not run in
the "imported module" context!

Thus:

* if a line of code runs when "__name__ = '__main__'", then use the
  fully-qualified name;

* if a line of code runs when "__name__ = '<script name>'", then use
  the non-qualified name.

For GC3Apps script, this boils down to the following simple rule:
only use the fully-qualified name in methods overriding parts of the
`SessionBasedScript` class.

Original comment by riccardo.murri@gmail.com on 6 May 2011 at 4:06

@GoogleCodeExporter
Copy link
Author

Workaround implemented for the remaining scripts in SVN r1440

Original comment by riccardo.murri@gmail.com on 6 May 2011 at 4:07

@GoogleCodeExporter
Copy link
Author

Original comment by riccardo.murri@gmail.com on 16 Jun 2011 at 3:16

  • Added labels: Milestone-Release1.1
  • Removed labels: Milestone-Release1.0

@GoogleCodeExporter
Copy link
Author

Can we close this ticket now ?

Sergio :)

Original comment by sergio.m...@gmail.com on 1 Jul 2011 at 11:33

@GoogleCodeExporter
Copy link
Author

Not sure - we have a workaround, is it a good enough "solution"?

Original comment by riccardo.murri@gmail.com on 2 Jul 2011 at 8:52

@GoogleCodeExporter
Copy link
Author

In SVN r1926 there's a better workaround, which consists of two lines
of boilerplate code, and requires no reasoning about what classes are
called where.

The trick is to "import self" and then use the
fully qualified name to run the script:

    if __name__ == "__main__":
        import gcrypto
        gcrypto.GCryptoScript().run()

Then the rest of the script runs as the `gcrypto` module, so there's
no need to qualify local names!

I think this fixes the issue; I'm going to keep the ticket open until
we have documented this.


Original comment by riccardo.murri@gmail.com on 10 Feb 2012 at 3:32

@GoogleCodeExporter
Copy link
Author

Note that the issue can be properly solved without any workaround starting with 
Python 3.3, which implements PEP 3155: http://www.python.org/dev/peps/pep-3155/

Original comment by riccardo.murri@gmail.com on 5 Mar 2012 at 2:03

@GoogleCodeExporter
Copy link
Author

This issue was closed by revision r2360.

Original comment by riccardo.murri@gmail.com on 19 Jun 2012 at 10:11

  • Changed state: Fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant