My favorites | Sign in
Project Home
READ-ONLY: This project has been archived. For more information see this post.
Search
for
  Advanced search   Search tips   Subscriptions
Issue 3: Models and importers for staStations
1 person starred this issue and may be notified of changes. Back to list
Status:  Fixed
Owner:  snagglepants
Closed:  Mar 2010


 
Project Member Reported by Ryan.Bis...@gmail.com, Mar 10, 2010
Here's my patch. I tried to follow along with t he modifications made to
staStationTypes, as it has a similar setup; the primary key being linked to
another table. I wasn't able to figure it out though......so I'm hoping you
guys can help me out if I did it wrong.

A question for a learning Django/Python programmer....

**BEGIN CODE

if row['stationTypeID']:
	invtype, created =
EVEInventoryType.objects.get_or_create(id=row['stationTypeID'])
	station.station_type, created =
EVEStationType.objects.get_or_create(type=invtype)

if row['corporationID']:
	station.corporation, created =
EVENPCCorporation.objects.get_or_create(id=row['corporationID'])

if row['solarSystemID']:
	station.solar_system = EVESolarSystem.objects.get(id=row['solarSystemID'])

**END CODE

I can see the difference between these three snippets, but why exactly is
this being done? Why do some columns need a different method to import than
others? I guess I am just unable to see any consistencies throughout the
importers, maybe you all can provide some guidance. I may not even be
asking the right question.
eve-db.rb.20100310.diff
5.8 KB   View   Download
Mar 10, 2010
Project Member #1 Ryan.Bis...@gmail.com
The code didn't come out like I wanted it too....I'd edit that out if I could. Here's
a py file containing the code I am referencing.
example.py
425 bytes   Download
Mar 10, 2010
Project Member #2 snagglepants
This is a good question, I think I can provide a reasonable answer for you.

In our first iteration, we didn't have dependency ordering and resolution for
importing. This meant that if I wanted mapDenormalized, I had to manually import all
of the dependencies (quite a few). If I didn't, I'd get errors for the get() queries.
The [sloppy] workaround was to use get_or_create(), making sure most of the model had
blank/nullable fields. If the dependency didn't exist yet, it'd at least create an
empty object that would be filled in with details later.

Our recently added dependency system makes sure that all of the pre-reqs are met
before importing a table (if the user asks for it via the -i command line argument),
and imports them in the "correct" order. We no longer need to use get_or_create() in
many cases, aside from one that will probably remain even with the new system:
Co-dependencies.

Sometimes you'll have two models that require each other. In this case, it's
completely unavoidable. You have to use get_or_create() or they aren't importable.

So the right thing to do would to be to go back and figure out which models don't
really need to be used with get_or_create(), setting fields back to blank/null=False,
etc. However, given the amount of work needing to be done to finish all of the
importers, it hasn't been much of a priority.

I hope that answers your question. If not, let me know. Thanks for the patch, I'll
start poking this today/tonight!
Owner: snagglepants
Labels: -Type-Defect Type-Enhancement
Mar 10, 2010
Project Member #3 snagglepants
As far as the patch goes, it's looking better, made the following changes when
merging in:

* DEPENDENCIES on staStations only need to contain the table names of the models that
are direct dependencies of the model. There's a recursive function that digs up the
complete dependency tree for the importer, so no need to go more than one level deep
in your list. Also, mapDenormalize not a dependency of this importer, removed it from
dependencies.
* Added mapRegions to dependencies.
* Renamed station_type field to 'type' to match the rest of the models. I started
yanking the stuff before underscores for the *_type fields since it's kind of
redundant for the EVEStation model to have a field that re-specifies the name of the
model. EVEStation.type is a lot less repetetive than EVEStation.station_type. Really
minor thing that I haven't specifically documented for developers, but just for
consistency's sake.
* Ditto with station_name. __unicode__ was pointing at self.name, so this would have
thrown an exception.
* Shortened the list_display list in admin.py to only contain used values that are
useful for quick browsing. Also moved 'name' to be the second value in the list.

Really tiny minor points, it looks like you've got a firm grasp on this now. If you
see consistency stuff like I mentioned in some of the older models, make sure you
open up an issue so I can practice what I preach.

We'll get you setup with commit access. Do you have a Google Talk account that we can
converse on periodically?
Mar 10, 2010
Project Member #4 snagglepants
(No comment was entered for this change.)
Status: Fixed

Powered by Google Project Hosting