My favorites | Sign in
Google
                
New issue | Search
for
| Advanced search | Search tips
Issue 423: AppEngine Deploy Blues 2: Missing index
16 people starred this issue and may be notified of changes. Back to list
Status:  Fixed
Owner:  ma...@google.com
Closed:  Nov 2008
Type-Defect
Priority-Medium
Log-1406709


Sign in to add a comment
 
Reported by mcansi, May 31, 2008
This bug seems quite elusive. I'll try to make this description as useful
as possible.

We used appcfg.py to upload an app that worked great with the dev-server.
When we tried to access it from appspot, the request failed with the
following error:

  NeedIndexError: no matching index found
  This query needs this index:
  - kind: Segment
    ancestor: yes
    properties:
    - name: rev

Our index.yaml looked like this:

  indexes:

  - kind: Segment
    ancestor: yes

  - kind: Segment
    ancestor: yes
    properties:
    - name: rev

Checking the "indexes" listing of the datastore online showed that both
indexes were created correctly:

  Entity and Indexes 	Status
  Segment

  Includes ancestors. 	Serving
  rev ↑
  Includes ancestors. 	Serving

The query code we used was:

  'WHERE ANCESTOR IS :tile AND rev >= :rev'

We tried a bunch of things to fix it. Let me first tell you the possible 
causes that we investigated WITHOUT success:

  - Making sure the Segment table had at least one entry didn't help.

  - Adding a version of the index with rev sorted descending didn't help.

  - Adding indexes with the other properties of "Segment" didn't help.

  - Waiting a while for the index to get built didn't help.

  - A query WITHOUT ancestor restrictions worked, but isn't good enough...


Now, to the procedure that DID fix it. Unfortunately we did a couple things
before retrying, so I don't know which of the following steps fixed things:

  We emptied index.yaml and ran the dev-server to rebuild it.
  Result (modulo comments):

    indexes:

    - kind: Segment
      ancestor: yes
      properties:
      - name: rev

  We ran appcfg.py update.

  We ran appcfg.py vacuum_indexes, which reported a Server Error!!!

  We checked the datastore indexes on appengine.google.com and found the 
  expected (vacuum seems to have worked anyways):

    Entity and Indexes  	Status
    Segment

    rev ↑
    Includes ancestors. 	Serving

  We ran appcfg.py update again.

  We ran appcfg.py vacuum_indexes, which did NOT return an error this time.

  Things worked fine after this.

From this it seemed like the extra index with just the ancestor (a superset
of the required index) was the problem, but adding this index back in now
does NOT break the app, i.e. this is probably not what causes it.

Somebody else seems to have had this problem. Logan reported the same issue
(all symptoms match) on Apr 23, 10:14pm in the project group, but I don't
think that post ever made it to the bug tracker.
Comment 1 by mcansi, May 31, 2008
Important addition!!!!!

The App just stopped working again and is giving me the NeedIndexError again! This
coincided with my re-addition of the superset (ancestor only) index going from status
"building" to "serving".

So, I think it might be clear what is going on:

With an index setup like this:

  indexes:

  - kind: Segment
    ancestor: yes

  - kind: Segment
    ancestor: yes
    properties:
    - name: rev

The second index seems to be hidden by the first one, since it is a subset of the
first. Deleting the first index exposes the second one.

After deleting the first index out of index.yaml and running "appcfg.py update", a
call to "appcfg.py vacuum_indexes" outputs this:

  Fetching index definitions diff.
  This index is no longer defined in your index.yaml file.

  ancestor: true
  kind: Segment
  properties: []

  Are you sure you want to delete this index? (N/y/a): y
  Deleting selected index definitions.
  Error 500: --- begin server output ---

  Server Error (500)
  A server error has occured.
  --- end server output ---

After this, the app works again, despite the error.

This problem only seems to be happening if an ancestor is involved.

I hope this helps.

Thanks,

  Markus
Comment 2 by gundlach, Jun 18, 2008
I've repro'd Markus's problem in a simpler context, which may help with tracking it
down.  (BTW Markus, bigsheetofpaper.com is *great* :) )

After completing the App Engine tutorial, I ran appcfg update.  Attempting to view
the root URL [which requests Greeting.all().order('-date').fetch(10)] gave a
NeedIndexError for the following index spec:
 
  - kind: Greeting
    properties:
    - name: date
      direction: desc

This was the content of my index.yaml file, minus comments:

  - kind: Greeting
    properties:
    - name: author
    - name: date
      direction: desc

  - kind: Greeting
    properties:
    - name: date
      direction: desc

Both indexes showed as "serving" on my App dashboard's Index page.

Taking a hint from Markus, I deleted the first entry in my index.yaml file, which was
an unused remnant from earlier code.  After another appcfg update, I no longer
received the error, and only one index is in my dashboard's list.

Interestingly, I then put back the deleted entry and re-updated, and the error does
not reappear.  Both indexes are listed again on the dashboard.

Hope this helps,
Michael [gundlach at you-can-guess-where]
Comment 3 by xiaoke.qiao, Jun 26, 2008
Just wait...
when the Data Store Indexes' status is "Serving", I think everything will be ok.
Hope this is the reason.
Actually, I don't know how it works...
Comment 4 by j...@google.com, Aug 19, 2008
(No comment was entered for this change.)
Status: Accepted
Owner: ma...@google.com
Comment 5 by akojevnikov, Sep 14, 2008
I have the same problem. In my app I have two indices, one is a superset of another (has an 
additional property):

# index A
- kind: UserRelease
  ancestor: yes
  properties:
  - name: artist_id
  - name: order
    direction: desc

# index B
- kind: UserRelease
  ancestor: yes
  properties:
  - name: order
    direction: desc

The query involving index A raises a NeedIndexError exception even though the index is available and 
its status is "Serving". 

If I vacuum index B, index A starts to work. When I put back index B, index A doesn't work again. 

The error occurs only in the production environment, everything works fine locally. It looks like 
index B somehow "hides" index A. This is clearly a bug, please fix it.
Comment 6 by akojevnikov, Sep 14, 2008
Forgot to mention. I didn't get this error until I started using ancestor indices. 
That is, if in the indices above I replace "ancestor: yes" with an additional 
property (e.g. "- name: user_id") and update the entities and queries accordingly, 
then both indices work fine.
Comment 7 by ryanmce, Sep 23, 2008
I am also having this issue, also using ancestor. For me, it doesn't seem to matter
which order the indexes are in index.yaml -- whichever index definition comes first
"blocks" the other index definition, so only one can be used.

This is especially frustrating because I just refactored a bunch of code to make use
of the ancestor relationship. Meh.
Comment 8 by ma...@google.com, Oct 01, 2008
(No comment was entered for this change.)
Labels: Log-1406709
Comment 9 by j...@thinkwave.com, Nov 10, 2008
This one really needs to be addressed...  
Comment 10 by ma...@google.com, Nov 20, 2008
Fixed in 1.1.6
Status: Fixed
Sign in to add a comment