Simple Site Search across Multiple Google App Engine Models

Search is a pretty important thing to have on a website. It is expected on every website and has become so prominent that it is many users main method of navigation on a site.
 
I have been working on a project management-esque application on Google App Engine, www.followtheproject.com , and have continually been putting off providing search functionality. I knew it was necessary from the beginning of development and have had a placeholder search input in the title bar and a search page for results, but no functionality at all, and very little guidance or ideas on how to provide that functionality across my application of many models such as Project, Account, Activity, File, Discussion, Comment etc.

This weekend I finally stumbled on a basic method to provide that functionality quickly and fairly easily. This method basically just “Gets it to work”, it is barebones with nothing fancy like auto completion or sorting. It is a good starting ground.

The starting point I used is found at http://www.billkatz.com/2008/8/A-SearchableModel-for-App-Engine on a blog post titled “A SearchableModel for App Engine” from 2008. I have no idea why it took me so long to find as I was searching for termes like “site search appengine” and “model search appengine” and it took a while to find a proper easy solution that is free.

Searching a Model, Easy

App Engine comes with a search module at google.appengine.ext.search. In the .py file that contains your models include this import:

 from google.appengine.ext import search 

The search module includes a class called search.SearchableModel which is a subclass of db.Model. Change desired searchable models to search.SearchableModel. 

class Activity(search.searchableModel):

This class provides functions that index an instance of the model whenever it is saved (with .put() or .save() ). Instances that are indexed can be searched using simple query syntax.

query = Activity.all().filter(“project =”, project).search(“some query string”)

I filter by project first because all of my searches will be project specific to whichever project is currently active. Optionally these results can be sorted by anything you would like using a .sort() call at the end of the query.

So how to search across multiple models?

Queries (as far as I know) can only be applied to one model at a time. To accomplish site-search over multiple models I simply did a separate a query for each model type I wanted and placed all the results in my values list to be sent to the template. In the template I list on results by model type, for my project management application this works nicely and the results can be somewhat sorted by category this way. I went with an order of importance based on my own judgment, which may change later. The order I used is Activities, Files, Discussions, People, Comments.
 
Next Steps

There is a lot to be desired with the built-in module. No autocomplete, no sorting by importance, and Bill Katz outlines a few more:

• If you try to index too much text, you might run into CPU quota issues on put() as it builds the index.  Background processing, if it gets offered in the future, would address this issue.

• There is a cap of 5000 indexed property values a single entity (one model instance) may have.  This could severely limit your keywords if you have a large number of indexed properties.  ListProperties take a toll.  Text and Blob properties aren't indexed, but your text property will generate keywords in SearchableModel.

• SearchableModel entities currently don't display well in the App Engine data viewers because of the size of __searchable_text_index.

• Multi-word searches seem broken for now because indexing the same property multiple times is not available until the next SDK release.

I believe that last one has been resolved now as the original post is from 2008 and I seem to be able to search multiple words. Try and design your app to sidestep these issues for now, and plan solutions for the future. I am doing that on mine.

Thanks and I hope this is useful to someone. It sure was to me this weekend! Please let me know if you stumble on this and have any input as I will take it into consideration.