Google Search Appliance software version 6.0
Posted June 2009
This guide provides Java programming information about how to use the Google Data API to create, retrieve, update, and delete information for one or more Google Search Appliance devices.
Use the information in this guide to create or learn about coding Java applications that programmatically set the administrative functions for the Admin Console of a search appliance.
The audience for this guide is Java programmers who want to programmatically configure a Google Search Appliance. Before using the Google Search Appliance Administrative API, read Getting Started to download and configure required software.
Note: This guide uses data source feed to indicate the search appliance's Feeds API (described in the Feeds Protocol Developer's Guide).
The organization of this guide corresponds to the navigation features of the Google Search Appliance:
The google-enterprise-gdata-api open source site provides ZIP files that contain the Java client library, source code and some sample applications for your reference. The information in this section helps you understand how to write your own applications based on the client library and how to run the provided open source sample applications. You can also use the sample applications as models for your own development.
Before starting, you need the following software:
After you download the software and acquire search appliance credentials, get started as follows:
gdata/java folder. The client library JAR files are present in the lib folder and the sample applications are present in the sample folder. Sample applications are located in the gdata/java/sample folder.
To see if you've installed the required software correctly, open a command prompt and change directory to the gdata/java folder. Enter the following command:
ant -f build-samples.xml sample.dashboard.run
This command displays build output messages and opens an example dashboard that demonstrates the features of the Google Search Appliance Administrative API. You can add your search appliance configuration information to the dashboard and see the dashboard running.
To find out how to run each sample, go to gdata/java/build-samples folder and view the build file for each sample, like dashboard.xml, commandline.xml, and aclclient.xml. Look for the samples run comment. Another example of how to run the commandline sample application, enter the following command in the gdata/java folder:
ant -f build-samples.xml -Dargs="retrieve --hostname=gsa_hostname --username=gsa_user --password=gsa_passwd --protocol=http --port=gsa_port config crawlURLs" sample.commandline.run
Substitute these parameters:
For example:
ant -f build-samples.xml -Dargs='retrieve --hostname=gsa42.corp.example.com --username=admin --password=greenparrot --protocol=http --port=8000 config crawlURLs' sample.commandline.run
This command displays the crawl URLs for a search appliance.
The commandline sample application requires that command line arguments be passed to Ant by using the -Dargs option in the commandline.xml file.
You can build your own applications using the client library.
Copy the following client library JAR files from the gdata/java/lib folder to your development folder and add the files to your classpath environmental system variable:
gdata-core-1.0.jar gdata-gsa-1.0.jar gdata-client-1.0.jar gdata-client-meta-1.0.jar gdata-gsa-meta-1.0.jar You can then use the JAR files in your application.
Before making API calls with the Java client library, you must construct a new
GsaClient object.
In the constructor that follows, replace gsaAddr with the host name of a search applaiance, gsaPort with port 8000, myUserId and myPassword with your Admin Console authentication information:
import com.google.enterprise.apis.client.GsaClient;
GsaClient myClient = new GsaClient("gsaAddr", gsaPort, "myUserId", "myPassword");
For example:
import com.google.enterprise.apis.client.GsaClient;
GsaClient myClient = new GsaClient("gsa42.corp.example.com", 8000, "admin", "greenparrot");
The sections that follow describe how to configure the Crawling and Indexing features of the Admin Console:
Retrieve and update crawl URL patterns on a search appliance using the
crawlURLs entry of the config feed.
| Property | Description | doNotCrawlURLs |
Do Not Crawl URLs with the following patterns, separate multiple URL patterns with new line delimiters. |
|---|---|
followURLs |
Follow and crawl only URLs with the following URL patterns, separate multiple URL patterns with new line delimiters. |
startURLs |
Start crawling from the following URLs, separate multiple URL patterns with new line delimiters. |
Retrieve information about the URL patterns that the search appliance is crawling as follows:
// Send a request and print the response
GsaEntry myEntry = myClient.getEntry("config", "crawlURLs");
System.out.println("Start URLs: " + myEntry.getGsaContent("startURLs"));
System.out.println("Follow URLs: " + myEntry.getGsaContent("followURLs"));
System.out.println("Do Not Crawl URLs: " + myEntry.getGsaContent("doNotCrawlURLs"));
Update the crawl URL settings on a
search appliance as follows -- in the example that follows, example.com is requested for crawling, and spreadsheets are requested to not be crawled.
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId(entryUrl);
// Add a property for adding crawl URLs to updateEntry
updateEntry.addGsaContent("startURLs", "http://www.example.com/");
updateEntry.addGsaContent("followURLs", "http://www.example.com/");
updateEntry.addGsaContent("doNotCrawlURLs", ".xls$");
// Send the request
myClient.updateEntry("config", "crawlURLs", updateEntry);
Retrieve, delete, and destroy data source feed information
for the search appliance using the feed feed.
The following parameters let you search for a string and
retrieve source statements.
| Parameter | Description | query |
The query string. When used to retrieve all feed information, the query parameter is overloaded to mean the feedDataStore. When getting information about a single feed, the parameter is a query. Each log statement contains a query string to be retrieved. |
|---|---|
startLine |
The starting log statement to retrieve. The default value is line 1. |
maxLines |
The maximum number of log statements to retrieve. The default value is 50 statements. |
Use the following properties to view data source feed records and content.
| Property | Description |
|---|---|
errorRecords |
The number of documents that had errors and were not added to the data source feed. |
feedDataSource |
The name of the data source. |
feedState |
Feed state: ACCEPTED:0, IN_PROGRESS:1, COMPLETED:2,
COMPLETED_WITH_ERROR:3, FAILED_IN_ERROR:4 |
feedTime |
The system's time stamp at the start of each stage, in milliseconds. |
feedType |
Feed type: FULL_FEED:0, INCREMENTAL:1, DELETED:2, METADATA_AND_URL:3 |
fromLine |
The starting line of a log. |
logContent |
The log's content. |
successRecords |
The number of documents in the search appliance index (the number of documents that were successfully indexed). |
toLine |
The ending line of a log. |
totalLines |
Total lines in a log. |
updateMethod |
Indicate to delete a data source feed. This value can only be delete. |
Note: You can only get information about data source feeds, and delete or destroy data source feeds. Inserting new data source feeds is not provided by this API. You can create new feeds using the Admin Console user interface.
Retrieve all data source feed information from a search appliance using the feed feed:
// Send a request and print the response
Map<String, String> queries = new HashMap<String, String>();
queries.put("query",feedDataSource);
GsaFeed myFeed = myClient.queryFeed("feed", queries);
for(GsaEntry myEntry : myFeed.getEntries()) {
//get information on each myEntry
System.out.println("Feed Name: " + myEntry.getGsaContent("entryID"));
System.out.println("Feed Data Source: " + myEntry.getGsaContent("feedDataSource"));
System.out.println("Feed Type: " + myEntry.getGsaContent("feedType"));
System.out.println("Feed State: " + myEntry.getGsaContent("feedState"));
System.out.println("Feed Time: " + myEntry.getGsaContent("feedTime"));
System.out.println("Error Records: " + myEntry.getGsaContent("errorRecords"));
System.out.println("Success Records: " + myEntry.getGsaContent("successRecords"));
System.out.println("Log Content: " + myEntry.getGsaContent("logContent"));
}
You can get the individual feed information from a search appliance as follows:
GsaEntry myEntry = myClient.getEntry("feed", FEED_NAME);
System.out.println("Feed Data Source: " + myEntry.getGsaContent("feedDataSource"));
System.out.println("Feed Type: " + myEntry.getGsaContent("feedType"));
System.out.println("Feed State: " + myEntry.getGsaContent("feedState"));
System.out.println("Feed Time: " + myEntry.getGsaContent("feedTime"));
System.out.println("Error Records: " + myEntry.getGsaContent("errorRecords"));
System.out.println("Success Records: " + myEntry.getGsaContent("successRecords"));
System.out.println("Log Content: " + myEntry.getGsaContent("logContent"));
Note: A feed log of all data source feeds can be retrieved only by getting individual feeds.
Delete a data source feed to remove all documents for a feed from the index on the search appliance. In the Admin Console, deleted feed sources are marked with Delete.
Delete data source feeds as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
// Add a property to updateEntry
updateEntry.addGsaContent("updateMethod", "delete");
myClient.updateEntry("feed", FEED_NAME, updateEntry);
Note: Delete data source feeds only of the
type
FULL_FEED or incremental. After deleting a data source feed, the deleted feed still
exists, and the feed type changes to DELETED.
After deleting a data source feed, you can destroy the feed so that the feed no longer exists on the search appliance:
myClient.deleteEntry("feed", FEED_NAME);
Retrieve and update trusted feed IP addresses
using the feedTrustedIP
entry of the config feed.
Retrieve the IP addresses of trusted feeds using the trustedIPs property.
| Property | Description |
|---|---|
trustedIPs |
Trusted IP addresses.
This value is a list of one or more IP addresses.
Specify all to indicate that the search appliance
trust all IP addresses. If the value is a list of IP addresses,
separate each IP address with white space.
|
Retrieve the trusted feed IP addresses as follows:
// Send the request and print the response
GsaEntry myEntry = myClient.getEntry("config ", "feedTrustedIP");
System.out.println("Trusted IP Addresses: " + myEntry.getGsaContent("trustedIPs"));
Update trusted feed IP addresses as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId(entryUrl);
// Add a property for the feeds trusted IP addresses
// to updateEntry
updateEntry.addGsaContent("trustedIPs", "127.0.0.1");
// Send the request
myClient.updateEntry("config", "feedTrustedIP", updateEntry);
Retrieve and update the crawl schedule for a search appliance.
| Property | Description |
|---|---|
crawlSchedule |
The crawl schedule is only available in scheduled crawl mode.
The value of crawlSchedule has
the format:
Day,Time,DurationWhere:
|
isScheduledCrawl |
Set to 1 if the search appliance is in scheduled crawl mode or set to 0 if the search appliance is in continuous crawl mode. |
Retrieve the crawl mode and get the crawl schedule as follows:
GsaEntry myEntry = myClient.getEntry("config", "crawlSchedule");
System.out.println("Is Scheduled Crawl: " + myEntry.getGsaContent("isScheduledCrawl"));
System.out.println("Crawl Schedule: " + myEntry.getGsaContent("crawlSchedule"));
Update the crawl schedule or crawl mode as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId(entryUrl);
// Add a property to updateEntry
updateEntry.addGsaContent("isScheduledCrawl", "1");
updateEntry.addGsaContent("crawlSchedule", "0,0300,360\n2,0000,1200");
// Send the request
myClient.updateEntry("config", "crawlSchedule", updateEntry);
Create, retrieve, update, and delete crawler access rules for a search appliance.
Crawler access rules instruct the crawler how to authenticate when crawling the protected content.
| Property | Description |
|---|---|
domain |
Windows domain for NTLM, or empty for HTTP Basic authorization. |
isPublic |
Indicates whether to allow users to view results of both
the public content (normally available to everyone)
and the secure (confidential) content. The value can
be 1 to enable users to view content as public, or 0 to require users to authenticate to view secure content. |
order |
Indicates that the crawler access rules are sequential.
The order indicates the sequence. The order is an integer
value starting from 1. |
password |
Password for authentication. |
urlPattern |
URL pattern that matches the protected files. |
username |
User name for authentication. |
Insert a new crawler access rule as follows:
// Create an entry to hold properties to insert
GsaEntry insertEntry = new GsaEntry();
// Add properties to insertEntry
insertEntry.addGsaContent("entryID", "#URL pattern for the new crawler access rule");
insertEntry.addGsaContent("domain", "domainone");
insertEntry.addGsaContent("isPublic", "1");
insertEntry.addGsaContent("username", "username");
insertEntry.addGsaContent("password", "password");
// Send the request
myClient.insertEntry("crawlAccessNTLM", insertEntry);
Retrieve a list of crawler access rules as follows:
// Send a request and print the response
GsaFeed myFeed = myClient.getFeed("crawlAccessNTLM");
for(GsaEntry myEntry : myFeed.getEntries()) {
System.out.println("URL Pattern: " + myEntry.getGsaContent("urlPattern"));
System.out.println("User Name: " + myEntry.getGsaContent("username"));
System.out.println("Order: " + myEntry.getGsaContent("order"));
System.out.println("Domain: " + myEntry.getGsaContent("domain"));
System.out.println("Is Public: " + myEntry.getGsaContent("isPublic"));
}
Retrieve an individual crawler access rule as follows:
// Send the request and print the response
GsaEntry myEntry = myClient.getEntry("crawlAccessNTLM", "urlPattern");
System.out.println("URL Pattern: " + myEntry.getGsaContent("urlPattern"));
System.out.println("User Name: " + myEntry.getGsaContent("username"));
System.out.println("Order: " + myEntry.getGsaContent("order"));
System.out.println("Domain: " + myEntry.getGsaContent("domain"));
System.out.println("Is Public: " + myEntry.getGsaContent("isPublic"));
Note:The password property doesn't appear when retrieving crawler access rules.
Update a crawler access rule as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId(entryUrl);
// Add properties to updateEntry
updateEntry.addGsaContent("urlPattern", "#new URL pattern");
updateEntry.addGsaContent("domain", "newdomain");
updateEntry.addGsaContent("isPublic", "0");
updateEntry.addGsaContent("order", "2");
updateEntry.addGsaContent("username", "newuser");
updateEntry.addGsaContent("password", "newpass");
// Send the request
myClient.updateEntry("crawlAccessNTLM", "urlPattern", updateEntry);
Delete a crawler access rule as follows:
myClient.deleteEntry("crawlAccessNTLM", "urlPattern");
Retrieve and update host load schedule information from the search appliance using the
hostLoad entry of the config feed.
| Property | Description |
|---|---|
defaultHostLoad |
The default web server host load, a float value. This value measures the relative load on the search appliance based on the number of connections that a search appliance can handle. You can set the defaultHostLoad to a decimal value starting at 0, where 0 indicates to not crawl the content from the number of URL patterns that you specify. Any decimal value of 1 or more sets the average number of connections per minute. A decimal value under 1 sets the percentage of time during which the search appliance opens connections. For more information, see the online help for Crawl and Index > Host Load Schedule. |
exceptionHostLoad |
Exceptions to the default web server host load are listed as multiple lines of text where each line is in the format:
hostName startTime endTime loadFactorWhere:
|
maxURLs |
Maximum number of URLs to crawl, an integer value. |
Retrieve information about the host load schedule from a search appliance as follows:
// Send the request and print the response
myEntry = myClient.getEntry("config", "hostLoad");
System.out.println("defaultHostLoad: " + myEntry.getGsaContent("defaultHostLoad"));
System.out.println("exceptionHostLoad: " + myEntry.getGsaContent("exceptionHostLoad"));
System.out.println("maxURLs: " + myEntry.getGsaContent("maxURLs"));
Update the host load schedule setting in a search appliance as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId(entryUrl);
// Add a property for the Host Load Schedule to updateEntry
updateEntry.addGsaContent("defaultHostLoad", "2.4");
updateEntry.addGsaContent("exceptionHostLoad", "* 3 5 1.2 \n www.example.com 1 6 3.6");
updateEntry.addGsaContent("maxURLs", "3000");
// Send the request
myClient.updateEntry("config", "hostLoad", updateEntry);
Increase or decrease the crawling frequency by specifying URL patterns.
| Property | Description |
|---|---|
archiveURLs |
URL patterns for pages that contain archival or rarely changing content. |
forceURLs |
URL patterns for pages to recrawl regardless of their response to If-Modified-Since request headers. |
frequentURLs |
URL patterns for pages that change often (typically more than once a day). |
Retrieve the freshness tuning configuration as follows:
GsaEntry myEntry = myClient.getEntry("config", "freshness");
System.out.println("Archive URLs: " + myEntry.getGsaContent("archiveURLs"));
System.out.println("Frequent URLs: " + myEntry.getGsaContent("frequentURLs"));
System.out.println("Force URLs: " + myEntry.getGsaContent("forceURLs"));
Update the settings for freshness tuning as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId(entryUrl);
// Add a property for updateEntry
updateEntry.addGsaContent("archiveURLs", "http://good/");
updateEntry.addGsaContent("frequentURLs", "http://frequent/");
updateEntry.addGsaContent("forceURLs", "http://force/");
// Send the request
myClient.updateEntry("config", "freshness", updateEntry);
If you discover that a set of URLs that you want to have in the search index are not being crawled you can inject a URL pattern into the queue of URLs that the search appliance is crawling. URLs may not appear in the index because changes were made to the web pages, or because a temporary error or misconfiguration was present when the crawler last tried to crawl the URL.
| Property | Description | recrawlURLs |
URL patterns to be recrawled. |
|---|
Recrawl URL patterns as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
// Add a property to updateEntry
updateEntry.addGsaContent("recrawlURLs", "http://recrawl/page.html");
// Send the request
myClient.updateEntry("command", "recrawlNow", updateEntry);
Retrieve, update, create, or delete the collections of documents on the search appliance.
| Property | Description |
|---|---|
collectionName |
The name of the collection to create, which is only required when creating a new collection. |
doNotCrawlURLs |
The URL patterns of content that you want to exclude from this collection. |
followURLs |
The URL patterns of content that you want to include in this collection. |
importData |
Indicates that the collection settings exported from the Admin Console are only required when creating a new collection from an import. |
insertMethod |
The method of creating a new method, which is only required when creating a new
collection.
Possible values: default, customize, import. |
Create a new collection as follows:
// Create an entry to hold properties to insert
GsaEntry insertEntry = new GsaEntry();
// Add a property to insertEntry
insertEntry.addGsaContent("collectionName", "new_collection");
insertEntry.addGsaContent("insertMethod", "default");
// Send the request
myClient.insertEntry("collection", insertEntry);
Create a new collection with a default setting as follows:
insertEntry.addGsaContent("insertMethod", "default");
Specify the settings for a new collection as follows:
// Add property for insertEntry
insertEntry.addGsaContent("collectionName", "new_collection");
insertEntry.addGsaContent("insertMethod", "customize");
insertEntry.addGsaContent("followURLs", "#url to follow");
insertEntry.addGsaContent("doNotCrawlURLs", "#url to not follow");
Retrieve a list of collections as follows:
// Send the request and print the response
GsaFeed myFeed = myClient.getFeed("collection");
for(GsaEntry myEntry : myFeed.getEntries()) {
System.out.println("Follow URLs: " + myEntry.getGsaContent("followURLs"));
System.out.println("Do Not Crawl URLs: " + myEntry.getGsaContent("doNotCrawlURLs"));
}
Retrieve the attributes of a single collection as follows:
// Send the request and print the response
GsaEntry myEntry = myClient.getEntry("collection", "default_collection");
System.out.println("Follow URLs: " + myEntry.getGsaContent("followURLs"));
System.out.println("Do Not Crawl URLs: " + myEntry.getGsaContent("doNotCrawlURLs"));
Update the attributes of a collection as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId(entryUrl);
// Add properties to updateEntry
updateEntry.addGsaContent("followURLs", "http://good/");
updateEntry.addGsaContent("doNotCrawlURLs", "http://frequent/");
// Send the request
myClient.updateEntry("collection", "default_collection", updateEntry);
Delete a collection as follows:
myClient.deleteEntry("collection", "new_collection");
The sections that follow describe how to configure the Serving features of the Admin Console:
Retrieve, update, insert, or delete front ends to remove URLs or a relative OneBox for the search appliance using the frontend feed. Retrieve a front end using the following properties.
| Property | Description |
|---|---|
frontendOnebox |
OneBox Modules that are relative to this front end. This value is a comma-separated list of OneBox names. The OneBox modules are triggered for this front end in the order that you specify. |
removeUrls |
Remove URLs that are relative to this front end. These are URL patterns that you do not want to appear in the search results for this front end. |
Retrieve all the front end information for a search appliance as follows:
// Send a request and print the response
GsaFeed myFeed = myClient.getFeed("frontend");
for(GsaEntry myEntry : myFeed.getEntries()) {
//get information for each myEntry
}
Get information about a front end as follows:
// Send a request and print the response
GsaEntry myEntry = myClient.getEntry("frontend", FRONTEND_NAME);
System.out.println("Front End OneBox: " + myEntry.getGsaContent("frontendOnebox"));
System.out.println("Remove URLs: " + myEntry.getGsaContent("removeUrls"));
Update the URLs to remove from the search results, and update a OneBox module in a front end as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId(entryUrl);
// Add properties to updateEntry
updateEntry.addGsaContent("frontendOnebox", "oneboxtwo");
updateEntry.addGsaContent("removeUrls", "http://www.example.com/");
// Send the request
myClient.updateEntry("frontend", FRONTEND_NAME, updateEntry);
Insert a front end and remove a URL from the search results as follows:
// Create an entry to hold properties to insert
GsaEntry insertEntry = new GsaEntry();
insertEntry.setId(entryUrl);
// Add properties to insertEntry
insertEntry.addGsaContent("entryID", FRONTEND_NAME);
insertEntry.addGsaContent("removeUrls", "http://www.example3.com/");
// Send the request
myClient.insertEntry("frontend", insertEntry);
Delete a front end from the search appliance as follows:
myClient.deleteEntry("frontend", FRONTEND_NAME);
Retrieve and update the XSLT template and other
output format-related properties for each language
of each front end using the frontend
entry of the outputFormat feed.
| Parameter | Description |
|---|---|
language |
Specify a language for the output format properties that you want to retrieve. Each
front end can contain multiple languages, and each language has its own
output format properties.
Each front end + language can have its own XSLT stylesheet. The language parameter enables you to retrieve and update a stylesheet for a front end associated with a language. Administrators who use the Admin Console set the language in their browser and the Admin Console then displays in that language (if the Admin Console has been translated into that language). Hence the language parameter for the outputFormat feed is limited to the values to which the Admin Console is translated. |
Use the following properties to access the XSLT template information.
| Property | Description |
|---|---|
isDefaultLanguage |
Set to 1 if the designated language is the default language for the specified
front end, set to 0 if not. |
isStyleSheetEdited |
Set to 0 if the style sheet is the default stylesheet that has not been previously edited.
Set to 1 if the style sheet has been edited. |
language |
When retrieving, the language is determined by the language
that is specified by the query parameter.
When updating, the language is passed as an entry property to
specify the language of the output stylesheet. |
restoreDefaultFormat |
Set to 1 to restore custom-edited XSLT code back to the default values. A 0 value
has no effect. |
styleSheetContent |
The output format XSLT code content. |
Note: For the update action,
the restoreDefaultFormat content is mutually exclusive
from the styleSheetContent. For each update action,
you should either restore the
output format XSLT stylesheet back to the original default values, set the
XSLT stylesheet to a custom format, or neither, but not both.
Retrieve the output format stylesheet information from a search appliance as follows:
Map<String, String> queryMap = new HashMap<String, String>();
// Initialize the query map
queryMap.put("language", "en");
GsaEntry myEntry = myClient.queryEntry("outputFormat", "default_frontend", queryMap);
System.out.println("Language: " + myEntry.getGsaContent("language"));
System.out.println("Default Language: " + myEntry.getGsaContent("isDefaultLanguage"));
System.out.println("Is the Style Sheet Edited: " + myEntry.getGsaContent("isStyleSheetEdited"));
System.out.println("XSLT Stylesheet Content: " + myEntry.getGsaContent("styleSheetContent"));
Update the output format stylesheet information in a search appliance as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId("default_frontend");
// The language parameter is passed as part of
// the entry because we cannot use a query parameter
updateEntry.addGsaContent("language", "en");
// Indicate that the XSLT stylesheet has default values
updateEntry.addGsaContent("isDefaultLanguage, "1");
// Add this line to update the style sheet content
updateEntry.addGsaContent("styleSheetContent", "{my new style sheet XSLT}");
// Or add this line to restore the stylesheet content to
// the default, which is mutually exclusive from the previous line
updateEntry.addGsaContent("restoreDefaultFormat", "1");
// Send the request and print the response
myClient.updateEntry("outputFormat", "default_frontend", updateEntry);
System.out.println("Output Format: " + updateEntry.getGsaContent("outputFormat"));
System.out.println("Default Front End: " + updateEntry.getGsaContent("default_frontend"));
KeyMatch settings let you promote specific web pages on your site. The following parameters let you find KeyMatches by search, and specify a starting line number and the number of lines to access.
| Parameter | Description |
|---|---|
query |
A query string to perform a full text search. For example, if you
set computer in the query parameter,
then you get all KeyMatch settings
that contain the word computer. |
startLine |
The starting line number of a result, default value is 0. |
maxLines |
The number of result lines in a response, default value is 50 lines. |
Use the following properties to set KeyMatch configurations.
| Property | Description |
|---|---|
| line_number | The line_number of the KeyMatch configuration rule. |
newLines |
The new KeyMatch configuration to update. This value may include
multiple KeyMatch statements. The line delimiter is \n. |
numLines |
The number of total result lines. |
originalLines |
The original KeyMatch configurations to change. The value may include
multiple
KeyMatch statements. The line delimiter is \n. |
startLine |
The starting line number of a KeyMatch configuration to change. The minimum value is 0. |
updateMethod |
The method to change a KeyMatch configuration. Possible values are:
|
Note: The format for a KeyMatch configuration rule is as follows:
Search_Terms,KeyMatch_Type,URL,Title
KeyMatch_Type can be: KeywordMatch, PhraseMatch, or ExactMatch.
Search_Term and URL fields
cannot be empty. The KeyMatch configuration conforms to the CSV format, which uses commas to separate values.
Retrieve KeyMatch settings as follows:
index Map<String, String> queryMap = new HashMap<String, String>();
// Initialize the query
map queryMap.put("query", "myQuery");
queryMap.put("startLine", "0");
queryMap.put("maxLines", "50");
// Send the request and print the response
GsaEntry myEntry = myClient.getEntry("keymatch", "myFrontend", queryMap);
Iterator i = myEntry.getAllGsaContents().entrySet().iterator();
while (i.hasNext()) {
Map.Entry me = (Map.Entry)i.next();
if (me.getKey().matches("\\d+")) {
System.out.println("The lines for " + my.getKey() + " are: " + my.getValue());
}
}
System.out.println("The number of lines are: " + myEntry.getGsaContent("numLines"));
The following example appends KeyMatch settings:
// Create an entry to hold properties to append
GsaEntry appendEntry = new GsaEntry();
appendEntry.setId("myFrontend");
appendEntry.addGsaContent("updateMethod", "append");
// Prepare new content
String newLines =
"image,KeywordMatch,http://images.google.com/,Google Image Search\n" +
"video,KeywordMatch,http://www.youtube.com/,Youtube\n" +
"rss feed,PhraseMatch,http://www.google.com/reader,Reader";
appendEntry.addGsaContent("newLines", newLines);
// Send the request to the search appliance
myClient.updateEntry("keymatch", "myFrontend", appendEntry);
The following example updates KeyMatch settings:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId("myFrontend");
updateEntry.addGsaContent("updateMethod", "update");
// Set the start line number
updateEntry.addGsaContent("startLine", 0);
// Provide the original content
String originalLines =
"image,KeywordMatch,http://images.google.com/,Google Image Search\n" +
"video,KeywordMatch,http://www.youtube.com/,Youtube\n" +
"rss feed,PhraseMatch,http://www.google.com/reader,Reader";
updateEntry.addGsaContent("originalLines", originalLines);
// Prepare new content
String newLines =
",,,\n" +
"video,KeywordMatch,http://video.google.com/,Video Search\n" +
"rss feed,PhraseMatch,http://www.example.com/,RSS example";
updateEntry.addGsaContent("newLines", newLines);
// Send the request to the search appliance
myClient.updateEntry("keymatch", "myFrontend", updateEntry);
Note: Delete a setting by
changing the statement to three commas (,,,).
The following example replaces KeyMatch settings:
// Create an entry to hold properties to replace
GsaEntry replaceEntry = new GsaEntry();
replaceEntry.setId("myFrontend");
replaceEntry.addGsaContent("updateMethod", "replace");
// Prepare new content
String newLines =
"image,KeywordMatch,http://images.google.com/,Google Image Search\n" +
"video,KeywordMatch,http://www.youtube.com/,Youtube\n" +
"rss feed,PhraseMatch,http://www.google.com/reader,Reader";
replaceEntry.addGsaContent("newLines", newLines);
// Send the request to the search appliance
myClient.updateEntry("keymatch", "myFrontend", replaceEntry);
Use related queries to associate alternative words or phrases with specified search terms. Related queries are also known as synonyms. Use the following parameters to search for a query and access lines from a starting line number and for a maximum amount of lines.
| Parameter | Description |
|---|---|
query |
A query string to perform a full-text search. |
startLine |
The starting line number of the result, the default value is to start at line 0. |
maxLines |
The number of result lines in the response, the default value is 50 lines. |
Use the following properties to access related queries.
| Property | Description |
|---|---|
| line number | The line number of the related query configuration rule (in all the rules). |
newLines |
The new related query configuration to add. This value may include
multiple lines of related query statements. The delimiter is \n. |
numLines |
The total number of result lines. |
originalLines |
The original related query configuration to change. This value may include
multiple lines of related query statements. The delimiter is \n. |
startLine |
The starting line number of the related query configuration to change. The minimum value is 0. |
updateMethod |
The method to use to change related query configurations. Possible values are:
|
Note: A related query configuration rule is in the following format:
Search_Terms,Related_Queries
The Search_Terms and the Related_Queries values
cannot be empty. The related query configuration rules conform to the CSV format,
which uses commas to separate values.
Retrieve related queries as follows:
Map<String, String> queryMap = new HashMap<String, String>();
// Initialize the query map
queryMap.put("query", "myQuery");
queryMap.put("startLine", "0");
queryMap.put("maxLines", "50");
// Send the request and print the response
GsaEntry myEntry = myClient.getEntry("synonym", "myFrontend", queryMap);
Iterator i = myEntry.getAllGsaContents().entrySet().iterator();
while (i.hasNext()) {
Map.Entry me = (Map.Entry)i.next();
if (me.getKey().matches("\\d+"))
{
System.out.println("The line " + my.getKey() + " is: " + my.getValue());
}
}
The following example appends related queries:
// Create an entry to hold properties to append
GsaEntry appendEntry = new GsaEntry();
appendEntry.setId("myFrontend");
appendEntry.addGsaContent("updateMethod", "append");
// Prepare new content
String newLines = "airplane,aircraft\n" + "google,googol\n" + "stock,security";
appendEntry.addGsaContent("newLines", newLines);
// Send the request to the search appliance
myClient.updateEntry("synonym", "myFrontend", appendEntry);
The following example updates related queries:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId("myFrontend");
updateEntry.addGsaContent("updateMethod", "update");
// Set the starting line number
updateEntry.addGsaContent("startLine", 0);
// Provide the original content
String originalLines = "airplane,aircraft\ngoogle,googol";
updateEntry.addGsaContent("originalLines", originalLines);
// Prepare new content
String newLines = "airplane,helicopter\n,";
updateEntry.addGsaContent("newLines", newLines);
// Send the request to the search appliance
myClient.updateEntry("synonym", "myFrontend", updateEntry);
Note: Delete a setting by changing the statement to a comma (,) value.
The following example replaces related queries:
// Create an entry to hold properties to replace
GsaEntry replaceEntry = new GsaEntry();
replaceEntry.setId("myFrontend");
replaceEntry.addGsaContent("updateMethod", "replace");
// Prepare new content
String newLines = "airplane,aircraft\n" + "google,googol\n" + "stock,security";
replaceEntry.addGsaContent("newLines", newLines);
// Send the request to the search appliance
myClient.updateEntry("synonym", "myFrontend", replaceEntry);
Retrieve and update OneBox module settings for the search appliance
using the config feed.
Retrieve OneBox information for a search appliance as follows:
// Send the request and print the response
GsaEntry myEntry = myClient.getEntry("config", "oneboxSetting");
System.out.println("Max Results: " + myEntry.getGsaContent("maxResults"));
System.out.println("Timeout: " + myEntry.getGsaContent("timeout"));
The properties for retrieving a OneBox are as follows:
| Property | Description |
|---|---|
maxResults |
Maximum number of results. |
timeout |
OneBox response timeout in milliseconds. |
Update the OneBox settings for a search appliance as follows -- in this example three results are requested and the timeout is set to 2000 milliseconds.
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId(entryUrl);
// Add properties for the OneBox settings to updateEntry
updateEntry.addGsaContent("maxResults", "3");
updateEntry.addGsaContent("timeout", "2000");
// Send the request
myClient.updateEntry("config", "oneboxSetting", updateEntry);
Retrieve and delete OneBox modules from the search appliance using the onebox feed.
| Property | Description |
|---|---|
logContent |
The logs content of OneBox logs. |
Note: Inserting a new OneBox module, updating an existing OneBox module, and retrieving a detailed configuration of a OneBox module are not supported by this API.
Retrieve information about all the OneBox modules from a search appliance
using the onebox feed:
// Send the request and print the response
GsaFeed myFeed = myClient.getFeed("onebox");
for(GsaEntry myEntry : myFeed.getEntries()) {
// Get information on each myEntry
System.out.println("OneBox Name: " + myEntry.getGsaContent("entryID"));
}
Note: Because detailed information about the OneBox configuration not supported by this API, the onebox feed provides only OneBox module names.
Retrieve an individual OneBox module's log information from a search appliance as follows:
// Send the request and print the response
GsaEntry myEntry = myClient.getEntry("onebox",ONEBOX_NAME);
System.out.println("OneBox Log: " + myEntry.getGsaContent("logContent"));
Note: You can only retrieve OneBox log entries individually.
Delete a OneBox module from a search appliance as follows:
myClient.deleteEntry("onebox", ONEBOX_NAME);
The sections that follow describe how to configure the Status and Reports features of the Admin Console:
Check crawl status, pause the crawl, or resume the crawl.
| Property | Description |
|---|---|
pauseCrawl |
Indicates: 1 if crawl is paused, 0 if crawling is occurring.
|
Retrieve the status of crawl as follows:
// Send the request and print the response
GsaEntry myEntry = myClient.getEntry("command", "pauseCrawl");
System.out.println("Pause Crawl: " + myEntry.getGsaContent("pauseCrawl"));
Pause or resume a crawl as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
// Add properties to updateEntry
updateEntry.addGsaContent("pauseCrawl", "0");
updateEntry.addGsaContent("doNotCrawlURLs", "http://frequent/");
// Send the request
myClient.updateEntry("command", "pauseCrawl", updateEntry);
Retrieve the serving status for the search appliance using the servingStatus entry
of the status feed.
| Property | Description |
|---|---|
queriesPerMinute
|
Average queries per minute served recently on the search appliance. |
Retrieve the current search appliance serving status as follows:
GsaEntry myEntry = myClient.getEntry("status", "servingStatus");
System.out.println("Queries Per Minute: " + myEntry.getGsaContent("queriesPerMinute"));
System status for the search appliance can be retrieved through
the systemStatus entry of the status feed.
| Property | Description |
|---|---|
cpuTemperature |
Temperature of the CPU.
Set to 0 if okay, 1 if caution, 2 if critical. |
diskCapacity |
Remaining disk capacity of a search appliance.
Set to 0 if okay, 1 if caution, 2 if critical. |
machineHealth |
Health of the motherboard.
Set to 0 if okay, 1 if caution, 2 if critical. |
overallHealth |
Overall health of the a search appliance.
Set to 0 if okay, 1 if caution, 2 if critical. |
raidHealth |
Health of the RAID array.
Set to 0 if okay, 1 if caution, 2 if critical. |
Note: Some health properties may not exist in certain versions of the search appliance.
Retrieve a the current search appliance system status as follows:
GsaEntry myEntry = myClient.getEntry("status", "systemStatus");
System.out.println("Overall Health: " + myEntry.getGsaContent("overallHealth"));
System.out.println("Disk Capacity: " + myEntry.getGsaContent("diskCapacity"));
System.out.println("RAID Health: " + myEntry.getGsaContent("raidHealth"));
System.out.println("CPU Temperature: " + myEntry.getGsaContent("cpuTemperature"));
System.out.println("Machine Health: " + myEntry.getGsaContent("machineHealth"));
Retrieve document status using the properties that follow.
| Property | Description |
|---|---|
crawledURLsToday |
The number of documents crawled since yesterday. (Note that the time pertains to the search appliance, not the computer sending this request.) |
crawlPagePerSecond |
Current crawling rate. |
errorURLsToday |
The document errors since yesterday. |
filteredBytes |
The document bytes that have been filtered. |
foundURLs |
The number of URLs found that match crawl patterns. |
servedURLs |
The total number of documents that have been served. |
Retrieve the document status as follows:
GsaEntry myEntry = myClient.getEntry("status", "documentStatus");
System.out.println("Served URLs: " + myEntry.getGsaContent("servedURLs"));
System.out.println("Crawled Pages Per Second: " + myEntry.getGsaContent("crawlPagePerSecond"));
System.out.println("Crawled URLs Today: " + myEntry.getGsaContent("crawledURLsToday"));
System.out.println("Found URLs: " + myEntry.getGsaContent("foundURLs"));
System.out.println("Filtered Bytes: " + myEntry.getGsaContent("filteredBytes"));
System.out.println("Error URLs Today: " + myEntry.getGsaContent("errorURLsToday"));
List crawled documents and retrieve the status of documents in a search appliance using the diagnostics feed.
The following tables list the document status values.
Note: Use the all to indicate any status value.
Successful Crawl:
| Success Value | Crawl Description |
|---|---|
| 1 | Crawled from remote server |
| 2 | Crawled from cache |
Crawl Errors:
| Errors | Retrieval Error |
|---|---|
| 7 | Redirect without a location header |
| 11 | Document not found (404) |
| 12 | Other HTTP 400 errors |
| 14 | HTTP 0 error |
| 15 | Permanent DNS failure |
| 16 | Empty document |
| 17 | Image conversion failed |
| 22 | Authentication failed |
| 25 | Conversion error |
| 32 | HTTP 500 error |
| 33 | The robots.txt file is unreachable |
| 35 | Temporary DNS failure |
| 36 | Connection failed |
| 37 | Connection timeout |
| 38 | Connection closed |
| 40 | Connection refused |
| 41 | Connection reset |
| 43 | No route to host |
| 50 | Other error |
Crawl Exclusions:
| Excluded | Description |
|---|---|
| 3 | Not in the URLs to crawl |
| 4 | In the URLs to not crawl |
| 5 | Off domain redirect |
| 6 | Long redirect chain |
| 8 | Infinite URL space |
| 9 | Unhandled protocol |
| 10 | URL is too long |
| 13 | The robots.txt file indicates to not index |
| 18 | Rejected by rewrite rules |
| 19 | Unknown extension |
| 20 | Disallowed by a meta tag |
| 24 | Disallowed by the robots.txt file |
| 26 | Unhandled content type |
| 27 | No filter for this content type |
| 34 | robots.txt forbidden |
Query parameters:
| Value | Description | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
collectionName |
Name of a collection that you want to list. The default value is the last used collection. | ||||||||||||||
flatList |
Indicates: false: (Default) List the files and
directories specified by the URL. true: List all files specified by a URL as a flat list. |
||||||||||||||
negativeState |
Indicates: false: (Default) Return documents with a
status equal to view.true: Return documents with a status equal
to view. |
||||||||||||||
pageNum |
The page you want to view. The files from a
URL may be separated into several pages to return. Page numbers start from
1. The default value is 1. |
||||||||||||||
sort |
The key field on which to sort:
|
||||||||||||||
uriAt |
The prefix of URL of a document that you want to list.
The default value is "". |
||||||||||||||
view |
The filter of document status. The values of view are described in Document Status Values.
The default value is all. |
List documents by sending an authenticated GET request to the root entry of the diagnostics feed.
A description entry,
a set of document status entries and a set
of directory status entries are returned.
Description entry properties:
| Property | Description |
|---|---|
| Entry Name | description |
numPages |
The total number of pages to return. |
uriAt |
URL prefix taken from the query parameters. |
Directory status entry properties:
| Property | Description |
|---|---|
| Entry Name | The URL of the directory. |
numCrawledURLs |
The number of crawled documents in this directory, |
numExcludedURLs |
The number of excluded URLs in this directory. |
numRetrievalErrors |
The number of retrieval error documents in this directory. |
type |
DirectoryContentData or HostContentData. |
Document status entry properties:
| Property | Description |
|---|---|
| Entry Name | The URL of the document. |
docState |
The status of this document. See Document Status Values for possible docState values. |
isCookieServerError |
Indicates if a cookie server error occurred. |
pageRank |
The page rank value. |
timeStamp |
The last time the search appliance processed this document. |
type |
FileContentData |
Example:
Map<String, String> queries = new HashMap<String, String>();
queries.put("uriAt","http://server.com/secured/test1/");
GsaFeed myFeed = myClient.queryFeed("diagnostics", queries);
for(GsaEntry entry : myFeed.getEntries()) {
System.out.println(entry.getGsaContent("entryID"));
if (entry.getGsaContent("entryID").equals("description")) {
System.out.println("Number of Pages: " + entry.getGsaContent("numPages"));
System.out.println("URI At: " + entry.getGsaContent("uriAt"));
} else if (entry.getGsaContent("type").equals("DirectoryContentData" )||
entry.getGsaContent("type") .equals("HostContentData")) {
System.out.println("Type: " + entry.getGsaContent("type"));
System.out.println("Number of Crawled URLs: " + entry.getGsaContent("numCrawledURLs"));
System.out.println("Number of Retrieval Errors: " + entry.getGsaContent("numRetrievalErrors"));
System.out.println("Number of Excluded URLs: " + entry.getGsaContent("numExcludedURLs"));
} else if (entry.getGsaContent("type").equals("FileContentData")) {
System.out.println("Type: " + entry.getGsaContent("type"));
System.out.println("Time Stamp: " + entry.getGsaContent("timeStamp"));
System.out.println("Document State: " + entry.getGsaContent("docState"));
System.out.println("Page Rank: " + entry.getGsaContent("pageRank"));
System.out.println("Is Cookie Server Error: " + entry.getGsaContent("isCookieServerError"));
}
}
Retrieve detailed information about a document by sending an authenticated GET request to a document status entry of the diagnostics feed. The parameter is as follows.
| Parameter | Description |
|---|---|
collectionName |
Name of the collection for which you want to view crawl diagnostics. |
A detailed document status entry is returned.
Detailed document status entry properties:
| Property | Description |
|---|---|
| Entry Name | The URL of the document. |
backwardLinks |
The number of backward links to this document. |
collectionList |
A list of collections that contain this document. |
contentSize |
The size of the document content. |
contentType |
The type of the document. |
crawlFrequency |
The frequency at which the document is being scheduled to crawl, with possible values of seldom, normal, and frequent. |
crawlHistory |
A multi-line history of the document crawl including the timestamp when the document was crawled, the document status code and description in the following format:timestamp status_code status_descriptionFor status code values, see Document Status Values. |
currentlyInflight |
If the document is currently in process. |
date |
The date of this document. |
forwardLinks |
The number of forward links for this document. |
isCached |
Indicates if the cached page for this document is ready. |
lastModifiedDate |
The last modified date of this document. |
latestOnDisk |
The timestamp of the version being served. |
pageRank |
The page rank value. |
GsaEntry entry = myClient.getEntry("diagnostics",
"http://server.com/secured/test1/doc_0_2.html");
System.out.println("Page Rank: " + entry.getGsaContent("pageRank"));
System.out.println("Collection List: " + entry.getGsaContent("collectionList"));
System.out.println("Forward Links: " + entry.getGsaContent("forwardLinks"));
System.out.println("Backward Links: " + entry.getGsaContent("backwardLinks"));
System.out.println("Is Cached: " + entry.getGsaContent("isCached"));
System.out.println("Document Date: " + entry.getGsaContent("date"));
System.out.println("Last Modified Date: " + entry.getGsaContent("lastModifiedDate"));
System.out.println("Latest Serving Version Timestamp: " +
entry.getGsaContent("latestOnDisk"));
System.out.println("Currently In Process: " + entry.getGsaContent("currentlyInflight"));
System.out.println("Content Size: " + entry.getGsaContent("contentSize"));
System.out.println("Content Type: " + entry.getGsaContent("contentType"));
System.out.println("Crawl Frequency: " + entry.getGsaContent("crawlFrequency"));
System.out.println("Crawl History: " + entry.getGsaContent("crawlHistory"));
Retrieve content statistics for each kind of document using the contentStatistics feed.
Common query parameters for all requests:
| Parameter | Description |
|---|---|
collectionName |
Name of the collection for which you want to view content statistics. |
Content statistics entry properties:
| Property | Description |
|---|---|
avgSize |
The average document size for this content type. |
| Entry Name | The MIME type of the documents, such as, plain/text. |
maxSize |
The maximum document size for the crawled files with this MIME type. |
minSize |
The minimum document size for the crawled files with this MIME type. |
numFiles |
The total number of crawled files for this MIME type. |
totalSize |
The total size of all crawled files for this MIME type. |
Retrieve content statistics for all crawled files
in a search appliance by sending an authenticated GET
request to the root entry of the contentStatistics feed.
A list of content statistics entries are returned.
GsaFeed myFeed = myClient.getFeed("contentStatistics");
for(GsaEntry entry : myFeed.getEntries()) {
System.out.println("Entry Name: " + entry.getGsaContent("entryID"));
System.out.println("Maximum Size: " + entry.getGsaContent("maxSize"));
System.out.println("Minimum Size: " + entry.getGsaContent("minSize"));
System.out.println("Total Size: " + entry.getGsaContent("totalSize"));
System.out.println("Average Size: " + entry.getGsaContent("avgSize"));
System.out.println("Number of Files: " + entry.getGsaContent("numFiles"));
}
Retrieve content statistics for a single crawled file
by sending an authenticated GET request to a content statistics
entry of the contentStatistics feed.
The following content statistics for a crawled file are returned:
GsaEntry entry = myClient.getEntry("contentStatistics", "text/html");
System.out.println("Maximum Size: " + entry.getGsaContent("maxSize"));
System.out.println("Minimum Size: " + entry.getGsaContent("minSize"));
System.out.println("Total Size: " + entry.getGsaContent("totalSize"));
System.out.println("Average Size: " + entry.getGsaContent("avgSize"));
System.out.println("Number of Files: " + entry.getGsaContent("numFiles"));
Generate, update, and delete search reports using the searchReport feed.
Search report entry properties:
| Property | Description | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
collectionName |
(Write only) The collection name--only use to create a search report. | ||||||||||||
diagnosticTerms |
The description of a search report. The default value is "". |
||||||||||||
| Entry Name | Search_Report_Name@Collection_Name | ||||||||||||
isFinal |
(Read only) Indicates if a search report contains a final result.
If so, it means the last update date is later
than the reportDate. |
||||||||||||
reportContent |
(Read only) The search report content. Only use for requests to get search report content when the content is ready. | ||||||||||||
reportCreationDate |
(Read only) The creation date of a search report. | ||||||||||||
reportDate |
The dates of each query that is collected in the search report. | ||||||||||||
reportName |
(Write only) The report name--only use to create a search report. | ||||||||||||
reportState |
(Read only) The status of a search report:
|
||||||||||||
topCount |
The number of top queries to generate. | ||||||||||||
withResults |
Indicates if a query should only count searches that have results. The default value is false. |
List search report entries by sending an authenticated GET request to the root entry of the searchReport feed. Query parameter:
| Parameter | Description |
|---|---|
collectionName |
Collection Name of search report. The default value is all.collections. |
A list of search report entries returns:
GsaFeed myFeed = myClient.getFeed("searchLog");
for(GsaEntry entry : myFeed.getEntries()) {
System.out.println("Entry Name: " + entry.getGsaContent("entryID"));
System.out.println("Report State: " + entry.getGsaContent("reportState"));
System.out.println("Report Creation Date: " + entry.getGsaContent("reportCreationDate"));
System.out.println("Report Date: " + entry.getGsaContent("reportDate"));
System.out.println("Is Final: " + entry.getGsaContent("isFinal"));
System.out.println("With Results: " + entry.getGsaContent("withResults"));
System.out.println("Top Count: " + entry.getGsaContent("topCount"));
System.out.println("Diagnostic Terms: " + entry.getGsaContent("diagnosticTerms"));
}
Create a new search report entry by sending an authenticated POST request to the root entry of the searchReport feed.
The possible date formats for reports are as follows.
| Purpose | Format |
|---|---|
| Date | date_month_day_year |
| Month | month_month_year |
| Year | year_year |
| Date range | range_month_day_year_month_day_year |
For example to specify the range of dates from 2 January 2009 to 23 September 2009, use this statement:
insertEntry.addGsaContent("reportDate", "range_1_2_2009_9_23_2009");
The following example generates and returns a new search report entry:
GsaEntry insertEntry = new GsaEntry();
insertEntry.addGsaContent("reportName", "bbb");
insertEntry.addGsaContent("collectionName", "default_collection");
insertEntry.addGsaContent("reportDate", "month_5_2009");
insertEntry.addGsaContent("withResults", "true");
insertEntry.addGsaContent("topCount", "100");
myClient.insertEntry("searchReport", insertEntry);
Retrieve the search report status and get search log content by sending an authenticated GET request to a search report entry of the searchReport feed.
A search report entry with log content (if content is ready) is returned:
GsaEntry entry = myClient.getEntry("searchReport", "bbb@default_collection");
System.out.println("Entry Name: " + entry.getGsaContent("entryID"));
System.out.println("Report State: " + entry.getGsaContent("reportState"));
System.out.println("Report Creation Date: " + entry.getGsaContent("reportCreationDate"));
System.out.println("Report Date: " + entry.getGsaContent("reportDate"));
System.out.println("Is Final: " + entry.getGsaContent("isFinal"));
System.out.println("With Results: " + entry.getGsaContent("withResults"));
System.out.println("Top Count: " + entry.getGsaContent("topCount"));
System.out.println("Diagnostic Terms: " + entry.getGsaContent("diagnosticTerms"));
status = entry.getGsaContent("reportState");
if (status.equals("2") || status.equals("3")) {
System.out.println("Report Content: " + entry.getGsaContent("reportContent"));
}
Update the search report status and get search report content by sending an authenticated PUT request to a
search report entry of the searchReport feed. There are no properties.
A search log entry is returned:
GsaEntry updateEntry = new GsaEntry();
myClient.updateEntry("searchReport","bbb@default_collection");
Delete a search report by sending an authenticated DELETE request to a search report entry of the searchReport feed.
The search report entry will be deleted:
myClient.deleteEntry("searchReport", "bbb@default_collection");
Generate, update, and delete a search log using the searchLog
feed. A search log lists all search queries for a specified time frame
in a format similar to a common log format (CLF).
Search log entry properties:
| Property | Description | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
collectionName |
(Write-only) The collection name--use only to create a search log. | ||||||||||||
| Entry Name | Search_Log_Name@Collection_Name | ||||||||||||
fromLine |
(Read only) The first line of a search log that is returned in the log content--only returned when getting search log content and the content is ready. | ||||||||||||
isFinal |
(Read only) If the search log contains the final result.
If so, it means the last update date is later than the reportDate. |
||||||||||||
logContent |
(Read only) A part of the content of the search log--only returned when getting search log content and the content is ready. | ||||||||||||
reportCreationDate |
(Read only) The creation date of a search log. | ||||||||||||
reportDate |
The dates of the queries in the search log. | ||||||||||||
reportName |
(Write-only) The report name--use only to create a search log. | ||||||||||||
reportState |
(Read only) Search log status:
|
||||||||||||
toLine |
(Read only) The last line of a search log that is returned in the log content--only returned when getting search log content and the content is ready. | ||||||||||||
totalLines |
(Read only) The number of lines of a search log that are returned in the log content--only returned when getting search log content and the content is ready. |
List search log entries by sending an authenticated GET request to the root entry of the searchLog feed.
| Parameter | Description |
|---|---|
collectionName |
Collection name of a search log. The default value is all.collections. |
A list of search log entries will be returned.
GsaFeed myFeed = myClient.getFeed("searchLog");
for(GsaEntry entry : myFeed.getEntries()) {
System.out.println("Entry Name: " + entry.getGsaContent("entryID"));
System.out.println("Report State: " + entry.getGsaContent("reportState"));
System.out.println("Report Creation Date: " + entry.getGsaContent("reportCreationDate"));
System.out.println("Report Date: " + entry.getGsaContent("reportDate"));
System.out.println("Is Final: " + entry.getGsaContent("isFinal"));
}
Create a new search log entry by sending an authenticated POST request to the root entry of the searchLog feed.
A new search log entry will be generated and returned.
GsaEntry insertEntry = new GsaEntry();
insertEntry.addGsaContent("reportName", "bbb");
insertEntry.addGsaContent("collectionName", "default_collection");
insertEntry.addGsaContent("reportDate", "date_3_25_2009");
myClient.insertEntry("searchLog", insertEntry);
Check the search log status and get search log content by sending an authenticated GET request to the search log entry of the searchLog feed using the following query parameters.
| Parameter | Description |
|---|---|
query |
Query string for the logContent.
The logContent contains many lines of logs. The query string applies to
each line, and only lines that contain the query string are returned. |
maxLines |
The maximum logContent lines to retrieve. The default value is 50 lines. |
startLine |
The first logContent lines to retrieve. The default value is 1 line. |
A search log entry with logContent, if content is ready, is returned.
Map<String, String> queries = new HashMap<String, String>();
queries.put("query","User");
queries.put("startLine","1");
queries.put("maxLine","10");
GsaEntry entry = myClient.queryEntry("searchLog", "bbb@default_collection", queries);
System.out.println("Entry Name: " + entry.getGsaContent("entryID"));
System.out.println("Report State: " + entry.getGsaContent("reportState"));
System.out.println("Report Creation Date: " + entry.getGsaContent("reportCreationDate"));
System.out.println("Report Date: " + entry.getGsaContent("reportDate"));
System.out.println("Is Final: " + entry.getGsaContent("isFinal"));
status = entry.getGsaContent("reportState");
if (status.equals("2") || status.equals("3")) {
System.out.println("Log Content: " + entry.getGsaContent("logContent"));
System.out.println("To Line: " + entry.getGsaContent("toLine"));
System.out.println("From Line: " + entry.getGsaContent("fromLine"));
System.out.println("Total Lines: " + entry.getGsaContent("totalLines"));
}
Update the search log status and get search log content by sending an authenticated PUT request to the search log entry of the searchLog feed. No properties are required.
GsaEntry updateEntry = new GsaEntry();
myClient.updateEntry("searchLog","bbb@default_collection");
Update the search log status and get search log content by sending an authenticated DELETE request to a search log entry of the searchLog feed.
The search log entry will be deleted.
myClient.deleteEntry("searchLog", "bbb@default_collection");
Retrieve lines from the event log for a search appliance by using the eventLog entry of the logs feed. The following parameters let you make a query, specify a starting line, and specify the number of event log statements to retrieve.
| Parameter | Description |
|---|---|
query |
Query string for the logContent.
The logContent contains many lines of
logs. The query string applies to each line, only lines that contain
the query string are returned. |
startLine |
The starting line number to retrieve from the event log. The default value is 1. |
maxLines |
The maximum number of lines in the event log to retrieve. The default value is 50 lines. |
Use the following properties to retrieve event log lines and event log content.
| Property | Description |
|---|---|
fromLine |
The starting line of a log. |
logContent |
The log's content. |
toLine |
The ending line of a log. |
totalLines |
Total lines of the log. |
Retrieve the event log information from a search appliance as follows:
Map<String, String> queries = new HashMap<String, String>();
queries.put("query","User");
queries.put("startLine","10");
queries.put("maxLine","2");
GsaEntry myEntry = myClient.queryEntry("logs", "eventLog", queries);
System.out.println("Log Content: " + myEntry.getGsaContent("logContent"));
System.out.println("Total Lines: " + myEntry.getGsaContent("totalLines"));
System.out.println("From Line: " + myEntry.getGsaContent("fromLine"));
System.out.println("To Line: " + myEntry.getGsaContent("toLine"));
The sections that follow describe how to configure the Connector Administration features of the Admin Console.
Add, retrieve, update, and delete a connector manager associated with a search appliance.
| Property | Description |
|---|---|
description |
A description of the connector manager. |
url |
The URL of the application server where the connector manager is installed. |
status |
The status of the connection between the search appliance and
the connector manager that is deployed on an application server.
The value can be Connected or Disconnected.
The Disconnected value may occur if the application server
is down or there are problems on the network. |
Add a connector manager to a search appliance as follows:
// Create an entry to hold properties to insert
GsaEntry insertEntry = new GsaEntry();
// Add property for insertEntry
insertEntry.addGsaContent("entryID", "ConnectorManagerOne");
insertEntry.addGsaContent("description", "Connector Manager One Description");
insertEntry.addGsaContent("url", "http://example.com:port/");
// Send the request
myClient.insertEntry("connectorManager", insertEntry);
Retrieve a list of connector managers as follows:
// Send the request and print the response
GsaFeed myFeed = myClient.getFeed("connectorManager");
for(GsaEntry myEntry : myFeed.getEntries()) {
System.out.println("Status: " + myEntry.getGsaContent("status"));
System.out.println("Description: " + myEntry.getGsaContent("description"));
System.out.println("URL: " + myEntry.getGsaContent("url"));
}
Retrieve an individual connector manager as follows:
// Send the request and print the response
GsaEntry myEntry = myClient.getEntry("connectorManager", "ConnectorManagerOne");
System.out.println("Status: " + myEntry.getGsaContent("status"));
System.out.println("Description: " + myEntry.getGsaContent("description"));
System.out.println("URL: " + myEntry.getGsaContent("url"));
Update the description and URL for a connector manager as follows:
// Create an entry to hold the properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId(entryUrl);
// Add properties to updateEntry
updateEntry.addGsaContent("description", "new description");
updateEntry.addGsaContent("url", "#new URL");
// Send the request
myClient.updateEntry("connectorManager", "ConnectorManagerOne", updateEntry);
Delete a connector manager as follows:
myClient.deleteEntry("connectorManager", "ConnectorManagerOne");
The sections that follow describe how to configure Federation features of the Admin Console:
Federation is also known as dynamic scalability. The federation feed provides Federation features.
Retrieve, update, create, or delete the federation node configuration and retrieve the node configuration of all nodes in the network on the Google Search Appliance.
| Property | Description | |||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
applianceId |
The ID of the search appliance, required to identify the node during node operations. | |||||||||||||||||||||||||||||||||
federationNetworkIP |
The private tunnel IP address (virtual address) for the node. This address must be an RFC 1918 address. Note: A federation works best when the IP addresses of the nodes are numerically near, such as 10.1.1.1, 10.1.1.2, 10.1.1.3, and so on. The search appliance disallows a federation for nodes that are not in the same /16 subnet. This is a problem only if there are more than 65534 nodes in a federation network. Federation nodes communicate on TCP port 10999. | |||||||||||||||||||||||||||||||||
hostname |
The host name of the search appliance. | |||||||||||||||||||||||||||||||||
nodeType |
The type of search appliance. Possible values: PRIMARY: The node merges results from other nodes.SECONDARY: The node serves results to the other nodes.PRIMARY_AND_SECONDARY:
The node acts as both a primary and secondary node. |
|||||||||||||||||||||||||||||||||
scoringBias |
The scoring bias value for this node. Valid values are integers between
-99 and 99. The scoring bias value reflects the weighting to be given to
results from this node. A higher value means a higher weighting. The
values and their equivalent in the Admin Console are:
|
|||||||||||||||||||||||||||||||||
secretToken |
The secret token that you use to establish a connection to this node. This token can be any non-empty string. The remote search appliance needs this token for the connection handshake. | |||||||||||||||||||||||||||||||||
Add a federation node as follows:
// Create an entry to hold properties to insert
GsaEntry insertEntry = new GsaEntry();
insertEntry.setId(entryUrl);
// In the following example code, add a secondary
// node with arbitrary values for the various settings.
// Add properties to insertEntry
insertEntry.addGsaContent("entryID", "node_appliance_id");
insertEntry.addGsaContent("nodeType", "SECONDARY");
insertEntry.addGsaContent("federationNetworkIP", "10.0.0.2");
insertEntry.addGsaContent("secretToken", "token");
insertEntry.addGsaContent("hostname", "corp.domain.x.com");
insertEntry.addGsaContent("scoringBias", "20");
// Send the request
myClient.insertEntry("federation", insertEntry);
Retrieve the configuration information about a federation node as follows:
// Send a request and print the response
GsaEntry myEntry = myClient.getEntry("federation", "applianceId");
String type = myEntry.getGsaContent("nodeType");
System.out.println("Node Type: " + type);
System.out.println("Federation Network IP: " + myEntry.getGsaContent("federationNetworkIP"));
System.out.println("Host Name: " + myEntry.getGsaContent("hostname"));
System.out.println("Secret Token: " + myEntry.getGsaContent("secretToken"));
System.out.println("Scoring Bias: " + myEntry.getGsaContent("scoringBias"));
if (type.equals("SECONDARY")) {
System.out.println("Remote Front End: " + myEntry.getGsaContent("remoteFrontend"));
System.out.println("Node Timeout: " + myEntry.getGsaContent("slaveTimeout"));
}
if (type.equals("PRIMARY") || type.equals("PRIMARY_AND_SECONDARY")) {
System.out.println("Secondary Nodes: " + myEntry.getGsaContent("secondaryNodes"));
}
Retrieve information on all federation nodes as follows:
// Send the request and print the response
GsaFeed myFeed = myClient.getFeed("federation");
for (GsaEntry gsaEntry : myFeed.getEntries()) {
// Process each entry
}
Update the configuration of a node as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
updateEntry.setId(entryUrl);
// Add properties to updateEntry
updateEntry.addGsaContent("entryID", "applianceId");
updateEntry.addGsaContent("nodeType", "PRIMARY");
updateEntry.addGsaContent("federationNetworkIP", "10.0.0.3");
updateEntry.addGsaContent("secretToken", "new_secret_token");
updateEntry.addGsaContent("hostname", "new_hostname");
updateEntry.addGsaContent("scoringBias", "20");
// Send the request
myClient.updateEntry("federation", "applianceId", updateEntry);
Delete a node as follows:
myClient.deleteEntry("federation", "applianceId");
The sections that follow describe how to configure Administration features of the Admin Console:
Retrieve license Information from the search appliance
using the licenseInfo entry of the info feed.
Note: You can only license information, but not update or install a new license through this API.
Retrieve license information using the following properties.
| Property | Description |
|---|---|
applianceID |
Provides the identification value for the Google Search Appliance software. This value is also known as the serial number for the search appliance. |
licenseID |
Provides the unique license identification value. |
licenseValidUntil |
Identifies when the search appliance software license expires. |
maxCollections |
Indicates the maximum number of collections. You can configure collections at the Crawl and Index > Collections page. |
maxFrontends |
Indicates the maximum number of front ends. You can configure front ends at the Serving > Front Ends page. |
maxPages |
Maximum number of content items that you can index with this product. Content items include documents, images, and content from the feeds interface. |
Retrieve the license Information from a search appliance as follows:
// Send the request and print the response
GsaEntry myEntry = myClient.getEntry("info", "licenseInfo");
System.out.println("Appliance ID: " + myEntry.getGsaContent("applianceID"));
System.out.println("License ID: " + myEntry.getGsaContent("licenseID"));
System.out.println("License Valid Until: " + myEntry.getGsaContent("licenseValidUntil"));
System.out.println("Maximum Front Ends: " + myEntry.getGsaContent("maxFrontends"));
System.out.println("Maximum Pages: " + myEntry.getGsaContent("maxPages"));
System.out.println("Maximum Collections: " + myEntry.getGsaContent("maxCollections"));
Reset the index for a search appliance using the following properties.
Caution: Resetting an index deletes all the documents in the index. Depending on the number of documents to crawl, crawling an index can take many days to complete.
| Property | Description | resetIndex |
1 if index is reset, 0 if index is not reset. |
|---|---|
resetStatusCode |
Status code for resetting index. |
resetStatusMessage |
Status message: ERROR, PROGRESS, READY. |
Retrieve the status of a reset index as follows:
// Send the request and print the response
GsaEntry myEntry = myClient.getEntry("command", "resetIndex");
System.out.println("Reset Index: " + myEntry.getGsaContent("resetIndex"));
System.out.println("Reset Status Code: " + myEntry.getGsaContent("resetStatusCode"));
System.out.println("Reset Status Message: " + myEntry.getGsaContent("resetStatusMessage"));
Reset the index as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
// Add a property to updateEntry
updateEntry.addGsaContent("resetIndex", "1");
myClient.updateEntry("command", "resetIndex", updateEntry);
Import or export a search appliance configuration using the importExport entry of the config feed.
The following is the common query parameter for all requests.
| Parameter | Description |
|---|---|
password |
The password of the exported configuration. |
Specify importExport entry properties.
| Property | Description |
|---|---|
password |
The password of the configuration file. |
xmlData |
The content of an exported configuration saved as XML data. |
Export a search appliance configuration by sending an
authenticated GET request to the importExport
entry of the config feed.
The following importExport entry is returned:
Map<String, String> queries = new HashMap<String, String>();
queries.put("password","12345678");
GsaEntry entry = myClient.queryEntry("config", "importExport");
System.out.println("XML Data: " + entry.getGsaContent("xmlData"));
Import a search appliance configuration
sending an authenticated PUT request to the importExport
entry of the config feed.
GsaEntry updateEntry = new GsaEntry();
updateEntry.addGsaContent("xmlData", "<config data>");
updateEntry.addGsaContent("password", "12345678");
myClient.updateEntry("config", "importExport", updateEntry);
Shut down or reboot the search appliance.
| Property | Description |
|---|---|
command |
Command sent to the search appliance. The command can
be shutdown or reboot. |
runningStatus |
Indicates the search appliance status:
|
Shut down or reboot the search appliance as follows:
// Create an entry to hold properties to update
GsaEntry updateEntry = new GsaEntry();
// Add a property to updateEntry
updateEntry.addGsaContent("command", "reboot");
myClient.updateEntry("command", "shutdown", updateEntry);