The Status and Reports > Crawl Status page provides information about the current status of a crawl. The chart is on a short time delay. The following information is available on the Crawl Status page:
Crawl Mode
The crawl mode indicates whether the search appliance is crawling and indexing
content at a fixed time interval or continuously. You can click the crawl
mode and change it. The two modes of crawling are:
- Continuous crawl. The crawler automatically locates
and indexes content when it is updated. By default, the crawl mode is set to continuous crawl.
- Full crawl. The crawler indexes the content at a scheduled time and duration.
For more information about the crawl modes, see the Crawl Schedule page.
Crawl Status (Table)
The Crawl Status table provides information about the following:
- URLs Found That Match Crawl Patterns - The total number of all urls found that match the crawl patterns that were specified in the Crawl and Index > Crawl URLs page. If the total number is far larger than what you expected, restrict the crawl pattern.
- Total Documents Being Served - The total number of URLs indexed at the time
of viewing this page.
- Current Crawling Rate - The number of pages being crawled per second.
- Document Bytes Filtered - The total number of bytes that have been processed. Recrawled documents are included in the total number.
- Documents Crawled Since Yesterday - The number of documents that have been crawled since yesterday.
- Document Errors Since Yesterday - The number of errors encountered since the crawling began yesterday.
If you have selected the continuous crawl mode in the Crawl and Index > Crawl Schedule page, you will see an lined box (on the right side of the table) that reports whether the crawl is paused or is running. If the system is crawling, you'll see "The crawling system is currently running." Below that is a Pause Crawl button. Click this button to temporarily suspend crawling. The status then reports: "The crawling system is currently paused." Click the Resume Crawl button to start the crawl again.
Note: You can change the frequency of crawling certain web servers on the Crawl and Index > Freshness Tuning page.
If the appliance is under full crawl mode, you can start a crawl by scheduling a new crawl job in the Crawl and Index > Crawl Schedule page.
Note: You can change the frequency of crawling certain web servers on the Crawl and Index > Freshness Tuning page.
Crawl Status (Graph)
The Crawl Status graphs shows the URL Tracker results. The x-axis represents two-hour segments on Universal Military Time (UMT). The y-axis shows the number of URLs crawled.
You can view the number of URLs that have been found and the number of URL that have been crawled in the following ways:
- Separate graphs. The first graph represents the number of URLs that have been crawled. The second graph represents the number of URLs that have been found.
- Single combined graph. The lines that represent the number of URLs that are found and crawled are presented in a single graph. The red line shows the number of URLs successfully crawled. The yellow line shows all found URLs, not including those that had errors, were excluded by follow-patterns, or were excluded by robots.txt.
Sometimes the yellow line may override the red line when they represent the same number of URLs.
You can test your search index by clicking the Test Center link in the horizontal blue bar at the top right of the page. This link
takes you directly to the search page where you can run sample queries to test
results. The link appears on all Admin Console pages.
Crawl Status