My favorites
|
Sign in
perlhobby
no summary
Project Home
Downloads
Wiki
Issues
Source
Search
Search within:
All wiki pages
Featured pages
Current pages
Deprecated pages
for
Updated
Jun 11, 2008
by
stefan.petrea
full_list_v08
list of features crawler
Source code
Full list of features for Crawler v0.8
Crawler v0.8
crawls links from the forum
if closed from interface remembers where it left off
is able to crawl multiple links at the same time
the user is able to customize the work time and the pause time
is able to use proxies
for each link provided by the user stores data in a differnt "data.txt"
has an easy to use interface
the user is able to download all the data.txt files from the web interface
features a 3-time retry on a link if failure occurs,and if those don't work starts using proxies
the user can delete easily from the interface all data associated with any link that was crawled
Sign in
to add a comment
Hosted by