[Athen] looking for spider-like software

Gunderson, Jon R jongund at illinois.edu
Mon Nov 28 07:41:15 PST 2016


There is a unix utility called "wget" that does this type of thing.


I believe there is a windows version.


-----Original Message-----
From: athen-list [mailto:athen-list-bounces at mailman13.u.washington.edu] On Behalf Of Prof Norm Coombs
Sent: Monday, November 28, 2016 12:20 AM
To: athen-list at u.washington.edu
Subject: [Athen] looking for spider-like software

I am looking for software like an old, old program called offsite explorer.
It would download a set of related web pages for me. Somehow I tole it where to go to start spidering web pages and how many levels down to include. It would get all the details and then save part of a web site for me. It let me select a sequence of related/connected web pages and then save them all for me.
Instead of having to save a lot of individual web pages, it collected them and saved them. I just set the parameters and waited.

It was free open source, but I'd also be willing to pay something if necessary.

athen-list mailing list
athen-list at mailman13.u.washington.edu

More information about the athen-list mailing list