[Athen] looking for spider-like software

Gunderson, Jon R jongund at illinois.edu
Mon Nov 28 07:41:15 PST 2016


Norm,

There is a unix utility called "wget" that does this type of thing.

http://gnuwin32.sourceforge.net/packages/wget.htm

I believe there is a windows version.

Jon


-----Original Message-----
From: athen-list [mailto:athen-list-bounces at mailman13.u.washington.edu] On Behalf Of Prof Norm Coombs
Sent: Monday, November 28, 2016 12:20 AM
To: athen-list at u.washington.edu
Subject: [Athen] looking for spider-like software

I am looking for software like an old, old program called offsite explorer.
It would download a set of related web pages for me. Somehow I tole it where to go to start spidering web pages and how many levels down to include. It would get all the details and then save part of a web site for me. It let me select a sequence of related/connected web pages and then save them all for me.
Instead of having to save a lot of individual web pages, it collected them and saved them. I just set the parameters and waited.

It was free open source, but I'd also be willing to pay something if necessary.
Norm

_______________________________________________
athen-list mailing list
athen-list at mailman13.u.washington.edu
http://mailman13.u.washington.edu/mailman/listinfo/athen-list



More information about the athen-list mailing list