| Forum Home | ||||
| Press F1 | ||||
| Thread ID: 106864 | 2010-01-26 02:44:00 | Webcrawler | PCT Joe (15018) | Press F1 |
| Post ID | Timestamp | Content | User | ||
| 852080 | 2010-01-26 02:44:00 | Hello all, I currently have disconnected the internet and am at the local libary, I'm wondering if anyone knows of a webcrawler that catalogs pages for instance i just downloaded all of playtech.co.nz by ctrl + s'ing the whole way. is there anything that could do this automaticaly so i can still access the pricing at home and update it automatically. Thanks, Joe |
PCT Joe (15018) | ||
| 852081 | 2010-01-26 02:46:00 | There was one around called Webreaper as I remember it. Still around too:- http://www.webreaper.net/ |
Sweep (90) | ||
| 852082 | 2010-01-26 03:20:00 | Wget can do it (--mirror). I suggest you ask the site operator for permission first though - mirroring an entire site without so much as a by-your-leave is rather rude (and sometimes illegal, depending on your location). | Erayd (23) | ||
| 852083 | 2010-01-26 04:04:00 | yea - plus there's one or two others (www.google.co.nz) that do it as well lol | bevy121 (117) | ||
| 852084 | 2010-01-26 04:37:00 | I like HTTRACK. But I usually get permission first, or download just a very small subset of a site. I also usually babysit it, and abort if it's downloading more than a few meg. |
annie (6010) | ||
| 852085 | 2010-01-26 22:29:00 | Thanks, but im still having problems as they download and name it category.php and since all are the same well... yeah... anyways ill try get in contact with playtech. |
PCT Joe (15018) | ||
| 1 | |||||