Forum Home
Press F1
 
Thread ID: 118021 2011-05-16 06:15:00 Automatic Webpage Capture bot (15449) Press F1
Post ID Timestamp Content User
1202465 2011-05-16 06:15:00 Is there software that will, say, capture a webpage or an element of a webpage every 10 minutes or so? bot (15449)
1202466 2011-05-16 07:53:00 What do you want to capture? Screen Grabs or do you want notification of any changes/updates in content? If content you can use something like Google alerts which emails changes in content. I know there are more advanced similar software for more regular updates, with more filters, etc, but require signing up. Unless some advanced scripting can be used. I used something similar a while back - but Google will block or warn of automated requests. Then can't search for a while.

I used to use Versionista which tracks changes, and I think sent out alerts. But requires online sign up. 		kahawai chaser (3545)
1202467 2011-05-16 09:08:00 What do you want to capture? Screen Grabs or do you want notification of any changes/updates in content? If content you can use something like Google alerts which emails changes in content. I know there are more advanced similar software for more regular updates, with more filters, etc, but require signing up. Unless some advanced scripting can be used. I used something similar a while back - but Google will block or warn of automated requests. Then can't search for a while.

I used to use Versionista which tracks changes, and I think sent out alerts. But requires online sign up.

I want a webpage to be refreshed and saved every 10 minutes. The reason for this is that I can't stay up that late!

PS it's for statistical reasons 		bot (15449)
1202468 2011-05-16 09:54:00 Theres a few suggestions and links to programs Listed here (www.elitetrader.com) - no idea if they work or are any good --- Give some a try and let everyone know :p:D

Edited: Try this in IrfanView ( if you want to use that) open IrfanView, options/Screen capture - looks like theres a timer option Shown here (www.thebest3d.com) as well as how to turn it into a movie if you want. 		wainuitech (129)
1202469 2011-05-16 09:57:00 Wget plus cron is ideal for this task.

Depending on what you need and how long you need it for, I may be happy to run a job on one of my servers and send you the result - PM me if you're interested. 		Erayd (23)
1202470 2011-05-16 10:51:00 Theres a few suggestions and links to programs Listed here (www.elitetrader.com) - no idea if they work or are any good --- Give some a try and let everyone know :p:D

Edited: Try this in IrfanView ( if you want to use that) open IrfanView, options/Screen capture - looks like theres a timer option Shown here (www.thebest3d.com) as well as how to turn it into a movie if you want.

I'll give IrfanView a crack - the page seems to refresh every minute.

I'd prefer something that actually captured data - as I want to copy and paste it into a document.

Perhaps if there was something that automatically went "Ctrl-S" in Chrome, waited, say, 10 secs, then went "Enter". Something like a macro program would do the trick.

Although Chrome would have to be in focus...

BTW, thanks for all you guys help so far :) 		bot (15449)
1202471 2011-05-16 10:58:00 @bot: if you're not familiar with wget, you should take Erayd up on his offer because it's exactly what you're looking for, a snapshot every X minutes (source and images) and not just a screenshot image. sal (67)
1202472 2011-05-16 11:57:00 @bot: if you're not familiar with wget, you should take Erayd up on his offer because it's exactly what you're looking for, a snapshot every X minutes (source and images) and not just a screenshot image.Welcome back Sal :D - long time, no see! Erayd (23)
1202473 2011-05-16 11:59:00 It's probably a bit late, but here's a AutoHotkey (www.autohotkey.com/download/) script that automates the job of saving and refreshing a webpage every 10 mins:


Var = 1
Loop {
Sleep, 100000
Send ^s
Sleep, 100000
Send Saved_webpage_v%Var%{Enter}
Sleep, 400000
Send ^r
Var += 1
}

Here's a couple I compiled earlier:

Save & refresh page every 1 minute -> SarpeOM.exe (b.spare.co.nz) (267KB)
Save & refresh page every 10 minutes -> SarpeTM.exe (b.spare.co.nz) (267KB)

Those are clean and legit executables, they only do what it says on the box (see code above).

First go to the webpage you want to save and do one run through "Ctrl+s" so it's saving to the right folder, run the ".exe" (it'll run from the tray) and then give focus to the relevant browser tab, and sorted. 		sal (67)
1202474 2011-05-16 12:12:00 It's probably a bit late, but here's a AutoHotkey (www.autohotkey.com/download/) script that automates the job of saving and refreshing a webpage every 10 mins:


Var = 1
Loop {
Sleep, 100000
Send ^s
Sleep, 100000
Send Saved_webpage_v%Var%{Enter}
Sleep, 400000
Send ^r
Var += 1
}

Here's a couple I compiled earlier:

Save & refresh page every 1 minute -> SarpeOM.exe (b.spare.co.nz) (267KB)
Save & refresh page every 10 minutes -> SarpeTM.exe (b.spare.co.nz) (267KB)

Those are clean and legit executables, they only do what it says on the box (see code above).

First go to the webpage you want to save and do one run through "Ctrl+s" so it's saving to the right folder, run the ".exe" (it'll run from the tray) and then give focus to the relevant browser tab, and sorted.

Thanks so much, this is just what I needed :)

:cookie: 		bot (15449)
1 2