Member Avatar for tondeuse34

Hey guys, i'm having some trouble using wget. What i'm trying do is log into a site, once its logged is to go to a certain link. And once it is there download all the files on the page, but check back later to see if any new ones are there. So far i've made up this:

#!/bin/sh

start() [--user=USER --password=PASS -A .pdf,.doc -r -Nc http://google.com]

new() [$new_file = wget --user=USER --password=PASS -N -A.pdf,.doc -r -Nc http://www.google.com]

if [start()] then
echo Downloading files...

if [new()] then
echo Found new files!...Downloadi

Whenever i attempt to use --user=USER and --password=PASS it does not log me in to the site i am using, that is my main obstacle right now. What do you guys suggest i do?, Thanks. Clipper34.

  • 2 Contributors
  • 1 Reply
  • 103 Views
  • 12 Hours Discussion Span
  • Latest Post Latest Post by eggi
Member Avatar for eggi

Hey There,

It might have to do with the user agent string that wget passes to the site (I think it's something like wget-version). You can manipulate the --user-agent= variable to pass anything, just be careful that you don't use Mozilla as they're litigation happy (more details on the options page on wget's site regarding getting sued by them)

That's just one possibility.

Hope it helps :)

, Mike

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.