I am new in Perl and generally in programming. I face some real problems here. I need a Perl script that can open a text file, read a series of URLs, get the content of the pages and save it to another file.

Thanks deeply for any guidance.

Recommended Answers

All 4 Replies

I face some real problems here. I need a Perl script that can open a text file, read a series of URLs, get the content of the pages and save it to another file.

Divide your task into steps, such as:

  1. open a text file
  2. read a series of URLs
  3. get the content of the pages for each URL
  4. save contents of pages to another file

Which of these steps gives you problems?

Could you attach an example of the text file you need to read? (Click on 'Files' in the Post menu here to upload a text file.)

Hi tonyprotop,

The script below can do what you want, but there are some questions like:
Is your output file also a text file or a html file, so that all the urls contents are saved and viewd as html file also. OR are you trying to parse the contents of all the urls?

What my script does is to open a file called urls.txt [you can call yours any name you like ] and read all the URLs on that text file, then I used the function get(), from module LWP::Simple [ Incase you don't have it, you might have to install it from CPAN ] to get the web pages url and return it contents, which was then written to another file called output.txt [ you can change that to .html if you like ].
Eventually, all the contents of the urls will be written to this output file and if you have it as HTML [i.e output.html] you will be able to see all the pages in a single html file.

#!/usr/bin/perl
use warnings;
use strict;
use LWP::Simple;

my $input_file = 'urls.txt';    # All my URLs are this text file
open my $fh2, '>:encoding(UTF-8)', 'output.txt'
  or die "can't open file: $!";    # file to write to
open my $fh, '<:encoding(UTF-8)', $input_file
  or die "can't open $input_file: $!";    # file to read from
binmode $fh;
while (<$fh>) {
    chomp;
    my $file = get($_) or die "can't get the webpage: $!";
    print {$fh2} $file;
}
close $fh  or die "can't close file: $!";
close $fh2 or die "can't close file: $!";

Hope this helps

commented: Good work. +8

it's perfect! Thank you! Thank you! You are the best!

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.