Member Avatar for Grep

Hey guys, quick question, I'm a beginner and put together a grep script that searches for certain content within text files in a directory. The result yields over 600 lines so i want to put them in a file called file.txt.

opendir (DIR, ".") or die "$!";
                                my @files = grep {/^[\w].+/} readdir DIR;
                                close DIR;
                                foreach my $file (@files) {
                                open(FH,"$file") or die "$!";
                                while (<FH>){ 
				if ($_ =~ /^searchtext/) {
open FILE, ">file.txt" or die $!; print FILE "$_";  }
                                }

                                close(FH);
close FILE;
                                }

This isn't currently working, it only puts in the first line in the text file. Can someone please offer a revision? Thanks.

Edited by Grep because: n/a
  • 3 Contributors
  • 6 Replies
  • 233 Views
  • 16 Hours Discussion Span
  • Latest Post Latest Post by Grep

Dani AI

Generated

Short summary and a robust, maintainable approach.

The reason the output file ended up with only one line is that it was being opened repeatedly inside the read/grep loop in write mode, which truncates the file each time. As pointed out, open the output once (or use append intentionally) and reuse that handle. For recursion, ’s idea of a directory-recursing subroutine is sound; a concise and safer way is to use File::Find and lexical filehandles. Also note the path bug @Fishmonger helped spot — use full paths (or File::Spec) or change directory before opening files.

A compact, reliable pattern (change $root and $pattern to suit the thread’s needs):

use strict;
use warnings;
use File::Find;
use File::Spec;

my $root    = 'data';             # start directory
my $outname = 'file.txt';
my $pattern = qr/^YourSearchText/; # adjust the regex

open my $outfh, '>', $outname or die "Can't open $outname: $!";

find({
  wanted => sub {
    my $path = $File::Find::name;
    return if -d $path;
    return unless -r $path;
    open my $infh, '<', $path or return;
    while (my $line = <$infh>) {
      print $outfh $line if $line =~ $pattern;
    }
    close $infh;
  },
  no_chdir => 1,
}, $root);

close $outfh;

Practical tips, distilled from the thread:

  • Open the output file once (before scanning) to avoid truncation and repeated opens. Use three-arg open and lexical handles for safety.
  • Use append ('>>') only when intentionally accumulating across runs.
  • If only some files are copied with xcopy on Windows, use robocopy or a Perl module such as File::Copy::Recursive for reliable recursive moves.
  • Watch for CRLF or BOM issues (Windows line endings can break ^ matches); normalize lines with chomp or strip \r.
  • For large datasets, tools like ripgrep or a streaming File::Find approach above are much faster than building giant in-memory file lists.

This keeps the logic clear, fixes the truncation/path problems seen here, and scales across nested folders without moving everything into one directory.

Recommended Answers

All 6 Replies

Member Avatar for k_manimuthu 
open FILE, ">file.txt" or die $!;

The above line creates "file.txt". You are using ">" write mode. So the file position is always beginning of the file. At the end of the process you may got only one matched line. To avoid this kind of circumstance you will open the file at "append mode". Try the below code.

open FILE, ">>file.txt" or die $!;
Member Avatar for Grep

Thanks for the help!

Member Avatar for Grep

One last bit of advice, I'm currently using a .bat file (I have 692 files spread over 15 folders) and placing them all into one folder. The xcopy command, however, only moves half of the files?

Is there a better way using perl to do some sort of recursive grep on the multiple folders so I don't have to move all my data into one folder to begin with? If it helps any, the name of the 15 folders always stays the same. If you could please demonstrate with this code I'd appreciate it:

opendir (DIR, ".") or die "$!";
	my @files = grep {/^[\w].+/} readdir DIR;
	close DIR;
	foreach my $file (@files) {
	open(FH,"$file") or die "$!";
		while (<FH>){ 
		if ($_ =~ /^TagString=V3/) {
		open FILE, ">>file.txt" or die $!; print FILE "$_";  }
		}

			close(FH);
			close FILE;
                                }
Member Avatar for d5e5

You could create a subroutine to which you pass each directory name and put your code in there. Try running the following first. It has a subroutine called 'something' that prints a message each time it is called. You could put your code in there.

#!/usr/bin/perl
#DoSomethingInDirAndSubdirs.pl
use strict;
use warnings;
my $startdir = '/home/david/Programming';
do_something_in_dir_and_subdirs($startdir);

sub do_something_in_dir_and_subdirs {
    my $dir = $_[0];
    opendir DH, $dir or die "Failed to open $dir: $!";
    my @d;
    my $res = something("Doing something in $dir");
    while ($_ = readdir(DH)) {
        next if $_ eq "." or $_ eq "..";
        my $fn = $dir . '/' . $_;
        if (-d $fn) {
            push @d, $fn;
        }
    }
    if (scalar @d == 0) { #If no directories found, $dir is lowest dir in this branch
        #We're done.
        return;
    }
    foreach (@d) {
        do_something_in_dir_and_subdirs($_); #Look for directories in directory
    }
}

sub something {
    my $msg = $_[0]; #Assign passed argument to variable
    print '-' x 75, "\n";
    print "$msg\n";
}
Member Avatar for Grep

Thanks D5E5. I'll give that a try. Another simple problem I came across is, I just realized I don't know how to open a different directory?!

opendir (DIR, ".") or die "$!";

to

opendir (DIR, "data/") or die "$!";

I do that and I receive a response saying:
"No such file or directory at a.pl line 5."

opendir (DIR, "data/") or die "$!";
                                my @files = grep {/^[\w].+/} readdir DIR;
                                close DIR;
                                foreach my $file (@files) {
                                open(FH,"$file") or die "$!";
                                while (<FH>){ 
				if ($_ =~ /^searchtext/) {
					open FILE, ">>file.txt" or die $!; print FILE "$_";  }
                                }

                                	close(FH);
					close FILE;
                                }
Member Avatar for Grep

problem solved. The error was telling me from the get go. Line 5 needed data/$file.
Thanks to Fishmonger.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.