Duplicate lines in a text file

Reply

Join Date: Jun 2009
Posts: 4
Reputation: kummetha is an unknown quantity at this point 
Solved Threads: 0
kummetha kummetha is offline Offline
Newbie Poster

Duplicate lines in a text file

 
0
  #1
Jun 25th, 2009
How to print the duplicates in a text file. Also the line number where the duplicate text has been found needs to be printed. pls help

KZ
Reply With Quote Quick reply to this message  
Join Date: May 2006
Posts: 38
Reputation: msvinaykumar is an unknown quantity at this point 
Solved Threads: 2
msvinaykumar msvinaykumar is offline Offline
Light Poster

Re: Duplicate lines in a text file

 
0
  #2
Jun 25th, 2009
Hi,

If you are working in Unix then, Use following command

<code>
cat -n file-name > file_name2
</code>

Then use
<code>
sort -u -k 2,4 file_name2
</code>
Reply With Quote Quick reply to this message  
Join Date: Jun 2009
Posts: 4
Reputation: kummetha is an unknown quantity at this point 
Solved Threads: 0
kummetha kummetha is offline Offline
Newbie Poster

Re: Duplicate lines in a text file

 
0
  #3
Jun 25th, 2009
No, I am not working in Unix. I am working with Windows.
KZ
Reply With Quote Quick reply to this message  
Join Date: May 2006
Posts: 38
Reputation: msvinaykumar is an unknown quantity at this point 
Solved Threads: 2
msvinaykumar msvinaykumar is offline Offline
Light Poster

Re: Duplicate lines in a text file

 
0
  #4
Jun 25th, 2009
Or use Uniq command

<code>
$ cat happybirthday.txt

Happy Birthday to You!

Happy Birthday to You!

Happy Birthday Dear vinay!

Happy Birthday to You!
</code>

<code>
$ sort happybirthday.txt | uniq -dc
3 Happy Birthday to You!
</code>
Reply With Quote Quick reply to this message  
Join Date: Mar 2006
Posts: 898
Reputation: KevinADC has a spectacular aura about KevinADC has a spectacular aura about 
Solved Threads: 67
KevinADC's Avatar
KevinADC KevinADC is offline Offline
Practically a Posting Shark

Re: Duplicate lines in a text file

 
0
  #5
Jun 25th, 2009
Originally Posted by kummetha View Post
How to print the duplicates in a text file. Also the line number where the duplicate text has been found needs to be printed. pls help

KZ
What have you tried? Where are you stuck?
Reply With Quote Quick reply to this message  
Join Date: Jun 2009
Posts: 4
Reputation: kummetha is an unknown quantity at this point 
Solved Threads: 0
kummetha kummetha is offline Offline
Newbie Poster

Re: Duplicate lines in a text file

 
0
  #6
Jun 25th, 2009
many a thanks msvinaykumar, i am looking for a perl script that works with windows.
KZ
Reply With Quote Quick reply to this message  
Join Date: Jun 2009
Posts: 10
Reputation: Prakash_8111 is an unknown quantity at this point 
Solved Threads: 0
Prakash_8111 Prakash_8111 is offline Offline
Newbie Poster

Re: Duplicate lines in a text file

 
0
  #7
Jun 26th, 2009
open(FILE,"1.txt") || die "$!"; ## 1.txt is a file name
%seen =();
$line=0 ;
while (<FILE>)
{
$seen{$_}++;
$line++;
print "$line : $_" if $seen{$_} > 1 ;
}
Reply With Quote Quick reply to this message  
Join Date: Jun 2009
Posts: 4
Reputation: kummetha is an unknown quantity at this point 
Solved Threads: 0
kummetha kummetha is offline Offline
Newbie Poster

Re: Duplicate lines in a text file

 
0
  #8
Jun 26th, 2009
thanks prakash, it did really work.
also if you could provide help for a script that reads a blank line in a txt file and prints the line above the blank line.
Eg
abcdef

william
marc
sumeet

nair

output:
abcdef
sumeet
thanks in advance
Reply With Quote Quick reply to this message  
Join Date: Mar 2006
Posts: 898
Reputation: KevinADC has a spectacular aura about KevinADC has a spectacular aura about 
Solved Threads: 67
KevinADC's Avatar
KevinADC KevinADC is offline Offline
Practically a Posting Shark

Re: Duplicate lines in a text file

 
0
  #9
Jun 26th, 2009
At some point you should start doing your own school work and the rest of the people participating in this thread you stop doing it for you.
Reply With Quote Quick reply to this message  
Reply

This thread is more than three months old.
Perhaps start a new thread instead?
Message:


Thread Tools Search this Thread



About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC