Compare two text files for difference

[ibpet11]

Hi guys,

I have a apps that I need your input inorder to improve efficiency, the apps can read a text file (A) and also another text file (B) then spit lines in both files, it can further compare to see if there is any common line using a unique column and then write it into a new text file (C).
Now I need assistance in this area, I want the apps to be able to leave the lines not in file (A) but in file (B) into a new text file (D) and also lines in file (B) not in file (A) into a new text file (E).

Please I need your urgent response, below is code

    If OFDialog_check.FileName <> "" And OFDialog_Base.FileName <> "" Then
        Dim sr1 = New StreamReader(OFDialog_check.FileName)
        Dim sr2 = New StreamReader(OFDialog_Base.FileName)
        line = ""
        line2 = ""
        'Dim new_records As String
        Dim i As Integer = 0
        Dim u As Integer = 0

        For Each lin In IO.File.ReadLines(OFDialog_Base.FileName)
            For Each lin2 In IO.File.ReadLines(OFDialog_check.FileName)

                Dim value = lin.Split(","c)(0)
                Dim value2 = lin2.Split(","c)(0)

                If value2 = value Then
                        u += 1
                        line += u.ToString + "_" + lin + vbNewLine
                        IO.File.WriteAllText("C:\Users\HON\Desktop\Result\new result.txt", line)

                End If

            Next
        Next

        sr1.Close()
        sr2.Close()

    End If

[Reverend Jim]

It would help to know something about the input data. Are the lines in each file unique (within that file) or are there possibly duplicate lines? How big are the files (can you state an approximate maximum number of lines?). You read file 2 once for each line in file 1. This is not very efficient. You also do a split on both lines in the inner loop. Also very inefficient. You would be better off reading and splitting the lines in both files into arrays (or dictionaries) then doing the loops to compare. If you can give me a little more information I'd be happy to help you out.

[ibpet11]

Thank you Reverend for ur response,
Yes each line in each file is unique,
I am developing d apps for file with possibly 12000 lines.
I will appreciate how to read into an array or dictionary in VB.net and then loop to compare.

Once again I appreciate your answer.

[Reverend Jim]

The following code creates two dictionaries (one for each input file). If you are unfamiliar with dictionaries, consider them like improved arrays. With a regular array, the index is a number (starting from 0 and incrementing by 1) and the value is whatever type you have declared the array to be. So for an array of strings, myarray(4) is the fifth (zero relative) string. With a dictionary, you declare the type of the index (or key) and the type of the value (which can be a basic type or an object). You index the dictionary by the key. However, unlike an array, the keys do not have to be sequential (only unique). As a benefit, instead of searching for a particular key, you can let the dictionary do it for you using the ContainsKey method. It may be clearer if you look at the code. You should be able to take this code and figure out how to do the rest. txtResults is a multi-line textbox to display the output of the comparison.

Public Class Form1

    Const TEST1 = "D:\temp\test1.txt"
    Const TEST2 = "D:\temp\test2.txt"

    Private Sub btnCompare_Click(sender As System.Object, e As System.EventArgs) Handles btnCompare.Click

        'Declare two dictionaries. The key for each will be the text from the input line up to,
        'but not including the first ",". The valus for each will be the entire input line.

        Dim file1 As New Dictionary(Of String, String)
        Dim file2 As New Dictionary(Of String, String)

        For Each line As String In System.IO.File.ReadAllLines(TEST1)
            Dim part() As String = line.Split(",")
            file1.Add(part(0), line)
        Next

        For Each line As String In System.IO.File.ReadAllLines(TEST2)
            Dim part() As String = line.Split(",")
            file2.Add(part(0), line)
        Next

        AddText("The following lines from " & TEST2 & " are also in " & TEST1)

        For Each key As String In file2.Keys
            If file1.ContainsKey(key) Then
                AddText(file2(key))
            End If
        Next

    End Sub

    Private Sub AddText(text As String)
        txtResults.Text &= text & vbCrLf
    End Sub

End Class

[TnTinMN]

Another option is to use the Linq extensions on the array class.

For example:

   Dim A() As String = {"a", "b", "c", "d", "e"}
   Dim B() As String = {"a", "g", "c", "h", "k"}


   Dim InA_notIn_B() As String = A.Except(B).ToArray 'yields: b, d, e
   Dim InB_notIn_A() As String = B.Except(A).ToArray 'yields: g, h, k

'another way

   Dim CommonTo_A_and_B() As String = A.Intersect(B).ToArray 'yields: a, c
   Dim Union_A_and_B() As String = A.Union(B).ToArray 'yields: a, b, c, d, e, g, h, k

   Dim InA_notIn_BV2() As String = A.Where(Function(s As String) Not CommonTo_A_and_B.Contains(s)).ToArray 'yields: b, d, e
   Dim InB_notIn_AV2() As String = B.Where(Function(s As String) Not CommonTo_A_and_B.Contains(s)).ToArray 'yields: g, h, k

[ibpet11]

Reverend, Thank you for your contribution, the output display at "txtResults" is also what am able to do at the moment, but I want what is in A and not in B to be in written out somewhere and what is in B not in A to be in a separate file.

I look forward to your further assistance.
Regards.

[ibpet11]

TnTinMN,

I seems not to fully understand what you are tryinig to say here, beacuse am dealing with lines in a text file in this case and will appreciate your input also cos I have been on hold on this apps.

Regards.

[TnTinMN]

Does this make things more clear? Perhaps I misunderstood your original request.

The Except method is described here: Click Here

   Dim A() As String = IO.File.ReadAllLines("fileA")
   Dim B() As String = IO.File.ReadAllLines("fileB")

   Dim InA_notIn_B() As String = A.Except(B).ToArray 
   Dim InB_notIn_A() As String = B.Except(A).ToArray


   If InA_notIn_B IsNot Nothing Then
      IO.File.WriteAllLines("InA_notInB", InA_notIn_B)
   End If

   If InB_notIn_A IsNot Nothing Then
      IO.File.WriteAllLines("InB_notIn_A", InB_notIn_A)
   End If

[Reverend Jim]

As I've said before, let the computer do as much for you as possible. In this case it means TnTinMN's suggestion is clearer and less prone to bugs.