Hey!

I have a problem.. I need to remove items from array that are duplicated here's my code:

private void Form2_Load(object sender, EventArgs e)
        {
            openFileDialog1.Title = "Open Links List";
            openFileDialog1.Filter = "Text File|*.txt";
            openFileDialog1.FileName = "";
            openFileDialog1.ShowDialog();

            var file = openFileDialog1.FileName;

            if (file != "")
            {
                StreamReader read = new StreamReader(file);
                var items = read.ReadToEnd();
                string[] urls = items.Split('\n');
                var url = RemoveDups(urls);

                read.Close();

                //StreamWriter write = new StreamWriter(file);

                foreach (string i in urls)
                {
                    //write.WriteLine(i);
                    MessageBox.Show(i);
                }

                //write.Close();

                /*StreamWriter writer = new StreamWriter(file);

                string currentLine;
                HashSet<string> previousLines = new HashSet<string>();

                while ((currentLine = read.ReadLine()) != null)
                {
                    if (previousLines.Add(currentLine))
                    {
                        writer.WriteLine(currentLine);
                    }
                }*/
            }
        }

        public string[] RemoveDups(string[] s)
        {
            var count = s.Count();

            string[] arr = new String[count];

            for (int i = 0; i < count; i++)
            {
                if (!arr.Contains(s[i]))
                {
                    arr[i] = s[i];
                }
            }

            return arr;

        }

Somehow it doesnt work

Try this:

class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("***original array***");
            string[] strs = new string[4] { "one", "two", "two", "three" };
            for (int i = 0; i < strs.Length; i++)
            {
                Console.WriteLine(strs[i]);
            }
            Console.WriteLine("***distinct array***");
            var Results = strs.Distinct();
            foreach (var item in Results)
            {
                Console.WriteLine(item);
            }
            Console.ReadLine();
        }
    }

Distinct is not an array, it is a method.
It puts all the distinct string values of the strs array into the Results array.
Arrays in C# are alwas with [] never with () as in VB.

Hm, I tried this:

public string[] RemoveDups(string[] s)
        {

            var new_list = s.Distinct();

            return new_list.ToArray();
        }

But it doesnt want to work..

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace DaniWebTest
{
    class Program
    {
        static void Main(string[] args)
        {
            String[] myString = new String[] { "http://google.com", "http://daniweb.com", "http://google.com", "http://webhostingtalkforums.com" };
            IEnumerable<String> disctinctUrl = myString.Distinct();
            foreach (String theString in disctinctUrl)
            {
                Console.WriteLine(theString);
            }
            Console.ReadLine();
        }

    }
}

I just ran the above code and it worked as predicted. IT only printed three URL's, all unique. Converting to an array won't affect the contents so much as to add entries that shouldn't be there.

Please try running that above code and see what happens.

Edited 5 Years Ago by Ketsuekiame: n/a

I modified it to:

public string[] RemoveDups(string[] s)
        {
            IEnumerable<string> rmv = s.Distinct();
            var count = s.Count();
            string[] urls = new String[count];

            int n = 0;
            foreach (string i in rmv)
            {
                urls[n] = i;
                n++;
            }

            return urls;
        }

But it still don't want to work. :/

Sorry it doesn't work.

Here's how I view them:

string[] new_urls = RemoveDups(urls);

                foreach (string u in new_urls)
                {
                    MessageBox.Show(u);
                }

If it isn't working, there is something wrong with your input, they must not be equal. Check *exactly* what is going in. The Distinct method *does* work and I've shown you code which works.

The only thing remaining is the fact that the input is different where you expect it to be the same.

My code looks like this:

private void Form2_Load(object sender, EventArgs e)
        {
            openFileDialog1.Title = "Open Links List";
            openFileDialog1.Filter = "Text File|*.txt";
            openFileDialog1.FileName = "";
            openFileDialog1.ShowDialog();

            var file = openFileDialog1.FileName;

            if (file != "")
            {
                StreamReader read = new StreamReader(file);
                var items = read.ReadToEnd();

                string[] urls = items.Split('\n');
                string[] new_urls = RemoveDups(urls);

                foreach (string u in new_urls)
                {
                    MessageBox.Show(u);
                }


            }
        }

        public string[] RemoveDups(string[] s)
        {
            return s.Distinct().ToArray<String>();
        }
    }

And that text file I'm opening looks like this:

http://www.webmastertalkforums.com/members/jaan.html
http://webhostingtalkforums.com
http://google.com
http://google.com

Ah, I was correct ;)

Your input is being split by '\n'. However, this leaves '\r' on the end of each line except the last line.

So if you breakpoint and look at your urls variable. You will see something like:

http://www.webmastertalkforums.com/members/jaan.html\r
http://webhostingtalkforums.com\r
http://google.com\r
http://google.com

So in computer terms, the last google.com is different to the google.com above it.

I recommend one of the following:
1. Split by "\r\n". This won't work for all files.
2. Split by '\n', replace any '\r' with String.Empty
3. Read each line in separately and remove any instance of '\r' and '\n'

That will solve your problem

Edited 5 Years Ago by Ketsuekiame: n/a

Hmmm.. I tried like this but it fails:

private void Form2_Load(object sender, EventArgs e)
        {
            openFileDialog1.Title = "Open Links List";
            openFileDialog1.Filter = "Text File|*.txt";
            openFileDialog1.FileName = "";
            openFileDialog1.ShowDialog();

            var file = openFileDialog1.FileName;

            if (file != "")
            {
                StreamReader read = new StreamReader(file);
                var items = read.ReadToEnd();

                string[] urls = items.Split('\n');

                var count = urls.Count();

                string[] ur = new String[count];

                int num = 0;
                foreach(string u in urls) 
                {
                    ur[num] = u.Replace('\r', ' ');
                }

                string[] new_urls = RemoveDups(ur);

                foreach (string u in new_urls)
                {
                    MessageBox.Show(u);
                }


            }
        }

        public string[] RemoveDups(string[] s)
        {
            var count = s.Count();

            int i = 0;
            string[] items = new String[count];

            foreach (string u in s)
            {
                try
                {
                    items[i] = u.Replace('\r', ' ');
                }
                catch (Exception e)
                {
                    items[i] = "Failed" + e.ToString();
                }
            }

            return items.Distinct().ToArray<String>();
        }

But now you're replacing '\r' with a space character, which is still different to no character at all. Please read what I put again :P

I dont't know how to remove \r :/

I tried:

string[] urls = items.Split('\r\n');

But it fails..

You need to replace '\r' with String.Empty

'\r\n' fails because '' indicates a single character \r is one character, \n is another character. You would need to use "" to indicate a string (multiple characters) and String.Empty is part of the String class.

Where you split the files, change this to items.Split('\n', StringSplitOptions.RemoveEmptyEntries); as you may have a blank line at the end of your file. (Or in between any other line)

I was hoping you'd figure it out yourself... ;) string[] urls = items.Split(new char[] { '\n' }, StringSplitOptions.RemoveEmptyEntries); is the actual line you need

As far as removing a duplicate array is concerned we need to use the array properties which will do the needful

Try with using HasSet collection:

class Program
    {
        static void Main(string[] args)
        {
            string[] array = { "a", "b", "c", "b", "d", "e", "c" };
            string[] result = RemovingDuplicates(array);
        }

        private static string[] RemovingDuplicates(string[] array)
        {
            HashSet<string> set = new HashSet<string>(array);
            string[] result = new string[set.Count];
            set.CopyTo(result);
            return result;
        }
    }
This article has been dead for over six months. Start a new discussion instead.