progressbar timing

[procomp65]

Hi
I have a data logger that records different lenghts of data. I download it in my application and create graphs.
I need to show a progressbar to indicate how long the download is going to take. I get data from the logger to indicate how long the data run was. I have timed it and it takes about 1second to download 1.68 minutes of data.
If the logger run was 44minutes, it takes 26 seconds to download.

when I set my progressBar.Max = 100 and timer,Interval = 260 milliseconds, the progressBar finishes in about 6 second instead of 26 seconds.
What am I doing wrong?

private void progressTimer(object sender, EventArgs e)
        {            
            subData1 = startValues1.Split(new char[] { ' ' });
            double progressTime = Convert.ToDouble(subData1[13]);            
            int tValue = Convert.ToInt16(progressTime /1.68);
            timer1.Interval = tValue;            
            // Increment the value of the ProgressBar a value of one each time.
            progressBar1.Increment(1);            
            // Determine if we have completed by comparing the value of the Value property to the Maximum value.
            if (progressBar1.Value == progressBar1.Maximum)
                // Stop the timer.
                timer1.Stop();

            
        }

[DdoubleD]

You cannot simply estimate time based on samples and, as I am sure you are aware, no progress for downloading from the web is extremely accurate. Given that is true all around, user's expect there to be plus/minus whatever they have experienced... In light of this, the best you can do is attempt to recalculate the download time at intervals based on the current sampling of progress...

Here are some links you might find useful:

http://stackoverflow.com/questions/1587333/c-calculate-download-upload-time-using-network-bandwidth
http://stackoverflow.com/questions/933242/smart-progress-bar-eta-computation
http://forums.asp.net/p/1438079/3280750.aspx#3280750

Also note that since the BackGroundWorker will only update when it receives a slice of CPU time, the thread might not be processing in exact increments.

[procomp65]

thanks for the reply. I know I will not get it exactly correct, I would just like to get the time a bit more precise. It is way out. I used timespan with the timer before and that solved most issues, but I can't find the code and I can't seem to figure it out now. If I use a while loop and thread.sleep it is almost dead on, but my main application locks up and do all sorts of funny things.
It seems as if the progress bar with the above code starts at the correct timer interval but then accelerate as it progress.
That does not make sense. If it waits for a cpu slice or tread in que the time should decrease and the bar should slow down. maybe a bug in .net4 beta2

[sknake]

Ehhhh this is a very bad way to go about calculating progress.

You should not estimate the product of any work based on the download time required except the download itself. If your internet connection is slow or the webserver is a little slower than usual then it will take slightly longer to download but the same amount of time to process the downloaded data. You should use the "file size" or size of data in memory to estimate the processing time required. As the file downloads you can recalculate every second based on the amount of data downloaded thus far.

You could also use another thread or pull one from the thread pool to increment your progress bar timer.

Also with the timer if you are executing every 260ms then you may have a problem. It is possible for a timer to call its' interval-delegate or tick event (depending on the type of timer used) and if your code takes longer than 260ms to run then you can very easily have the same code running in two different threads. That being said I didn't see any effort in your code to synchronize threads -- and that might lead to very hard to diagnose problems.

Can you upload your project?

[sknake]

I started with an example from MSDN and a few changes. This class will give you a downloader (http) that supports reporting progress and the finished buffer. In this case all of the download goes to a memorystream which is not safe as the downloaded file could be very large. You may consider changing this to a FileStream writing to a temporary file.

Downloader.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.Web;
using System.Runtime.Remoting.Messaging;
using Microsoft.Internal;
using System.Threading;
using System.IO;

namespace daniweb.progress2
{
  public class Downloader
  {
    const int BUFFER_SIZE = 1024;
    const int DefaultTimeout = 2 * 60 * 1000; //2 min

    private Uri uri;

    public event DownloadProgressChanged OnDownloadProgressChanged;
    public event DownloadFinished OnDownloadFinished;

    private Downloader()
    {
    }
    public Downloader(string URL)
      : this(new Uri(URL, UriKind.Absolute))
    {
    }
    public Downloader(Uri URL)
      : this()
    {
      this.uri = URL;
    }

    public void Run()
    {
      HttpWebRequest req = (HttpWebRequest)WebRequest.Create(this.uri);
      req.Method = "GET";
      RequestState reqState = new RequestState(req);
      Action<RequestState> del = new Action<RequestState>(RunDelegate);
      del.BeginInvoke(reqState, new AsyncCallback(RunDelegateCallback), reqState);
    }

    private void RunDelegate(RequestState reqState)
    {
      //Hang on to the async result so we can implement a download timeout
      IAsyncResult ar = (IAsyncResult)reqState.request.BeginGetResponse(new AsyncCallback(ResponseCallback), reqState);
      //Implement a timeout
      ThreadPool.RegisterWaitForSingleObject(ar.AsyncWaitHandle, new WaitOrTimerCallback(TimeoutCallback), reqState.request, DefaultTimeout, true);
      //Block until we're finished running
      reqState.mre.WaitOne();
    }

    private void RunDelegateCallback(IAsyncResult ar)
    {
      AsyncResult result = (AsyncResult)ar;
      Action<RequestState> del = (Action<RequestState>)result.AsyncDelegate;
      try
      {
        del.EndInvoke(ar);
      }
      catch (Exception ex)
      {
        //Dont know what you want to do here
        Console.WriteLine(ex.Message);
#if (DEBUG)
        if (System.Diagnostics.Debugger.IsAttached)
          System.Diagnostics.Debugger.Break();
#endif
      }
      RequestState reqState = (ar.AsyncState as RequestState);
      if (reqState != null)
      {
        try
        {
          reqState.response.Close();
        }
        catch { }
      }
    }

    private void ResponseCallback(IAsyncResult ar)
    {
      RequestState reqState = (RequestState)ar.AsyncState;
      try
      {
        HttpWebRequest myHttpWebRequest = reqState.request;
        reqState.response = (HttpWebResponse)myHttpWebRequest.EndGetResponse(ar);

        // Read the response into a Stream object.
        Stream responseStream = reqState.response.GetResponseStream();
        reqState.streamResponse = responseStream;

        // Begin the Reading of the contents of the HTML page and print it to the console.
        IAsyncResult asynchronousInputRead = responseStream.BeginRead(reqState.BufferRead, 0, BUFFER_SIZE, new AsyncCallback(ReadCallBack), reqState);
        return;
      }
      catch (WebException ex)
      {
        //Dont know what you want to do here
        Console.WriteLine(ex.Message);
#if (DEBUG)
        if (System.Diagnostics.Debugger.IsAttached)
          System.Diagnostics.Debugger.Break();
#endif
      }
      reqState.mre.Set();
    }

    private void ReadCallBack(IAsyncResult asyncResult)
    {
      RequestState reqState = (RequestState)asyncResult.AsyncState;

      try
      {
        Stream responseStream = reqState.streamResponse;
        int read = responseStream.EndRead(asyncResult);
        // Read the HTML page and then print it to the console.
        if (read > 0)
        {
          reqState.requestData.Write(reqState.BufferRead, 0, read);
          IAsyncResult asynchronousResult = responseStream.BeginRead(reqState.BufferRead, 0, BUFFER_SIZE, new AsyncCallback(ReadCallBack), reqState);
          var del = this.OnDownloadProgressChanged;
          if (del != null)
          {
            ProgressCancelEventArgs args = new ProgressCancelEventArgs(0, reqState.response.ContentLength, reqState.requestData.Length);
            del.Invoke(this, args);
            if (args.Cancel)
            {
              responseStream.Close();
            }
          }
          return;
        }
        else
        {
          var del = this.OnDownloadFinished;
          if (del != null)
          {
            del.Invoke(this, new DownloadFinishedEventArgs(reqState.requestData));
          }
          responseStream.Close();
        }

      }
      catch (Exception ex)
      {
        //Dont know what you want to do here
        Console.WriteLine(ex.Message);
#if (DEBUG)
        if (System.Diagnostics.Debugger.IsAttached)
          System.Diagnostics.Debugger.Break();
#endif
      }
      reqState.mre.Set();
    }


    private class RequestState
    {
      // This class stores the State of the request.
      const int BUFFER_SIZE = 1024;
      public MemoryStream requestData;
      public byte[] BufferRead;
      public HttpWebRequest request;
      public HttpWebResponse response;
      public Stream streamResponse;
      public ManualResetEvent mre;
      public RequestState()
      {
        BufferRead = new byte[BUFFER_SIZE];
        requestData = new MemoryStream();
        mre = new ManualResetEvent(false);
      }
      public RequestState(HttpWebRequest request)
        : this()
      {
        this.request = request;
      }
    }
    private static void TimeoutCallback(object state, bool timedOut)
    {
      if (timedOut)
      {
        HttpWebRequest request = (state as HttpWebRequest);
        if (request != null)
        {
          request.Abort();
        }
      }
    }

  }

  public delegate void DownloadProgressChanged(object sender, ProgressCancelEventArgs e);
  public delegate void DownloadFinished(object sender, DownloadFinishedEventArgs e);
  public class DownloadFinishedEventArgs : EventArgs
  {
    private Stream _stream;
    public Stream Data { get { return _stream; } }
    private DownloadFinishedEventArgs()
      : base()
    {
    }
    public DownloadFinishedEventArgs(Stream data)
      : this()
    {
    }
  }
  public class ProgressEventArgs : EventArgs
  {
    private long min;
    private long max;
    private long pos;

    public long Minimum { get { return min; } }
    public long Maximum { get { return max; } }
    public long Position { get { return pos; } }

    public ProgressEventArgs()
      : base()
    {
    }
    public ProgressEventArgs(long Min, long Max, long Pos)
      : this()
    {
      this.min = Min;
      this.max = Max;
      this.pos = Pos;
    }
  }
  public class ProgressCancelEventArgs : ProgressEventArgs
  {
    public bool Cancel { get; set; }
    public ProgressCancelEventArgs()
      : base()
    {
    }
    public ProgressCancelEventArgs(long Min, long Max, long Pos)
      : base(Min, Max, Pos)
    {
    }
  }
}

Calling it:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;

namespace daniweb.progress2
{
  public partial class Form1 : Form
  {
    public Form1()
    {
      InitializeComponent();
    }

    private void buttonStart_Click(object sender, EventArgs e)
    {
      const string url = @"http://mozilla-mirror.3347.voxcdn.com/pub/mozilla.org/firefox/releases/3.5.5/win32/en-US/Firefox%20Setup%203.5.5.exe";
      Downloader d = new Downloader(url);
      d.OnDownloadFinished += new DownloadFinished(d_OnDownloadFinished);
      d.OnDownloadProgressChanged += new DownloadProgressChanged(d_OnDownloadProgressChanged);
      d.Run();
    }

    void d_OnDownloadProgressChanged(object sender, ProgressCancelEventArgs e)
    {
      if (this.progressBarDownloadProgress.InvokeRequired)
      {
        this.progressBarDownloadProgress.Invoke(new MethodInvoker(
          delegate()
          {
            d_OnDownloadProgressChanged(sender, e);
          }));
      }
      else
      {
        progressBarDownloadProgress.Value = (e.Maximum > 0 ? Convert.ToInt32(e.Position * 100.0 / e.Maximum) : 0);
      }
    }

    void d_OnDownloadFinished(object sender, EventArgs e)
    {
      progressBarDownloadProgress.Value = progressBarDownloadProgress.Maximum;
    }

    private void Form1_Load(object sender, EventArgs e)
    {
      progressBarDownloadProgress.Minimum = 0;
      progressBarDownloadProgress.Maximum = 100;
      progressBarDownloadProgress.Value = 0;
    }
  }
}

[edit]
One more thing -- The progress is calculated by the response's length IF the webserver provided it. Sometimes the server does not send the response length so the progress in the code above will always be 0 until the download finishes. You should use a marquee progress bar if the denominator is zero in the form's progress changed event.
[/edit]

[procomp65]

Thanks Sknake
I will try and incorporate your code in my app. I am actually downloading data from a piece of hardware at a fixed 9600 baud. I can see beforehand how much data there is stored before downloading and based on that I tried to work out the time nessecary to download and display that via a progressbar. What I still don't understand is that the progress bar start slow and accelerate towards the end, why is that?

Here are more detailed code

private void port_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {            
            try
            {
                serialPort1.BaseStream.ReadTimeout = 1000;                
                //serialPort1.ReadTimeout = 3000;
                Thread.Sleep(500);
                startValues1 = serialPort1.ReadTo(ASCIIEncoding.ASCII.GetString(new byte[] { 5 }));
                //ValidInput(startValues1);                 
                this.BeginInvoke(new EventHandler(progressTimer));
                startValues2 = serialPort1.ReadTo(ASCIIEncoding.ASCII.GetString(new byte[] { 5 }));
                // ValidInput(startValues2);                
                
                startValues3 = serialPort1.ReadTo(ASCIIEncoding.ASCII.GetString(new byte[] { 5 }));
                //ValidInput(startValues3);                            
                tempData = serialPort1.ReadTo(ASCIIEncoding.ASCII.GetString(new byte[] { 5 }));
                //ValidInput(tempData);
                Thread.Sleep(500);
                startValues4 += serialPort1.ReadExisting();
                //ValidInput(startValues4);
                this.BeginInvoke(new EventHandler(processData));
            }
            catch(IOException)
            {
                MessageBox.Show("PUR is busy downloading, please wait!");
                serialPort1.DiscardInBuffer();
                this.BeginInvoke(new EventHandler(errorHandling));
            }
            catch (TimeoutException)
            {
                MessageBox.Show("PUR disconnected,switched off or wrong port selected");
                serialPort1.DiscardInBuffer();
                this.BeginInvoke(new EventHandler(errorHandling));
               
            }

            catch (IndexOutOfRangeException)
            {
                MessageBox.Show(" Data corrupt or recorder switched off during download");
                serialPort1.DiscardInBuffer();
                this.BeginInvoke(new EventHandler(errorHandling));
            }

            catch (InvalidOperationException)
            {
                MessageBox.Show("Com Port disconnected");
                serialPort1.DiscardInBuffer();
                this.BeginInvoke(new EventHandler(errorHandling));
            }
               
                
        }
       
        #endregion        

        #region Processdata
                
        private void progressTimer(object sender, EventArgs e)
        {            
            subData1 = startValues1.Split(new char[] { ' ' });
            double progressTime = Convert.ToDouble(subData1[13]);            
            int tValue = Convert.ToInt16( progressTime /1.68)  ;
            timer1.Interval = tValue;
            
            // Increment the value of the ProgressBar a value of one each time.
            progressBar1.Increment(1);            
            // Determine if we have completed by comparing the value of the Value property to the Maximum value.
            if (progressBar1.Value == progressBar1.Maximum)
            {
                // Stop the timer.
                timer1.Stop();
                
            }
            
        }

[Geekitygeek]

without knowing what values you are pulling from your serial port its hard to say. But your interv al is being set based on the value in subData1[13]. If the value in that array item gets smaller the interval will get smaller and the progress bar will accelerate. Why are you basing the time on a value in the data? Shouldnt it be based on the length of the data?
I'd suggest throwing a breakpoint in at line 60 and tracking the values in subData1, progressTime and tValue to see if they are the values you are expecting.

[procomp65]

Hi Ryshad
Thanks for your reply. The value I get over the serial port is the total time in minutes that the recorder has logged data. I have timed the port and got to the conclusion that over 9600 baud It takes 1 second to download 1.68 minutes worth of recorder data. On this value I tried to time the progress bar.
I am really at a dead stop now and will appreciate it if you can think of another more effiecent way or method to do the progress timing.
I have changed the code and method of calculation. It is a bit better now but still the timer accelerates.
I have set the timer to a value of 1000 milliseconds. I take value [13] of the array which is the total recording time of the logger. Divide it by the 1.68 minutes to see howmany seconds I need to download over the serial port. I now set the progressbar.maximum to the seconds needed to download and time each increment at 1000ms. In theory it should work now but it does not.

[procomp65]

This is my code now

private void progressTimer(object sender, EventArgs e)
        {            
            subData1 = startValues1.Split(new char[] { ' ' });
            double progressTime = Convert.ToDouble(subData1[13]);
            double progressTime1 = progressTime * 0.1;
            int tValue = Convert.ToInt16( progressTime1 /1.68)  ;
            timer1.Interval = 1000;
            progressBar1.Maximum = tValue;            
            //Increment the value of the ProgressBar a value of one each time.
            progressBar1.Increment(1);            
            // Determine if we have completed by comparing the value of the Value property to the Maximum value.
            if (progressBar1.Value == progressBar1.Maximum)
           {
                // Stop the timer.
                timer1.Stop();
                
           }
            
        }


private void timer1_Tick(object sender, EventArgs e)
        {
            timer1.Tick += new EventHandler(progressTimer);
        }

Please check and see if I am using the timer correctly. Maybe progressTimer() does not use the timer.tick

[Geekitygeek]

Your approaching the timer all wrong.
Every time it ticks you are adding a new event handler. When you add the Event Handler, it doesnt just run the method once, it tells the event dispatcher to run it every time the event occurs. The second time it ticks, the event is added again. This doesnt overwrite the first event handler, it adds it a second time. So the next time the timers Tick event is raised your progressTimer method will run twice for that single tick.
This is why your bar accelerates, each tick adds an extra event handler so each tick the bar gets incremented more and more times.

The correct way to use the timer would be to initialise everything first, so set your progress bar's maximum, set the interval on the timer, etc. Then call Start(). Inside the Tick event you should only process code that is affacted by time. In this case the increment on the progressbar. You can then do the check to see if the timer needs to stop:

private void startTimer(object sender, EventArgs e)
        {            
            subData1 = startValues1.Split(new char[] { ' ' });
            double progressTime = Convert.ToDouble(subData1[13]);
            double progressTime1 = progressTime * 0.1;
            int tValue = Convert.ToInt16( progressTime1 /1.68)  ;
            timer1.Interval = 1000;
            progressBar1.Maximum = tValue;  
            timer1.Start();
        }


private void timer1_Tick(object sender, EventArgs e)
        {
            //Increment the value of the ProgressBar a value of one each time.
            progressBar1.Increment(1);            
            // Determine if we have completed by comparing the value of the Value property to the Maximum value.
            if (progressBar1.Value == progressBar1.Maximum)
           {
                // Stop the timer.
                timer1.Stop();
           }
        }

[procomp65]

Thank Ryshad
This is the second time that you bail me out. Really appreciate it