Long running processes

[Fortinbra]

My web application has the ability to process data from word docs and PDF files. We recently had a client who attempted to run files that are of a significantly larger size than anything we tested with. 50MB-500MB PDFs compared to our test files of 1MB-30MB files. If they don't get a timeout error, they get a System.OutofMemoryException. I know what to do about the out of memory exception, but I can't tell the client they can't use those files.

I'd like to run the processing of the files as a background process, but I need to show progress on the files to the user. We're using Aspose.Total and .NET 4, what are everyone else's thoughts on how to accomplish this? We use MSSQL as the backend, and out application supports 1000+ concurrent users, so performance is key.

[riteshbest]

My web application has the ability to process data from word docs and PDF files. We recently had a client who attempted to run files that are of a significantly larger size than anything we tested with. 50MB-500MB PDFs compared to our test files of 1MB-30MB files. If they don't get a timeout error, they get a System.OutofMemoryException. I know what to do about the out of memory exception, but I can't tell the client they can't use those files.

I'd like to run the processing of the files as a background process, but I need to show progress on the files to the user. We're using Aspose.Total and .NET 4, what are everyone else's thoughts on how to accomplish this? We use MSSQL as the backend, and out application supports 1000+ concurrent users, so performance is key.

Hi Fortinbra,
If you want to perform background processing, implement background worker.
Also, you can show progress:

worker.WorkerReportsProgress = true;

and then implement ProgressChanged event handler to indicate the progress.

private void bw_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
    //Your Code
}

Please find below link.
Check Link: Implement Background Worker

[Fortinbra]

We have something similar in place where the event handler updates a database, and we have an AJAX enabled data grid that is updating on a 5sec timer from that database. But in this case I have nothing in the database I can update.

I'm not sure how to display the progress to the user, when everything is happening server side, and the server can't say anything to the client except as a response.

[Fortinbra]

From another source, it was recommended that we create an entire separate process, outside the website, that shares a database with the site, and does all the long processing. That way it can be offloaded to a separate server all together.

[riteshbest]

From another source, it was recommended that we create an entire separate process, outside the website, that shares a database with the site, and does all the long processing. That way it can be offloaded to a separate server all together.

Yes you can do this with a windows service.
Basically the idea is..

•You submit a request for processing the doc/pdf file in database table with some status as not started.
•Then your windows service picks up the request from database table which are not started and update them as in progress status.
•Once the processing is complete succesfully /unsuccesfuly your service updated the database table with status as Completed / Failed.

but you will have problems with that:
•what if the sql fails? should there be any response to the client
•if it fails, how do you ensure the file on a later request

Background processing is good way to keep track of current status. You can also, cancel operation if needed and show status as operation aborted.
You can show progress using status label.
Check: http://forums.asp.net/t/1266633.aspx/1