We're a community of 1076K IT Pros here for help, advice, solutions, professional growth and fun. Join us!
1,075,619 Members — Technology Publication meets Social Media
Username:
Password:
Lost login information?
Start New Discussion Reply to this Discussion

Insert large CSV into database - 504 error

I have an ASP page written in JavaScript that writes a CSV file into a database table.

while (!oFile.AtEndOfStream)
{
    var s = oFile.ReadLine();
    var sArr = s.split(",");

    var sPostcode = String(sArr[0]).replace(re, "");
    var sSuburb = String(sArr[1]).replace(re, "");

    ...stored procedure to write into database...
}

It works fine for smaller csv files (1,000 lines), but for larger files, it comes across a 504 error. Does anyone know a fix to this?

Help is very much appreciated.

2
Contributors
2
Replies
1 Day
Discussion Span
5 Months Ago
Last Updated
3
Views
Question
Answered
f_atencia
Junior Poster in Training
59 posts since Aug 2009
Reputation Points: 10
Solved Threads: 4
Skill Endorsements: 0

HTTP Error 504 it's gateway timeout.

How are you trying to insert the data into the database?

You said you are using JS and ASP. The JS is reading the file and sending the data to the ASP server page to insert into the database? Is that right?
If so, then the your http request is pretty big and would take some time to complete.

Assuming you are using IIS, you can try to change the limits of the http request, like this:

<httpRuntime maxRequestLength="102400" executionTimeout="3600" />

This will set the max length to 100MB and the timeout to 60 minutes.

AleMonteiro
Master Poster
752 posts since Aug 2010
Reputation Points: 129
Solved Threads: 140
Skill Endorsements: 23

Increased the buffer and seems to be working.

f_atencia
Junior Poster in Training
59 posts since Aug 2009
Reputation Points: 10
Solved Threads: 4
Skill Endorsements: 0
Question Answered as of 5 Months Ago by AleMonteiro

This question has already been solved: Start a new discussion instead

Post: Markdown Syntax: Formatting Help
 
You
 
© 2013 DaniWeb® LLC
Page rendered in 0.3063 seconds using 2.67MB