1,105,395 Community Members

Insert large CSV into database - 504 error

Member Avatar
f_atencia
Junior Poster in Training
60 posts since Aug 2009
Reputation Points: 0 [?]
Q&As Helped to Solve: 4 [?]
Skill Endorsements: 0 [?]
 
0
 

I have an ASP page written in JavaScript that writes a CSV file into a database table.

while (!oFile.AtEndOfStream)
{
    var s = oFile.ReadLine();
    var sArr = s.split(",");

    var sPostcode = String(sArr[0]).replace(re, "");
    var sSuburb = String(sArr[1]).replace(re, "");

    ...stored procedure to write into database...
}

It works fine for smaller csv files (1,000 lines), but for larger files, it comes across a 504 error. Does anyone know a fix to this?

Help is very much appreciated.

Member Avatar
AleMonteiro
Master Poster
758 posts since Aug 2010
Reputation Points: 115 [?]
Q&As Helped to Solve: 143 [?]
Skill Endorsements: 29 [?]
 
0
 

HTTP Error 504 it's gateway timeout.

How are you trying to insert the data into the database?

You said you are using JS and ASP. The JS is reading the file and sending the data to the ASP server page to insert into the database? Is that right?
If so, then the your http request is pretty big and would take some time to complete.

Assuming you are using IIS, you can try to change the limits of the http request, like this:

<httpRuntime maxRequestLength="102400" executionTimeout="3600" />

This will set the max length to 100MB and the timeout to 60 minutes.

Member Avatar
f_atencia
Junior Poster in Training
60 posts since Aug 2009
Reputation Points: 0 [?]
Q&As Helped to Solve: 4 [?]
Skill Endorsements: 0 [?]
 
0
 

Increased the buffer and seems to be working.

Question Answered as of 1 Year Ago by AleMonteiro
You
This question has already been solved: Start a new discussion instead
Post:
Start New Discussion
Tags Related to this Article