944,047 Members | Top Members by Rank

Ad:
  • MS SQL Discussion Thread
  • Marked Solved
  • Views: 16626
  • MS SQL RSS
You are currently viewing page 1 of this multi-page discussion thread
Sep 22nd, 2006
0

Get data out of excel file stored as an image

Expand Post »
Greetings,

I've posted this question in other news groups (msdn and a few others) without any response. I hope I don't get into any trouble for posting this question here, but I reach out for your help.

I have an excel file stored in a data base table as an image data type. I would like to somehow read the data from the excel file and extract its data into a data base table using t-sql.

I've used sql statements such as the following but this requires the excel file to reside as a physical file on the sql servers file system.

MS SQL Syntax (Toggle Plain Text)
  1. INSERT INTO SpreadsheetDataDump (f1,f2,f3,f4,f5,f6)
  2. SELECT f1,f2,f3,f4,f5,f6
  3. FROM OPENDATASOURCE
  4. ('Microsoft.Jet.OLEDB.4.0','Data Source="D:\spreadsheets\test.xls";Extended Properties="Excel 8.0;IMEX=1;HDR=NO"')
  5. ...WorksheetName$

In my case, I do not want the excel file to reside on the file system because I require many different users to submit excel files. I just cannot afford to expose the file system to so many users, so I've created a web interface for the users to upload the excel file into the database.

As I mentioned, the excel file gets inserted into a table as an image data type. Now I want to select this image data using t-sql and somehow extract the excel spreadsheet data. If I can't do this, can I at least parse the image data into a physical file somewhere on the SQL server? I don't mind parsing it to disk at this point because by now the process is handled by SQL and the end user is out of picture.

Can this be done? Or do I have to parse out the image data as an excel file somewhere on the sql servers file system?

If I have to parse the image data to the file system, can this be done in t-sql? i.e. does t-sql have the ability to stream image data to the file system either natively or by extended stored procedures? (I want to avoid writing a dll wrapper to do this.)

I'm using SQL Server 2000 on Windows 2000 server.

Thanks for any help you can provide.

-Mike
Similar Threads
Reputation Points: 12
Solved Threads: 2
Junior Poster
M_K_Higa is offline Offline
102 posts
since Sep 2006
Feb 7th, 2007
0

Re: Get data out of excel file stored as an image

Ok folks, I didn't get any replies but I see a few views -- 811 at this point. I completed this task a while ago, but forgot about this thread.

If anyone is interested in how I coded this task, please reply to this thread. If I see anyone interested, I'll post my code. Hopefully the folks who viewed this thread are seeking a solution to the same issues I was facing.
Reputation Points: 12
Solved Threads: 2
Junior Poster
M_K_Higa is offline Offline
102 posts
since Sep 2006
Mar 19th, 2007
0

Re: Get data out of excel file stored as an image

Click to Expand / Collapse  Quote originally posted by M_K_Higa ...
Ok folks, I didn't get any replies but I see a few views -- 811 at this point. I completed this task a while ago, but forgot about this thread.

If anyone is interested in how I coded this task, please reply to this thread. If I see anyone interested, I'll post my code. Hopefully the folks who viewed this thread are seeking a solution to the same issues I was facing.
I'm definitely interested in seeing how you accomplished this; I'm in this position myself now and would rather not have to reinvent the wheel if i didn't have to...

thanks!
Reputation Points: 10
Solved Threads: 1
Newbie Poster
juncmayl is offline Offline
1 posts
since Mar 2007
Mar 19th, 2007
0

Re: Get data out of excel file stored as an image

Ok. Hang on; it's going to take a while for me to gather all of the information.
Reputation Points: 12
Solved Threads: 2
Junior Poster
M_K_Higa is offline Offline
102 posts
since Sep 2006
Mar 19th, 2007
0

Re: Get data out of excel file stored as an image

First of all, let me recap the situation. I need an interface that allows users to upload excel data into a sql database.

1) I need a way to get excel data into my sql database.

2) I use the following sql statement to get the excel data into my sql database…

sql Syntax (Toggle Plain Text)
  1. INSERT INTO SpreadsheetDataDump (f1,f2,f3,f4,f5,f6)
  2. SELECT f1,f2,f3,f4,f5,f6
  3. FROM OPENDATASOURCE
  4. ('Microsoft.Jet.OLEDB.4.0','Data Source="D:\spreadsheets\test.xls";Extended Properties="Excel 8.0;IMEX=1;HDR=NO"')
  5. ...WorksheetName$

Keep in mind, in this scenario, the excel file must exist on the sql server.

3) Users need a way to upload the excel file onto the sql server.

4) I cannot grant users direct access to the sql server’s file system.

Task: How does the user get the excel file onto the sql server file system without granting access to the sql server’s file system?

In essence, there are three steps in this process. 1. Get excel file into sql table. 2. Put excel file (stored as Image) onto disc. 3. Insert excel data into sql database.

Step-1) I use a web (asp.net) interface to upload the excel file into a sql table as an Image data type. I’m going to defer you to http://aspnet.4guysfromrolla.com/articles/120606-1.aspx, which explains the process of getting a binary file uploaded into a sql table. You’ll get a lot more stuff out of this article, but we’re concerned with the part about getting the file uploaded into the database. I thought it was a neat article, so I'm giving you the link instead of my code.

I use a similar table structure in my database. This is the table structure I use in my code:

sql Syntax (Toggle Plain Text)
  1. CREATE TABLE BinaryDataFiles(
  2. [PkID] [INT] IDENTITY(1,1) NOT NULL,
  3. [FileName] [nvarchar](255) ,
  4. [FileSize] [REAL],
  5. [FileData] [image],
  6. [UploadDate] [DATETIME] DEFAULT (getdate()),
  7. [ContentType] [nvarchar](30)
  8. )

Step-2) Here’s the part that actually gets the excel file onto the sql server’s file system. I use a console application to dump the excel file (stored as an Image) onto disc. This console application is called from a sql stored procedure.

Here’s the code that streams the image data to file. It compiles to a file called StreamImageData.exe.

c# Syntax (Toggle Plain Text)
  1. using System;
  2. using System.Collections.Generic;
  3. using System.Text;
  4. using System.Data;
  5. using System.Data.SqlClient;
  6. using System.Diagnostics;
  7. using System.IO;
  8. using System.Configuration;
  9.  
  10. namespace StreamImageData {
  11. class Program {
  12. static int Main(string[] args) {
  13. try {
  14. string ConStr = "Data Source=theServer;Initial Catalog=theDatabase;Persist Security Info=True;User ID=theUser;Password=thePassword";
  15. SqlConnection dbConn = new SqlConnection(ConStr);
  16. string sql_str = "select FileSize, FileName, FileData, ContentType from BinaryDataFiles where (PkID = @PkID)";
  17. SqlCommand cmdGetFile = new SqlCommand(sql_str, dbConn);
  18. SqlParameter paramPkID = new SqlParameter("@PkID", SqlDbType.Int);
  19. cmdGetFile.Parameters.Add(paramPkID);
  20. paramPkID.Direction = ParameterDirection.Input;
  21. paramPkID.Value = args[0].ToString(); // The primary key of the blob.
  22. string fileName = args[1].ToString(); // The output file name.
  23. dbConn.Open();
  24. SqlDataReader dr;
  25. dr = cmdGetFile.ExecuteReader();
  26. if(dr.Read()) {
  27. // Create random data to write to the file.
  28. byte[] dataArray = (byte[])dr["FileData"];
  29. using(FileStream fileStream = new FileStream(fileName, FileMode.Create)) {
  30. // Write the data to the file, byte by byte.
  31. for(int i = 0; i < dataArray.Length; i++) {
  32. fileStream.WriteByte(dataArray[i]);
  33. }
  34. }
  35. }
  36. return 0;
  37. } catch(Exception ex) {
  38. Debug.Write(ex.ToString());
  39. return -1;
  40. }
  41. }
  42. }
  43. }

This application expects two arguments – the primary key for the record which stores the image data and an output file name.

I use a stored procedure to execute StreamImageData.exe. The stored procedure goes something like this...

sql Syntax (Toggle Plain Text)
  1. USE [YourDatabase]
  2. GO
  3. CREATE PROCEDURE [dbo].[BinaryDataFiles_StreamImage2File](
  4. @FileName as VARCHAR(256)
  5. ,@ID as VARCHAR(64)
  6. ,@Debug as BIT = 0 --defaults to 0 (off)
  7. )
  8. AS
  9. SET nocount on
  10.  
  11. DECLARE @TheResult INT
  12. DECLARE @command_string as nvarchar(2000)
  13. SET @command_string = '"d:\apps\bin\StreamImageData.exe" '
  14. + @ID + ' ' + @FileName
  15.  
  16. EXEC @TheResult = master.dbo.xp_cmdshell @command_string
  17. print @TheResult
  18.  
  19. DECLARE @ItExists as INT
  20. exec master..xp_fileexist @FileName, @ItExists out
  21. if @ItExists = 0 -- It failed; the file doesn't exist.
  22. BEGIN
  23. -- Dump some kind of fail message
  24. END

Step-3) Now that the excel file is parsed onto the sql server’s file system, you can insert the excel data into your database table.
Once again, I use some code that looks like the following to get the excel data into the sql table.

sql Syntax (Toggle Plain Text)
  1. INSERT INTO SpreadsheetDataDump (f1,f2,f3,f4,f5,f6)
  2. SELECT f1,f2,f3,f4,f5,f6
  3. FROM OPENDATASOURCE
  4. ('Microsoft.Jet.OLEDB.4.0','Data Source="D:\spreadsheets\test.xls";Extended Properties="Excel 8.0;IMEX=1;HDR=NO"')
  5. ...WorksheetName$

And this is what the SpreadsheetDataDump table’s structure looks like…

sql Syntax (Toggle Plain Text)
  1. CREATE TABLE BinaryDataFiles(
  2. [f1] [nvarchar](255) ,
  3. [f2] [nvarchar](255) ,
  4. [f3] [nvarchar](255) ,
  5. [f4] [nvarchar](255) ,
  6. [f5] [nvarchar](255) ,
  7. [f6] [nvarchar](255)
  8. )

These are the fundamental components of getting this task done. I have all three of these steps launch automatically, so to the user, it seems like the spreadsheet data automatically populated into the database. In the end, I also delete the image data out of the table and delete the excel file from the file system. Once the sql data is put into the table, my application had no reason to keep the file or image data.

There's a lot more that I've done in my production code. I didn't post all of it because I don't know what applies to your situation, so instead of cluttering this up with unnecessary code, I choose to provide the basics. I hope this helps - if not, please let me know your specifics and I will answer them as best as I can.

Also note that in this case, the spreadsheet data is being thrown into varchar fields. You may need to tweak your data if you want it in a different data type other that character data.
Reputation Points: 12
Solved Threads: 2
Junior Poster
M_K_Higa is offline Offline
102 posts
since Sep 2006
Nov 2nd, 2007
0

Re: Get data out of excel file stored as an image

Is there any way we can specify the file name and location in Datasource as a variable?

Like can we replace the following with a variable?

'Data Source="D:\spreadsheets\test.xls";Extended Properties="Excel 8.0;IMEX=1;HDR=NO"'
Reputation Points: 10
Solved Threads: 1
Newbie Poster
tatini is offline Offline
6 posts
since Nov 2007
Nov 2nd, 2007
0

Re: Get data out of excel file stored as an image

Yes -- you can build the statement and then use the execute command to execute the statement.

I do something similar to the following in my actual code...

SQL Syntax (Toggle Plain Text)
  1. DECLARE @text_01 as VARCHAR(1000)
  2. DECLARE @text_02 as VARCHAR(500)
  3. DECLARE @spreadsheet as VARCHAR(250)
  4. DECLARE @worksheet as VARCHAR(250)
  5. DECLARE @sql_statement as VARCHAR(2000)
  6.  
  7. SET @spreadsheet = 'c:\path\test.xls'
  8. SET @worksheet = 'Sheet1'
  9.  
  10. SET @text_01 = 'INSERT INTO SpreadsheetDataDump (f1,f2,f3,f4,f5,f6) SELECT f1,f2,f3,f4,f5,f6 '
  11.  
  12. SET @text_02 = ' FROM OPENDATASOURCE(''Microsoft.Jet.OLEDB.4.0'',''Data Source="'
  13. + @spreadsheet + '";Extended Properties="Excel 8.0;IMEX=1;HDR=NO"'')...'
  14. + @worksheet + '$'
  15.  
  16.  
  17. SET @sql_statement = @text_01 + @text_02
  18.  
  19. exec (@sql_statement )

In my actual code, the @spreadsheet and @worksheet variables are passed in as parameters. I've declared them in this sample code for clarity.

Hope this helps.
Reputation Points: 12
Solved Threads: 2
Junior Poster
M_K_Higa is offline Offline
102 posts
since Sep 2006
Nov 2nd, 2007
0

Re: Get data out of excel file stored as an image

thanks a lot.
Started on the same path before posting that question. Acheieved the result in a similar way.
Reputation Points: 10
Solved Threads: 1
Newbie Poster
tatini is offline Offline
6 posts
since Nov 2007
Nov 2nd, 2007
0

Re: Get data out of excel file stored as an image

All is done and working but when i import the data from the excel to sql table, i see a change in the data.
the excel sheet has a column with numbers stored as text. they have a 0 as the first digit. this is gone once the sheet is imported into the column with data type varchar.

is there a way out of this?
Reputation Points: 10
Solved Threads: 1
Newbie Poster
tatini is offline Offline
6 posts
since Nov 2007
Nov 2nd, 2007
0

Re: Get data out of excel file stored as an image

Yes -- this is a hurdle that we have to face. You have to wirte a second routine that will cast your data into correct data types.

I tweak my data by getting the table containing the spreadsheet data and inserting it into a second table. The second table is defined with correct data types, so I have to cast anything that is not a character data type.
Reputation Points: 12
Solved Threads: 2
Junior Poster
M_K_Higa is offline Offline
102 posts
since Sep 2006

This thread is solved

Either the thread starter or a moderator has marked this thread as solved. You can most likely trust the responses and answers given. There is most likely no reason for any further responses to be posted here. If you have a related question, please start a new thread in this forum instead.

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in MS SQL Forum Timeline: Odd issue with LIKE operator
Next Thread in MS SQL Forum Timeline: Identity column





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC