 |  |  |  |  |  |  |  |
Transfer csv files into a mysql database
Please support our MySQL advertiser: PostgreSQL or MySQL? Compare and contrast the two most popular open source databases
 |
Hey everybody,
My company decided to improve the management of some data contained in csv files by loading their content in a mysql database.
I heard that this is part of the process called ETL. Is that right?
If that's so, which tool can I use in order to do what I describe above.
thanks.
My company decided to improve the management of some data contained in csv files by loading their content in a mysql database.
I heard that this is part of the process called ETL. Is that right?
If that's so, which tool can I use in order to do what I describe above.
thanks.
Last edited by Kei; Aug 6th, 2008 at 5:58 pm.
Hi kei,
As a matter of fact you have two ways to do it depending on your technical background. You can either write a script yourself or use an ETL tool that will write the most part of the script for you.
Are you more of a technical user or more of a business user ?
As a matter of fact you have two ways to do it depending on your technical background. You can either write a script yourself or use an ETL tool that will write the most part of the script for you.
Are you more of a technical user or more of a business user ?
Talend, provider of open source data integration software.
•
•
Join Date: Dec 2004
Posts: 4,185
Reputation: 
Solved Threads: 481
MySQL documentation - LOAD DATA INFILE Syntax. Search page for references of csv for faster location
Learn to see in another's calamity the ills which you should avoid.
Publilius Syrus
(~100 BC)
LJC - London Java Community, Graduate & Undergraduate Software Development Community, JAVAWUG (Java Web User Group), The London Android Group
Publilius Syrus
(~100 BC)
LJC - London Java Community, Graduate & Undergraduate Software Development Community, JAVAWUG (Java Web User Group), The London Android Group
Indeed, if it is a one-shot project, the fastest way to do it is indeed with :
LOAD DATA INFILE 'path/file.csv' INTO TABLE your_table;
However, if you plan on doing it on a regular basis (if you receive, these very csv files regularly) you may want to use, indeed, an ETL tool.
LOAD DATA INFILE 'path/file.csv' INTO TABLE your_table;
However, if you plan on doing it on a regular basis (if you receive, these very csv files regularly) you may want to use, indeed, an ETL tool.
Talend, provider of open source data integration software.
Thanks gtd and peter.
As you said gtd, I would like to do it 3 or 4 times a week.
I will do with "load data infile" now and I will wait for your other suggestion concerning the etl tool.
Why would it be better to use an etl tool if I plan to do it "on a regular basis" ?
As you said gtd, I would like to do it 3 or 4 times a week.
I will do with "load data infile" now and I will wait for your other suggestion concerning the etl tool.
Why would it be better to use an etl tool if I plan to do it "on a regular basis" ?
It would not be better but simpler, because you can automate the task with an etl tool, thus allowing you to run the task on a user-defined basis.
Talend, provider of open source data integration software.
•
•
Join Date: Dec 2004
Posts: 4,185
Reputation:
Solved Threads: 481
This may help you with choosing ETL tool
Learn to see in another's calamity the ills which you should avoid.
Publilius Syrus
(~100 BC)
LJC - London Java Community, Graduate & Undergraduate Software Development Community, JAVAWUG (Java Web User Group), The London Android Group
Publilius Syrus
(~100 BC)
LJC - London Java Community, Graduate & Undergraduate Software Development Community, JAVAWUG (Java Web User Group), The London Android Group
Thanks for the link peter, it is very interesting.
but the full study need to be purchased...
Anyway, this survey seems a bit old (more than 2 years ago) and since I can't afford to dedicate a budget to the operations mentioned above, I have to choose an open source etl tool which pretty much narrows down the choice.
I will think about it and give my feedback afterwards.
but the full study need to be purchased...
Anyway, this survey seems a bit old (more than 2 years ago) and since I can't afford to dedicate a budget to the operations mentioned above, I have to choose an open source etl tool which pretty much narrows down the choice.
I will think about it and give my feedback afterwards.
|
Similar Threads
Other Threads in the MySQL Forum
- Previous Thread: SELECT DISTINCT only returning ONE record
- Next Thread: how to integrate mysql database server with apache server
| Thread Tools | Search this Thread |
You need to join our community in order to build your list of favorite forums. You can visit the forum index for a listing of all forums.
Related Forum Features
Latest MySQL Posts
- Today's Posts
- Unanswered Threads
agplv3 alfresco amazon api artisticlicense aws bizspark breathalyzer camparingtocolumns changingprices cmg communityjournalism contentmanagement contractors copyright count court crm database design developer development distinct drupal dui ec2 email enterprise eudora facebook form foss gartner gnu government gpl greenit groupware hiring hyperic images innerjoins insert ip joebrockmeier join keyword keywords kickfire laptop law legal license licensing linux maintenance managing mariadb matchingcolumns metron micromanage microsoft microsoftexchange mindtouch montywidenius mozilla multiple music mysql mysqlcolumnupdating mysqldatetimeordermax() mysqlindex mysqlinternalqueries mysqlquery mysqlsearch news open-xchange opendatabasealliance opengovernment opensource oracle penelope priceupdating query referencedesign reorderingcolumns resultset saas select sharepoint simpledb sourcecode spotify sql sugarcrm syntax techsupport thunderbird transparency virtualization
