L
L
lPolar2015-02-24 20:38:31
Python
lPolar, 2015-02-24 20:38:31

How to quickly port .csv to Oracle?

Hello!
Question - what are the RODBC/Pandas+Sqlalchemy productivity tools for sending .csv files to Oracle?
Let's say it looks like this in Python:

import pandas as pd
from sqlalchemy import create_engine
e = create_engine("oracle://user:[email protected]")
df = pd.read_csv("data.csv")
df.to_sql("data",e)

And like this - in R:
library(RODBC)
conn = odbcConnect("dsn","user","pass")
tab = read.csv("data.csv")
sqlSave(conn,tab,"data",fast=TRUE)

However, both work quite slowly - a table of 100k rows and 120 columns is loaded in pandas - 20 minutes, in R - 20 minutes.
Is there any way to improve performance?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
P
protven, 2015-02-24
@protven

Use the tools provided by Oracle itself. In this case, it is best to use the SQL*Loader utility .
In my experience, if there are no locks on the table and the database is running on a relatively fast server, 100k rows should be inserted in 2-4 minutes.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question