如何将CSV文件数据导入PostgreSQL表

步骤:

Need to connect a PostgreSQL database in the terminal psql -U postgres -h localhost Need to create a database create database mydb; Need to create a user create user siva with password 'mypass'; Connect with the database \c mydb; Need to create a schema create schema trip; Need to create a table create table trip.test(VendorID int,passenger_count int,trip_distance decimal,RatecodeID int,store_and_fwd_flag varchar,PULocationID int,DOLocationID int,payment_type decimal,fare_amount decimal,extra decimal,mta_tax decimal,tip_amount decimal,tolls_amount int,improvement_surcharge decimal,total_amount ); Import csv file data to postgresql COPY trip.test(VendorID int,passenger_count int,trip_distance decimal,RatecodeID int,store_and_fwd_flag varchar,PULocationID int,DOLocationID int,payment_type decimal,fare_amount decimal,extra decimal,mta_tax decimal,tip_amount decimal,tolls_amount int,improvement_surcharge decimal,total_amount) FROM '/home/Documents/trip.csv' DELIMITER ',' CSV HEADER; Find the given table data select * from trip.test;

正如Paul提到的，导入在pgAdmin中起作用:

右键单击表→导入

选择一个本地文件，格式和编码。

这是一个德文pgAdmin GUI截图:

使用DbVisualizer也可以做类似的事情(我有许可证，但不确定是否有免费版本)。

右键单击表→导入表数据…

在Python中，你可以使用这段代码自动创建带有列名的PostgreSQL表:

import pandas, csv

from io import StringIO
from sqlalchemy import create_engine

def psql_insert_copy(table, conn, keys, data_iter):
    dbapi_conn = conn.connection
    with dbapi_conn.cursor() as cur:
        s_buf = StringIO()
        writer = csv.writer(s_buf)
        writer.writerows(data_iter)
        s_buf.seek(0)
        columns = ', '.join('"{}"'.format(k) for k in keys)
        if table.schema:
            table_name = '{}.{}'.format(table.schema, table.name)
        else:
            table_name = table.name
        sql = 'COPY {} ({}) FROM STDIN WITH CSV'.format(table_name, columns)
        cur.copy_expert(sql=sql, file=s_buf)

engine = create_engine('postgresql://user:password@localhost:5432/my_db')

df = pandas.read_csv("my.csv")
df.to_sql('my_table', engine, schema='my_schema', method=psql_insert_copy)

它的速度也相对较快。我可以在大约4分钟内导入330多万行。

如果你需要一个简单的机制来导入文本/解析多行CSV内容，你可以使用:

CREATE TABLE t   -- OR INSERT INTO tab(col_names)
AS
SELECT
   t.f[1] AS col1
  ,t.f[2]::int AS col2
  ,t.f[3]::date AS col3
  ,t.f[4] AS col4
FROM (
  SELECT regexp_split_to_array(l, ',') AS f
  FROM regexp_split_to_table(
$$a,1,2016-01-01,bbb
c,2,2018-01-01,ddd
e,3,2019-01-01,eee$$, '\n') AS l) t;

DBFiddle演示

如何将CSV文件数据导入PostgreSQL表

步骤:

Need to connect a PostgreSQL database in the terminal psql -U postgres -h localhost Need to create a database create database mydb; Need to create a user create user siva with password 'mypass'; Connect with the database \c mydb; Need to create a schema create schema trip; Need to create a table create table trip.test(VendorID int,passenger_count int,trip_distance decimal,RatecodeID int,store_and_fwd_flag varchar,PULocationID int,DOLocationID int,payment_type decimal,fare_amount decimal,extra decimal,mta_tax decimal,tip_amount decimal,tolls_amount int,improvement_surcharge decimal,total_amount ); Import csv file data to postgresql COPY trip.test(VendorID int,passenger_count int,trip_distance decimal,RatecodeID int,store_and_fwd_flag varchar,PULocationID int,DOLocationID int,payment_type decimal,fare_amount decimal,extra decimal,mta_tax decimal,tip_amount decimal,tolls_amount int,improvement_surcharge decimal,total_amount) FROM '/home/Documents/trip.csv' DELIMITER ',' CSV HEADER; Find the given table data select * from trip.test;

看看这篇短文吧。

解决方案如下:

创建你的表:

CREATE TABLE zip_codes
(ZIP char(5), LATITUDE double precision, LONGITUDE double precision,
CITY varchar, STATE char(2), COUNTY varchar, ZIP_CLASS varchar);

将数据从CSV文件复制到表中:

COPY zip_codes FROM '/path/to/csv/ZIP_CODES.txt' WITH (FORMAT csv);