使用列名导出 SQLite答案

【问题标题】：SQLite export with column names使用列名导出 SQLite
【发布时间】：2010-11-16 22:48:06
【问题描述】：

是否有任何 SQLite 命令或第三方工具允许数据库转储在 INSERT INTO 语句中包含列名？

而不是

INSERT INTO "MyTable" VALUES ('A', 'B');

我想看看

INSERT INTO "MyTable" (Column1, Column2) VALUES ('A', 'B');

SQLite 中的.dump 命令只提供第一个版本。

【问题讨论】：

标签： sql sqlite

【解决方案1】：

让我再试一次。

将列名和 INSERT 语句转储到文件中。

sqlite> .output test.data
sqlite> pragma table_info(test);
sqlite> .dump test
sqlite> .quit

$ cat test.data
0|test_id|int|0||1
1|test_name|varchar(35)|0||0
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE test (test_id int primary key, test_name varchar(35));
INSERT INTO "test" VALUES(1,'Wibble');
INSERT INTO "test" VALUES(2,'Wobble');
INSERT INTO "test" VALUES(3,'Pernicious');
COMMIT;

现在运行这个 awk 脚本

/\|/ {
  split($0, col_name, "|");
  column_names[++n] = col_name[2];
}
/INSERT INTO \"[A-Za-z].*\"/ {
  insert_part = match($0, /INSERT INTO \"[A-Za-z].*\"/);
  printf("%s ", substr($0, RSTART, RLENGTH));

  printf("(");
  for (i = 1; i <= n; i++) {
    if (i == 1) {
      printf("%s", column_names[i]);
    }
    else {
      printf(", %s", column_names[i]);
    }
  }
  printf(") ");

  values_part = substr($0, RLENGTH+1, length($0) - RSTART);
  printf("%s\n", values_part);


}

我们得到

$ awk -f dump_with_col_names.awk test.data
INSERT INTO "test" (test_id, test_name)  VALUES(1,'Wibble');
INSERT INTO "test" (test_id, test_name)  VALUES(2,'Wobble');
INSERT INTO "test" (test_id, test_name)  VALUES(3,'Pernicious');

【讨论】：

不幸的是，这一次只能用于一个表，如果数据中的任何位置有竖线字符 (|)，则会失败。

【解决方案2】：

这并不能回答问题。我在这里写这篇文章是因为这是我处理类似问题的方式。一种方法是分别转储结构和数据。对于您在数据文件之外描述的插入：

sqlite> .headers on
sqlite> .mode insert MyTable
sqlite> .output MyTable_data.sql
sqlite> select * from MyTable;
sqlite> .quit

【讨论】：

【解决方案3】：

有一个用于导入/导出的 SQLite 扩展模块来自/到 SQL 源文本的数据库信息和导出为 CSV 文本。 http://www.ch-werner.de/sqliteodbc/html/impexp_8c.html

例如，在 Ubuntu 中，您的步骤是：

从 ubuntu 仓库安装模块
```
sudo apt install libsqlite3-mod-impexp
```
在sqlite命令行提示符运行中加载模块
```
.load libsqlite3_mod_impexp
```
导出数据库到dump.sql文件
```
select export_sql('dump.sql','1');
```

我的数据库的结果示例是

INSERT OR REPLACE INTO "camera" ("name","reviews") VALUES('BenQ GH700', NULL);  
INSERT OR REPLACE INTO "camera" ("name","reviews") VALUES('Canon EOS 40D', NULL);

【讨论】：

在 Ubuntu 18.04 上，您需要将 .so 附加到 .load 参数。

【解决方案4】：

我创建了这个 shell 脚本：

#!/bin/sh

SQLITE=sqlite3

if  [ -z "$1" ] ; then
        echo usage: $0  sqlite3.db
        exit
fi

DB="$1"

TABLES=`"$SQLITE" "$DB" .tables`
echo "-- $TABLES" 
echo 'BEGIN TRANSACTION;'

for TABLE in $TABLES ; do
        echo 
        echo "-- $TABLE:";
        COLS=`"$SQLITE" "$DB" "pragma table_info($TABLE)" |
        cut -d'|' -f2 `
        COLS_CS=`echo $COLS | sed 's/ /,/g'`
        echo -e ".mode insert\nselect $COLS_CS from $TABLE;\n" |
        "$SQLITE" "$DB" |
        sed "s/^INSERT INTO table/INSERT INTO $TABLE ($COLS_CS)/"
done
echo 'COMMIT;';

两个问题：

表和列必须具有“正常”名称（即字母数字下划线），
数据不能包含字符串'\nINSERT INTO 表”。

【讨论】：

【解决方案5】：

我快速浏览了源代码。我没有看到任何明显的方法来做到这一点。但是我编写了一个快速而肮脏的 awk 脚本来插入列名。

从这个转储开始：

PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE test (test_id int primary key, test_name varchar(35));
INSERT INTO "test" VALUES(1,'Wibble');
INSERT INTO "test" VALUES(2,'Wobble');
INSERT INTO "test" VALUES(3,'Pernicious');
COMMIT;

我运行了这个 awk 脚本

/CREATE TABLE/ {
  # Extract the part between parens. This part contains the 
  # column definitions.
  first_col = match($0, /\(.*\)/ );
  if (first_col) {
     num_columns = split(substr($0, RSTART + 1, RLENGTH), a, ",");
     for (i = 1; i <= num_columns; i++) {
       sub(/^ /, "", a[i]);
       split(a[i], names, " ");
       column_names[i] = names[1];
     }
  }
}
/INSERT INTO \"[A-Za-z].*\"/ {
  insert_part = match($0, /INSERT INTO \"[A-Za-z].*\"/);
  printf("%s ", substr($0, RSTART, RLENGTH));

  printf("(");
  for (j = 1; j <= num_columns; j++) {
    if (j == 1) {
        printf("%s", column_names[j]);
    }
    else {
        printf(", %s", column_names[j]);
    }
  }
  printf(") ");

  values_part = substr($0, RLENGTH+1, length($0) - RSTART);
  printf("%s\n", values_part);

}

这给了我这个输出。

INSERT INTO "test" (test_id, test_name)  VALUES(1,'Wibble');
INSERT INTO "test" (test_id, test_name)  VALUES(2,'Wobble');
INSERT INTO "test" (test_id, test_name)  VALUES(3,'Pernicious');

【讨论】：

【解决方案6】：

这是一个适用于任意数量表的 Perl 版本：

#!/usr/bin/perl

use strict;
use warnings;

my @column_names;
my $col_reset = 1;

while(<>)
{
  if (/^\d+\|/) {
    if ($col_reset)
    {
      @column_names = ();
      $col_reset = 0;
    }
    my @col_info = split(/\|/);
    push @column_names, $col_info[1];
  }

  if(/INSERT INTO/) {
    m/(INSERT INTO \"?[A-Za-z_]+\"?) (.*)/ or die $_;
    my $insert_part = $1;
    my $values_part = $2;
    print $insert_part." (".join(",", @column_names).") ".$values_part."\n";
    $col_reset = 1;
  }
}

这就是我生成数据库中每个表的转储的方式：

grep 'CREATE TABLE' /tmp/school.sql.final \
| awk '{ print $3 }' \
| while read table; do
    echo -e "pragma table_info($table);\n.dump $table"
done | sqlite3 school.db \
> /tmp/school.sql.final-with-table-info

【讨论】：

【解决方案7】：

结合shell管道的纯SQL方案：

创建文件dump_sqlite.sql:

.headers off
.mode list

select
    "select " || """" || "insert into " || tab || " (" ||
    "" || group_concat(col_name) || ") VALUES (" || """ || " ||
    group_concat(col_val, " || "","" || ") || 
    " || "")" || """" || " FROM " || tab || ";" as stmt
from (
select
    m.name as tab,
    ti.name as col_name,
    case
        when ti.type like "text" or ti.type like "varchar%" 
            then """"" || " || "coalesce(quote(`" || ti.name || "`), 'NULL')" || " || """""
        else "coalesce(`" || ti.name || "`, 'NULL')"
    end as col_val,
    ti.type as coltype
from sqlite_master as m,
-- https://stackoverflow.com/a/54962853
PRAGMA_TABLE_INFO(m.name) as ti
where m.type = 'table' and m.name not like 'sqlite_%'
) group by tab
;

然后执行shell命令：

sqlite3 your.db < ./dump_sqlite.sql | sqlite3 your.db

如果您想将输出保存到文件中（而不是将其打印到控制台），将最终重定向添加到文件。

sqlite3 your.db < ./dump_sqlite.sql | sqlite3 your.db > dump.sql

【讨论】：

【解决方案8】：

另一个 AWK 脚本，它直接从“sqlite3 data.db .dump”的输出中用于任意数量的表

它使用了这样一个事实，即 CREATE 语句现在每列都打印在自己的行上

BEGIN {
        state = "default"  # Used to know if we are in the middle of a table declaration
        print_definitions = 1 # Wether to print CREATE statements or not
}

state == "default" && match($0, /^CREATE TABLE ([A-Za-z0-9_]+)/, a) {
        tablename = a[1]
        state = "definition"
        if (print_definitions)
                print
        next
}

state == "definition" && /^);$/ {
        state = "default"
        if (print_definitions)
                print
        next
}

state == "definition" && ! ( /PRIMARY/ || /UNIQUE/ || /CHECK/ || /FOREIGN/) {
        if (length(columnlist [tablename]))
                columnlist[tablename] = columnlist[tablename] ", "
        columnlist[tablename] = columnlist[tablename] $1
        if (print_definitions)
                print
        next
}

state == "default" && match($0, /^(INSERT INTO ")([A-Za-z0-9_]+)"(.*)$/, a) {
        print a[1] a[2] "\" (" columnlist[a[2]] ")" a[3]
}

【讨论】：

我不确定这是否能回答问题。

【解决方案9】：

如果您不介意 GUI。您可以使用DB Browser for SQLite。检查选项Keep column names in INSERT INTO

【讨论】：

【解决方案10】：

Louis L. 解决方案不适合我，所以我用 sqlite3 版本 3.8.7.1 的转储测试了这个 gawk 解决方案

表 CREATE 语句类似于例如

CREATE TABLE "strom" (
  "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
  "otec" integer NOT NULL,
  "nazev" text NOT NULL,
  "ikona" text NULL,
  "barva" text NULL
);

但也可能看起来像这个

CREATE TABLE "changes" (
  "version" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
  "last_change" text NOT NULL DEFAULT (datetime('now','localtime')),
  `ref` text NOT NULL,
  "ref_id" text NULL,
  "action" text NOT NULL
, "data" text NOT NULL DEFAULT '');

#!/usr/bin/gawk -f

# input is sqlite3 dump, tested with sqlite3 version 3.8.7.1
# output are INSERT statements including column names
# i.e. not e.g.
# INSERT INTO "changes" VALUES(1,'2016-07-19 17:46:12','cenik','10','UPDATE','');
# like in standard dump
# but
# INSERT INTO "changes" ("version", "last_change", "ref", "ref_id", "action", "data") VALUES(1,'2016-07-19 17:46:12','cenik','10','UPDATE','');
# BEGIN TRANSACTION and COMMIT are included in output

BEGIN {
        state = "default"  # default/definition/insert let us know wether we are in CREATE or INSERT statement
        print_definitions = 0 # wether to print CREATE statements or not
}

state == "default" && match($0, /^CREATE TABLE \"([A-Za-z0-9_]+)\" *\($/, a) {
        tablename = a[1]
    state = "definition"
        if (print_definitions)
                print
        next
}

state == "definition" && /^ *); *$/ {
        state = "default"
        if (print_definitions)
                print
        next
}

state == "definition" && ! ( /^[\ ]{1,2}PRIMARY/ || /UNIQUE/ || /CHECK/ || /^[\ ]{1,2}FOREIGN KEY.*REFERENCES/) {
        if (length(columnlist [tablename]))
                columnlist[tablename] = columnlist[tablename] ", "
        if (match($0, /(\".*\")\s/, b))
        columnlist[tablename] = columnlist[tablename] b[1]
    if (match($0, /`(.*)`\s/, c))
        columnlist[tablename] = columnlist[tablename] "\""c[1]"\""
        if (print_definitions)
                print
}

state == "definition" && /^.*); *$/ {
        state = "default"
        next
}

state == "default" && match($0, /^(INSERT INTO ")([A-Za-z0-9_]+)"(.*)/, a) {
        print a[1] a[2] "\" (" columnlist[a[2]] ")" a[3]
    state = "insert"
    if (/^.*); *$/) 
        state = "default"
}

state == "insert" && ! /^INSERT INTO/{
    print
}

state == "insert" && /^.*); *$/ {
        state = "default"
    next
}

state == "default" && (/^ *BEGIN TRANSACTION;/ || /^ *COMMIT;/) {
    print
}

【讨论】：

【解决方案11】：

简单的 python 脚本可以解决问题

import sqlite3
infile="your_file.sqlite3"
table="your_table"

conn = sqlite3.connect(infile)
conn.row_factory = sqlite3.Row

c = conn.cursor()
res = c.execute("SELECT * FROM " + table)
curr_row = -1

for row in res:
   curr_row += 1
   if curr_row == 0:
      col_names = sorted(row.keys())
      s = "INSERT INTO " + table + " ("
      for col_name in col_names:
        s+=col_name + ","
      prefix = s[:-1] + ") VALUES ("

   s = ""
   for col_name in col_names:
     col_val = row[col_name]
     if isinstance(col_val,int) or isinstance(col_val,float):
       s+= str(row[col_name]) +","
     else:
       s+= "'" + str(row[col_name]) +"',"
   print prefix,s[:-1],");"

【讨论】：

这不能正确地转义名称、字符串或 blob。