插入从选择查询中检索的 MySQL 结果集？

Question

我在 服务器 x 上有一个 MySQL 数据库，另一个在 服务器 y 上。

我正在尝试从位于服务器 x 上的名为 testx 的表中获取记录，并将 INSERT 它们放入名为 testy 的表中。所以我所做的是执行SELECT 语句并将其存储到resultset 中。然后我尝试在resultsetwhile 循环中迭代INSERT 语句。这是我的示例代码：

private static void cloneTableAndAlter() throws ClassNotFoundException, SQLException, InstantiationException, IllegalAccessException, InterruptedException {
        Connection connForSource = getConnectionForSource();
        Connection connForTarget = getConnectionForTarget();

        if (connForSource != null && connForTarget != null) {
            try {
                Statement st = connForSource.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY,
                        java.sql.ResultSet.CONCUR_READ_ONLY);

                String selectStatement = "SELECT field1, field2 FROM dbx.testx where time between ('2016-09-01 00:00:00') and ('2016-09-03 23:59:59');";

                String insertStatement = "INSERT INTO testy(" + "field1," + "field2)" + 
                        "VALUES(?,?) " +
                        "ON DUPLICATE KEY UPDATE field1 = VALUES(field1);"; <----field1 is a unique key but not primary

                st.setFetchSize(Integer.MIN_VALUE);

                ResultSet resultSetForSelect = st.executeQuery(selectStatement);
                PreparedStatement preparedStatement = connForTarget.prepareStatement(insertStatement);
                int count = 0;
                while (resultSetForSelect.next()) {
                    ++count;
                    TableDetails tableDetails = setTableDetails(resultSetForSelect); <--- i'm getting the value from the resultset and setting it to my DTO class.
                    getTableDetails(preparedStatement, tableDetails); <--- setting the values back during the insert from the DTO
                    preparedStatement.executeUpdate();
                    System.out.println(count + "rows affected");
                }
            } catch (SQLException e) {
                e.printStackTrace();
            }
        }
    }

注意：源表（testx）有超过一百万条记录。

我可以插入，但我觉得插入有点慢，每秒插入 45-50 次。我要去哪里迟钝？

有什么方法可以优化此操作并增加插入次数，或者这是插入大型数据集的本质吗？

任何帮助都将不胜感激。

Answer 1

你可以换一种方式：

用这个 Concats 创建一个 Select

select GROUP_CONCAT(
    CONCAT(" ('",field1,"','",field2,"')")) as vals
from my_table;

结果如下所示：

结果

mysql> select GROUP_CONCAT(
    -> CONCAT(" ('",field1,"','",field2,"')")) as vals
    -> from my_table;
+---------------------------------------------------------------------------------------------------------+
| vals                                                                                                    |
+---------------------------------------------------------------------------------------------------------+
|  ('O1','AC'), ('O1','PT'), ('O2','PT'), ('O3','MI'), ('O3','PT'), ('O4','EG'), ('O4','PT'), ('O5','PT') |
+---------------------------------------------------------------------------------------------------------+
1 row in set (0,00 sec)

mysql>

你可以在插入语句中直接连接这个，你只读取一行，只写和执行一个语句。

您的代码

    String selectStatement = "SELECT GROUP_CONCAT(CONCAT(" ('",field1,"','",field2,"')")) as vals
         FROM dbx.testx where time between ('2016-09-01 00:00:00') and ('2016-09-03 23:59:59');";

    ResultSet resultSetForSelect = st.executeQuery(selectStatement);


    String insertStatement = "INSERT INTO testy(" + "field1," + "field2)" + 
                            + String from result +
                            "ON DUPLICATE KEY UPDATE field1 = VALUES(field1);"; <----field1 is a unique key but not primary


# execute one time