【问题标题】:How to control the number of Oracle's request in Node.js?Node.js中如何控制Oracle的请求数?
【发布时间】:2020-10-16 03:12:59
【问题描述】:

我正在尝试使用 Node.js 和“oracledb”节点执行以下操作:

  1. 请查阅数据库 A 以获取在特定日期发布的所有账单。
  2. 将结果分配给变量list
  3. list 上使用函数.map(),并在该函数内部通过公共密钥查询数据库B,以获取list 的每一项的客户信息。

问题是:数据库 B 的请求是一起完成的,所以如果有 1000 个账单要映射,它只返回 100 个并将其余的视为错误。可能和同时请求的数量有关。

因此,鉴于详细信息,我想知道是否有办法划分请求数量(例如当时 100 个),或任何其他解决方案。

ps.:我提前为我的错误道歉。我也为没有在代码上演示而道歉。

【问题讨论】:

  • 查看Bluebird's Promise.map() 中的concurrency 选项。
  • 不知道你是不是在并行使用100个连接?这可能不太理想。总体而言,这听起来像是您可以在 PL/SQL 中完成的任务,如果不是在 SQL 中并且效率更高。
  • 我同意克里斯的观点。如果没有看到一些代码,很难说出发生了什么。前100个成功需要多长时间?其余的错误是什么?您是否为任一数据库使用连接池(更重要的是用于数据库 B)?如果有,尺寸是多少?您将 UV_THREADPOOL_SIZE 设置为什么?一个不好的解决方案是使用类似 async.queue 的东西,一个好的解决方案是使用类似 PL/SQL 的东西。我将尝试为您整理一个通用示例...

标签: node.js request node-oracledb


【解决方案1】:

这是一个示例,说明如何利用 v2.2.0(最近发布)中的新 executeMany 和全局临时表来最大程度地减少往返行程。

鉴于这些对象:

-- Imagine this is the table you want to select from based on the common keys
create table t (
  common_key number,
  info       varchar2(50)
);

-- Add 10,000 rows with keys 1-10,000 and random data for info
insert into t (common_key, info)
select rownum,
  dbms_random.string('p', 50)
from dual
connect by rownum <= 10000;

commit;

-- Create a temp table
create global temporary table temp_t (
  common_key number not null
)
on commit delete rows;

以下应该有效:

const oracledb = require('oracledb');
const config = require('./dbConfig.js');
const startKey = 1000;
const length = 2000;

// Uses a promise to simulate async work.
function getListFromDatabaseA() {
  return new Promise((resolve) => {
    const list = [];
    const count = length - startKey;

    for (let x = 0; x < count; x += 1) {
      list.push(startKey + x);
    }

    resolve(list);
  });
}

// The list returned from A likely isn't in the right format for executeMany.
function reformatAsBinds(list) {
  const binds = [];

  for (let x = 0; x < list.length; x += 1) {
    binds.push({
      key: list[x]
    });
  }

  return binds;
}

async function runTest() {
  let conn;

  try {
    const listFromA = await getListFromDatabaseA();

    const binds = reformatAsBinds(listFromA);

    conn = await oracledb.getConnection(config);

    // Send the keys to the temp table with executeMany for a single round trip.
    // The data in the temp table will only be visible to this session and will
    // be deleted automatically at the end of the transaction.
    await conn.executeMany('insert into temp_t (common_key) values (:key)', binds);

    // Now get your common_key and info based on the common keys in the temp table.
    let result = await conn.execute(
     `select common_key, info 
      from t 
      where common_key in (
        select common_key
        from temp_t
      )
      order by common_key`
    );

    console.log('Got ' + result.rows.length + ' rows');

    console.log('Showing the first 10 rows');

    for (let x = 0; x < 10; x += 1) {
      console.log(result.rows[x]);
    }
  } catch (err) {
    console.error(err);
  } finally {
    if (conn) {
      try {
        await conn.close();
      } catch (err) {
        console.error(err);
      }
    }
  }
}

runTest();

在我发布上面的解决方案之后,我认为我应该提供一个替代方案,让密钥从 Node.js 传输到“内存中”的数据库。您必须进行测试并查看解释计划,看看哪个是最适合您的选择(取决于许多因素)。

鉴于这些对象:

-- This is the same as before
create table t (
  common_key number,
  info       varchar2(50)
);

-- Add 10,000 rows with keys 1-10,000 and random data for info
insert into t (common_key, info)
select rownum,
  dbms_random.string('p', 50)
from dual
connect by rownum <= 10000;

-- But here, we use a nested table instead of a temp table
create or replace type number_ntt as table of number;

这应该可行:

const oracledb = require('oracledb');
const config = require('./dbConfig.js');
const startKey = 1000;
const length = 2000;

// Uses a promise to simulate async work.
function getListFromDatabaseA() {
  return new Promise((resolve) => {
    const list = [];
    const count = length - startKey;

    for (let x = 0; x < count; x += 1) {
      list.push(startKey + x);
    }

    resolve(list);
  });
}

async function runTest() {
  let conn;

  try {
    const listFromA = await getListFromDatabaseA();

    const binds = {
      keys: {
        type: oracledb.NUMBER,
        dir: oracledb.BIND_IN,
        val: listFromA
      },
      rs: {
        type: oracledb.CURSOR,
        dir: oracledb.BIND_OUT
      }
    };

    conn = await oracledb.getConnection(config);

    // Now get your common_key and info based on what's in the temp table.
    let result = await conn.execute(
     `declare

        type number_aat is table of number index by pls_integer;

        l_keys    number_aat;
        l_key_tbl number_ntt := number_ntt();

      begin

        -- Unfortunately, we have to bind in with this data type, but
        -- it can't be used as a table...
        l_keys := :keys;

        -- So we'll transfer the data to another array type that can. This
        -- variable's type was created at the schema level so that it could
        -- be seen by the SQL engine.
        for x in 1 .. l_keys.count
        loop
          l_key_tbl.extend();
          l_key_tbl(l_key_tbl.count) := l_keys(x);
        end loop;

        open :rs for
          select common_key, info 
          from t 
          where common_key in (
            select column_value
            from table(l_key_tbl)
          )
          order by common_key;

      end;`,
      binds
    );

    const resultSet = result.outBinds.rs;

    console.log('Showing the first 10 rows');

    for (x = 0; x < 10; x += 1) {
      let row = await resultSet.getRow();
      console.log(row);
    }
  } catch (err) {
    console.error(err);
  } finally {
    if (conn) {
      try {
        await conn.close();
      } catch (err) {
        console.error(err);
      }
    }
  }
}

runTest();

绑定的格式不同(这里稍微简单一些)。另外,因为我正在执行 PL/SQL,所以我需要一个外绑定游标/结果集类型。

请参阅这篇关于嵌套表的基数的帖子: http://www.oracle-developer.net/display.php?id=427

如果您同时尝试这两种方法,请留下一些反馈,说明哪种效果更好。

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 2012-09-16
    • 1970-01-01
    • 2016-01-28
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2011-12-10
    • 2012-01-08
    相关资源
    最近更新 更多