按特定值匹配数据集中的实例记录答案

【问题标题】：Matching instance records in a data set by specific value按特定值匹配数据集中的实例记录
【发布时间】：2021-04-27 17:55:46
【问题描述】：

我有一个解决方案，它运行良好，但性能不佳，需要一些时间才能运行。让我们从最初的两个查询（都是双连接）返回的内容开始：

第一组数据如下所示 - 我们称它们为line_items。如您所见，line_items 没有 dh_first_name 键/值。


 [
   [
    {
    pb_id: "133599.0",
    pbbname: "CUSTOMER",
    opl_amount: "101.0",
    ops_type: "P",
    ops_stop_id: 269802,
    ops_order_id: 133599,
    ops_driver1: 11,
    ops_delivered_time: null
    },
    {
    pb_id: "133599.0",
    pbbname: "CUSTOMER",
    opl_amount: "11.62",
    ops_type: "P",
    ops_stop_id: 269802,
    ops_order_id: 133599,
    ops_driver1: 11,
    ops_delivered_time: null
    },
    {
    pb_id: "133590.0",
    pbbname: "CUSTOMER",
    opl_amount: "79.0",
    ops_type: "P",
    ops_stop_id: 269780,
    ops_order_id: 133590,
    ops_driver1: 104,
    ops_delivered_time: null
    },
    {
    pb_id: "133220.0",
    pbbname: "CUSTOMER",
    opl_amount: "625.0",
    ops_type: "D",
    ops_stop_id: 269011,
    ops_order_id: 133220,
    ops_driver1: 62,
    ops_delivered_time: "2021-04-01T12:35:00.000-05:00"
    },
    {
    pb_id: "133357.0",
    pbbname: "CUSTOMER",
    opl_amount: "550.0",
    ops_type: "D",
    ops_stop_id: 269290,
    ops_order_id: 133357,
    ops_driver1: 92,
    ops_delivered_time: "2021-04-01T09:38:00.000-05:00"
    },
    {
    pb_id: "133219.0",
    pbbname: "CUSTOMER",
    opl_amount: "1267.06",
    ops_type: "P",
    ops_stop_id: 269008,
    ops_order_id: 133219,
    ops_driver1: 43,
    ops_delivered_time: null
    },
    {
    pb_id: "133577.0",
    pbbname: "CUSTOMER",
    opl_amount: "150.0",
    ops_type: "P",
    ops_stop_id: 269754,
    ops_order_id: 133577,
    ops_driver1: 94,
    ops_delivered_time: null
    },
    {
    pb_id: "133503.0",
    pbbname: "CUSTOMER",
    opl_amount: "79.0",
    ops_type: "P",
    ops_stop_id: 269592,
    ops_order_id: 133503,
    ops_driver1: 104,
    ops_delivered_time: null
    },
    {
    pb_id: "133643.0",
    pbbname: "HALLMARK CARDS BERMAN BLAKE",
    opl_amount: "79.0",
    ops_type: "P",
    ops_stop_id: 269895,
    ops_order_id: 133643,
    ops_driver1: 104,
    ops_delivered_time: null
    }
   ]
 ]

现在，让我们看一下来自第二个双重联接的下一组数据，即line_stops。它看起来像这样：


    [
    {
    pb_id: "133633.0",
    pbbname: "CUSTOMER",
    pb_net_rev: "250.0",
    ops_driver1: 59,
    ops_stop_id: 269869,
    dh_first_name: "FIRST",
    dh_last_name: "LAST",
    ops_delivered_time: "2021-04-02T13:07:00.000-05:00"
    },
    {
    pb_id: "133127.0",
    pbbname: "CUSTOMER",
    pb_net_rev: "1147.0",
    ops_driver1: 102,
    ops_stop_id: 268801,
    dh_first_name: "FIRST",
    dh_last_name: "LAST",
    ops_delivered_time: null
    },
    {
    pb_id: "133144.0",
    pbbname: "CUSTOMER",
    pb_net_rev: "650.0",
    ops_driver1: 71,
    ops_stop_id: 268836,
    dh_first_name: "FIRST",
    dh_last_name: "LAST",
    ops_delivered_time: "2021-04-01T14:38:00.000-05:00"
    },
    {
    pb_id: "133144.0",
    pbbname: "CUSTOMER",
    pb_net_rev: "650.0",
    ops_driver1: 71,
    ops_stop_id: 268837,
    dh_first_name: "FIRST",
    dh_last_name: "LAST",
    ops_delivered_time: null
    },
    {
    pb_id: "133188.0",
    pbbname: "CUSTOMER",
    pb_net_rev: "700.0",
    ops_driver1: 71,
    ops_stop_id: 268924,
    dh_first_name: "FIRST",
    dh_last_name: "LAST",
    ops_delivered_time: "2021-04-01T08:04:00.000-05:00"
    },
    ]

我目前正在做的是循环遍历它们并根据这些 values 匹配它们。

ops_stop_id, ops_driver_1, pb_id

如果这三个匹配，那么我需要在特定驱动程序的名称下构造它们，该名称只能来自具有dh_first_name 的实例。该数据结构完成后如下所示：

    {
    FIRST LAST: [
    {
    pb_id: "133599.0",
    pbbname: "CUSTOMER",
    opl_amount: "101.0",
    ops_type: "P",
    ops_stop_id: 269802,
    ops_order_id: 133599,
    ops_driver1: 11,
    ops_delivered_time: null
    },
    {
    pb_id: "133599.0",
    pbbname: "CUSTOMER",
    opl_amount: "11.62",
    ops_type: "P",
    ops_stop_id: 269802,
    ops_order_id: 133599,
    ops_driver1: 11,
    ops_delivered_time: null
    },
    {
    pb_id: "133536.0",
    pbbname: "CUSTOMER",
    opl_amount: "45.0",
    ops_type: "P",
    ops_stop_id: 269665,
    ops_order_id: 133536,
    ops_driver1: 11,
    ops_delivered_time: null
    },
    {
    pb_id: "133536.0",
    pbbname: "CUSTOMER",
    opl_amount: "5.18",
    ops_type: "P",
    ops_stop_id: 269665,
    ops_order_id: 133536,
    ops_driver1: 11,
    ops_delivered_time: null
    },
    {
    pb_id: "133522.0",
    pbbname: "CUSTOMER",
    opl_amount: "150.0",
    ops_type: "P",
    ops_stop_id: 269637,
    ops_order_id: 133522,
    ops_driver1: 11,
    ops_delivered_time: null
    },
    {
    pb_id: "133619.0",
    pbbname: "CUSTOMER",
    pb_net_rev: "550.0",
    ops_driver1: 11,
    ops_stop_id: 269841,
    dh_first_name: "FIRST",
    dh_last_name: "LAST",
    ops_delivered_time: "2021-04-02T11:41:00.000-05:00"
    }
    ],

您将看到两条记录的混合，匹配的参数组织正确。

这就是我目前解决问题的方法！


    merger = {}
      line_items.each do |lines, i|
        line_stops.each do |stops|
            if (lines.ops_stop_id == stops.ops_stop_id && lines.ops_driver1 == stops.ops_driver1 && lines.pb_id == stops.pb_id)
              stops_arr.push(stops)
              merger[stops.dh_first_name + ' ' + stops.dh_last_name] = (merger[stops.dh_first_name + ' ' + stops.dh_last_name] ||= []) << lines
            end
         end
      end

      line_stops.each do |stops|
            if (!stops_arr.include?(stops))
              stops_arr.push(stops)
              merger[stops.dh_first_name + ' ' + stops.dh_last_name] = (merger[stops.dh_first_name + ' ' + stops.dh_last_name] ||= []) << stops
            end
        end

这太慢了，我认为这条线是罪魁祸首：

(merger[stops.dh_first_name + ' ' + stops.dh_last_name] ||= []) << stops

【问题讨论】：

如果您从数据库中获取此数据，那么您可能应该在此处而不是在 Ruby 中执行此操作。
我担心这就是答案，但我只是更舒服地编写脚本通过它......但我想除了通过查询来优化之外没有真正的方法来优化.就通过 SQL 查询实现逻辑而言，我什至不知道从哪里开始。
我会试一试，如果进展不顺利，请询问有关架构、模型、数据示例和预期输出的问题。
您能否提供一个可运行的 JSON 变体示例？它可以是抽象的，不需要限制所有那些不能告诉我们太多的属性（显然它们对您的业务很重要，但我不关心 pb_net_rev :-)）或者提供架构和预期结果？所以我们可以帮助您在 SQL 中执行此操作。
我猜stops_arr 是一个数组，对吧？如果是真的，那么我猜罪魁祸首是逻辑：!stops_arr.include?(stops)。将stops_arr 更改为哈希怎么样。

标签： ruby-on-rails ruby algorithm activerecord

【解决方案1】：

你的代码的时间复杂度是O(lines.size * stops.size)

这里是我的提案，关于O(lines.size + stops.size)的时间复杂

def merge_key(stops)
  stops.dh_first_name + ' ' + stops.dh_last_name
end

# Note that hash_key code below maybe not good enough
def hash_key(lines)
  "#{lines.ops_stop_id} #{lines.ops_driver1} #{lines.pb_id}"
end

merger = Hash.new { |hash, key| hash[key] = [] }
stops_hash = Hash.new

# O(line_stops.size)
line_stops.each do |stops|
  merge_key = merge_key(stops)
  next if merger.hash_key?(merge_key) # since in your code, you not add dup stops, right ?

  merger[merge_key] << stops
  stops_hash[hash_key(stops)] = merge_key
end

# O(line_items.size)
line_items.each do |lines, i|
  if merge_key = stops_hash[hash_key(lines)]
    merger[merge_key].unshift(lines) # since in your code, lines add before stops, right ? 
  end
end

【讨论】：