【问题标题】:Google Apps Script for creating normalised Data from unnormalised Data in Google Sheets用于从 Google 表格中的非规范化数据创建规范化数据的 Google Apps 脚本
【发布时间】:2023-03-19 08:30:01
【问题描述】:

在 Google 表格中,我在一个 Google 电子表格('Category1')、('Category2')、('Category3')中有 3 个原始数据工作表。这些工作表由我的业务人员不断更新,但不幸的是,这些数据不是规范化的形式,无法运行有效的查询。

我想创建一个脚本,该脚本自动生成此原始信息的标准化输出('Category1 Output')、('Category2 Output')、('Category3 Output'),当有人更改时自动更新原始标签。

在下面的谷歌表中,我提供了一个类别需要看起来的示例。 'Category1' 工作表是每个人不断更新的原始工作表。 “Category1Output”是最终输出工作表,当在“Category1”工作表中进行编辑时会自动更新。

Google Sheet Link

【问题讨论】:

  • 您可以添加任何详细信息,例如:使用的代码,遇到的错误问题吗? How do I ask a good question?, How to create a Minimal, Complete, and Verifiable example 向社区展示你的尝试。
  • @jwlon81 我想谈谈这个任务。没有问题,只是澄清。我想你也在悉尼?无论如何,StackOverflow 聊天首先?如果我能弄清楚如何让它工作!
  • 确定 Ted - 你想澄清什么?
  • @jwlon81 我的错;直到现在才注意到你的回复。 1)您有没有想过如果出现新的/删除的/修改的产品该怎么办? (也许重建?) 2)触发器是否仅适用于“周”列(目录、显示、ESP、机械师)中的四个“关键”数据字段之一? 3) 是否有 53 周(闰年)? FWIW,代码完成“构建”输出表(不包括只是连接的“连接”)。 Next = trigger - 这是更难的部分,因为代码必须“找到”输出表上的相关行。你有什么特别的想法吗?我认为需要一些思考?
  • @jwlon81 触发器初稿。下周再说。

标签: javascript google-apps-script google-sheets


【解决方案1】:

提问者在定义的列式布局中具有三张工作表 - 每个数据集基本上有几行和几列(每个时间段一个)。这些表格没有被替换,但寻求一个总结版本,可以使用过滤来有效地关注相关数据。因此,每张工作表都会从列格式转换为逐行格式。

这个过程本身很简单。源数据包含 64 个产品,每个产品有 8 个数据行。输出记录为@1,350。

提问者的代码在数据到输出格式的转换上挂了。每个产品使用 8 行数据很重要,代码包括检查数据总行数除以 8 的商是否为整数。此外,源表和输出表的名称按名称 (getSheetByName) 调用,因此代码可以轻松应用于任何命名的输入表和任何命名的输出表。唯一的条件是两张表都必须事先存在。

对提问者的代码打嗝的初步解决是成功的,并且使用 getDataRangegetValues before 循环的方法大大提高了性能。有两个循环;一个垂直方向,在数据行中移动;第二个是水平方向的,穿过与时间相关的列。然而,性能最初非常低效,代码在完成之前就超时了。

我修改了代码以构建单个二维数组,并在循环结束时将其保存到输出表一次。这对性能产生了巨大的影响。完成的总时间从几分钟减少到不到 5 秒。


function so5243560403() {

    // Build a clean Output Sheet
    var ss = SpreadsheetApp.getActiveSpreadsheet();
    var SourceSheet = ss.getSheetByName("Category2");
    var Target = ss.getSheetByName("Category3Output");

    // Various variables
    var SourceHeaderRow = 9;
    var RowsPerProduct = 8;
    var ProductLengthTruncate = 11;
    var SourceArray = [];
    var i = 0;
    var w = 0;

    // get the bottom of the column
    var ColAvals = SourceSheet.getRange("A" + (SourceHeaderRow + 1) + ":A").getValues();
    var ColAlast = ColAvals.filter(String).length;
    //Logger.log("Last row in column A with data"+ColAlast);  //DEBUG
    var NumberofProducts = ColAlast / RowsPerProduct;
    var lastRow = SourceSheet.getLastRow();

    // Count the products and confirm eight rows each
    var prodtest = isInt1(NumberofProducts);
    if (!prodtest) {
        // Logger.log("NOT an integer!");
        SpreadsheetApp.getUi().alert("Number of Rows divided by rows by Product isn't an integer");
        return false;
    }

    // Get data to clear Target ready for new data
    var TargetlastRow = Target.getLastRow();
    var TargetlastColumn = Target.getLastColumn();
    // clear the content before re-building
    Target.getRange(2, 1, TargetlastRow, TargetlastColumn).clear({
        contentsOnly: true
    });

    // Get ALL the data on the SourceSheet
    var SourceRange = SourceSheet.getDataRange();
    var SourceValues = SourceRange.getValues();

    // create loop for rows of data; first row of data in array=9
    for (i = SourceHeaderRow; i < (SourceHeaderRow + ColAlast); i = i + 8) {

        // create loop for weeks (Week 1=Col5, Week 2=Col6... Week 52=Col56, etc) (actual column numbers are +1)
        for (w = 1; w < 53; w++) {

            // Test to see whether there's a value for Display; the only field ALWAYS populated
            if (SourceValues[i + 1][w + 4]) {

                // Get Product and data fields
                var Prodlen = SourceValues[i][3].length;
                var prodedit = SourceValues[i][3].substring(11, (SourceValues[i][3].length));
                var product = prodedit.trim();
                var catalogue = SourceValues[i][w + 4];
                var display = SourceValues[i + 1][w + 4];
                var ESP = SourceValues[i + 3][w + 4];
                var mechanic = SourceValues[i + 6][w + 4];
                var join1 = product+" | "+display+" | "+mechanic;
                var join2 = display+" | "+product+" | "+mechanic;
                // Start building an array
                SourceArray.push([w, product, catalogue, display, ESP, mechanic,join1,join2]);

            } // end if data exists - process this week

        } // end w - this week loop

    } // end i - this row loop 


    // Copy the data from the array to the Target sheet

    // count number of rows
    var SourceArraylen = SourceArray.length;

    // first row is #2, allowing for header row
    // first column = A
    // number of rows = length of array
    // number of columns = 6 (the fields puched to the array
    var TargetRange = Target.getRange(2, 1, SourceArraylen, 8);

    // set the array values on the Target sheet
    TargetRange.setValues(SourceArray);
}

function isInt1(value) {
    return !isNaN(value) && parseInt(Number(value)) == value && !isNaN(parseInt(value, 10));
}

更新

提问者代码的第二个元素处理在对“类别”表进行更改时将数据更新到“输出表”。提问者的更新代码没问题,但缺少将类别表上的源范围转换为在输出表上建立目标范围。

解决方案涉及基于数学数字序列的量规。在这种情况下,数学序列是源工作表上产品的行号;每个产品占8行,第一行是#10,所以顺序是10,18,26,34....

onEdit 返回已更改单元格的范围,getRowgetColumn 可用于建立已更改单元格的坐标。挑战在于,了解实际更改的行号,确定实际行号代表的行序列中的数字(以及产品名称)。更改的行也极不可能(八比一)与产品编号的第一行重合。

因此有必要将算法应用于数学序列 - 两次。确定数字序列中第 n 个数字的公式是 An = A1 + (D x(n-1)),其中 A1 是第一行的数字(在我们的例子中为 10),D= 每个数字之间的差在序列中(在我们的例子中为 8),并且 n = 序列中的数字(在我们的例子中,更改的行号)。

第一遍是在实际变化的行所代表的数字(产品组)序列中建立位置编号。结果很可能不是整数,即它与产品组的第一行不一致。因此,将结果向下舍入到最接近的整数,并再次处理算法。

不过这一次我们知道了序列号的位置,我们通过求解找到该数字的值。在这种情况下,公式为 ((An-A1)/D)+1。这将返回 Source 表中与相关产品组的第一行相对应的行号。我们使用它来识别更改的字段类型(类别、显示等)。

列号表示周数。第 1 周从 F 列开始,因此get column 使我们能够确定更改是否发生在一周列中(或者是否发生在 F 列的左侧)。如果在左侧,则“不是我的问题”,如果在 F 或更高,则需要注明。

最后,我们为目标表执行getRangeValue,并在 A 列中查找周数的匹配项,并在 B 列中查找截断的产品名称。这为 setValue 提供了新值的坐标从 OnEdit 跟踪。


 function OnEdit(e) {

    // Update relevant Outputsheets on changes in Category sheets

    var ss = SpreadsheetApp.getActiveSpreadsheet();

    // Establish variables

    var s1 = "Category1";
    var s2 = "Category2";
    var s3 = "Category3";
    var tsuffix = "Output";
    //Logger.log("Sheet information");//DEBUG
    //Logger.log("The sheets to track are s1= "+s1+", s2 = "+s2+", and s3 = "+s3+", and the Output suffix is "+tsuffix+". For example s1output = "+s1+tsuffix);// DEBUG

    var TargetSheet = "";
    var weekscolumnstart = 6; // Column F
    var startrow = 10; // applies to the Source sheet
    var rowsperProduct = 8; // applies to the source sheet
    var changedfield = 0;
    var changedfieldname = "";
    var n = 0;

    // Collect data from the event
    var range = e.range;
    var oldValue = e.oldValue;
    var value = e.value
    var source = e.source;
    var sheet = source.getActiveSheet();
    var ssname = sheet.getName();
    // Logger.log("Range: "+range.getA1Notation()+", old value = "+oldValue+", new value = "+value+", source = "+source+", ss = "+sheet+", sheet name = "+ssname); //DEBUG

    // get the co-ordinates of the change
    var SourceRow = range.getRow();
    var SourceColumn = range.getColumn();
    // Logger.log("the Column is "+SourceColumn+", and the Row is "+SourceRow);// DEBUG


    // the weeks range to the right, from column F (va = weekscolumnstart). So by knowing the column number of the even, we can calculate the week number that applied to the change.
    var weeknumber = (SourceColumn - weekscolumnstart + 1);

    switch (ssname) { // the field references are used in a GetValue statement where the column is a reference to a specific column 
        case s1:
            TargetSheet = s1 + tsuffix;
            //Logger.log("The Source sheet was "+ssname+", so the Target sheet is "+TargetSheet);// DEBUG
            break;
        case s2:
            TargetSheet = s2 + tsuffix;
            //Logger.log("The Source sheet was "+ssname+", so the Target sheet is "+TargetSheet);// DEBUG
            break;
        case s3:
            TargetSheet = s3 + tsuffix;
            //Logger.log("The Source sheet was "+ssname+", so the Target sheet is "+TargetSheet);// DEBUG
            break;
        default:
            TargetSheet = 0;
            //Logger.log("The change was made in a sheet that we don't need to track.");
    } // end switch


    // get product and other change information if the change is on a tracked sheet and in a relevant column.
    // evalue for the event on a non-relevant sheet or in a non-relevant column
    if (TargetSheet == 0 || weeknumber <= 0) {
        // do nothing 
    } else {
        //Logger.log("before calculating line number; the TargetSheet is "+TargetSheet);
        // The source has eight rows of data per Product; there is no predictability about which one of the eight will be chnaged for a given product.
        // However the sequence of all the rows follows a mathenmatical sequence, so by knowing the row, it is possible to determine the product grouping
        // And by knowing the product grouping, it is possible to determine the first row of the product group.
        // 
        // The formula for the position of a number n a mathematical sequence is: = an=a1+d(n-1)
        // where an = the "nth" number in the sequence (equates to the nth Product); a1 = the start row (var=startrow); d = difference between each group (var=rowsperProduct) and n=the actual row number.
        // In the first instance we know the row number from the event data, so we work backwards to solve for the position of that number in the sequence.
        // 
        // 1) calculate the starting row for this product
        // 2) (Row number - starting row) divided by rowsperProduct) plus one.
        // 3) There's only a one-in eight chance that it is an integer, so round down to get first row of this product sequence
        // 4) Then we work forwards; since we know the nth number, we can calculate the row number for the first row for that product.
        // 5) starting row plus (rowsperproduct x (seqwuence number minus 1))
        // By knowing the first row in the product sequence, and the row number that was chnaged, we can calculate which data set was chnaged.
        var productseq = (((SourceRow - startrow) / rowsperProduct) + 1);
        var productseqround = Math.floor(productseq);
        var productline = (startrow + (rowsperProduct * (productseqround - 1)));
        //Logger.log("the row number is "+SourceRow+", but the sequence number for this product is "+productseqround+", and the startrow for this product group = "+productline); //DEBUG

        // identify the field that has changed
        // Source Row number less Productline 
        // if 0 = Catalogue
        // if 1= Display
        // if 3 = ESP
        // if 6 = Mechanic

        var LineNumber = (SourceRow - productline);
        //Logger.log("the calculated Line number = "+LineNumber); //DEBUG

        switch (LineNumber) { // the field references are used in a GetValue statement where the column is a reference to a specific column 
            case 0:
                changedfield = 3;
                changedfieldname = "Catalogue";
                //Logger.log("the changed field was "+changedfieldname); // DEBUG
                break;
            case 1:
                changedfield = 4;
                changedfieldname = "Display";
                //Logger.log("the changed field was "+changedfieldname); // DEBUG
                break;
            case 3:
                changedfield = 5;
                changedfieldname = "ESP";
                //Logger.log("the changed field was "+changedfieldname); // DEBUG
                break;
            case 6:
                changedfield = 6;
                changedfieldname = "Mechanic";
                //Logger.log("the changed field was "+changedfieldname); // DEBUG
                break;
            default:
                //Logger.log("the changed field was none of the above");
                changedfield = 0;
        } //end switch

    } //end if


    // OK, let's get this party started..
    // evaluate the sheet
    if (TargetSheet == 0) {
        //Logger.log("Do nothing because it's not on a sheet we need to worry about"); //DEBUG
    }
    // evaluate the week applying to the change
    else if (weeknumber <= 0) {
        //Logger.log("whatever was changed wasn't one of the key fields"); //DEBUG
    }
    //evaluate the changed field
    else if (changedfield == 0) {
        //Logger.log("Do nothing because it's not a field that we're not worried about"); //DEBUG
    }
    // looks OK to go ahead  
    else {
        //Logger.log("the field was changed for week# "+weeknumber+", lets find the product");

        // trim the Product Code for searching on the Output Sheet  
        var LongProdName = sheet.getRange(productline, 4).getValue();
        var Prodedit = LongProdName.substring(11, (LongProdName.length));
        var ShortProdName = Prodedit.trim();
        //Logger.log("the Product Name is "+LongProdName+", shortened to: "+ShortProdName);// DEBUG

        // test for existence of the TargetSheet  
        var sheeterror = 1; // use this variable as the canary in the coal mine. Set to 1, prima facie error
        var target = SpreadsheetApp.getActiveSpreadsheet().getSheetByName(TargetSheet);
        if (target != null) { // test for exitentce of target; 
            sheeterror = 0; // if target sheet exists, then set sheeterror to zero; that is, sound the all clear
        }
        if (sheeterror != 0) { // now test for a sheet erorr; anything other than zero means there is a problem
            SpreadsheetApp.getUi().alert("WARNING#1: \n\n The Output Sheet: " + TargetSheet + " does NOT exist.\n\n Product: " + LongProdName + ", \nWeek: " + weeknumber + ",\nField: " + changedfieldname + ", \nold value = " + oldValue + " \n new value = " + value + ".\n Date: " + (new Date()));
            // Logger.log("ERROR: The Outout sheet:" + TargetSheet + " doesn't exist. Data changed on sheet:" + ssname + ", Product: " + LongProdName + ", Week# " + weeknumber + ", Field: " + changedfieldname + ", old value=" + oldValue + ", new value=" + value + ", Date " + (new Date())); //DEBUG
            return false;
        }


        // set the data range for the Output sheet and get the data
        var TargetRange = target.getDataRange();
        var TargetValues = TargetRange.getValues();

        // setup the search string
        // Logger.log("target Range = "+TargetRange.getA1Notation()+", search string = '"+ShortProdName+"', week# is "+weeknumber);  // DEBUG
        // Logger.log("TargetValues length = "+TargetValues.length);
        // so lets find a match
        var outputmatch = 1; // use this variable as the canary in the coal mine for not finding a match. Set to 1 = prima facie error
        for (n = 0; n < TargetValues.length; ++n) {
            // iterate row by row and match the week (Column A) and Name (Column B)
            //Logger.log("n = "+n+", product = "+ShortProdName+", week = "+weeknumber);
            if (TargetValues[n][1] == ShortProdName && TargetValues[n][0] == weeknumber) {

                // when we find the result (row number), add plus one to accout for the array starting at zero.
                var result = n + 1;
                // Logger.log("Found a match, n = "+result);  //DEBUG

                // create the co-ordinates for the output cell
                // row number = result, column = chnagedfield calculated earlier
                // Logger.log("update range: row = "+result+", column = "+changedfield); //DEBUG
                var updatecell = target.getRange(result, changedfield);
                //Logger.log("The update cell is "+updatecell.getA1Notation());  // DEBUG

                // Update the cell for the new value
                updatecell.setValue(value);

                // Fix values for Display/Mechanic if they were updated
                if (changedfieldname == "Display") {
                    var displayvalue = value;
                } else {
                    var displayvalue = TargetValues[n][3];
                }
                if (changedfieldname == "Mechanic") {
                    var mechanicvalue = value;
                } else {
                    var mechanicvalue = TargetValues[n][5];
                }

                // define the join1 parameters  
                var join1 = TargetValues[n][1] + " | " + displayvalue + " | " + mechanicvalue; // Bundle, Display, Mechanic
                // set the range for join 1
                var updatejoin1 = target.getRange(result, 7);
                // update join1
                updatejoin1.setValue(join1);

                // define the join2 parameters  
                var join2 = displayvalue + " | " + TargetValues[n][1] + " | " + mechanicvalue; // Display, Bundle, Mechanic
                // set the range for join 2
                var updatejoin2 = target.getRange(result, 8);
                // update join2
                updatejoin2.setValue(join2);

                // the outputmatch value to zero 
                outputmatch = 0;
                //Logger.log("The update cell is "+updatecell.getA1Notation()+", and the new value is "+ value); //DEBUG
                //Logger.log("SUMMARY: Data changed on sheet:" + ssname + ", saved to Output sheet:" + TargetSheet + ", range: " + range.getA1Notation() + ", Product: " + LongProdName + ", Week# " + weeknumber + ", Field: " + changedfieldname + ", old value=" + oldValue + ", new value=" + value + ", Date " + (new Date())); //DEBUG
                return false;

            }

        } // end for n
        if (outputmatch != 0) { // now test for a faliure to update the output sheet; anything other than zero means there is a problem
            // create an error message if we were unable to find a match and could not update the output sheet field
            SpreadsheetApp.getUi().alert("WARNING#2: There was an unidentified problem.\n\n Output Sheet: " + TargetSheet + " does NOT appear to have been updated.\n\n Product: " + LongProdName + ", \nWeek: " + weeknumber + ",\nField: " + changedfieldname + ", \nold value = " + oldValue + " \n new value = " + value + ".\n Date: " + (new Date()));
            return false;
        }
    } // end if

}

【讨论】:

  • 感谢您抽出时间与我一起应对这一挑战。除了让代码工作之外,我现在对触发器的工作方式也有了很好的理解。非常感谢
猜你喜欢
  • 2013-12-11
  • 1970-01-01
  • 2016-05-27
  • 2016-07-23
  • 1970-01-01
  • 2017-01-14
  • 2013-01-18
  • 2012-11-18
  • 2010-10-06
相关资源
最近更新 更多