Postgresql 中的 RRULE 解析答案

【问题标题】：RRULE parsing in PostgresqlPostgresql 中的 RRULE 解析
【发布时间】：2013-01-26 03:14:00
【问题描述】：

目标：从一个 RRULE 字符串（即FREQ=WEEKLY;INTERVAL=2;COUNT=8;WKST=SU;BYDAY=TU,TH）和一个起始时间戳生成一个时间戳列表，每个时间戳代表一个事件发生。由于这些时间会随用户的心血来潮而改变，因此需要 postgresql，因为 (1) 它具有严格的业务逻辑质量和 (2) 触发器（值更改时自动更新行）

替代解决方案：

我最终使用了 plpythonu（用于 postgresql 的 Python 语言）。 dateutil library 有一个很棒的规则解析器。

mydatabase=# CREATE FUNCTION parse_occurrences(rule text, start timestamp) RETURNS timestamp[] AS
mydatabase-# 'from dateutil.rrule import *
mydatabase'# from dateutil.parser import *
mydatabase'# import datetime
mydatabase'# dates = list(rrulestr(rule, dtstart=parse(start)))
mydatabase'# return [str(a)[0:10] for a in dates]'
mydatabase-# LANGUAGE plpythonu;
CREATE FUNCTION
mydatabase=# SELECT parse_occurrences('FREQ=WEEKLY;INTERVAL=2;COUNT=8;WKST=SU;BYDAY=TU,TH'::text, now()::timestamp);                                                                  
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 {"2013-02-14 00:00:00","2013-02-26 00:00:00","2013-02-28 00:00:00","2013-03-12 00:00:00","2013-03-14 00:00:00","2013-03-26 00:00:00","2013-03-28 00:00:00","2013-04-09 00:00:00"}

（原始）努力：将C-library libical 与Postgresql C-Extensions 连接起来。这需要专门的 C 程序来执行以下操作：(1) 将 postgresql 数据类型转换为 C 数据类型，(2) 执行所有必要的 C 库函数，以及 (3) 使用头文件以 postgresql 格式返回数据“ postgres.h"。

连接器文件：ical_recur.c

#include "postgres.h"
#include "icalrecur.h"
#include <time.h> /* for time() */
#include "fmgr.h"

#ifdef PG_MODULE_MAGIC
PG_MODULE_MAGIC;
#endif

PG_FUNCTION_INFO_V1(get_occurrences);

Datum
get_occurrences(PG_FUNCTION_ARGS)
{
        //char*        rule; /* rule string */
        time_t      start; /* start time */
        int         count;
        char        *rrule;

       // *rule = PG_GETARG_CHAR(0);
        start = (time_t) PG_GETARG_INT32(1);
        count = (int) PG_GETARG_INT32(2);
        *rrule = PG_GETARG_CHAR(0);

        time_t   *result[count]; /* output array */

        icalrecur_expand_recurrence(rrule, start, count, *result);

        PG_RETURN_INT32(*result);
}

准备连接器文件：

步骤1：编译到目标文件，创建共享对象，复制到postgresql寻找C-extensions的位置

sudo gcc -I/usr/local/libical/lib -lical -I/usr/include/postgresql/9.2/server -fpic -c ical_recur.c
sudo gcc -shared -L/usr/local/libical/lib -lical -o ical_recur.so ical_recur.o
sudo cp ical_recur.so /usr/lib/postgresql/9.2/lib/

第 2 步：添加 C 找到的 libical lib 文件夹并重新加载配置

sudo echo "/usr/local/libical/lib" >> /etc/ld.so.conf.d/libc.conf
sudo ldconfig

测试连接器文件：

第 1 步：加载 psql 并创建函数

psql
mydatabase=# CREATE FUNCTION get_occurrences(text, integer, integer) RETURNS int[]
mydatabase=- AS '$libdir/ical_recur', 'get_occurrences'
mydatabase=- LANGUAGE C STRICT;
CREATE FUNCTION
mydatabase=#

当前包版：

C 函数使 postgresql 服务器崩溃。

psql (9.2.3)
Type "help" for help.

mydatabase=# SELECT get_occurrences('FREQ=WEEKLY;INTERVAL=2;COUNT=8;WKST=SU;BYDAY=TU,TH', now()::timestamp, 5);
The connection to the server was lost. Attempting reset: Failed.
!>

日志...

2013-02-11 22:03:33 UTC LOG:  server process (PID 22733) was terminated by signal 11: Segmentation fault

2013-02-11 22:03:33 UTC DETAIL:  Failed process was running: SELECT get_occurrences('FREQ=WEEKLY;INTERVAL=2;COUNT=8;WKST=SU;BYDAY=TU,TH', now()::timestamp, 5);
2013-02-11 22:03:33 UTC LOG:  terminating any other active server processes
2013-02-11 22:03:33 UTC WARNING:  terminating connection because of crash of another server process
2013-02-11 22:03:33 UTC DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2013-02-11 22:03:33 UTC HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2013-02-11 22:03:33 UTC LOG:  all server processes terminated; reinitializing
2013-02-11 22:03:33 UTC FATAL:  the database system is in recovery mode
2013-02-11 22:03:33 UTC LOG:  database system was interrupted; last known up at 2013-02-11 21:47:26 UTC
2013-02-11 22:03:33 UTC LOG:  database system was not properly shut down; automatic recovery in progress
2013-02-11 22:03:33 UTC LOG:  redo starts at 0/1903A0C
2013-02-11 22:03:33 UTC LOG:  record with zero length at 0/190E1E0
2013-02-11 22:03:33 UTC LOG:  redo done at 0/190E1B8
2013-02-11 22:03:33 UTC LOG:  last completed transaction was at log time 2013-02-11 22:03:29.641161+00
2013-02-11 22:03:33 UTC LOG:  database system is ready to accept connections
2013-02-11 22:03:33 UTC LOG:  autovacuum launcher started

更新：

我已经更新了方法来解决一些建议。

#include "postgres.h"
#include "icalrecur.h"
#include <time.h> /* for time() */
#include "fmgr.h"

#ifdef PG_MODULE_MAGIC
PG_MODULE_MAGIC;
#endif

PG_FUNCTION_INFO_V1(get_occurrences);

Datum
get_occurrences(PG_FUNCTION_ARGS)
{
        time_t start = (time_t) PG_GETARG_INT32(1); /* convert int to time_t */
        int count = PG_GETARG_INT32(2);
        char rrule = PG_GETARG_CHAR(0);

        char *_rrule = &rrule; /* icalrecur wants a pointer to the rrule */

        time_t  result[count]; /* instantiate the output array */

        int success = icalrecur_expand_recurrence(_rrule, start, count, result);

        /* convert time_t values to int */
        int *output = malloc(sizeof(result));

        int i;
        for(i = 0; i < (int) (sizeof(result) / sizeof(result[0]) - 1); i++){
                output[i] = (int) result[i];
        }

        if(success != 1){
                PG_RETURN_INT32(0);
        } else {
                PG_RETURN_INT32(output);
        }
}

用..构建..

sudo gcc -Wall -Wextra -l/usr/local/libical/lib/ical -I/usr/local/libical/include/libical -I/usr/include/postgresql/9.2/server -fpic -c ical_recur.c
sudo gcc -Wall -shared -static -L/usr/local/libical/lib -lical -o ical_recur.so ical_recur.o
sudo cp libical.so /usr/lib/postgresql/9.2/lib/

进入数据库并执行..

mydb=# CREATE FUNCTION get_occurrences(text, integer, integer) RETURNS int[]
AS '$libdir/ical_recur', 'get_occurrences'
LANGUAGE C STRICT;
CREATE FUNCTION
mydb=# SELECT get_occurrences('FREQ=WEEKLY;INTERVAL=2;COUNT=8;WKST=SU;BYDAY=TU,TH', 1360690024, 5);
The connection to the server was lost. Attempting reset: Failed.
!> \q

相同的日志输出。 documentation注解参数：

int icalrecur_expand_recurrence(char * rule, time_t start, int count, time_t * array)

问题：我们怎样才能让它在不导致 postgres 崩溃的情况下工作？

【问题讨论】：

类似“过于本地化”的定义。
可能，但我认为从 postgres 中的 rrule 字符串生成事件发生是发布和讨论的好东西。
这篇文章是否可以更简洁，即更新可以压缩成一个单一的、有凝聚力的、简洁的帖子吗？感觉就像阅读一个信息非常缓慢的故事。
我们开始吧，删掉 180 行 :) 很抱歉播放的内容！
够简洁了吧？ :)

标签： python postgresql icalendar rrule libical

【解决方案1】：

显示的代码中有两个明显的指针相关缺陷：

第一：

    *rrule = PG_GETARG_CHAR(0);

在这个执行点，rrule 是一个未初始化的指针，这条指令试图将一个字符写入它指向的随机位置。这通常会导致您看到的那种崩溃。

第二：

time_t   *result[count]; /* output array */

icalrecur_expand_recurrence(rrule, start, count, *result);

result 应该是time_t 的数组，而不是time_t* 的数组，并且它应该作为result 传递，而不是*result，此时再次未初始化。

【讨论】：

但是现在它有两个新问题：这个 malloc() 永远不会被释放并且 PG_RETURN_INT32 不应该有一个指针传递给它
我最终通过 dateutil.rrule 库使用了 python :) 用 5 行解决了它。也许效率不高，但它完成了工作。谢谢你的帮助，丹尼尔。
在您的笔记中，postgresql 文档提到当有一个可变长度的 var 时通过引用传递。 postgresql.org/docs/9.2/static/xfunc-c.html#AEN52593 并且我还在某处读到，这里并不需要真正释放 mem（尽管可能不是最佳实践），因为一旦函数调用结束，postgres 就会自动执行它。

【解决方案2】：

现在您可以为 postgresql 使用 this 扩展名。

示例用法：

SELECT * FROM
     unnest(
         rrule_get_occurrences('FREQ=WEEKLY;INTERVAL=1;WKST=MO;UNTIL=20200101T045102Z;BYDAY=SA;BYHOUR=10;BYMINUTE=51;BYSECOND=2'::rrule,
             '2019-12-07 10:51:02+00'::timestamp with time zone)
     );

          unnest
 ------------------------
  2019-12-07 10:51:02+00
  2019-12-14 10:51:02+00
  2019-12-21 10:51:02+00
  2019-12-28 10:51:02+00
 (4 rows)

【讨论】：