如何在单元测试中计算 sqlalchemy 查询

Question

在 Django 中，我经常断言应该进行的查询数量，以便单元测试捕获新的 N+1 查询问题

from django import db
from django.conf import settings
settings.DEBUG=True

class SendData(TestCase):
    def test_send(self):
        db.connection.queries = []
        event = Events.objects.all()[1:]
        s = str(event) # QuerySet is lazy, force retrieval
        self.assertEquals(len(db.connection.queries), 2)

在 SQLAlchemy 中通过设置 echo 标志启用对 STDOUT 的跟踪 引擎

engine.echo=True

编写计算 SQLAlchemy 查询次数的测试的最佳方法是什么？

class SendData(TestCase):
    def test_send(self):
        event = session.query(Events).first()
        s = str(event)
        self.assertEquals( ... , 2)

Answer 1

为此我创建了一个上下文管理器类：

class DBStatementCounter(object):
    """
    Use as a context manager to count the number of execute()'s performed
    against the given sqlalchemy connection.

    Usage:
        with DBStatementCounter(conn) as ctr:
            conn.execute("SELECT 1")
            conn.execute("SELECT 1")
        assert ctr.get_count() == 2
    """
    def __init__(self, conn):
        self.conn = conn
        self.count = 0
        # Will have to rely on this since sqlalchemy 0.8 does not support
        # removing event listeners
        self.do_count = False
        sqlalchemy.event.listen(conn, 'after_execute', self.callback)

    def __enter__(self):
        self.do_count = True
        return self

    def __exit__(self, *_):
        self.do_count = False

    def get_count(self):
        return self.count

    def callback(self, *_):
        if self.do_count:
            self.count += 1

Answer 2

使用 SQLAlchemy Core Events 记录/跟踪执行的查询（您可以从单元测试中附加它，这样它们就不会影响您在实际应用程序中的性能：

event.listen(engine, "before_cursor_execute", catch_queries)

现在您编写函数catch_queries，其方式取决于您的测试方式。例如，您可以在测试语句中定义此函数：

def test_something(self):
    stmts = []
    def catch_queries(conn, cursor, statement, ...):
        stmts.append(statement)
    # Now attach it as a listener and work with the collected events after running your test

上述方法只是一个灵感。对于扩展情况，您可能希望在每次测试后清空事件的全局缓存。原因是在 0.9（当前开发）之前没有 API 可以删除事件侦听器。因此，创建一个访问全局列表的全局侦听器。

Answer 3

使用flask_sqlalchemy.get_debug_queries() btw 的方法怎么样。这是Flask Debug Toolbar 内部使用的方法，检查其source

from flask_sqlalchemy import get_debug_queries


def test_list_with_assuring_queries_count(app, client):
    with app.app_context():
        # here generating some test data
        for _ in range(10):
            notebook = create_test_scheduled_notebook_based_on_notebook_file(
                db.session, owner='testing_user',
                schedule={"kind": SCHEDULE_FREQUENCY_DAILY}
            )
            for _ in range(100):
                create_test_scheduled_notebook_run(db.session, notebook_id=notebook.id)
    with app.app_context():
        # after resetting the context call actual view we want asserNumOfQueries
        client.get(url_for('notebooks.personal_notebooks'))
        assert len(get_debug_queries()) == 3

请记住，要重置上下文和计数，您必须在要测量的确切内容之前调用 with app.app_context()。

Answer 4

@omar-tarabai's solution 的略微修改版本，在退出上下文时移除事件侦听器：

from sqlalchemy import event

class QueryCounter(object):
    """Context manager to count SQLALchemy queries."""

    def __init__(self, connection):
        self.connection = connection.engine
        self.count = 0

    def __enter__(self):
        event.listen(self.connection, "before_cursor_execute", self.callback)
        return self

    def __exit__(self, *args, **kwargs):
        event.remove(self.connection, "before_cursor_execute", self.callback)

    def callback(self, *args, **kwargs):
        self.count += 1

用法：

with QueryCounter(session.connection()) as counter:
     session.query(XXX).all()
     session.query(YYY).all()

print(counter.count)  # 2