【问题标题】:GAE NBD entities still seen in Datastore viewer after they are deleted from Python and Memchach is flushedGAE NBD 实体在从 Python 中删除并刷新 Memchach 后仍可在 Datastore 查看器中看到
【发布时间】:2016-10-20 11:41:06
【问题描述】:

下面是一个最小的 Python 示例,展示了向 NDB添加实体,然后删除所有实体。

但是,尽管实体似乎已被删除,Datastore Viewer 仍然显示它们 - 即使在我刷新 Memcache. 之后也是如此

我应该进行哪些更改才能删除 NDB 实体?

我的代码:

from google.appengine.ext import ndb
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app

from models import Data

KEY = "fastsimon"
datum_key = dict()

class InvalidHandler(webapp.RequestHandler):
    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.out.write('Invalid entry')

class GetHandler(webapp.RequestHandler):
    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.out.write('Get!\n')
        name = self.request.get('name')
        out_str = "Should not be seen"
        try:
            ancestor_key = datum_key[name]
            qry = Data.owner_query(ancestor_key)
            data = qry.fetch()
            out_str = data[-1].value
        except KeyError:
            out_str = "None"
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.out.write(out_str)

class EndHandler(webapp.RequestHandler):
    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.out.write('End!')
        qry = Data.query().iter(keys_only=True)
        print "before delete"
        list_of_keys = list()
        for q in qry:
            print "ndb.Key('Datum',q):",ndb.Key('Datum',q.integer_id())
            list_of_keys.append(ndb.Key('Datum',q.integer_id()))
        print "list_of_keys:",list_of_keys
        ndb.delete_multi(list_of_keys)
        print "after delete"
        for q in qry:
            print "ndb.Key('Datum',q):",ndb.Key('Datum',q.integer_id())

class SetHandler(webapp.RequestHandler):
    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.out.write('Set!\n')
        name=self.request.get('name')
        data = Data(parent=ndb.Key("Datum", KEY),
                    name=name,
                    value=self.request.get('value'))
        print "data:",data
        ret_key = data.put()
        datum_key[name] = ret_key

def main():
    application = webapp.WSGIApplication([
            ('/get', GetHandler),
            ('/set', SetHandler),
            ('/end', EndHandler)],     
            debug=False) 
    global app
    app = application
    run_wsgi_app(app)

if __name__ == 'main':
    main()

models.py:

from google.appengine.ext import ndb

class Data(ndb.Model):
    name = ndb.StringProperty() # Upto 500 characters
    value = ndb.TextProperty(required=True) # Unlimited length

    @classmethod
    def owner_query(cls, parent_key):
        return cls.query(ancestor=parent_key).order(cls.name)

我采取的步骤:

  1. 已清除数据存储:

  1. 在浏览器中执行了以下操作:
  2. http://localhost:8080/set?name=a&value=1
  3. http://localhost:8080/set?name=b&value=2
  4. http://localhost:8080/end

调试行是:

data: Data(key=Key('Datum', 'fastsimon', 'Data', None), name=u'a', value=u'1')
INFO     2016-10-20 11:29:34,213 module.py:788] default: "GET /set?name=a&value=1 HTTP/1.1" 200 5
data: Data(key=Key('Datum', 'fastsimon', 'Data', None), name=u'b', value=u'2')
INFO     2016-10-20 11:29:50,442 module.py:788] default: "GET /set?name=b&value=2 HTTP/1.1" 200 5
before delete
ndb.Key('Datum',q): Key('Datum', 4661104668049408)
ndb.Key('Datum',q): Key('Datum', 5787004574892032)
list_of_keys: [Key('Datum', 4661104668049408), Key('Datum', 5787004574892032)]
after delete
INFO     2016-10-20 11:30:04,125 module.py:788] default: "GET /end HTTP/1.1" 200 4

但是,当我转到 Datastore 查看器并按下 Flush Memcache 时,我仍然在那里看到实体:

我应该进行哪些更改才能删除 NDB 实体?

【问题讨论】:

  • 您还应该打印查询中使用的祖先键并检查数据存储中的实体是否与该祖先匹配
  • 您的 Data.owner_queryEndHandler 查询不匹配:一个是祖先查询,另一个不是,因此它们产生的结果不会重叠。
  • EndHandler 的目的是删除所有属于我的实体,所以 - 我还需要检查祖先,@DanCornilescu 吗?此外,after delete 行之后没有打印任何内容的事实是否表明实体已被删除?
  • 而您在EndHandler 中构建的list_of_keys 可能包含不存在的密钥:在执行ndb.Key('Datum',q.integer_id()) 时,您将创建一个没有祖先的密钥,这如果有祖先,则不会匹配原始的 q 键。如果密钥不存在,ndb.delete_multi() 不会抱怨。
  • 您为什么不直接调用ndb.delete_multi(qry) - 因为qry 已经包含您要删除的实际密钥?

标签: python google-app-engine google-cloud-datastore app-engine-ndb


【解决方案1】:

根据@DanCornilescu 的优秀建议,我更改了EndHandler 的代码,它现在像冠军一样删除了实体。

代码:

from google.appengine.ext import ndb
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app

from models import Data

KEY = "fastsimon"
datum_key = dict()

class InvalidHandler(webapp.RequestHandler):
    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.out.write('Invalid entry')

class GetHandler(webapp.RequestHandler):
    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.out.write('Get!\n')
        name = self.request.get('name')
        out_str = "Should not be seen"
        try:
            ancestor_key = datum_key[name]
            qry = Data.owner_query(ancestor_key)
            data = qry.fetch()
            out_str = data[-1].value
        except KeyError:
            out_str = "None"
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.out.write(out_str)


class EndHandler(webapp.RequestHandler):
    def show_entities(self):
        for q in datum_key.values():
            k = ndb.Key('Datum', 'fastsimon', 'Data',q.integer_id())
            print "entity for",k,"=>", ndb.Key('Datum', 'fastsimon', 'Data',q.integer_id()).get()

    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.out.write('End!')
        print "datum_key.values():",datum_key.values()
        print "before delete"
        self.show_entities()
        ndb.delete_multi(datum_key.values())
        print "after delete"
        self.show_entities()


class SetHandler(webapp.RequestHandler):
    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.out.write('Set!\n')
        name=self.request.get('name')
        data = Data(parent=ndb.Key("Datum", KEY),
                    name=name,
                    value=self.request.get('value'))
        ret_key = data.put()
        datum_key[name] = ret_key
        print "data:",data

def main():
    application = webapp.WSGIApplication([
            ('/get', GetHandler),
            ('/set', SetHandler),
            ('/end', EndHandler)],     
            debug=False) 
    global app
    app = application
    run_wsgi_app(app)

if __name__ == 'main':
    main()

调试行:

data: Data(key=Key('Datum', 'fastsimon', 'Data', 4943129400573952), name=u'z', value=u'55')
INFO     2016-10-20 13:40:17,008 module.py:788] default: "GET /set?name=z&value=55 HTTP/1.1" 200 5
data: Data(key=Key('Datum', 'fastsimon', 'Data', 6069029307416576), name=u'x', value=u'88')
INFO     2016-10-20 13:40:28,556 module.py:788] default: "GET /set?name=x&value=88 HTTP/1.1" 200 5
datum_key.values(): [Key('Datum', 'fastsimon', 'Data', 4943129400573952), Key('Datum', 'fastsimon', 'Data', 6069029307416576)]
before delete
entity for Key('Datum', 'fastsimon', 'Data', 4943129400573952) => Data(key=Key('Datum', 'fastsimon', 'Data', 4943129400573952), name=u'z', value=u'55')
entity for Key('Datum', 'fastsimon', 'Data', 6069029307416576) => Data(key=Key('Datum', 'fastsimon', 'Data', 6069029307416576), name=u'x', value=u'88')
after delete
entity for Key('Datum', 'fastsimon', 'Data', 4943129400573952) => None
entity for Key('Datum', 'fastsimon', 'Data', 6069029307416576) => None
INFO     2016-10-20 13:40:40,517 module.py:788] default: "GET /end HTTP/1.1" 200 4

Datastore 查看器不显示任何实体。

【讨论】:

    【解决方案2】:

    一般来说,当您为实体使用祖先时,您需要小心查询 - 指定和不指定祖先键的查询会产生不同(非重叠)的结果。见Ancestor Queries

    问题的根本原因是,如果各个实体有祖先,您在 list_of_keys 中构建的要删除的密钥与原始密钥不匹配,因此传递给 ndb.delete_multi 的内容可能会丢失一些您打算删除的键。

    ndb.Key('Datum',q.integer_id())没有祖先的实体生成一个密钥,如果q 是具有祖先的实体的密钥,则该密钥与q 密钥不匹配。

    一个简单的解决方案是不转换您要删除的密钥,而是将它们直接传递给ndb.delete_multi(),例如您可以简单地使用ndb.delete_multi([key for key in qry]) 而不是ndb.delete_multi(list_of_keys)

    【讨论】:

    • 我不认为我可以通过qry,因为它是QueryIterator 类型。
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2014-12-15
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多