通过 Python 搜索 API 搜索间歇性超时

Question

我们有一个应用程序，它基本上只是用于请求创建团队驱动器的表单提交。它托管在 Google App Engine 上。

此超时错误来自表单中的单个字段，该字段仅对电子邮件地址进行了预先输入。域上的所有名称都在数据存储中建立索引，大约 300k 实体 - 没有直接从目录 api 中提取任何内容。搜索 10 秒后（通过 Python Google Search API），它将超时。这目前是间歇性的，但错误的频率一直在增加。

Error: line 280, in get_result raise _ToSearchError(e) Timeout: Failed to complete request in 9975ms

基本上，加快搜索速度会解决问题。我查看了代码，我认为那里没有任何改进的余地。我不确定增加实例类是否会改善这一点，它目前是 F2。或者，也许还有另一种方法可以提高索引效率。但是，我不完全确定如何做到这一点。任何想法将不胜感激。

搜索代码：

class LookupUsersorGrpService(object):

    '''
    lookupUsersOrGrps accepts various params and performs  search

    ''' 
    def lookupUsersOrGrps(self,params):
        search_results_json = {}
        search_results = []
        directory_users_grps = GoogleDirectoryUsers()
        error_msg = 'Technical error'
        query = ''

        try:

            #Default few values if not present
            if ('offset' not in params) or (params['offset'] is None):      
                params['offset'] = 0
            else:
                params['offset'] = int(params['offset'])

            if ('limit' not in params) or (params['limit'] is None): 
                params['limit'] = 20
            else:
                params['limit'] = int(params['limit'])

            #Search related to field name
            query = self.appendQueryParam(q=query, p=params,  qname='search_name',  criteria=':',  pname='query', isExactMatch=True,splitString=True)

            #Search related to field email
            query = self.appendQueryParam(q=query, p=params,  qname='search_email',  criteria=':',  pname='query', isExactMatch=True, splitString=True)

            #Perform search
            log.info('Search initialized :\"{}\"'.format(query) )

            # sort results by name ascending
            expr_list = [search.SortExpression(expression='name', default_value='',direction=search.SortExpression.ASCENDING)]
            # construct the sort options
            sort_opts = search.SortOptions(expressions=expr_list)

            #Prepare the search index
            index = search.Index(name= "GoogleDirectoryUsers",namespace="1") 
            search_query = search.Query(
                    query_string=query.strip(),
                    options=search.QueryOptions(
                                limit=params['limit'],
                                offset=params['offset'],
                                sort_options=sort_opts,
                                returned_fields = directory_users_grps.get_search_doc_return_fields()
                                ))                
            #Execute the search query
            search_result = index.search(search_query)

            #Start collecting the values
            total_cnt = search_result.number_found
            params['limit'] = len(search_result.results)

            #Prepare the response object    
            for teamdriveDoc in search_result.results:
                teamdriveRecord = GoogleDirectoryUsers.query(GoogleDirectoryUsers.email==teamdriveDoc.doc_id).get()
                if teamdriveRecord:
                    if teamdriveRecord.suspended == False:
                        search_results.append(teamdriveRecord.to_dict())

            search_results_json.update({"users" : search_results})
            search_results_json.update({"limit" : params['limit'] if len(search_results)>0 else '0'})
            search_results_json.update({"total_count" : total_cnt if len(search_results)>0 else '0'})
            search_results_json.update({"status" : "success"})

        except Exception as e:
            log.exception("Error in performing search")
            search_results_json.update({"status":"failed"})
            search_results_json.update({"description":error_msg})             

        return search_results_json   



    ''' Retrieves the given param from dict and adds to query if exists    
    '''
    def appendQueryParam(self, q='', p=[], qname=None, criteria='=', pname=None, 
        isExactMatch = False, splitString = False, defaultValue=None):        

        if (pname in p) or (defaultValue is not None):
            if len(q) > 0:
                q += ' OR '
            q += qname 
            if criteria:
                q += criteria 

            if defaultValue is None:
                val = p[pname]
            else:
                val = defaultValue            

            if splitString:
                val = val.replace("", "~")[1: -1]

            #Helps to retain passed argument as it is, example email
            if isExactMatch:
                q +=  "\"" +val + "\""
            else:
                q +=  val

        return q

Answer 1

索引实例的 search 方法接受 deadline 参数，因此您可以使用它来增加您愿意等待搜索响应的时间：

search_result = index.search(search_query, deadline=30)

documentation 没有为 deadline 指定可接受的值，但其他 App Engine 服务倾向于接受最长 60 秒的值。