【问题标题】:Fetch data from Ignite using AsCacheQueryable with Generic query perfromance efficient使用具有通用查询性能的 AsCacheQueryable 从 Ignite 获取数据
【发布时间】:2020-11-26 14:18:38
【问题描述】:

我正在尝试在我的类 CacheHandler 中添加 Where 函数以从 Ignite 缓存中获取数据 我正在寻找使通用查询与 AsCacheQueryableWhere 子句一起使用以从 ignite 缓存返回数据的方法

//调用

var data = CacheHandler.Where2<LocalDb>(x => x.Value.localId== localId).FirstOrDefault();

//功能

    public ICache<string, T> GetCache<T>()
    {            
        var cacheName = CacheNames.GetCacheNameFromType<T>();
        
        if (cacheName == "")
        {
            throw new Exception("Invalid Cache name");
        }

        return _ignite.GetCache<string, T>(cacheName);
    }

    public List<T> Where2<T>(Func<ICacheEntry<string, T>, bool> query)
    {
        // 'System.Collections.Generic.List<Apache.Ignite.Core.Cache.ICacheEntry<string, T>>'
        var queryResult = GetCache<T>().AsCacheQueryable().Where(x => query(x));

        if (queryResult == null)
        {
            return new List<T>();
        }

        return queryResult.Select(x => x.Value).ToList();
    }

使用上述代码上下文忽略以下错误中的表名称。但是尝试获取数据时所有表的类似错误。 我在模型中的所有字段都有[QuerySqlField]

我收到以下错误:

    ""Hwb_Entity_Cache"".ENTITY AS _T0 WHERE _T0._KEY,[*] _T0._VAL "; SQL statement:
select _T0._VAL from "Hwb_Entity_Cache".ENTITY as _T0 where _T0._KEY, _T0._VAL [42000-197] (6e3f4055)
Apache.Ignite.Core.Common.IgniteException: Failed to parse query. Syntax error in SQL statement "SELECT _T0._VAL FROM ""Hwb_Entity_Cache"".ENTITY AS _T0 WHERE _T0._KEY,[*] _T0._VAL "; SQL statement:
select _T0._VAL from "Hwb_Entity_Cache".ENTITY as _T0 where _T0._KEY, _T0._VAL [42000-197]
 ---> Apache.Ignite.Core.Common.JavaException: class org.apache.ignite.IgniteCheckedException: Failed to parse query. Syntax error in SQL statement "SELECT _T0._VAL FROM ""Hwb_Entity_Cache"".ENTITY AS _T0 WHERE _T0._KEY,[*] _T0._VAL "; SQL statement:
select _T0._VAL from "Hwb_Entity_Cache".ENTITY as _T0 where _T0._KEY, _T0._VAL [42000-197]
    at org.apache.ignite.internal.processors.platform.utils.PlatformUtils.unwrapQueryException(PlatformUtils.java:520)
    at org.apache.ignite.internal.processors.platform.cache.PlatformCache.runFieldsQuery(PlatformCache.java:1321)
    at org.apache.ignite.internal.processors.platform.cache.PlatformCache.processInStreamOutObject(PlatformCache.java:940)
    at org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutObject(PlatformTargetProxyImpl.java:79)
Caused by: javax.cache.CacheException: Failed to parse query. Syntax error in SQL statement "SELECT _T0._VAL FROM ""Hwb_Entity_Cache"".ENTITY AS _T0 WHERE _T0._KEY,[*] _T0._VAL "; SQL statement:
select _T0._VAL from "Hwb_Entity_Cache".ENTITY as _T0 where _T0._KEY, _T0._VAL [42000-197]
    at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:817)
    at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:412)
    at org.apache.ignite.internal.processors.platform.cache.PlatformCache.runFieldsQuery(PlatformCache.java:1315)
    ... 2 more
Caused by: class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to parse query. Syntax error in SQL statement "SELECT _T0._VAL FROM ""Hwb_Entity_Cache"".ENTITY AS _T0 WHERE _T0._KEY,[*] _T0._VAL "; SQL statement:
select _T0._VAL from "Hwb_Entity_Cache".ENTITY as _T0 where _T0._KEY, _T0._VAL [42000-197]
    at org.apache.ignite.internal.processors.query.h2.QueryParser.parseH2(QueryParser.java:584)
    at org.apache.ignite.internal.processors.query.h2.QueryParser.parse0(QueryParser.java:210)
    at org.apache.ignite.internal.processors.query.h2.QueryParser.parse(QueryParser.java:131)
    at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1103)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2406)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2402)
    at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2919)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$1(GridQueryProcessor.java:2422)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2460)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2396)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2323)
    at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:802)
    ... 4 more

有人可以建议在这里更正传递和阅读查询的方式吗?

【问题讨论】:

  • 这可能是一个错误。您能否提供一个我可以运行和调试的最小复制器?
  • 如果您在缓存中添加少量记录,并尝试使用上述功能进行检索,因为它会为您重现。不过,您将不得不更改查询。我在GetCache 中添加了一个函数,另外请注意,当我将查询传递为`var queryResult = GetCache().AsCacheQueryable().Where(query);` 而不是谓词时。它似乎工作

标签: c# .net linq generics ignite


【解决方案1】:

AsCacheQueryable 是 Ignite.NET LINQ 提供程序的一部分 - 它将 LINQ 表达式树 转换为 Ignite SQL 方言。上面代码的问题是Func&lt;ICacheEntry&lt;string, T&gt;, bool&gt; query - 它应该是Expression&lt;Func&lt;ICacheEntry&lt;string, T&gt;, bool&gt;&gt; query。这对我有用:

public List<T> Where2<T>(Expression<Func<ICacheEntry<string, T>, bool>> query)
{
    var queryResult = GetCache<T>().AsCacheQueryable().Where(query);

    return queryResult.Select(x => x.Value).ToList();
}

【讨论】:

  • 是的,正如我在上面的评论中提到的,这现在可以工作了。但是当我对其进行一些性能测试时,这似乎会随着数据的增长而降级在我直接运行的地方,这要快得多@987654325 @ 任何线索为什么一定会发生这种情况?
  • 嗯.. 你所做的是正确的,我可以看到它是高效的。忽略我之前的评论。我之前没有在我的函数中添加Expression,所以它表现不佳。现在确实如此。谢谢老哥!
  • 您能否详细解释一下不将Expression 添加到函数签名中也会影响性能。
  • @Sohan 例如,这类似于 EntityFramework 的工作方式。 Expression 可以在运行时分析并转换为 SQL 查询。原始Func&lt;&gt; 只是一个函数指针 - 它不能转换为 SQL => 这可能会导致错误或性能下降,因为必须在本地提取数据以使用该 Func 过滤,而不是服务器端SQL 过滤。
  • 谢谢!有意义
猜你喜欢
  • 2022-07-23
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2015-08-28
  • 1970-01-01
相关资源
最近更新 更多