如何避免 Terracotta BigMemory Go 中的数据丢失答案

【问题标题】：How to avoid lost data in Terracotta BigMemory Go如何避免 Terracotta BigMemory Go 中的数据丢失
【发布时间】：2013-09-03 17:41:16
【问题描述】：

我正在测试 BigMemory Go 的性能，发现数据丢失。

这是单元测试：

public class BigMemoryGoPerformanceTest {

    public static void main(String[] args) throws IOException {
        BigMemoryGoPerformanceTest test = new BigMemoryGoPerformanceTest();
        test.testBigMemoryGoPerformance(args[0], Long.valueOf(args[1]), Long.valueOf(args[2]));
    }

    @DataProvider(name = "bigMemoryGoPerformance")
    public Object[][] bigMemoryGoPerformance() {
        return new Object[][] {
                   { "very small", 250000, 1 }
                  ,{ "small", 1000000, 2 }
                  ,{ "large", 10000000, 2 }
                  ,{ "very large", 50000000, 4 }
        };
    }

    @Test(dataProvider="bigMemoryGoPerformance")
    public void testBigMemoryGoPerformance(String testName, long maxRegisters, long externalMemory) throws IOException {

        Configuration managerConfiguration = new Configuration();
        managerConfiguration.updateCheck(true)
            .monitoring(Configuration.Monitoring.AUTODETECT)
            .name("config")
            .cache(new CacheConfiguration()
                .name("testperformance")
                .maxBytesLocalHeap(128, MemoryUnit.MEGABYTES)
                .maxBytesLocalOffHeap(externalMemory, MemoryUnit.GIGABYTES)
                .eternal(true)
            );

        CacheManager manager = CacheManager.create(managerConfiguration);
        Cache cache = manager.getCache("testperformance");

        try {
            System.out.println("First pass: insert " + maxRegisters + " nodes.");
            long mms = System.currentTimeMillis();
            for (long i = 0; i < maxRegisters; i++) {
                ItemForTesting item = new ItemForTesting();
                item.id = i;
                item.lastWay = i;
                item.latitude = new Double(i);
                item.longitude = new Double(i);
                cache.put( new Element(i, item) );
            }
            long timeInMMS = System.currentTimeMillis() - mms;
            System.out.println(testName + " --> Inserted " + maxRegisters + " registers in " + timeInMMS + " mms. Performance: " + ((long)(maxRegisters * 1000d / timeInMMS)) + " regs per seconds writing." );


            System.out.println("Second pass: reading " + maxRegisters + " nodes.");
            mms = System.currentTimeMillis();
            for (long i = 0; i < maxRegisters; i++) {
                Element element = cache.get(i);
                ItemForTesting item = (ItemForTesting)element.getObjectValue(); // <--- Null point exception !!!!!
            }
            timeInMMS = System.currentTimeMillis() - mms;
            System.out.println(testName + " --> Read " + maxRegisters + " registers in " + timeInMMS + " mms. Performance: " + ((long)(maxRegisters * 1000d / timeInMMS)) + " regs per seconds reading." );

            System.out.println("Third pass: updating " + maxRegisters + " nodes.");
            mms = System.currentTimeMillis();
            for (long i = 0; i < maxRegisters; i++) {
                Element element = cache.get(i);
                ItemForTesting item = (ItemForTesting)element.getObjectValue();
                item.latitude = item.latitude +1;
                cache.put( new Element(i, item) );
            }
            timeInMMS = System.currentTimeMillis() - mms;
            System.out.println(testName + " --> Updated " + maxRegisters + " registers in " + timeInMMS + " mms. Performance: " + ((long)(maxRegisters * 1000d / timeInMMS)) + " regs per seconds reading." );

            System.out.println("Fourth pass: deleting " + maxRegisters + " nodes.");
            mms = System.currentTimeMillis();
            for (long i = 0; i < maxRegisters; i++) {
                Element element = cache.get(i);
                ItemForTesting item = (ItemForTesting)element.getObjectValue();
                item.latitude = item.latitude +1;
                cache.remove( new Element(i, item) );
            }
            timeInMMS = System.currentTimeMillis() - mms;
            System.out.println(testName + " --> Removed " + maxRegisters + " registers in " + timeInMMS + " mms. Performance: " + ((long)(maxRegisters * 1000d / timeInMMS)) + " regs per seconds reading." );


        } finally {
            if (manager != null) manager.shutdown();
        }

    }

}

在“非常大”中，使用 50.000.000 次迭代，在第二遍读取数据时，返回 null。所以数据丢失了！！配置设置为永恒=真。如何避免数据丢失？

怎么了？

谢谢！

【问题讨论】：

标签： ehcache terracotta ehcache-bigmemory

【解决方案1】：

我怀疑您需要固定元素以避免它们过期。尝试如下所示的配置：

<cache name="myBigMemoryGoStore"
    maxBytesLocalHeap="1M"
    eternal="true"
    maxBytesLocalOffHeap="2G">
    <persistence strategy="none"/>
    <pinning store="inCache" />
    <sizeOfPolicy maxDepth="100000" maxDepthExceededBehavior="continue"/>

http://terracotta.org/documentation/4.0/bigmemorygo/configuration/data-life

【讨论】：

【解决方案2】：

由于堆/堆外压力，数据可能会被驱逐。如上所述，您可以使用固定来确保您永远不会丢失数据。但仅当您确定缓存大小时才应使用固定，因为固定缓存可能会增长到无限大小或导致 OOME，具体取决于您可能使用的版本。

关于测试 BigMemory Go 性能的另一个注意事项，我建议使用多线程测试来获得更多吞吐量，而不是仅仅使用大量迭代。

干杯，里达夫

【讨论】：