Jedis (Redis) 减速

Question

我在 redis 中插入大量文本以逐行存储频率。但是，jedis/redis 在执行一定次数的操作后会变慢，并且需要很长时间才能执行操作，并且程序以错误结束： java.lang.OutOfMemoryError .

这是我的主要测试文件： 公共类临时{

private ExecutorService executor;

public temp() {
    executor = Executors.newFixedThreadPool(5);
}

public static void main(String[] args) {
    temp ob = new temp();

    System.out.println("starting");
    for(long i =0;i<10000000;i++) {
        if (i%10000 == 0) {
            System.out.println(i);
        }
        String x = Integer.toString(new Random().nextInt());
        ob.executor.submit(new Runner1("abra"+x));
        ob.executor.submit(new Runner2("delhi"+x));
    }

    try {
        if (ob.executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS)) {
            System.out.println("completed");
        }
    } catch (Exception ex) {
        ex.printStackTrace();
    }

}


}

这是我的两个跑步者： 亚军1：

public class Runner1 implements Runnable {
//private static RedisClientUtil redisClient = null;
private String key;
private static  Integer count = 0;

public Runner1(String key) {
    this.key = key;
}

public void run() {
    try {

        ArrayList<ArrayList<Object>> cmd = new ArrayList<ArrayList<Object>>();

        String offer_title = this.key + " this is thread1";
        String offer_title_words[] = offer_title.split(" ");
        for (String word : offer_title_words) {
            // INCR the frequency in reddis
            cmd.add(GenerateUtils.getArrayList("incrBy", "test"+word, 1));
        }

        List<Object> responses = RedisbenchmarkTest.getLocalhostJedisPool().executePipelinedAndReturnResponses(0,cmd);
        cmd = null;
        responses = null;

        updateNumberOfRowsInserted();

    } catch (Exception ex) {
        ex.printStackTrace();
    }
}

    private synchronized void updateNumberOfRowsInserted() {
        //logging
       count++;
        if(count%10000==0)
            System.out.println("Thread 1 : " + count);
    }



}

亚军 2：

public class Runner2 implements Runnable {
//private static RedisClientUtil redisClient = null;
private String key;
private static Integer count = 0;

public Runner2(String key) {
    this.key = key;
}

public void run() {
    try {

        ArrayList<ArrayList<Object>> cmd = new ArrayList<ArrayList<Object>>();

        String offer_title = this.key + " this is thread2";
        String offer_title_words [] = offer_title.split(" ");
        for (String word  : offer_title_words) {
            // INCR the category_word in reddis
            cmd.add(GenerateUtils.getArrayList("incrBy","test1"+word,1));
        }                 RedisbenchmarkTest.getLocalhostJedisPool().executePipelinedWithoutReturningResponses(0,cmd);
        updateNumberOfRowsInserted();

    } catch (Exception ex) {
        ex.printStackTrace();
    }
}

private synchronized void updateNumberOfRowsInserted() {
    //logging
    count++;
    if(count%10000==0)
        System.out.println("Thread 2 : " + count);
}

}

这是我的 Redis 客户端：

public class RedisbenchmarkTest {

   private static RedisClientUtil localhostJedisPool;


 private static final JedisPoolConfig standardJedisPoolConfig = new JedisPoolConfig() {{
    setMaxTotal(500);
    setMaxIdle(20);
    setMaxWaitMillis(500);
    setTestOnBorrow(false);
    setTestOnReturn(false);
}};

 private static final int semiLowTimeout = 500;

 static {
    initialize();
 }

 public static void initialize() {
    localhostJedisPool = new RedisClientUtil(
            standardJedisPoolConfig
            , "localhost"
            , 6379
            , semiLowTimeout
    );
  }



   public static RedisClientUtil getLocalhostJedisPool() {
      if (localhostJedisPool == null) {
          initialize();
   }
   return localhostJedisPool;
   }
  }

Answer 1

• 您应该在 redis.conf 中禁用 bgsave 如果 redis 启动 bgsave，通常在保存完成之前会出现巨大的延迟。
• 您应该确保 redis.conf 中的 maxmemory 足够高。我猜您的实例已满。

Answer 2

Redis 不是内存不足的，而是您的 Java 进程。

您没有完整发布实际错误，我不是 Java 人，但Caused by: java.lang.OutOfMemoryError: Java heap space 可能会为您提供一些答案。