Log4j 2 JSON 模式布局 + 记录 JSON 有效负载

Question

我正在使用 ELK 堆栈以及 log4j 2 via sl4j 和 json 模式布局 来记录消息。我的所有日​​志都记录为json 消息。此外，在我的一个日志中，我正在尝试记录从第三方服务收到的json 响应。但是此响应 json 正文未附加到 json 结构中。但它宁愿附加为包含转义字符的字符串。

最终日志是如何被注销的。

 {
    "timeMillis": 1471862316416,
    "thread": "FioranoMQ Pubsub Session Thread",
    "level": "INFO",
    "loggerName": "com.mlp.eventing.bridge.fiorano.TopicMessageListener",
    "message": "{\"Msgtype\":\"SentToRabbitMqTest\",\"MessageData\":\"10\",\"opration\":\"devide\"}",
    "endOfBatch": false,
    "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
    "threadId": 28,
    "threadPriority": 5
}

在上面的消息段中附加为转义字符串，而不是整个json 结构。我的预期输出应该是

{
    "timeMillis": 1471862316416,
    "thread": "FioranoMQ Pubsub Session Thread",
    "level": "INFO",
    "loggerName": "com.mlp.eventing.bridge.fiorano.TopicMessageListener",
    "message": {
        "Msgtype": "SentToRabbitMqTest",
        "MessageData": "10",
        "opration": "devide"
    },
    "endOfBatch": false,
    "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
    "threadId": 28,
    "threadPriority": 5
}

我希望使用 shipper.conf 中的 json 的 grok 过滤器提取消息段中的字段

以下是我的配置：- log4j2.xml

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="info"> <!-- log4j internals tracing -->
    <properties>
        <property name="pattern">%d{yyyy-MM-dd HH:mm:ss.SSS} | %-5.5p | %-20.20C:%-5.5L | %msg%n</property>
        <property name="filePath">/opt/mlp/logs</property> 
        <property name="fileName">logs</property>
    </properties>
    <Appenders>
        <RollingFile name="RollingFile" fileName="${filePath}/${fileName}.log"
                     filePattern="${filePath}/${fileName}-%d{yyyy-MM-dd}-%i.log" append="true">
            <JSONLayout complete="false" compact="true" eventEol="true" />  
            <PatternLayout>
                <pattern>${pattern}</pattern>
            </PatternLayout>
            <Policies>
                <SizeBasedTriggeringPolicy size="1000 KB"/> 
            </Policies>l
        </RollingFile>
        <Console name="STDOUT" target="SYSTEM_OUT">
            <PatternLayout>
                <pattern>${pattern}</pattern>
            </PatternLayout>
        </Console>
    </Appenders>
    <Loggers>
        <Root level="debug">
            <AppenderRef ref="RollingFile"/>
            <AppenderRef ref="STDOUT"/>
        </Root>
    </Loggers>
</Configuration>

示例代码sn-p

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

class A {
private static final Logger LOG = LoggerFactory.getLogger(Main.class);

public void testMethod()  {

JSONObject responseJson = callService();// json simple object
LOG.info(responseJson);

}

}

maven 依赖

<dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.21</version>
        </dependency>

        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-slf4j-impl</artifactId>
            <version>2.6.2</version>
        </dependency>

        <!-- end adding sl4j 2 for the message bridge -->

        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.6.2</version>
        </dependency>

        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.6.2</version>
        </dependency>

        <!--
        to enable json support for log4j enable following libraries
        -->

        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-core</artifactId>
            <version>2.7.5</version>
        </dependency>

        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.7.5</version>
        </dependency>

        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-annotations</artifactId>
            <version>2.7.5</version>
        </dependency>

Answer 1

为了能够不转义地记录 JSON，您应该使用 Log4j2 记录器而不是 Slf4j。此功能从 Log4j 2.11 开始可用。

Log4j 记录器能够记录ObjectMessage，它将被转换为嵌套的 JSON。 ObjectMessage 构造函数接受 Map，因此 JSONObject 必须转换为 map（例如，在 Jackson ObjectMapper 的帮助下）。

在布局配置中添加objectMessageAsJsonObject="true":

<JSONLayout complete="false" compact="true" eventEol="true" objectMessageAsJsonObject="true" />

完整的工作示例：

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.message.ObjectMessage;
import org.json.JSONObject;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

public class test {

    public static void main(String[] args) throws IOException {
        HashMap<Object, Object> map = new HashMap<>();
        map.put("foo", "bar");
        JSONObject jsonObject = new JSONObject(map);

        Map<String, Object> newMap = new ObjectMapper().readValue(jsonObject.toString(), new TypeReference<Map<String, Object>>() {});

        Logger log4jLogger = LogManager.getLogger("mainLogger");
        log4jLogger.info(new ObjectMessage(newMap));
    }
}

这将产生：

{"thread":"main","level":"INFO","loggerName":"mainLogger","message":{"foo":"bar"},"endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","instant":{"epochSecond":1548434758,"nanoOfSecond":572000000},"threadId":1,"threadPriority":5}