如何在 Dropwizard 的文件中记录 HTTP 请求？答案

【问题标题】：How to log HTTP requests in file in Dropwizard?如何在 Dropwizard 的文件中记录 HTTP 请求？
【发布时间】：2020-11-29 04:56:54
【问题描述】：

我的 Dropwizard（版本 1.3.8）应用程序将 HTTP 请求记录到文件中时遇到问题。我按照 Dropwizard 的 documentation 配置 YAML 文件，我的 config.yml 看起来像这样：

logging:
  level: INFO
  loggers:
    com.nikolas.master_thesis: DEBUG

  appenders:
    - type: file
      currentLogFilename: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log
      threshold: ALL
      queueSize: 512
      archive: true
      archivedLogFilenamePattern: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log
      archivedFileCount: 5
      timeZone: UTC
      logFormat: "%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx"
      # logFormat: "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
#      logFormat: “%h %l %u %t “%r“ %s %b “%{Referer}i“ “%{User-Agent}i““ combined
#      logFormat: '%h %l %u %t '%r' %\\>s %b '%{Referer}i' '%{User-Agent}i''
#      logFormat: "%d{HH:mm:ss.SSS} [%thread] %-5level %com.nikolas.master_thesis.examples.`config-kt`.logger{36} - %msg%n"
#      logFormat: “%h %l %u %t “%r“ %s %b “%{Referer}i“ “%{User-Agent}i““
#      logFormat: "%h %l %u %t \"%r\" %s %O \"%{Referer}i\""
#      logFormat: “%h %l %u %t “%r“ %\\>s %b “%{Referer}i“ “%{User-Agent}i““
#      logFormat: "%h %l %u %t \"%r\" \"%>s\" %O \"%{Referer}i\""
#      logFormat: "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
#      logFormat: combined
#      logFormat: "%h %l %u %t \"%r\" %>s %b" /etc/httpd
#      logFormat: “%h %l %u %t “%r” %s %b “%{User-Agent}i””
#      logFormat: "%h %l %u %t \"%r\" %>s %b" /etc/httpd
#      logFormat: "%h %l %u %t \"%r\" %>s %b"
#        logFormat: "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""
      bufferSize: 8KB
      immediateFlush: true
      # filterFactories:
      #   - type: URI
    
    - type: console
      threshold: ALL
      queueSize: 512
      discardingThreshold: 0
      timeZone: UTC
      target: stdout
      logFormat: "%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx"

问题是 HTTP 请求是这样记录在控制台中的：

192.168.1.5 - - [09/Aug/2020:10:08:05 +0000] "GET /api/orders HTTP/1.1" 200 14094 "-" "Apache-HttpClient/4.5.10 (Java/1.8.0_161)" 710

...我也想在文件中记录它，但它根本不记录它！我尝试使用注释过的logFormats，但对于每个注释过的logFormat 值，我都会收到错误消息。例如。对于logFormat: "%h %l %u %t \"%r\" %>s %b"，当我在终端中运行它时（我使用RMI 参数从远程机器收集JMX 样本）：

java -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=1616 -Dcom.sun.management.jmxremote.rmi.port=1616 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -jar ./target/DW-Bookshop-1.0-SNAPSHOT.jar server config.yml

...我得到 NumberFormatException：

java.lang.NumberFormatException: For input string: ">"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Integer.parseInt(Integer.java:580)
    at java.lang.Integer.parseInt(Integer.java:615)
    at ch.qos.logback.core.pattern.FormatInfo.valueOf(FormatInfo.java:73)
    at ch.qos.logback.core.pattern.parser.Parser.T(Parser.java:136)
    at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:96)
    at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
    at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
    at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
    at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
    at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
    at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
    at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
    at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
    at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
    at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
    at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
    at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
    at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
    at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
    at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
    at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
    at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
    at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
    at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
    at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
    at ch.qos.logback.core.pattern.parser.Parser.parse(Parser.java:91)
    at ch.qos.logback.core.pattern.PatternLayoutBase.start(PatternLayoutBase.java:83)
    at io.dropwizard.logging.AbstractAppenderFactory.buildLayout(AbstractAppenderFactory.java:243)
    at io.dropwizard.logging.AbstractOutputStreamAppenderFactory.build(AbstractOutputStreamAppenderFactory.java:25)
    at io.dropwizard.logging.DefaultLoggingFactory.configure(DefaultLoggingFactory.java:143)
    at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:83)
    at io.dropwizard.cli.Cli.run(Cli.java:78)
    at io.dropwizard.Application.run(Application.java:93)
    at com.nikolas.master_thesis.DropwizardMasterThesisApplication.main(DropwizardMasterThesisApplication.java:35)

我也查看了this 的答案，但没有运气！另外，我正在运行 Fedora 31 操作系统，并且正在使用 Java 8。

非常感谢任何建议/帮助。

更新1：
我深入研究了配置文档，发现this 链接下的模式和 logFormat 键下的编辑值如下：

logFormat:  "%h %l %u [%t] '%r' %s %b '%i{Referer}' '%i{User-Agent}'"

还发现 Logback 项目使用 PatternLayout 类并在终端中得到了这个输出（因为起始行和其他行像往常一样）：

09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.xml]
09:27:33,966 |-INFO in ch.qos.logback.classic.BasicConfigurator@316a598d - Setting up default configuration.
09:27:36,978 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@216914 - Propagating INFO level on Logger[ROOT] onto the JUL framework
09:27:37,008 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@216914 - Propagating DEBUG level on Logger[com.nikolas.master_thesis] onto the JUL framework
09:27:37,010 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - Setting bufferSize to [8 KB]
09:27:37,027 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1805845895 - No compression will be used
09:27:37,028 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1805845895 - Will use the pattern /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log for the active file
09:27:37,050 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern '/home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log'.
09:27:37,050 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
09:27:37,068 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Thu Aug 13 09:27:06 CEST 2020
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [h]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [h] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [l]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [l] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [u]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [u] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [s]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [s] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [b]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [b] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [i]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [i] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [i]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [i] is not a valid conversion word
09:27:37,074 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - Active log file name: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log
09:27:37,074 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - File property is set to [/home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log]
09:27:37,075 |-INFO in ch.qos.logback.classic.AsyncAppender[async-file-appender] - Attaching appender named [file-appender] to AsyncAppender.
09:27:37,076 |-INFO in ch.qos.logback.classic.AsyncAppender[async-file-appender] - Setting discardingThreshold to 0
09:27:37,077 |-INFO in ch.qos.logback.classic.AsyncAppender[async-console-appender] - Attaching appender named [console-appender] to AsyncAppender.
09:27:37,077 |-INFO in ch.qos.logback.classic.AsyncAppender[async-console-appender] - Setting discardingThreshold to 0

显然我需要创建 logback.xml 文件，因为这一行指出：

09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.xml]

是否知道该 logback.xml 文件需要位于 MAven 项目中的哪个位置以及我应该使用 logFormat 的哪个值？

【问题讨论】：

如果您从日志格式中删除 > 符号是否有效？
不，然后它抱怨{符号。
我想说的是您需要验证String.format(...) 可以处理您的自定义格式。但是，如果即使使用控制台输出的模式也无法正常工作，则很可能是其他原因，例如缺少文件的写访问权限等。
@philonous：我想你是对的，但它的 (logFormat) 占位符似乎没有得到应有的利用。我已经更新了我的问题，所以请看一下。
一个logback.xml文件通常位于src/main/resources下

标签： java logging yaml dropwizard

【解决方案1】：

要将 http 请求记录在文件中，您必须设置 server: requestLog: 配置。 logging config 用于配置应用程序的日志。

在以下位置查看该配置以及 logback-access 和 logback-access-pattern 参考的文档： https://www.dropwizard.io/en/latest/manual/configuration.html#request-log

这是我的例子：

...
logging:
  level: INFO
  appenders:
    - type: file
      currentLogFilename: /logs/api.log
      archivedLogFilenamePattern: /logs/api-%d{yyyy-MM-dd}.log.gz
      archivedFileCount: 7
      queueSize: 2048
  loggers:
    com.mstn: DEBUG
server:
    applicationConnectors:
      - type: http
        port: 8003
    adminConnectors:
      - type: http
        port: 9003
    requestLog:
      appenders:
        - type: file
          currentLogFilename: /logs/api-requests.log
          archivedLogFilenamePattern: /logs/api-requests-%d{yyyy-MM-dd}.log.gz
          archivedFileCount: 7
          queueSize: 2048
database:
...

【讨论】：