log4j + 设置 DEBUG 模式级别后，日志仍然显示为 INFO

Question

我们将资源管理器服务的log4j配置为DEBUG模式如下

yarn.server.resourcemanager.appsummary.logger=DEBUG,RMSUMMARY

然后我们重新启动资源管理器服务（从 ambari 仪表板）

从日志中我们仍然看到日志级别为 INFO 而不是 DEBUG

示例：

2021-07-02 06:00:53,620 INFO  resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(458)) - NodeManager from node worker02.bd-gontamopigure-1.com(cmPort: 45454 httpPort: 8042) registered with capability: <memory:16384, vCores:3>, assigned nodeId worker02.bd-gontamopigure-1.com:45454
2021-07-02 06:00:53,620 INFO  rmnode.RMNodeImpl (RMNodeImpl.java:handle(440)) - worker02.bd-gontamopigure-1.com:45454 Node Transitioned from NEW to RUNNING
2021-07-02 06:00:53,621 INFO  capacity.CapacityScheduler (CapacityScheduler.java:addNode(1364)) - Added node worker02.bd-gontamopigure-1.com:45454 clusterResource: <memory:32768, vCores:6>

所以我分享了 log4j 配置

知道有什么问题：

yarn.server.resourcemanager.appsummary.logger=DEBUG,RMSUMMARY

以及为什么尽管我们配置了 DEBUG 却得到 INFO

这里所有的log4j配置

#Relative to Yarn Log Dir Prefix
yarn.log.dir=.
#
# Job Summary Appender
#
# Use following logger to send summary to separate file defined by
# hadoop.mapreduce.jobsummary.log.file rolled daily:
# hadoop.mapreduce.jobsummary.logger=INFO,JSA
#
hadoop.mapreduce.jobsummary.logger=${hadoop.root.logger}
hadoop.mapreduce.jobsummary.log.file=hadoop-mapreduce.jobsummary.log
log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
# Set the ResourceManager summary log filename
yarn.server.resourcemanager.appsummary.log.file=hadoop-mapreduce.jobsummary.log
# Set the ResourceManager summary log level and appender
yarn.server.resourcemanager.appsummary.logger=${hadoop.root.logger}

# To enable AppSummaryLogging for the RM,
# set yarn.server.resourcemanager.appsummary.logger to
# LEVEL,RMSUMMARY in hadoop-env.sh
yarn.server.resourcemanager.appsummary.logger=DEBUG,RMSUMMARY
# Appender for ResourceManager Application Summary Log
# Requires the following properties to be set
#    - hadoop.log.dir (Hadoop Log directory)
#    - yarn.server.resourcemanager.appsummary.log.file (resource manager app summary log filename)
#    - yarn.server.resourcemanager.appsummary.logger (resource manager app summary log level and appender)
log4j.appender.RMSUMMARY=org.apache.log4j.RollingFileAppender
log4j.appender.RMSUMMARY.File=${yarn.log.dir}/${yarn.server.resourcemanager.appsummary.log.file}
log4j.appender.RMSUMMARY.MaxFileSize=256MB
log4j.appender.RMSUMMARY.MaxBackupIndex=20
log4j.appender.RMSUMMARY.layout=org.apache.log4j.PatternLayout
log4j.appender.RMSUMMARY.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
log4j.appender.JSA.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2}: %m%n
log4j.appender.JSA.DatePattern=.yyyy-MM-dd
log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
log4j.logger.org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary=${yarn.server.resourcemanager.appsummary.logger}
log4j.additivity.org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary=false

# Appender for viewing information for errors and warnings
yarn.ewma.cleanupInterval=300
yarn.ewma.messageAgeLimitSeconds=86400
yarn.ewma.maxUniqueMessages=250
log4j.appender.EWMA=org.apache.hadoop.yarn.util.Log4jWarningErrorMetricsAppender
log4j.appender.EWMA.cleanupInterval=${yarn.ewma.cleanupInterval}
log4j.appender.EWMA.messageAgeLimitSeconds=${yarn.ewma.messageAgeLimitSeconds}
log4j.appender.EWMA.maxUniqueMessages=${yarn.ewma.maxUniqueMessages}

# Audit logging for ResourceManager
rm.audit.logger=${hadoop.root.logger}
log4j.logger.org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger=${rm.audit.logger}
log4j.additivity.org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger=false
log4j.appender.RMAUDIT=org.apache.log4j.DailyRollingFileAppender
log4j.appender.RMAUDIT.File=${yarn.log.dir}/rm-audit.log
log4j.appender.RMAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.RMAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
log4j.appender.RMAUDIT.DatePattern=.yyyy-MM-dd

# Audit logging for NodeManager
nm.audit.logger=${hadoop.root.logger}
log4j.logger.org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger=${nm.audit.logger}
log4j.additivity.org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger=false
log4j.appender.NMAUDIT=org.apache.log4j.DailyRollingFileAppender
log4j.appender.NMAUDIT.File=${yarn.log.dir}/nm-audit.log
log4j.appender.NMAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.NMAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
log4j.appender.NMAUDIT.DatePattern=.yyyy-MM-dd

Answer 1

了解日志级别。

例如，如果您设置 DEBUG，您将看到所有 INFO、ERROR、WARN 等。

见下图来自this post。

如果您不想看到 INFO，请将您的日志级别设置为 WARN。