流利的不从头开始采摘日志

Question

我正在尝试使用 fluentd 获取我的 docker 容器日志。应用程序和 fluentd 进程都通过 supervisord 启动，并且都在同一个容器中，但 fluentd 只占用一半的应用程序日志。我需要从头开始获取日志。在下面添加 fluentd conf：-

<source>
 type tail
 path /var/log/*
 path_key path
 format none
 read_from_head true
 <parse>
   @type grok
    <grok>
      pattern (?<logtm>%{MONTHDAY}-%{MONTH}-%{YEAR} %{HOUR}:%{MINUTE}:%{SECOND}) %{LOGLEVEL:loglevel} \[%{DATA:thread}] %{GREEDYDATA:message}
    </grok>
    <grok>
      pattern %{GREEDYDATA:timestamp} %{LOGLEVEL:loglevel} %{GREEDYDATA:message}
    </grok>
    <grok>
      pattern %{URIHOST:remote_host} - - \[%{HTTPDATE:request_time}] "%{WORD:method} %{NOTSPACE:request_page} %{GREEDYDATA:message}/%{NUMBER:http_version}\" %{NUMBER:response_code} %{NUMBER:bytes} %{INT:time_taken} %{QS:referrer} %{QS:user_agent}
    </grok>
  </parse>
 keep_time_key true
# time_format yyyy-MM-dd HH:mm:ss.SSSZ
 tag graylog2.*
</source>
<filter **>
  @type record_transformer
  enable_ruby
  <record>
   logtm ${record["logtm"]}
   thread ${record["thread"]}
   instance "#{Socket.gethostname}"
   namespace "#{ENV.fetch('INSTANCE_PREFIX'){'default'}}"
   app "#{ENV.fetch('APPZ_APP_NAME'){'wordpress'}}"
   level ${if record["loglevel"] == "EMERG";record["level"] = "0" ; record["loglevel"] == "ALERT";record["level"] = "1";elsif record["loglevel"] == "CRIT" || record["loglevel"] == "SEVERE"; record["level"]= "2" ;elsif record["loglevel"] == "ERROR" ; record["level"]= "3" ;elsif record["loglevel"] == "WARN" || record["loglevel"] == "WARNING" ; record["level"]= "4" ;elsif record["loglevel"] == "NOTICE" ; record["level"]= "5";elsif record["loglevel"] == "INFO" ||  record["loglevel"] == nil ||  record["loglevel"] == 0 ; record["level"]= "6";else record["loglevel"] == "DEBUG" || record["loglevel"] == "debug"; record["level"]= "7";end}
  </record>
</filter>
<match **>
 type graylog
 host "#{ENV.fetch('LOG_HOST'){'GL'}}"
 port "#{ENV.fetch('LOG_PORT'){12201}}"
# BufferedOutput config
 flush_interval 5s
 num_threads 2
 # ...
</match>

注意：- 当我将 /var/log/* 更改为 /var/log/log_file_name.log 时，它工作正常。

Answer 1

您可以尝试使用 in_tail 中的 POS 文件

路径 /var/log/httpd-access.log

pos_file /var/log/td-agent/httpd-access.log.pos