https://www.cnblogs.com/xiaobaozi-95/p/9214307.html
一、ELK收集Nginx日志
#由于10.192.27.111 上已经安装了ES 接下来安装Kibana [root@web01 soft]# rpm -ivh kibana-6.6.0-x86_64.rpm [root@web01 ~]# rpm -qc kibana #查看Kibana配置文件 /etc/kibana/kibana.yml [root@web01 ~]# grep "^[a-z]" /etc/kibana/kibana.yml #修改后的配置文件 server.port: 5601 server.host: "0.0.0.0" elasticsearch.hosts: ["http://localhost:9200"] kibana.index: ".kibana" [root@web01 ~]# [root@web01 ~]# systemctl start kibana [root@web01 ~]# systemctl status kibana [root@web01 ~]# netstat -lntup|grep 5601 tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 69594/node [root@web01 ~]#
#测试Nginx负载均衡日志 [root@web01 ~]# ab -n 100 -c 100 http://10.192.27.111:6443/ [root@web01 ~]# tailf /var/log/nginx/k8s-access.log 10.192.27.111 10.192.27.114:6443 - [07/Dec/2019:11:45:45 +0800] 200 86 10.192.27.111 10.192.27.100:6443 - [07/Dec/2019:11:45:45 +0800] 200 86 10.192.27.111 10.192.27.114:6443 - [07/Dec/2019:11:45:45 +0800] 200 86 10.192.27.111 10.192.27.100:6443 - [07/Dec/2019:11:45:45 +0800] 200 86 10.192.27.111 10.192.27.114:6443 - [07/Dec/2019:11:45:45 +0800] 200 86 10.192.27.111 10.192.27.100:6443 - [07/Dec/2019:11:45:45 +0800] 200 86 10.192.27.111 10.192.27.114:6443 - [07/Dec/2019:11:45:45 +0800] 200 86 10.192.27.111 10.192.27.100:6443 - [07/Dec/2019:11:45:45 +0800] 200 86 10.192.27.111 10.192.27.114:6443 - [07/Dec/2019:11:45:45 +0800] 200 86 10.192.27.111 10.192.27.100:6443 - [07/Dec/2019:11:45:45 +0800] 200 86
#安装filebeat [root@web01 soft]# rpm -ivh filebeat-6.6.0-x86_64.rpm 警告：filebeat-6.6.0-x86_64.rpm: 头V4 RSA/SHA512 Signature, 密钥 ID d88e42b4: NOKEY 准备中... ################################# [100%] 正在升级/安装... 1:filebeat-6.6.0-1 ################################# [100%] [root@web01 soft]# rpm -qc filebeat /etc/filebeat/filebeat.yml /etc/filebeat/modules.d/apache2.yml.disabled /etc/filebeat/modules.d/auditd.yml.disabled /etc/filebeat/modules.d/elasticsearch.yml.disabled /etc/filebeat/modules.d/haproxy.yml.disabled /etc/filebeat/modules.d/icinga.yml.disabled /etc/filebeat/modules.d/iis.yml.disabled /etc/filebeat/modules.d/kafka.yml.disabled /etc/filebeat/modules.d/kibana.yml.disabled /etc/filebeat/modules.d/logstash.yml.disabled /etc/filebeat/modules.d/mongodb.yml.disabled /etc/filebeat/modules.d/mysql.yml.disabled /etc/filebeat/modules.d/nginx.yml.disabled /etc/filebeat/modules.d/osquery.yml.disabled /etc/filebeat/modules.d/postgresql.yml.disabled /etc/filebeat/modules.d/redis.yml.disabled /etc/filebeat/modules.d/suricata.yml.disabled /etc/filebeat/modules.d/system.yml.disabled /etc/filebeat/modules.d/traefik.yml.disabled [root@web01 soft]# cd [root@web01 ~]# egrep -v "#|^$" /etc/filebeat/filebeat.yml #修改后的配置文件 filebeat.inputs: - type: log #log模式 enabled: True paths: - /var/log/nginx/k8s-access.log #日志目录 filebeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: false setup.template.settings: index.number_of_shards: 3 #三个副本 setup.kibana: output.elasticsearch: hosts: ["localhost:9200"] #es地址 processors: - add_host_metadata: ~ - add_cloud_metadata: ~ [root@web01 ~]#
简单的访问一下：http://10.192.27.111:5601/
上面是收集简单的Nginx日志，接下来我们收集json格式的日志
例如：Nginx日志如下
user nginx; worker_processes 1; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; log_format json '{ "time_local": "$time_local", ' '"remote_addr": "$remote_addr", ' '"referer": "$http_referer", ' '"request": "$request", ' '"status": $status, ' '"bytes": $body_bytes_sent, ' '"agent": "$http_user_agent", ' '"x_forwarded": "$http_x_forwarded_for", ' '"up_addr": "$upstream_addr",' '"up_host": "$upstream_http_host",' '"upstream_time": "$upstream_response_time",' '"request_time": "$request_time"' ' }'; access_log /var/log/nginx/access.log json; sendfile on; #tcp_nopush on; keepalive_timeout 65; #gzip on; include /etc/nginx/conf.d/*.conf; }
filebeat删减版日志
filebeat.inputs: - type: log enabled: true paths: - /var/log/nginx/access.log json.keys_under_root: true #关键字顶级 json.overwrite_keys: true # setup.kibana: host: "10.192.27.111:5601" output.elasticsearch: hosts: ["10.192.27.111:9200"] index: "nginx-%{[beat.version]}-%{+yyyy.MM}" #定义索引名称 setup.template.name: "nginx" #定义模板名称 setup.template.pattern: "nginx-*" #模板正则匹配 setup.template.enabled: false #不使用系统模板 setup.template.overwrite: true #覆盖
重启服务然后再次访问 步骤更上面一致
详细情况可以参考官网：https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html
日志拆分 收集多个日志 和多台机器日志汇总
可以将单台机器的Nginx配置文件（日志那部分配置也可以）拷到其它节点上，同时filebeat.yaml也一样 重启服务
filebeat.inputs: - type: log enabled: true paths: - /var/log/nginx/access.log json.keys_under_root: true json.overwrite_keys: true tags: ["access"] #打tag - type: log enabled: true paths: - /var/log/nginx/error.log tags: ["error"] setup.kibana: host: "10.192.27.111:5601" output.elasticsearch: hosts: ["10.192.27.111:9200"] #index: "nginx-%{[beat.version]}-%{+yyyy.MM}" indices: - index: "nginx-access-%{[beat.version]}-%{+yyyy.MM}" when.contains: tags: "access" #判断条件 - index: "nginx-error-%{[beat.version]}-%{+yyyy.MM}" when.contains: tags: "error" setup.template.name: "nginx" setup.template.pattern: "nginx-*" setup.template.enabled: false setup.template.overwrite: true
重启服务然后再次访问 步骤更上面一致
使用filledeat modules配置 官方网址 https://www.elastic.co/guide/en/beats/filebeat/6.6/configuration-filebeat-modules.html 使用模版配置nginx正常日志 社区论坛： https://discuss.elastic.co/t/filebeat-module-custom-index/181350 客户端（收集web服务器日志） 10.192.27.100 1、安装filebeat [root@web01 ~]# rpm -ivh filebeat-6.6.0-x86_64.rpm 警告：filebeat-6.6.0-x86_64.rpm: 头V4 RSA/SHA512 Signature, 密钥 ID d88e42b4: NOKEY 准备中... ################################# [100%] 正在升级/安装... 1:filebeat-6.6.0-1 ################################# [100%] [root@web01 soft]# rpm -qc filebeat /etc/filebeat/filebeat.yml /etc/filebeat/modules.d/apache2.yml.disabled /etc/filebeat/modules.d/auditd.yml.disabled /etc/filebeat/modules.d/elasticsearch.yml.disabled /etc/filebeat/modules.d/haproxy.yml.disabled /etc/filebeat/modules.d/icinga.yml.disabled /etc/filebeat/modules.d/iis.yml.disabled /etc/filebeat/modules.d/kafka.yml.disabled /etc/filebeat/modules.d/kibana.yml.disabled /etc/filebeat/modules.d/logstash.yml.disabled /etc/filebeat/modules.d/mongodb.yml.disabled /etc/filebeat/modules.d/mysql.yml.disabled /etc/filebeat/modules.d/nginx.yml.disabled /etc/filebeat/modules.d/osquery.yml.disabled /etc/filebeat/modules.d/postgresql.yml.disabled /etc/filebeat/modules.d/redis.yml.disabled /etc/filebeat/modules.d/suricata.yml.disabled /etc/filebeat/modules.d/system.yml.disabled /etc/filebeat/modules.d/traefik.yml.disabled [root@web01 soft]# cd 2、配置文件 [root@web01 ~]# egrep -v "#|^$" /etc/filebeat/filebeat.yml #修改后的配置文件 filebeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: true reload.period: 10s setup.kibana: host: "10.192.27.111:5601" output.elasticsearch: hosts: ["10.192.27.111:9200"] indices: - index: "nginx-access-%{[beat.version]}-%{+yyyy.MM}" when.contains: fileset.name: "access" - index: "nginx-error-%{[beat.version]}-%{+yyyy.MM}" when.contains: fileset.name: "error" setup.template.name: "nginx" setup.template.pattern: "nginx-*" setup.template.enabled: false setup.template.overwrite: true 3、filebeat modules配置 使用nginx模版配置需要安装2个插件，默认从官网下载速度太慢，可以提前下载然后离线安装 https://www.elastic.co/guide/en/elasticsearch/plugins/6.6/ingest-geoip.html https://www.elastic.co/guide/en/elasticsearch/plugins/6.6/plugin-management-custom-url.html 在线安装： [root@web01 ~]# /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-user-agent [root@web01 ~]# /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-geoip 离线下载安装： [root@web01 ~]# wget https://artifacts.elastic.co/downloads/elasticsearch-plugins/ingest-user-agent/ingest-user-agent-6.6.0.zip [root@web01 ~]# wget https://artifacts.elastic.co/downloads/elasticsearch-plugins/ingest-geoip/ingest-geoip-6.6.0.zip [root@web01 ~]# /usr/share/elasticsearch/bin/elasticsearch-plugin install file:///root/ingest-geoip-6.6.0.zip [root@web01 ~]# /usr/share/elasticsearch/bin/elasticsearch-plugin install file:///root/ingest-user-agent-6.6.0.zip 注意：6.7之后这两个插件默认集成到了elasticsearch，不需要单独安装了 激活nginx模块： [root@web01 ~]# filebeat modules enable nginx Enabled nginx [root@web01 ~]# filebeat modules list Enabled: nginx Disabled: apache2 auditd elasticsearch haproxy icinga iis kafka kibana logstash mongodb mysql osquery postgresql redis suricata system traefik [root@web01 ~]# [root@web01 ~]# egrep -v "#|^$" /etc/filebeat/modules.d/nginx.yml - module: nginx access: enabled: true var.paths: ["/var/log/nginx/access.log"] error: enabled: true var.paths: ["/var/log/nginx/error.log"] 4、重启服务 [root@web01 ~]# systemctl start filebeat [root@web01 ~]# systemctl status filebeat ● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch. Loaded: loaded (/usr/lib/systemd/system/filebeat.service; disabled; vendor preset: disabled) Active: active (running) since 二 2020-04-28 21:28:28 CST; 8s ago Docs: https://www.elastic.co/products/beats/filebeat Main PID: 10646 (filebeat) CGroup: /system.slice/filebeat.service └─10646 /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat ... 4月 28 21:28:28 web01 systemd[1]: Started Filebeat sends log files to Logstash or directly to Elasticsearch.. [root@master01 filebeat]# tailf /var/log/filebeat/ #日志 [root@web01 ~]# netstat -anput | grep filebeat #查看端口连接情况 tcp 0 0 10.192.27.100:55290 10.192.27.111:9200 ESTABLISHED 3124/filebeat [root@web01 ~]# 服务器端（es kibana） 10.192.27.111 省略 参考：https://www.cnblogs.com/linux985/p/11995364.html https://www.cnblogs.com/linux985/p/12010657.html 5.3 使用模块收集系统日志message和secure日志 如果不需要转换，也可以直接按普通日志模式收集message和secure日志 5.4 导入kibana视图 默认如果使用filbeat模版导入视图会把所有的服务都导入进去,而我们实际上并不需要这么多视图， 而且默认的视图模版只能匹配filebeat-*开头的索引，所以这里我们有2个需要需要解决： 1.通过一定处理只导入我们需要的模版 2.导入的视图模版索引名称可以自定义 解决方法： 1.备份一份filebeat的kibana视图，删除不需要的视图模版文件 2.修改视图文件里默认的索引名称为我们需要的索引名称 cp -a /usr/share/filebeat/kibana /root find . -type f ! -name "*nginx*"|xargs rm -rf sed -i 's#filebeat\-\*#nginx\-\*#g' Filebeat-nginx-overview.json 替换索引名称 filebeat setup --dashboards -E setup.dashboards.directory=/root/kibana/ 5.5 使用模块收集mysql日志和慢日志 5.6 使用模块收集mongo日志和redis日志