array(2) { ["docs"]=> array(10) { [0]=> array(10) { ["id"]=> string(3) "428" ["text"]=> string(77) "Visual Studio 2017 单独启动MSDN帮助（Microsoft Help Viewer）的方法" ["intro"]=> string(288) "目录 ECharts 异步加载 ECharts 数据可视化在过去几年中取得了巨大进展。开发人员对可视化产品的期望不再是简单的图表创建工具，而是在交互、性能、数据处理等方面有更高的要求。 chart.setOption({ color: [ " ["username"]=> string(8) "DonetRen" ["tagsname"]=> string(55) "Visual Studio 2017|MSDN帮助|C#程序|.NET|Help Viewer" ["tagsid"]=> string(23) "[401,402,403,"300",404]" ["catesname"]=> string(0) "" ["catesid"]=> string(2) "[]" ["createtime"]=> string(10) "1511400964" ["_id"]=> string(3) "428" } [1]=> array(10) { ["id"]=> string(3) "427" ["text"]=> string(42) "npm -v；报错 cannot find module "wrapp"" ["intro"]=> string(288) "目录 ECharts 异步加载 ECharts 数据可视化在过去几年中取得了巨大进展。开发人员对可视化产品的期望不再是简单的图表创建工具，而是在交互、性能、数据处理等方面有更高的要求。 chart.setOption({ color: [ " ["username"]=> string(4) "zzty" ["tagsname"]=> string(50) "node.js|npm|cannot find module "wrapp“|node" ["tagsid"]=> string(19) "[398,"239",399,400]" ["catesname"]=> string(0) "" ["catesid"]=> string(2) "[]" ["createtime"]=> string(10) "1511400760" ["_id"]=> string(3) "427" } [2]=> array(10) { ["id"]=> string(3) "426" ["text"]=> string(54) "说说css中pt、px、em、rem都扮演了什么角色" ["intro"]=> string(288) "目录 ECharts 异步加载 ECharts 数据可视化在过去几年中取得了巨大进展。开发人员对可视化产品的期望不再是简单的图表创建工具，而是在交互、性能、数据处理等方面有更高的要求。 chart.setOption({ color: [ " ["username"]=> string(12) "zhengqiaoyin" ["tagsname"]=> string(0) "" ["tagsid"]=> string(2) "[]" ["catesname"]=> string(0) "" ["catesid"]=> string(2) "[]" ["createtime"]=> string(10) "1511400640" ["_id"]=> string(3) "426" } [3]=> array(10) { ["id"]=> string(3) "425" ["text"]=> string(83) "深入学习JS执行--创建执行上下文（变量对象，作用域链，this）" ["intro"]=> string(288) "目录 ECharts 异步加载 ECharts 数据可视化在过去几年中取得了巨大进展。开发人员对可视化产品的期望不再是简单的图表创建工具，而是在交互、性能、数据处理等方面有更高的要求。 chart.setOption({ color: [ " ["username"]=> string(7) "Ry-yuan" ["tagsname"]=> string(33) "Javascript|Javascript执行过程" ["tagsid"]=> string(13) "["169","191"]" ["catesname"]=> string(0) "" ["catesid"]=> string(2) "[]" ["createtime"]=> string(10) "1511399901" ["_id"]=> string(3) "425" } [4]=> array(10) { ["id"]=> string(3) "424" ["text"]=> string(30) "C# 排序技术研究与对比" ["intro"]=> string(288) "目录 ECharts 异步加载 ECharts 数据可视化在过去几年中取得了巨大进展。开发人员对可视化产品的期望不再是简单的图表创建工具，而是在交互、性能、数据处理等方面有更高的要求。 chart.setOption({ color: [ " ["username"]=> string(9) "vveiliang" ["tagsname"]=> string(0) "" ["tagsid"]=> string(2) "[]" ["catesname"]=> string(8) ".Net Dev" ["catesid"]=> string(5) "[199]" ["createtime"]=> string(10) "1511399150" ["_id"]=> string(3) "424" } [5]=> array(10) { ["id"]=> string(3) "423" ["text"]=> string(72) "【算法】小白的算法笔记：快速排序算法的编码和优化" ["intro"]=> string(288) "目录 ECharts 异步加载 ECharts 数据可视化在过去几年中取得了巨大进展。开发人员对可视化产品的期望不再是简单的图表创建工具，而是在交互、性能、数据处理等方面有更高的要求。 chart.setOption({ color: [ " ["username"]=> string(9) "penghuwan" ["tagsname"]=> string(6) "算法" ["tagsid"]=> string(7) "["344"]" ["catesname"]=> string(0) "" ["catesid"]=> string(2) "[]" ["createtime"]=> string(10) "1511398109" ["_id"]=> string(3) "423" } [6]=> array(10) { ["id"]=> string(3) "422" ["text"]=> string(64) "JavaScript数据可视化编程学习（二）Flotr2，雷达图" ["intro"]=> string(288) "目录 ECharts 异步加载 ECharts 数据可视化在过去几年中取得了巨大进展。开发人员对可视化产品的期望不再是简单的图表创建工具，而是在交互、性能、数据处理等方面有更高的要求。 chart.setOption({ color: [ " ["username"]=> string(7) "chengxs" ["tagsname"]=> string(28) "数据可视化|前端学习" ["tagsid"]=> string(9) "[396,397]" ["catesname"]=> string(18) "前端基本知识" ["catesid"]=> string(5) "[198]" ["createtime"]=> string(10) "1511397800" ["_id"]=> string(3) "422" } [7]=> array(10) { ["id"]=> string(3) "421" ["text"]=> string(36) "C#表达式目录树（Expression）" ["intro"]=> string(288) "目录 ECharts 异步加载 ECharts 数据可视化在过去几年中取得了巨大进展。开发人员对可视化产品的期望不再是简单的图表创建工具，而是在交互、性能、数据处理等方面有更高的要求。 chart.setOption({ color: [ " ["username"]=> string(4) "wwym" ["tagsname"]=> string(0) "" ["tagsid"]=> string(2) "[]" ["catesname"]=> string(4) ".NET" ["catesid"]=> string(7) "["119"]" ["createtime"]=> string(10) "1511397474" ["_id"]=> string(3) "421" } [8]=> array(10) { ["id"]=> string(3) "420" ["text"]=> string(47) "数据结构 队列_队列实例：事件处理" ["intro"]=> string(288) "目录 ECharts 异步加载 ECharts 数据可视化在过去几年中取得了巨大进展。开发人员对可视化产品的期望不再是简单的图表创建工具，而是在交互、性能、数据处理等方面有更高的要求。 chart.setOption({ color: [ " ["username"]=> string(7) "idreamo" ["tagsname"]=> string(40) "C语言|数据结构|队列|事件处理" ["tagsid"]=> string(23) "["246","247","248",395]" ["catesname"]=> string(12) "数据结构" ["catesid"]=> string(7) "["133"]" ["createtime"]=> string(10) "1511397279" ["_id"]=> string(3) "420" } [9]=> array(10) { ["id"]=> string(3) "419" ["text"]=> string(47) "久等了,博客园官方Android客户端发布" ["intro"]=> string(288) "目录 ECharts 异步加载 ECharts 数据可视化在过去几年中取得了巨大进展。开发人员对可视化产品的期望不再是简单的图表创建工具，而是在交互、性能、数据处理等方面有更高的要求。 chart.setOption({ color: [ " ["username"]=> string(3) "cmt" ["tagsname"]=> string(0) "" ["tagsid"]=> string(2) "[]" ["catesname"]=> string(0) "" ["catesid"]=> string(2) "[]" ["createtime"]=> string(10) "1511396549" ["_id"]=> string(3) "419" } } ["count"]=> int(200) } 222 awstats分析nginx日志 - 爱码网

awstats分析nginx日志

1.首先修改nginx日志格式为combined格式：
# log_format combined
# '$remote_addr - $remote_user [$time_local] '
# '"$request" $status $apache_bytes_sent '
# '"$http_referer" "$http_user_agent"';
# log_format main
# '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';

# access_log logs/access.log main;
access_log logs/access.log combined ;

2 安装awstats
tar -zxvf awstats-7.0.tar.gz
mkdir -p /usr/local/awstats
mv ./awstats-7.0/* /usr/local/awstats
cd/usr/local/awstats/tools
[[email protected] tools]# perl awstats_configure.pl
awstats分析nginx日志
awstats分析nginx日志awstats分析nginx日志

需要注意的地方是：

Check for web server install<wbr><wbr>这项填none。<br> Define config file name to create<wbr><wbr>这项填配置文件名字,一般用域名。</wbr></wbr></wbr></wbr>
其余默认即可。

3.修改建成的配置文件astats.www.nginx.log
LogFile="LogFile="/usr/local/awstats/tools/logresolvemerge.pl /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log|"

4 修改完awstats的配置文件以后要更新
perl /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.nginx.log
或
[[email protected] logs]# /usr/local/awstats/tools/awstats_updateall.pl now
Running '"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=www.nginx.log -configdir="/etc/awstats"' to update config www.nginx.log
Create/Update database for config "/etc/awstats/awstats.www.nginx.log.conf" by AWStats version 7.0 (build 1.971)
From data in log file "/usr/local/awstats/tools/logresolvemerge.pl /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log|"...
Phase 1 : First bypass old records, searching new record...
Direct access to last remembered record has fallen on another record.
So searching new records from beginning of log file...
Jumped lines in file: 0
Parsed lines in file: 48864
Found 0 dropped records,
Found 0 comments,
Found 0 blank records,
Found 0 corrupted records,
Found 48864 old records,
Found 0 new qualified records.

5 执行成功以后，就可以利用 awstats 的工具将统计的结果生成静态文件，访问web网页的时候，就是读取的这个静态文件
[[email protected] awstats]# perl /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.nginx.log -lang=cn -dir=/data/webroot/awstats/ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl
Launch update process : "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -update -configdir=
Build main page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output
Build alldomains page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=alldomains
Build allhosts page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=allhosts
Build lasthosts page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=lasthosts
Build unknownip page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=unknownip
Build allrobots page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=allrobots
Build lastrobots page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=lastrobots
Build session page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=session
Build urldetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=urldetail
Build urlentry page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=urlentry
Build urlexit page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=urlexit
Build osdetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=osdetail
Build unknownos page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=unknownos
Build browserdetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=browserdetail
Build unknownbrowser page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=unknownbrowser
Build refererse page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=refererse
Build refererpages page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=refererpages
Build keyphrases page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=keyphrases
Build keywords page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=keywords
Build errors404 page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=errors404
20 files built.
Main HTML page is 'awstats.www.nginx.log.html'.

根据提示可以看到主要生成的静态页面为awstats.www.nginx.log.html
我们访问web的时候的链接是：http://IP/awstats/awstats.www.nginx.log.html

6.以下是测试时报错：
]# perl /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=.www.nginx.log
Error: AWStats database directory defined in config file by 'DirData' parameter (/var/lib/awstats) does not exist or is not writable.
Setup ('/etc/awstats/awstats..www.nginx.log.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).
解决：建一个awstats用于记录数据的目录
[[email protected] awstats]# mkdir /var/lib/awstats

另一种常见报错就是日志格式问题：
[[email protected] logs]# /usr/local/awstats/tools/awstats_updateall.pl now
Running '"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=www.nginx.log -configdir="/etc/awstats"' to update config www.nginx.log
Create/Update database for config "/etc/awstats/awstats.www.nginx.log.conf" by AWStats version 7.0 (build 1.971)
From data in log file "/usr/local/awstats/tools/logresolvemerge.pl /usr/local/nginx/logs/*.pcstars_access.log|"...
Phase 1 : First bypass old records, searching new record...
Direct access to last remembered record has fallen on another record.
So searching new records from beginning of log file...
AWStats did not find any valid log lines that match your LogFormat parameter, in the 50th first non commented lines read of your log.
Your log file /usr/local/awstats/tools/logresolvemerge.pl /usr/local/nginx/logs/*.pcstars_access.log| must have a bad format or LogFormat parameter setup does not match this format.
Your AWStats LogFormat parameter is:
1
This means each line in your web server log file need to have "combined log format" like this:
111.22.33.44 - - [10/Jan/2001:02:14:14 +0200] "GET / HTTP/1.1" 200 1234 "http://www.fromserver.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"
And this is an example of records AWStats found in your log file (the record number 50 in your log):
59.36.213.132 - - [14/Sep/2011:00:00:17 +0800] GET /lm/3008-p610.html HTTP/1.1 302 5 -Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) -
Setup ('/etc/awstats/awstats.www.nginx.log.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).

7 进行nginx日志分割，这个在这里不做解释

8 将下列写入计划任务，执行时间根据实际生产需求
/usr/local/awstats/tools/awstats_updateall.pl now
perl /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.nginx.log -lang=cn -dir=/data/webroot/awstats/ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl

最后浏览http://IP/awstats/awstats.www.nginx.log.html


相关文章：

猜你喜欢
相关资源
相似解决方案
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode