1. 查询mysql数据库编码是否为utf8:
    MariaDB [(none)]> show databases;
    +--------------------+
    | Database           |
    +--------------------+
    | Syslog             |
    | information_schema |
    | mysql              |
    | performance_schema |
    +--------------------+
    4 rows in set (0.00 sec)
    #########################
    MariaDB [(none)]> show create database Syslog;
    +----------+-------------------+
    | Database | Create Database   |
    +----------+-------------------+
    | Syslog   | CREATE DATABASE `Syslog` /*!40100 DEFAULT CHARACTER SET utf8mb4 */ |
    +----------+--------------------+
    1 row in set (0.00 sec)
    ##########################
    MariaDB [(none)]> status
    mysql  Ver 15.1 Distrib 10.3.29-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
    Connection id:          1793
    Current database:
    Current user:           root@localhost
    SSL:                    Not in use
    Current pager:          stdout
    Using outfile:          ''
    Using delimiter:        ;
    Server:                 MariaDB
    Server version:         10.3.29-MariaDB-0+deb10u1 Debian 10
    Protocol version:       10
    Connection:             Localhost via UNIX socket
    Server characterset:    utf8mb4
    Db     characterset:    utf8mb4
    Client characterset:    utf8mb4
    Conn.  characterset:    utf8mb4
    UNIX socket:            /var/run/mysqld/mysqld.sock
    Uptime:                 1 hour 23 min 21 sec

    编辑/etc/mysql/mariadb.conf.d/下面的50-mysql-clients.cnf50-server.cnf

    编辑50-server.cnf
    # this is only for the mysqld standalone daemon
    [mysqld]
    ..........
    ..........
    ..........
    # MySQL/MariaDB default is Latin1, but in Debian we rather default to the full
    # utf8 4-byte character set. See also client.cnf
    #
    character-set-server  = utf8mb4
    collation-server      = utf8mb4_general_ci
    init_connect          = 'SET NAMES utf8mb4'
    #character-set-server = utf8
    #collation-server      =utf8_general_ci
    #init_connect          = 'SET NAMES utf8'
    编辑50-mysql-clients.cnf
    [mysql]
    # Default is Latin1, if you need UTF-8 set this (also in server section)
    default-character-set = utf8mb4
    #default-character-set = latin1
    #default-character-set = utf8

    如不是utf8:

    Vim /usr/share/doc/rsyslog-7.4.7/mysql-createDB.sql
    修改:create database syslog default character set utf8 collate utf8mb4; //utf8_general_ci
    CREATE DATABASE /*!32312 IF NOT EXISTS*/`Syslog` /*!40100 DEFAULT CHARACTER SET utf8 */;
    USE `Syslog`;
    /*Table structure for table `SystemEvents` */
    DROP TABLE IF EXISTS `SystemEvents`;
    CREATE TABLE `SystemEvents` (
    `ID` int(10) unsigned NOT NULL AUTO_INCREMENT,
    `CustomerID` bigint(20) DEFAULT NULL,
    `ReceivedAt` datetime DEFAULT NULL,
    `DeviceReportedTime` datetime DEFAULT NULL,
    `Facility` smallint(6) DEFAULT NULL,
    `Priority` smallint(6) DEFAULT NULL,
    `FromHost` varchar(60) DEFAULT NULL,
    `FromIP` varchar(60) DEFAULT NULL,
    `Message` text DEFAULT NULL,
    `NTSeverity` int(11) DEFAULT NULL,
    `Importance` int(11) DEFAULT NULL,
    `EventSource` varchar(60) DEFAULT NULL,
    `EventUser` varchar(60) DEFAULT NULL,
    `EventCategory` int(11) DEFAULT NULL,
    `EventID` int(11) DEFAULT NULL,
    `EventBinaryData` text DEFAULT NULL,
    `MaxAvailable` int(11) DEFAULT NULL,
    `CurrUsage` int(11) DEFAULT NULL,
    `MinUsage` int(11) DEFAULT NULL,
    `MaxUsage` int(11) DEFAULT NULL,
    `InfoUnitID` int(11) DEFAULT NULL,
    `SysLogTag` varchar(60) DEFAULT NULL,
    `EventLogType` varchar(60) DEFAULT NULL,
    `GenericFileName` varchar(60) DEFAULT NULL,
    `SystemID` int(11) DEFAULT NULL,
    `processid` varchar(60) NOT NULL DEFAULT '',
    `checksum` int(11) unsigned NOT NULL DEFAULT 0,
    PRIMARY KEY (`ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=13144 DEFAULT CHARSET=utf8mb4;
    CREATE DATABASE /*!32312 IF NOT EXISTS*/`Syslog` /*!40100 DEFAULT CHARACTER SET utf8 */;
    USE `Syslog`;
    /*Table structure for table `SystemEventsProperties` */
    DROP TABLE IF EXISTS `SystemEventsProperties`;
    CREATE TABLE `SystemEventsProperties` (
    `ID` int(10) unsigned NOT NULL AUTO_INCREMENT,
    `SystemEventID` int(11) DEFAULT NULL,
    `ParamName` varchar(255) DEFAULT NULL,
    `ParamValue` text DEFAULT NULL,
    PRIMARY KEY (`ID`)
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

    重启mysqld服务

    service mysql restart
  2. 如果数据库,数据表编码都为utf8:
    修改loganalyzer网站include/functions_common.php文件,找到return htmlentities字段的行。
    将return htmlentities($mystr,ENT_NOQUOTES,$content['HeaderDefaultEncoding'],"utf-8");
    修改为:return htmlentities($mystr,ENT_NOQUOTES,"utf-8");

    重启httpd服务

    service apache2 restart
  3. 增加登录验证
    auth();
    function auth ()
    {
         $valid_passwords = array ("admin" => "admin");
         $valid_users = array_keys($valid_passwords);
         $user = @$_SERVER['PHP_AUTH_USER'];
         $pass = @$_SERVER['PHP_AUTH_PW'];
         $validated = (in_array($user, $valid_users)) && ($pass == $valid_passwords[$user]);
         if (!$validated) {
           header('WWW-Authenticate: Basic realm="My Realm"');
           header('HTTP/1.0 401 Unauthorized');
           die ("<script>window.location.href='/';</script>");
         }
    }
  4. 添加ip字段
  • php文件添加字段
    vim ./include/constants_logstream.php
    //354行添加
    $dbmapping['monitorware']['DBMAPPINGS'][FROMIP] = "FromIP";
  • 视图添加字段
    wKioL1U_MEeTUV4XAAFqvnEvSis527.jpg
    wKioL1U_MEuBlr3VAATEgLuhTig369.jpg
    wKioL1U_MEXy5_SyAAJ_TkyFc9E304.jpg
    wKioL1U_MM3CofwAAAE7UAH1FgE872.jpg
  • 数据库添加字段
    mysql> USE Syslog;
    mysql> ALTER TABLE SystemEvents ADD FromIP VARCHAR(60) DEFAULT NULL AFTER FromHost;
  • 修改/etc/rsyslog.conf 文件添加模板
    #template
    #$template RemoteLogs,"/var/log/logdata/%FROMHOST-IP%_%$YEAR%-%$MONTH%-%$DAY%.log"
    #*.* ?RemoteLogs
    #:fromhost-ip, !isequal, "127.0.0.1" ?RemoteLogs
    #:fromhost-ip, isequal, "192.168.10.20" ?RemoteLogs
    #:fromhost-ip, startswith, "192.168.1." /var/log/network1.log
    #$AllowedSender TCP, 127.0.0.1, 10.110.50.0/24, *.yourdomain.com
    #$AllowedSender TCP, 192.168.10.0/24
    #$template RemoteLogs,"/var/log/rsyslog/logdata"
    #*.notice   ?RemoteLogs
    #############################
    #bash_log
    #$template bash_mysql_insert,"insert into SystemEvents (Message, Facility, FromHost,FromIP,Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag,EventUser) values('%msg:F,124:6%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%,'%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%,'%syslogtag%','%msg:F,124:4%');",SQL
    #$template Bash-history,"/var/log/remote_log/bash-log/%hostname%_%fromhost-ip%_log_%app-name%_%$YEAR%-%$MONTH%-%$DAY%.log"
    #if ($msg contains "normal" and $app-name == "bash-log")  then {
    #  action(type="omfile" DynaFile="Bash-history")
    #  action(type="ommysql" server="localhost" serverport="3306" db="Syslog" uid="root" pwd="root" template="bash_mysql_insert")
    #  stop
    #}
    ###################
    $template insertpl,"insert into SystemEvents (Message, Facility, FromHost, FromIP,Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%,'%HOSTNAME%', '%fromhost-ip%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL
    *.notice     :ommysql:localhost,Syslog,rsyslog,mULHyeznrat0;insertpl
    #*.notice   action(type="ommysql" server="localhost" db="Syslog" uid="rsyslog" pwd="mULHyeznrat0" template="insertpl")
    "%msg:F,124:4%":  代表将msg信息以|为分割,取出第四个区域的内容,F代表分割,124是|的ASCII码
    "%msg:1:2%":   读取从pos从1到2的数据
    "%msg:::lowercase%" :将整个消息转换为小写
    "%msg:10:$%" : 截取pos从10到最后的消息内容
    "%msg:R:.*Sev:. (.*) [.*–end%" : 取"*Sev:."和[.*之间内的所有内容
    ------------------------------------------------------------------------------
    R,<regexp -type>,<submatch>,<nomatch>,<match -number>
    regexp-type :
    ------------------------------------------------------------------------------
    -->"BRE" (基本的正则表达式)或"ERE" (扩展的正则表达式)
    0-9是标识submatch的,0标识所有内容,1-9标识相应序号的内容。(和shell类似,其实就是正则表达式中括号的用法)
    如果相同的实例被匹配到,那么第一个匹配的是0,之后的匹配依次排列
    -->nomatch :
    标识没有匹配到的时候写什么
    实例:
    "%msg:R,ERE,1,FIELD:for (vlan[0-9]*):--end%"
    "%msg:R,ERE,1,FIELD,1:for (vlan[0-9]*):--end%"
    "F,44" : 设置分隔符(用ACSII表示),针对分隔符,在引用时,0代表没找到,匹配的实例从1开始数
    "%msg:F,59:3%" : 以分号为分隔符,提取第三个区域的内容
    "%msg:F,59,5:3,9%" : 以分号为分隔符,从第5个子串中,提取3-6位的字符
    "%msg:F,32+:2%" : 如果多个分隔符(比如:日志中有n个空格作为分割,那么可以添加一个+)
  1. 解决中文显示乱码的问题
  • 视图首选项改为utf8
    wKioL1U_LP7yUp2mAAIQECySOKQ095.jpg
  • 注意rsyslog模板连接mariadb/mysql数据库用的账号密码,最好是安装rsyslog-mysql插件时系统创建的账号rsyslog