mentat-reporter.py
This Mentat module is a script providing periodical event reports to target abuse groups.
This script is implemented using the pyzenkit.zenscript
framework and
so it provides all of its core features. See the documentation for more in-depth
details.
It is further based on mentat.script.fetcher
module, which provides
database fetching and message post-processing capabilities.
Usage examples
# Display help message and exit.
mentat-reporter.py --help
# Run in debug mode (enable output of debugging information to terminal).
mentat-reporter.py --debug
# Run with insanely increased logging level.
mentat-reporter.py --log-level debug
# Run in TEST DATA mode and MAIL TEST mode, force all reports to go to
# 'admin@domain.org'. In test data mode only events tagged with 'Test'
# category will be processed (useful for debugging). In mail test mode
# all generated reports will be redirected to configured admin email (root
# by default) instead of original contact, which is again useful for
# debugging or testing.
mentat-reporter.py --mail-test-mode --test-data --mail-to admin@domain.org
# Force reporter to use different email report template and localization.
mentat-reporter.py --template-id another --locale cs
Available script commands
report
(default)Generate report containing overall Mentat system performance statistics within configured time interval thresholds.
Brief overview of reporting algorithm
Reporting algorithm follows these steps:
For all abuse groups found in database:
For all event severities (
low
,medium
,high
,critical
):Fetch reporting configuration settings.
Fetch events with given severity, that appeared in database in given time window and belonging to that particular group.
Filter events with configured reporting filters.
Remove events from detectors with low credibility.
Threshold already reported events.
Fetch relapsed events.
Generate summary and/or extra reports and store them to database.
Send reports via email to target abuse contacts.
Configuration
The application supports multiple means for adjusting the internal configurations. When appropriate the default values for each configuration is hardcoded in module source code. However there are several options to change the value:
Override the internal default value when instantinating the application object by passing different value to object constructor.
Pass the different value by configuration file.
Pass the different value by command line option.
The configuration values are assigned from the sources mentioned above in that particular order, so the value given by command line option overwrites the value written in configuration file.
Command line options
Configuration can be passed down to application by command line options. These options have the highest priority and will overwrite any other configuration values. Depending on the base object of the application different set of options is available.
Common application options
Following configuration options are available for all applications based on
pyzenkit.baseapp
:
--help
Display help and usage description and exit (flag).
--debug
Run in debug mode (flag).
Input various status information to
stderr
.Type:
boolean
, default:False
--quiet
Run in quiet mode (flag).
Do not write anything to
stdout
orstderr
.Type:
boolean
, default:False
--verbose
Increase application output verbosity (flag, repeatable).
Type:
boolean
, default:False
--name alternative-name
Alternative name for application instead of default
$0
.This value will be used to generate names for log, runlog, pid, status and other application files.
Type:
string
, default:$0
--config-file file-name
Name of the configuration file.
Type:
string
, default: autodetected--config-file-silent
Do not complain in case given configuration file does not exist (flag).
Type:
boolean
, default:False
--config-dir file-name
Name of the configuration directory.
Type:
string
, default: autodetected--config-file-silent
Do not complain in case given configuration directory does not exist (flag).
Type:
boolean
, default:False
--log-file file-name
Name of the log file.
Type:
string
, default: autodetected--log-level level
Logging level [
debug
,info
,warning
,error
,critical
].Type:
string
, default:info
--runlog-dir dir-name
Name of the runlog directory.
Type:
string
, default: autodetected--runlog-dump
Dump runlog to stdout when done processing (flag).
Type:
boolean
, default:False
--runlog-log
Write runlog to logging service when done processing (flag)
Type:
boolean
, default:False
--pstate-file file-name
Name of the persistent state file.
Type:
string
, default: autodetected--pstate-dump
Dump persistent state to stdout when done processing (flag).
Type:
boolean
, default:False
--pstate-log
Write persistent state to logging service when done processing (flag).
Type:
boolean
, default:False
--action action
Execute given quick action and exit. List of available actions can be displayed with
--help
option.Type:
string
, default:None
--user name-or-id
Name/gid of the system user for process permissions.
Type:
string
, default:None
--group name-or-id
Name/gid of the system group for process permissions.
Type:
string
, default:None
Common script options
Following configuration options are available on top of common application options
for all applications based on pyzenkit.zenscript
:
--regular
Operational mode: regular script execution (flag).
Conflicts with
--shell
option.Type:
boolean
, default:False
--shell
Operational mode: manual script execution from shell (flag).
Conflicts with
--regular
option.Type:
boolean
, default:False
--command name
Name of the script command to be executed.
Type:
string
, default: autodetected--interval interval
Execution interval. This value should correspond with related cron script.
Type:
string
, default:daily
--adjust-thresholds
Round-up time interval threshols to interval size (flag).
Type:
boolean
, default:False
--time-high time
Upper time interval threshold.
Type:
float
, default: time.time
Common fetcher script options
Following configuration options are available on top of common script options
for all applications based on mentat.script.fetcher
:
--db-host host-name
Name of the host running the database.
Type:
string
, default:localhost
--db-port number
Port number for connecting to database.
Type:
integer
, default:27017
--db-timeout miliseconds
Socket timeout in miliseconds for communicating with database.
Type:
integer
, default:3600000
--database db-name
Name of the database to connect to.
Type:
string
--collection col-name
Name of the collection to connect to.
Type:
string
Configuration files and directories
Configuration can be passed down to application using a combination of configuration file or configuration directory. The configuration file
The available configuration keys are very similar to command line options and the
names differ only in the use of _
character instead of -
. However there is
a certain set of configuration keys that is available only through command line
options and not through configuration file and vice versa.