mentat.reports.event module

Library for generating event reports.

The implementation is based on mentat.reports.base.BaseReporter.

class mentat.reports.event.EventReporter(logger, reports_dir, templates_dir, global_fallback, locale, timezone, eventservice, sqlservice, mailer, event_classes_dir, groups_dict, settings_dict, whoismodule, thresholding=True)

Bases: BaseReporter

Implementation of reporting class providing Mentat event reports.

static aggregate_events(events)

Aggregate given list of events to dictionary structure that can be used to generate report message.

Parameters:

events (dict) – Structure containing events as mentat.idea.internal.Idea objects.

Returns:

Dictionary structure of aggregated events.

Return type:

dict

aggregate_relapsed_events(relapsed)

Parameters:

events (dict) – Dictionary of events aggregated by threshold key.

Returns:

Events aggregated by source.

Return type:

dict

cleanup(ttl)

Cleanup thresholding cache and remove all records with TTL older than given value.

Parameters:

time_h (datetime.datetime) – Upper cleanup time threshold.

Returns:

Number of removed records.

Return type:

int

fetch_severity_events(abuse_group, severity, time_l, time_h, testdata=False)

Fetch events with given severity for given abuse group within given time iterval.

Parameters:

• abuse_group – Abuse group model object.

• severity (str) – Event severity level to fetch.

• time_l (datetime.datetime) – Lower time interval boundary.

• time_h (datetime.datetime) – Upper time interval boundary.

• testdata (bool) – Switch to use test data for reporting.

Returns:

List of events matching search criteria.

Return type:

list

filter_event(filter_rules, event, to_db=True)

Filter given event according to given list of filtering rules.

Parameters:

• filter_rules (list) – Filters to be used.

• mentat.idea.internal.Idea – Event to be filtered.

• to_db (bool) – Save hit to db.

Returns:

True in case any filter matched, False otherwise.

Return type:

bool

filter_events(main_group, events)

Filter given list of IDEA events according to given abuse group settings. Events are aggregated by resolved abuses and source IP addresses.

Parameters:

• main_group (str) – Abuse group.

• events (list) – List of IDEA events as mentat.idea.internal.Idea objects.

Returns:

Tuple with list of events that passed filtering, aggregation of them, filtering log as a dictionary and number of passed events.

Return type:

tuple

filter_events_by_credibility(events_aggr)

Filter given dictionary of IDEA events aggregated by the source IP address by detector credibility. If the resulting credibility is less than 0.5, the event is discarded from the report.

Parameters:

events_aggt (dict) – Dictionary of IDEA events as mentat.idea.internal.Idea objects.

Returns:

Tuple with filtered dictionary, number of events passed, number of events discarded.

Return type:

tuple

filter_one_event(src, event, main_group, fltlog)

Compute and filter resolved abuses for an event with only one source IP address.

Parameters:

• src (ipranges.IP/Net/Range) – Source IP address

• event (mentat.idea.internal.Idea) – Event to be filtered.

• main_group (str) – Abuse group.

• fltlog (dict) – Filtering log.

Returns:

List of resolved abuses, list of fallback groups and filtering log as dictionary.

Return type:

tuple

static j2t_idea_path_valueset(message_s, jpath_s)

Calculate and return set of all values on all given jpaths in all given messages. Messages and jpaths can also be single values.

static prepare_structured_data(events_reg_aggr, events_rel_aggr, settings)

Prepare structured data for report column

Parameters:

• events_reg_aggr (list) – List of events as mentat.idea.internal.Idea objects.

• events_rel_aggr (list) – List of relapsed events as mentat.idea.internal.Idea objects.

Returns:

Structured data that can be used to generate report message

Return type:

dict

relapse_events(abuse_group, severity, time_h)

Detect IDEA event relapses for given abuse group settings.

Parameters:

• abuse_group (mentat.datatype.sqldb.GroupModel) – Abuse group.

• severity (str) – Severity for which to perform reporting.

• time_h (datetime.datetime) – Upper reporting time threshold.

Returns:

List of events that relapsed.

Return type:

list

render_report(report, settings, template_vars=None, srcip=None)

report(abuse_group, severity, time_l, time_h, template_vars=None, testdata=False)

Perform reporting for given most specific abuse group, event severity and time window.

Parameters:

• abuse_group (mentat.datatype.internal.GroupModel) – Abuse group.

• severity (str) – Severity for which to perform reporting.

• time_l (datetime.datetime) – Lower reporting time threshold.

• time_h (datetime.datetime) – Upper reporting time threshold.

• template_vars (dict) – Dictionary containing additional template variables.

• testdata (bool) – Switch to use test data for reporting.

report_extra(parent_rep, result, events, group_chain, fallback_groups, settings, severity, time_l, time_h, template_vars=None, testdata=False)

Generate extra reports from given events for given abuse group, severity and period.

Parameters:

• parent_rep (mentat.datatype.sqldb.EventReportModel) – Parent summary report.

• result (dict) – Reporting result structure with various usefull metadata.

• events (dict) – Dictionary structure with IDEA events to be reported.

• group_chain (list) – List of resolved abuse groups.

• fallback_groups (list) – List of fallback abuse groups.

• settings (mentat.reports.event.ReportingSettings) – Reporting settings.

• severity (str) – Severity for which to perform reporting.

• time_l (datetime.datetime) – Lower reporting time threshold.

• time_h (datetime.datetime) – Upper reporting time threshold.

• template_vars (dict) – Dictionary containing additional template variables.

• testdata (bool) – Switch to use test data for reporting.

report_summary(result, events, group_chain, fallback_groups, settings, severity, time_l, time_h, original_group_only, template_vars=None, testdata=False)

Generate summary report from given events for given abuse group, severity and period.

Parameters:

• result (dict) – Reporting result structure with various usefull metadata.

• events (dict) – Dictionary structure with IDEA events to be reported.

• group_chain (list) – List of resolved abuse groups.

• fallback_groups (list) – List of fallback abuse groups.

• settings (mentat.reports.event.ReportingSettings) – Reporting settings.

• severity (str) – Severity for which to perform reporting.

• time_l (datetime.datetime) – Lower reporting time threshold.

• time_h (datetime.datetime) – Upper reporting time threshold.

• original_group_only (bool) – Check if there is only the most specific abuse group.

• template_vars (dict) – Dictionary containing additional template variables.

• testdata (bool) – Switch to use test data for reporting.

threshold_events(events_aggr, abuse_group, group_chain, severity, time_h)

Threshold given list of IDEA events according to given abuse group settings.

Parameters:

• events_aggr (dict) – Aggregation of IDEA events as mentat.idea.internal.Idea objects by source.

• mentat.datatype.sqldb.GroupModel – Abuse group.

• severity (str) – Severity for which to perform reporting.

• time_h (datetime.datetime) – Upper reporting time threshold.

Returns:

List of events that passed thresholding.

Return type:

list

update_thresholding_cache(events, settings, severity, time_h)

Parameters:

• events (dict) – Dictionary structure with IDEA events that were reported.

• settings (mentat.reports.event.ReportingSettings) – Reporting settings.

• severity (str) – Severity for which to perform reporting.

• time_h (datetime.datetime) – Upper reporting time threshold.

mentat.reports.event.REPORT_EMAIL_TEXT_WIDTH = 90

Width of the report email text.

mentat.reports.event.REPORT_SUBJECT_EXTRA = '[{:s}] {:s} - Notice about possible problems regarding host {:s}'

Subject for extra report emails.

mentat.reports.event.REPORT_SUBJECT_SUMMARY = '[{:s}] {:s} - Notice about possible problems in your network'

Subject for summary report emails.

mentat.reports.event.json_default(val)

Helper function for JSON serialization of non basic data types.