[bugfix] HTML faild validator.w3.org checks

Implements checks for mass channel activity to ensure proper functionality.

Introduces methods to verify if mass channels are active and to set errors when they are not connected.

Enhances reliability of data logging by avoiding unnecessary processing when mass channels are inactive.

.gitignore

@@ -213,3 +213,7 @@ flycheck_*.el
 
 /__simulate_fail.json
 /mailing_list.yaml
+
+.vscode/
+
+*.code-workspace

Dockerfile

@@ -0,0 +1,40 @@
+FROM python:3
+
+# metadata
+LABEL maintainer="Kasper D. Fischer <kasper.fischer@rub.de>"
+LABEL version="0.2-docker"
+LABEL description="Docker image for the survBot application"
+
+# install required system packages
+RUN apt update && apt install -y bind9-host iputils-ping
+
+# create user and group and home directory
+RUN groupadd -r survBot && useradd -r -g survBot survBot
+RUN mkdir -p /home/survBot && chown -R survBot:survBot /home/survBot
+
+# change working directory
+WORKDIR /usr/src/app
+
+# install required python packages
+RUN --mount=type=bind,source=requirements.txt,target=/tmp/requirements.txt \
+    pip install --no-cache-dir --requirement /tmp/requirements.txt
+
+# copy application files
+COPY survBot.py utils.py write_utils.py LICENSE README.md ./
+
+# copy configuration files
+VOLUME /usr/src/app/conf
+COPY parameters.yaml mailing_list.yam[l] simulate_fail.jso[n] conf/
+RUN ln -s conf/simulate_fail.json simulate_fail.json
+
+# copy www files
+VOLUME /usr/src/app/www
+COPY logo.pn[g] stylesheets/desktop.css stylesheets/mobile.css www/
+RUN ln -s www/survBot_out.html www/index.html
+
+# change ownership of working directory
+RUN chown -R survBot:survBot /usr/src/app
+
+# run the application as user survBot
+USER survBot
+CMD [ "python", "./survBot.py", "-html", "www", "-parfile", "conf/parameters.yaml" ]

README.md

@@ -3,7 +3,7 @@
 version: 0.2
 
 survBot is a small program used to track station quality channels of DSEBRA stations via PowBox output over SOH channels
- by analysing contents of a Seiscomp3 datapath.
+ by analyzing contents of a Seiscomp data archive.
 
 ## Requirements
 
@@ -40,16 +40,53 @@ The GUI can be loaded via
 python survBotGui.py
 ```
 
+### Docker
+
+To run the program in a Docker container, first build the image:
+
+```shell script
+docker build -t survbot .
+```
+
+Then run the container:
+
+```shell script
+docker run -v /path/to/conf-dir:/usr/src/app/conf -v /path/to/output:/usr/src/app/www survbot
+```
+
+The directory `/path/to/conf-dir` should contain the `parameters.yaml` file, and the directory `/path/to/output` will contain the output HTML files.
+
+### Configuration of the e-mail server settings
+
+The e-mail server settings can be configured in the `parameters.yaml` file. The following settings are available:
+
+* `mailserver`: the address of the mail server
+* `auth_type`: the authentication type for the mail server (`None`, `SSL`, `TLS`)
+* `port`: the port of the mail server
+* `user`: the username for the mail server (if required)
+* `password`: the password for the mail server (if required)
+
+The `user` and `password` fields are optional, and can be left empty if the mail server does not require authentication. The `auth_type` field can be set to `None` if no authentication is required, `SSL` if the mail server requires SSL authentication, or `TLS` if the mail server requires TLS authentication. If the `user` or `password` fields are set to `Docker` ore `ENV` the program will try to read the values from the docker secrets `mail_user` and `mail_password` or environment variables `MAIL_USER` and `MAIL_PASSWORD` respectively. Docker secrets are only available in Docker Swarm mode, i.e. if the program is run as a service.
+
 ## Version Changes
-- surveillance of mass, clock and gaps
+
-- individual mailing lists for different stations
+### 0.2
-- html mail with recent status information
+
-- updated web page design
+* surveillance of mass, clock and gaps
-- restructured parameter file
+* individual mailing lists for different stations
-- recognize if PBox is disconnected
+* html mail with recent status information
+* updated web page design
+* restructured parameter file
+* recognize if PBox is disconnected
+
+### 0.2-docker
+
+* added Dockerfile for easy deployment
+* added more settings for connection to a mail server
 
 ## Staff
 
-Original author: M.Paffrath (marcel.paffrath@rub.de)
+Original author: M.Paffrath (<marcel.paffrath@rub.de>)
+Contributions: Kasper D. Fischer (<kasper.fischer@rub.de>)
 
-June 2023
+Jan 2025

__init__.py

@@ -1,3 +1,3 @@
 # survBot is a small program used to track station quality channels of DSEBRA stations via PowBox output
 # over SOH channels by analysing contents of a Seiscomp3 datapath.
-__version__ = "0.2"
+__version__ = "0.2-docker"

mailing_list.yaml

@@ -2,7 +2,7 @@
 # "mail.address@provider.com, mail.address2@provider2.com":
 #   - 1Y.GR01
 #   - 1Y.GR02
-# "mail.address3@provder.com":
+# "mail.address3@provider.com":
 #   - 1Y.GR03
 
 #"kasper.fischer@rub.de":

parameters.yaml

@@ -1,14 +1,15 @@
 # Parameters file for Surveillance Bot
-datapath: "/data/SDS/"                    # SC3 Datapath
+datapath: "/data/SDS/"                    # path to SDS data archive
 networks: ["1Y", "HA", "MK"]              # select networks, list or str
 stations: "*"                             # select stations, list or str
 locations: "*"                            # select locations, list or str
-stations_blacklist: ["TEST", "EREA", "DOMV"]      # exclude these stations
+stations_blacklist: ["ATHR", "DOMV", "EREA", "GR19", "GR27", "HAVD", "LAKA", "LFKM", "TEST"]      # exclude these stations
 networks_blacklist: []                    # exclude these networks
 interval: 60               # Perform checks every x seconds
 n_track: 360               # wait n_track * intervals before performing an action (i.e. send mail/end highlight status)
-timespan: 3                # Check data of the recent x days
+timespan: 7                # Check data of the recent x days
-verbosity: 0               # verbosity flag
+verbosity: 0               # verbosity flag for program console output (not logging)
+logging_level: WARN        # set logging level (info, warning, debug)
 track_changes: True        # tracks all changes since GUI startup by text highlighting (GUI only)
 warn_count: False          # show number of warnings and errors in table
 min_sample: 5              # minimum samples for raising Warn/FAIL
@@ -39,6 +40,7 @@ POWBOX:
 THRESHOLDS:
   pb_thresh: 0.2               # Threshold for PowBox Voltage check +/- (V)
   max_temp: 50                 # max temperature for temperature warning
+  critical_temp: 65            # max temperature for critical warning (fail)
   low_volt: 12                 # min voltage for low voltage warning
   high_volt: 14.8              # max voltage for over voltage warning
   unclassified: 5              # min voltage samples not classified for warning
@@ -54,7 +56,8 @@ THRESHOLDS:
 #
 # For each channel a factor 'unit' for unit conversion (e.g. to SI) can be provided, as well as a 'name'
 # and 'ticks' [ymin, ymax, ystep] for plotting.
-# 'warn' and 'fail' plot horizontal lines in corresponding colors (can be str in TRESHOLDS, int/float or iterable)
+# 'warn' and 'fail' plot horizontal lines in corresponding colors (can be str in THRESHOLDS, int/float or iterable)
+# keyword "pb_SOH2" or "pb_SOH3" can be used to extract warning values from above POWBOX parameter definition
 #
 # 'transform' can be provided for plotting to perform arithmetic operations in given order, e.g.:
 #  transform: - ["*", 20]
@@ -73,12 +76,12 @@ CHANNELS:
     unit: 1e-6
     name: "PowBox 230V/12V (V)"
     ticks: [0, 5, 1]
-    warn: [2, 3, 4, 4.5, 5]
+    warn: "pb_SOH2"
   EX3:
     unit: 1e-6
     name: "PowBox Router/Charger (V)"
     ticks: [0, 5, 1]
-    warn: [2, 2.5, 3, 4, 5]
+    warn: "pb_SOH3"
   VEI:
     unit: 1e-3
     name: "Datalogger (V)"
@@ -114,6 +117,11 @@ data_channels: ["HHZ", "HHN", "HHE"]
 
 
 # ---------------------------------------- OPTIONAL PARAMETERS ---------------------------------------------------------
+# deactivate powbox surveillance for stations (e.g. for solar powered), key: station, value: status message (abbr.)
+no_pbox_stations: {'GR33': 'SOL', 'GR27B': 'DCN', 'RP01': 'noPBox', 'RP02': 'noPBox', 'RP03': 'noPBox', 'RP04': 'noPBox', 'RP05B': 'noPBox', 'RP06': 'noPBox', 'RP07': 'noPBox', 'RP08': 'noPBox'}
+
+# deactivate mass position surveillance for stations without connected mass channels (e.g. STS-2 instruments), key: station, value: status message (abbr.)
+no_mass_stations: {'BIA': 'DCN', 'KNEZ': 'DCN', 'KRUS': 'DCN', 'OHR': 'DCN', 'OTOV': 'DCN', 'SKO': 'DCN', 'STIP': 'DCN', 'VAY': 'DCN', 'ZUPA': 'DCN'}
 
 # add links to html table with specified key as column and value as relative link, interpretable string parameters:
 # nw (e.g. 1Y), st (e.g. GR01A), nwst_id (e.g. 1Y.GR01A)
@@ -121,22 +129,31 @@ add_links:
   # for example: slmon: {"URL": "path/{nw}_{st}.html", "text": "link"}
   slmon: {"URL": "../slmon/{nw}_{st}.html", "text": "show"}
   24h-plot: {"URL": "../scheli/{nw}/{st}.png", "text": "plot"}
+  ppsd: {"URL": "../ppsd/{nw}.{st}.html", "text": "show"}
 
 # add station-independent links below html table (list items separated with -)
 add_global_links:
   # for example: - {"text": "our homepage", "URL": "https://www.rub.de"}
   - {"text": "show recent events on map",
-     "URL": "https://fdsnws.geophysik.ruhr-uni-bochum.de/map/?lat=39.5&lon=21&zoom=7&baselayer=mapnik"}
+     "URL": "https://fdsnws.geophysik.ruhr-uni-bochum.de/map/?lat=39.5&lon=21&zoom=7&baselayer=osm"}
 
 # html logo at page bottom (path relative to html directory)
-html_logo: "figures/Logo_RUB_BLAU_rgb.png"
+html_logo: "logo.png"
 
 # E-mail notifications
 EMAIL:
-  mailserver: "localhost"
+  # specify mail server and credentials
-  addresses: ["marcel.paffrath@rub.de", "kasper.fischer@rub.de"]  # list of mail addresses for info mails
+  # port, auth_type, user and password are only required if mailserver is not set to "localhost"
-  sender: "webmaster@geophysik.ruhr-uni-bochum.de" # mail sender
+  # user and password can be set to "ENV" or "DOCKER" to read from environment variables or docker secrets
-  stations_blacklist: ['GR33']                     # do not send emails for specific stations
+  mailserver: "smtp.example.org"      # mail server
-  networks_blacklist: []                           # do not send emails for specific network
+  auth_type: "SSL"                    # mail authentication type, can be "SSL", "TLS" or "None"
+  port: 465                           # mail port, default 465 for SSL, 587 for TLS
+  user: "DOCKER"                      # mail user, read from environment variable if set to "ENV" or from docker secret if set to "DOCKER"
+  password: "DOCKER"                  # mail password, read from environment variable if set to "ENV" or from docker secret if set to "DOCKER"
+  # specify mail recipients, sender and blacklists
+  addresses: ["test@example.org"]  # list of mail addresses for info mails
+  sender: "<test@example.org>"              # mail sender
+  stations_blacklist: []                                          # do not send emails for specific stations
+  networks_blacklist: []                                          # do not send emails for specific network
   # specify recipients for single stations in a yaml: key = email-address, val = station list (e.g. [1Y.GR01, 1Y.GR02])
-  external_mail_list: "mailing_list.yaml"
+  external_mail_list: "conf/mailing_list.yaml"

requirements.txt

@@ -0,0 +1,36 @@
+Brotli==1.1.0
+certifi==2025.1.31
+cffi==1.17.1
+charset-normalizer==3.4.1
+contourpy==1.3.1
+cycler==0.12.1
+decorator==5.2.1
+fonttools==4.56.0
+greenlet==3.1.1
+h2==4.2.0
+hpack==4.1.0
+hyperframe==6.1.0
+idna==3.10
+kiwisolver==1.4.8
+lxml==5.3.1
+matplotlib==3.8.4
+munkres==1.1.4
+numpy==1.26.4
+obspy==1.4.1
+packaging==24.2
+pillow==11.1.0
+pip==25.0.1
+pycparser==2.22
+pyparsing==3.2.1
+PySocks==1.7.1
+python-dateutil==2.9.0.post0
+PyYAML==6.0.2
+requests==2.32.3
+scipy==1.15.2
+setuptools==75.8.2
+six==1.17.0
+SQLAlchemy==1.4.54
+unicodedata2==16.0.0
+urllib3==2.3.0
+wheel==0.45.1
+zstandard==0.23.0

stylesheets/desktop.css

@@ -1,6 +1,5 @@
 body {
     background-color: #ffffff;
-    place-items: center;
     text-align: center;
     padding-bottom: 30px;
     font-family: "Helvetica", "sans-serif";
@@ -17,7 +16,7 @@ td {
 }
 
 th {
-    background-color: #999;
+    background-color: #17365c;
     color: #fff;
     border-radius: 2px;
     padding: 3px 1px;

stylesheets/mobile.css

@@ -1,6 +1,5 @@
 body {
     background-color: #ffffff;
-    place-items: center;
     text-align: center;
     padding-bottom: 30px;
     font-family: "Helvetica", "sans-serif";
@@ -17,10 +16,10 @@ td {
 }
 
 th {
-    background-color: #999;
+    background-color: #17365c;
     color: #fff;
     border-radius: 3px;
-    padding: 10px, 2px;
+    padding: 10px 2px;
     position: sticky;
     top: 0;
 }

submit_bot.sh

@@ -1,19 +1,24 @@
 #!/bin/bash
-ulimit -s 8192
 
 #$ -l low
-#$ -l h_vmem=5G
+#$ -l h_vmem=2.5G
+#$ -l mem=2.5G
+#$ -l h_stack=INFINITY
 #$ -cwd
 #$ -pe smp 1
-#$ -N survBot_bg
+#$ -binding linear:1
-#$ -l os=*stretch
+#$ -N survBot
+#$ -o /data/www/~kasper/survBot/survBot_bg.log
+#$ -e /data/www/~kasper/survBot/survBot_bg.err
+#$ -m e
+#$ -M kasper.fischer@rub.de
 
 source /opt/anaconda3/etc/profile.d/conda.sh
-conda activate py37
+conda activate survBot
 
 # environment variables for numpy to prevent multi threading
 export MKL_NUM_THREADS=1
 export NUMEXPR_NUM_THREADS=1
 export OMP_NUM_THREADS=1
 
-python survBot.py -html '/data/www/~marcel/'
+python survBot.py -html '/data/www/~kasper/survBot'

survBot.py

@@ -1,12 +1,13 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
-__version__ = '0.1'
+__version__ = '0.3'
-__author__ = 'Marcel Paffrath'
+__author__ = ['Marcel Paffrath <marcel.paffrath@rub.de>', 'Kasper D. Fischer <kasper.fischer@rub.de>']
 
 import os
 import io
 import copy
+import logging
 import traceback
 import yaml
 import argparse
@@ -22,7 +23,8 @@ from obspy.clients.filesystem.sds import Client
 
 from write_utils import get_html_text, get_html_link, get_html_row, html_footer, get_html_header, get_print_title_str, \
     init_html_table, finish_html_table, get_mail_html_header, add_html_image
-from utils import get_bg_color, get_font_color, modify_stream_for_plot, set_axis_yticks, set_axis_color, plot_axis_thresholds
+from utils import get_bg_color, get_font_color, modify_stream_for_plot, set_axis_yticks, set_axis_color, plot_axis_thresholds, \
+    connect_to_mail_server
 
 try:
     import smtplib
@@ -31,27 +33,48 @@ try:
 
     mail_functionality = True
 except ImportError:
-    print('Could not import smtplib or mail. Disabled sending mails.')
+    logging.warning('Could not import smtplib or mail. Disabled sending mails.')
     mail_functionality = False
 
 pjoin = os.path.join
 UP = "\x1B[{length}A"
 CLR = "\x1B[0K"
-deg_str = '\N{DEGREE SIGN}C'
+DEG_STR = '\N{DEGREE SIGN}C'
 
 
-def read_yaml(file_path, n_read=3):
+def read_yaml(file_path: str, n_read: int = 3) -> dict:
     for index in range(n_read):
         try:
             with open(file_path, "r") as f:
                 params = yaml.safe_load(f)
+            set_logging_level(params)
         except Exception as e:
-            print(f'Could not read parameters file: {e}.\nWill try again {n_read - index - 1} time(s).')
+            logging.warning(f'Could not read parameters file: {e}.\nWill try again {n_read - index - 1} time(s).')
             time.sleep(10)
             continue
         return params
 
 
+def set_logging_level(params: dict) -> None:
+    logging_levels = {'info': logging.INFO,
+                      'warning': logging.WARNING,
+                      'warn': logging.WARNING,
+                      'debug': logging.DEBUG,
+                      'error': logging.ERROR,
+                      'critical': logging.CRITICAL}
+    logging_level_str = params.get('logging_level')
+    if not logging_level_str:
+        logging.warning('Could not set logging level. Parameter not set')
+        return
+    if not isinstance(logging_level_str, str):
+        logging.warning(
+            f'Could not set logging level. Parameter logging_level = {logging_level_str} could not be interpreted.')
+        return
+    logging.info(f'Setting logging level to {logging_level_str}')
+    logging_level = logging_levels.get(logging_level_str.lower())
+    logging.basicConfig(level=logging_level)
+
+
 def nsl_from_id(nwst_id):
     nwst_id = get_full_seed_id(nwst_id)
     network, station, location = nwst_id.split('.')
@@ -115,7 +138,6 @@ class SurveillanceBot(object):
         self.parameters['channels'] = channels
         self.reread_parameters = self.parameters.get('reread_parameters')
         self.dt_thresh = [int(val) for val in self.parameters.get('dt_thresh')]
-        self.verbosity = self.parameters.get('verbosity')
         self.stations_blacklist = self.parameters.get('stations_blacklist')
         self.networks_blacklist = self.parameters.get('networks_blacklist')
         self.refresh_period = self.parameters.get('interval')
@@ -196,8 +218,7 @@ class SurveillanceBot(object):
         for filename in self.filenames:
             # if file already read and last modification time is the same as of last read operation: continue
             if self.filenames_read_last_modif.get(filename) == os.path.getmtime(filename):
-                if self.verbosity > 0:
+                logging.info(f'Continue on file {filename}')
-                    print('Continue on file', filename)
                 continue
             try:
                 # read only header of wf_data
@@ -207,13 +228,13 @@ class SurveillanceBot(object):
                     st_new = read(filename, dtype=float)
                 self.filenames_read_last_modif[filename] = os.path.getmtime(filename)
             except Exception as e:
-                print(f'Could not read file {filename}:', e)
+                logging.warning(f'Could not read file {filename}: {e}')
                 continue
             self.dataStream += st_new
         self.gaps = self.dataStream.get_gaps(min_gap=self.parameters['THRESHOLDS'].get('min_gap'))
         self.dataStream.merge()
 
-        # organise data in dictionary with key for each station
+        # organize data in dictionary with key for each station
         for trace in self.dataStream:
             nwst_id = get_nwst_id(trace)
             if not nwst_id in self.data.keys():
@@ -234,8 +255,7 @@ class SurveillanceBot(object):
             if stream:
                 nsl = nsl_from_id(nwst_id)
                 station_qc = StationQC(self, stream, nsl, self.parameters, self.keys, qc_starttime,
-                                       self.verbosity, print_func=self.print,
+                                       print_func=self.print, status_track=self.status_track.get(nwst_id))
-                                       status_track=self.status_track.get(nwst_id))
                 analysis_print_result = station_qc.return_print_analysis()
                 station_dict = station_qc.return_analysis()
             else:
@@ -331,7 +351,7 @@ class SurveillanceBot(object):
             first_exec = False
 
     def console_print(self, itemlist, str_len=21, sep='|', seplen=3):
-        assert len(sep) <= seplen, f'Make sure seperator has less than {seplen} characters'
+        assert len(sep) <= seplen, f'Make sure separator has less than {seplen} characters'
         sl = sep.ljust(seplen)
         sr = sep.rjust(seplen)
         string = sl
@@ -388,13 +408,13 @@ class SurveillanceBot(object):
                 st = modify_stream_for_plot(st, parameters=self.parameters)
                 st.plot(fig=fig, show=False, draw=False, block=False, equal_scale=False, method='full',
                         starttime=starttime, endtime=endtime)
-                # set_axis_ylabels(fig, self.parameters, self.verbosity)
+                # set_axis_ylabels(fig, self.parameters)
-                set_axis_yticks(fig, self.parameters, self.verbosity)
+                set_axis_yticks(fig, self.parameters)
                 set_axis_color(fig)
-                plot_axis_thresholds(fig, self.parameters, self.verbosity)
+                plot_axis_thresholds(fig, self.parameters)
             except Exception as e:
-                print(f'Could not generate plot for {nwst_id}:')
+                logging.error(f'Could not generate plot for {nwst_id}: {e}')
-                print(traceback.format_exc())
+                logging.error(traceback.format_exc())
             if len(fig.axes) > 0:
                 ax = fig.axes[0]
                 ax.set_title(f'Plot refreshed at (UTC) {UTCDateTime.now().strftime("%Y-%m-%d %H:%M:%S")}. '
@@ -402,7 +422,10 @@ class SurveillanceBot(object):
                 for ax in fig.axes:
                     ax.grid(True, alpha=0.1)
                 for fnout in fnames_out:
-                    fig.savefig(fnout, dpi=150., bbox_inches='tight')
+                    try:
+                        fig.savefig(fnout, dpi=150., bbox_inches='tight')
+                    except IOError as e:
+                        logging.warning('Could not save figure with IO error. Disk quota exceeded?\nError message: {e}')
                 # if needed save figure as virtual object (e.g. for mailing)
                 if save_bytes:
                     fnames_out[-1].seek(0)
@@ -458,7 +481,7 @@ class SurveillanceBot(object):
                 # add degree sign for temp
                 if check_key == 'temp':
                     if not type(message) in [str]:
-                        message = str(message) + deg_str
+                        message = str(message) + DEG_STR
 
                 html_class = self.get_html_class(hide_keys_mobile, status=status, check_key=check_key)
                 item = dict(text=str(message), tooltip=str(detailed_message), color=bg_color,
@@ -515,17 +538,16 @@ class SurveillanceBot(object):
                 # write footer with optional logo
                 logo_file = self.parameters.get('html_logo')
                 if not os.path.isfile(pjoin(self.outpath_html, logo_file)):
-                    print(f'Specified file {logo_file} not found.')
+                    logging.info(f'Specified file {logo_file} not found.')
                     logo_file = None
 
                 outfile.write(html_footer(footer_logo=logo_file))
 
         except Exception as e:
-            print(f'Could not write HTML table to {fnout}:')
+            logging.info(f'Could not write HTML table to {fnout}:')
-            print(traceback.format_exc())
+            logging.debug(traceback.format_exc())
 
-        if self.verbosity:
+        logging.info(f'Wrote html table to {fnout}')
-            print(f'Wrote html table to {fnout}')
 
     def update_status_message(self):
         timespan = timedelta(seconds=int(self.parameters.get('timespan') * 24 * 3600))
@@ -540,7 +562,6 @@ class SurveillanceBot(object):
         string.replace('\n', clear_end)
         print(string, end=clear_end, **kwargs)
         self.print_count += n_nl + 1  # number of newlines + actual print with end='\n' (no check for kwargs end!)
-        # print('pc:', self.print_count)
 
     def clear_prints(self):
         print(UP.format(length=self.print_count), end='')
@@ -548,14 +569,12 @@ class SurveillanceBot(object):
 
 
 class StationQC(object):
-    def __init__(self, parent, stream, nsl, parameters, keys, starttime, verbosity, print_func, status_track=None):
+    def __init__(self, parent, stream, nsl, parameters, keys, starttime, print_func, status_track=None):
         """
         Station Quality Check class.
         :param nsl: dictionary containing network, station and location (key: str)
         :param parameters: parameters dictionary from parameters.yaml file
         """
-        if status_track is None:
-            status_track = {}
         self.parent = parent
         self.stream = stream
         self.nsl = nsl
@@ -565,7 +584,6 @@ class StationQC(object):
         # make a copy of parameters object to prevent accidental changes
         self.parameters = copy.deepcopy(parameters)
         self.program_starttime = starttime
-        self.verbosity = verbosity
         self.last_active = False
         self.print = print_func
 
@@ -576,6 +594,9 @@ class StationQC(object):
             status_track = {}
         self.status_track = status_track
 
+        self.powbox_active = self.is_pbox_activated_check()
+        self.mass_active = self.is_mass_activated_check()
+
         self.start()
 
     @property
@@ -598,8 +619,7 @@ class StationQC(object):
         current_status = self.status_dict.get(key)
 
         # change this to something more useful, SMS/EMAIL/PUSH
-        if self.verbosity:
+        logging.info(f'{UTCDateTime()}: {detailed_message}')
-            self.print(f'{UTCDateTime()}: {detailed_message}', flush=False)
 
         # if error, do not overwrite with warning
         if current_status.is_error:
@@ -628,7 +648,8 @@ class StationQC(object):
         send_mail = False
         new_error = StatusError(count=count, show_count=self.parameters.get('warn_count'))
         if disc:
-            new_error.set_disconnected()
+            msg = disc if type(disc) == str else None
+            new_error.set_disconnected(msg)
         current_status = self.status_dict.get(key)
         if current_status.is_error:
             current_status.count += count
@@ -638,8 +659,7 @@ class StationQC(object):
             if self.status_track.get(key) and not self.status_track.get(key)[-1]:
                 self.parent.write_html_figure(self.nwst_id, save_bytes=True)
 
-        if self.verbosity:
+        logging.info(f'{UTCDateTime()}: {detailed_message}')
-            self.print(f'{UTCDateTime()}: {detailed_message}', flush=False)
 
         # do not send error mail if this is the first run (e.g. program startup) or state was already error (unchanged)
         if self.search_previous_errors(key) is True:
@@ -671,7 +691,7 @@ class StationQC(object):
 
         # simulate an error specified in json file (dictionary: {nwst_id: key} )
         if self._simulated_error_check(key) is True:
-            print(f'Simulating Error on {self.nwst_id}, {key}')
+            logging.info(f'Simulating Error on {self.nwst_id}, {key}')
             return True
 
         previous_errors = self.status_track.get(key)
@@ -696,26 +716,22 @@ class StationQC(object):
     def send_mail(self, key, status_type, additional_message=''):
         """ Send info mail using parameters specified in parameters file """
         if not mail_functionality:
-            if self.verbosity:
+            logging.info('Mail functionality disabled. Return')
-                print('Mail functionality disabled. Return')
             return
 
         mail_params = self.parameters.get('EMAIL')
         if not mail_params:
-            if self.verbosity:
+            logging.info('parameter "EMAIL" not set in parameter file. Return')
-                print('parameter "EMAIL" not set in parameter file. Return')
             return
 
         stations_blacklist = mail_params.get('stations_blacklist')
         if stations_blacklist and self.station in stations_blacklist:
-            if self.verbosity:
+            logging.info(f'Station {self.station} listed in blacklist. Return')
-                print(f'Station {self.station} listed in blacklist. Return')
             return
 
         networks_blacklist = mail_params.get('networks_blacklist')
         if networks_blacklist and self.network in networks_blacklist:
-            if self.verbosity:
+            logging.info(f'Station {self.station} of network {self.network} listed in blacklist. Return')
-                print(f'Station {self.station} of network {self.network} listed in blacklist. Return')
             return
 
         sender = mail_params.get('sender')
@@ -724,10 +740,8 @@ class StationQC(object):
         if add_addresses:
             # create copy of addresses ( [:] ) to prevent changing original, general list with addresses
             addresses = addresses[:] + list(add_addresses)
-        server = mail_params.get('mailserver')
         if not sender or not addresses:
-            if self.verbosity:
+            logging.info('Mail sender or addresses not (correctly) defined. Return')
-                print('Mail sender or addresses not (correctly) defined. Return')
             return
         dt = self.get_dt_for_action()
         text = f'{key}: Status {status_type} longer than {dt}: ' + additional_message
@@ -744,8 +758,10 @@ class StationQC(object):
         html_str = self.add_html_mail_body(text)
         msg.add_alternative(html_str, subtype='html')
 
-        # send message via SMTP server
+        # connect to server, send mail and close connection
-        s = smtplib.SMTP(server)
+        s = connect_to_mail_server(mail_params)
+        if not s: # if connection failed
+            return
         s.send_message(msg)
         s.quit()
 
@@ -790,20 +806,16 @@ class StationQC(object):
                         yield address
             # file not existing
             except FileNotFoundError as e:
-                if self.verbosity:
+                logging.warning(e)
-                    print(e)
             # no dictionary
             except AttributeError as e:
-                if self.verbosity:
+                logging.warning(f'Could not read dictionary from file {eml_filename}: {e}')
-                    print(f'Could not read dictionary from file {eml_filename}: {e}')
             # other exceptions
             except Exception as e:
-                if self.verbosity:
+                logging.warning(f'Could not open file {eml_filename}: {e}')
-                    print(f'Could not open file {eml_filename}: {e}')
         # no file specified
         else:
-            if self.verbosity:
+            logging.info('No external mail list set.')
-                print('No external mail list set.')
 
         return []
 
@@ -877,9 +889,8 @@ class StationQC(object):
         timespan = self.parameters.get('timespan') * 24 * 3600
         self.analysis_starttime = self.program_starttime - timespan
 
-        if self.verbosity > 0:
+        logging.info(150 * '#')
-            self.print(150 * '#')
+        logging.info('This is StationQC. Calculating quality for station'
-            self.print('This is StationQT. Calculating quality for station'
                        ' {network}.{station}.{location}'.format(**self.nsl))
         self.voltage_analysis()
         self.pb_temp_analysis()
@@ -907,7 +918,7 @@ class StationQC(object):
             if key == 'last active':
                 items.append(fancy_timestr(message))
             elif key == 'temp':
-                items.append(str(message) + deg_str)
+                items.append(str(message) + DEG_STR)
             else:
                 items.append(str(message))
         return items
@@ -946,9 +957,8 @@ class StationQC(object):
         clock_quality_warn_level = self.parameters.get('THRESHOLDS').get('clockquality_warn')
         clock_quality_fail_level = self.parameters.get('THRESHOLDS').get('clockquality_fail')
 
-        if self.verbosity > 1:
+        logging.info(40 * '-')
-            self.print(40 * '-')
+        logging.info('Performing Clock Quality check')
-            self.print('Performing Clock Quality check', flush=False)
 
         clockQuality_warn = np.where(clock_quality < clock_quality_warn_level)[0]
         clockQuality_fail = np.where(clock_quality < clock_quality_fail_level)[0]
@@ -992,9 +1002,8 @@ class StationQC(object):
         low_volt = self.parameters.get('THRESHOLDS').get('low_volt')
         high_volt = self.parameters.get('THRESHOLDS').get('high_volt')
 
-        if self.verbosity > 1:
+        logging.info(40 * '-')
-            self.print(40 * '-')
+        logging.info('Performing Voltage check')
-            self.print('Performing Voltage check', flush=False)
 
         overvolt = np.where(voltage > high_volt)[0]
         undervolt = np.where(voltage < low_volt)[0]
@@ -1020,9 +1029,12 @@ class StationQC(object):
             self.warn(key, detailed_message=detailed_message, count=n_undervolt,
                       last_occurrence=self.get_last_occurrence(trace, undervolt))
 
-    def pb_temp_analysis(self, channel='EX1'):
+    def pb_temp_analysis(self, channel='EX1', t_max_default=50, t_crit_default=70):
         """ Analyse PowBox temperature output. """
         key = 'temp'
+        if not self.powbox_active:
+            self.set_pbox_inactive_error(key)
+            return
         st = self.stream.select(channel=channel)
         trace = self.get_trace(st, key)
         if not trace:
@@ -1033,23 +1045,31 @@ class StationQC(object):
         # average temp
         timespan = min([self.parameters.get('timespan') * 24 * 3600, int(len(temp) / trace.stats.sampling_rate)])
         nsamp_av = int(trace.stats.sampling_rate) * timespan
-        av_temp_str = str(round(np.nanmean(temp[-nsamp_av:]), 1)) + deg_str
+        av_temp_str = str(round(np.nanmean(temp[-nsamp_av:]), 1)) + DEG_STR
         # dt of average
         dt_t_str = str(timedelta(seconds=int(timespan))).replace(', 0:00:00', '')
         # current temp
         cur_temp = round(temp[-1], 1)
-        if self.verbosity > 1:
+        logging.info(40 * '-')
-            self.print(40 * '-')
+        logging.info('Performing PowBox temperature check (EX1)')
-            self.print('Performing PowBox temperature check (EX1)', flush=False)
+        logging.info(f'Average temperature at {np.nanmean(temp)}\N{DEGREE SIGN}')
-            self.print(f'Average temperature at {np.nanmean(temp)}\N{DEGREE SIGN}', flush=False)
+        logging.info(f'Peak temperature at {max(temp)}\N{DEGREE SIGN}')
-            self.print(f'Peak temperature at {max(temp)}\N{DEGREE SIGN}', flush=False)
+        logging.info(f'Min temperature at {min(temp)}\N{DEGREE SIGN}')
-            self.print(f'Min temperature at {min(temp)}\N{DEGREE SIGN}', flush=False)
+        max_temp = thresholds.get('max_temp', t_max_default)
-        max_temp = thresholds.get('max_temp')
+        max_temp_crit = thresholds.get('critical_temp', t_crit_default)
         t_check = np.where(temp > max_temp)[0]
-        if len(t_check) > 0:
+        t_check_crit = np.where(temp > max_temp_crit)[0]
+        tcheck_message_template = ('Trace {id}: Temperature over {tmax}' + f'\N{DEGREE SIGN}'
+                                   + '! Current temperature: {temp}' + f'\N{DEGREE SIGN}')
+        if len(t_check_crit) > 0:
+            self.error(key=key,
+                      detailed_message=tcheck_message_template.format(id=trace.get_id(), tmax=max_temp, temp=cur_temp)
+                                       + self.get_last_occurrence_timestring(trace, t_check_crit),
+                      last_occurrence=self.get_last_occurrence(trace, t_check_crit))
+        elif len(t_check) > 0:
             self.warn(key=key,
-                      detailed_message=f'Trace {trace.get_id()}: '
+                      detailed_message=tcheck_message_template.format(id=trace.get_id(), tmax=max_temp_crit,
-                                       f'Temperature over {max_temp}\N{DEGREE SIGN} at {trace.get_id()}!'
+                                                                      temp=cur_temp)
                                        + self.get_last_occurrence_timestring(trace, t_check),
                       last_occurrence=self.get_last_occurrence(trace, t_check))
         else:
@@ -1061,6 +1081,11 @@ class StationQC(object):
         """ Analyse datalogger mass channels. """
         key = 'mass'
 
+        # skip processing if mass is not active
+        if not self.mass_active:
+            self.set_mass_inactive_error(key)
+            return
+        
         # build stream with all channels
         st = Stream()
         for channel in channels:
@@ -1098,23 +1123,26 @@ class StationQC(object):
             self.error(key=key,
                       detailed_message=f'Fail status for mass centering. Highest val (abs) {common_highest_val}V',)
 
-        if self.verbosity > 1:
+        logging.info(40 * '-')
-            self.print(40 * '-')
+        logging.info('Performing mass position check')
-            self.print('Performing mass position check', flush=False)
+        logging.info(f'Average mass position at {common_highest_val}')
-            self.print(f'Average mass position at {common_highest_val}', flush=False)
 
     def pb_power_analysis(self, channel='EX2', pb_dict_key='pb_SOH2'):
         """ Analyse EX2 channel of PowBox """
         keys = ['230V', '12V']
+        if not self.powbox_active:
+            for key in keys:
+                self.set_pbox_inactive_error(key)
+            return
+
         st = self.stream.select(channel=channel)
         trace = self.get_trace(st, keys)
         if not trace:
             return
 
         voltage = trace.data * self.get_unit_factor(channel)
-        if self.verbosity > 1:
+        logging.info(40 * '-')
-            self.print(40 * '-')
+        logging.info('Performing PowBox 12V/230V check (EX2)')
-            self.print('Performing PowBox 12V/230V check (EX2)', flush=False)
         voltage_check, voltage_dict, last_val = self.pb_voltage_ok(trace, voltage, pb_dict_key, channel=channel)
 
         if voltage_check:
@@ -1128,16 +1156,19 @@ class StationQC(object):
     def pb_rout_charge_analysis(self, channel='EX3', pb_dict_key='pb_SOH3'):
         """ Analyse EX3 channel of PowBox """
         keys = ['router', 'charger']
-        pb_thresh = self.parameters.get('THRESHOLDS').get('pb_1v')
+        if not self.powbox_active:
+            for key in keys:
+                self.set_pbox_inactive_error(key)
+            return
+
         st = self.stream.select(channel=channel)
         trace = self.get_trace(st, keys)
         if not trace:
             return
 
         voltage = trace.data * self.get_unit_factor(channel)
-        if self.verbosity > 1:
+        logging.info(40 * '-')
-            self.print(40 * '-')
+        logging.info('Performing PowBox Router/Charger check (EX3)')
-            self.print('Performing PowBox Router/Charger check (EX3)', flush=False)
         voltage_check, voltage_dict, last_val = self.pb_voltage_ok(trace, voltage, pb_dict_key, channel=channel)
 
         if voltage_check:
@@ -1243,6 +1274,10 @@ class StationQC(object):
         with each voltage value associated to the different steps specified in POWBOX > pb_steps. Also raises
         self.warn in case there are unassociated voltage values recorded.
         """
+        
+        if not self.powbox_active:
+            return
+
         pb_thresh = self.parameters.get('THRESHOLDS').get('pb_thresh')
         pb_ok = self.parameters.get('POWBOX').get('pb_ok')
         # possible voltage levels are keys of pb voltage level dict
@@ -1270,7 +1305,7 @@ class StationQC(object):
 
         # Warn in case of voltage under OK-level (1V)
         if len(under) > 0:
-            # try calculate number of occurences from gaps between indices
+            # try calculate number of occurrences from gaps between indices
             n_occurrences = len(np.where(np.diff(under) > 1)[0]) + 1
             voltage_dict[-1] = under
             self.status_other(detailed_message=f'Trace {trace.get_id()}: '
@@ -1305,6 +1340,19 @@ class StationQC(object):
         """ get UTCDateTime from trace and index"""
         return trace.stats.starttime + trace.stats.delta * index
 
+    def is_pbox_activated_check(self):
+        return self.station not in self.parameters.get('no_pbox_stations', [])
+
+    def set_pbox_inactive_error(self, key):
+        msg = self.parameters.get('no_pbox_stations')[self.station]
+        self.error(key, detailed_message=f'PowBox not connected', disc=msg)
+
+    def is_mass_activated_check(self):
+        return self.station not in self.parameters.get('no_mass_stations', [])
+
+    def set_mass_inactive_error(self, key):
+        msg = self.parameters.get('no_mass_stations')[self.station]
+        self.error(key, detailed_message=f'Mass channels not connected', disc=msg)
 
 class Status(object):
     """ Basic Status class. All status classes are derived from this class."""
@@ -1359,21 +1407,23 @@ class StatusOK(Status):
 
 
 class StatusWarn(Status):
-    def __init__(self, message='WARN', count=1, last_occurence=None, detailed_messages=None, show_count=False):
+    def __init__(self, message='WARN', count=1, last_occurrence=None, detailed_messages=None, show_count=False):
-        super(StatusWarn, self).__init__(message=message, count=count, last_occurrence=last_occurence,
+        super(StatusWarn, self).__init__(message=message, count=count, last_occurrence=last_occurrence,
                                          detailed_messages=detailed_messages, show_count=show_count)
         self.set_warn()
 
 
 class StatusError(Status):
-    def __init__(self, message='FAIL', count=1, last_occurence=None, detailed_messages=None, show_count=False):
+    def __init__(self, message='FAIL', count=1, last_occurrence=None, detailed_messages=None, show_count=False):
-        super(StatusError, self).__init__(message=message, count=count, last_occurrence=last_occurence,
+        super(StatusError, self).__init__(message=message, count=count, last_occurrence=last_occurrence,
                                           detailed_messages=detailed_messages, show_count=show_count)
         self.set_error()
         self.default_message = message
 
-    def set_disconnected(self, message='DCN'):
+    def set_disconnected(self, message=None):
         self.connection_error = True
+        if not message:
+            message = 'DCN'
         self.message = message
 
     def set_connected(self):
@@ -1382,8 +1432,8 @@ class StatusError(Status):
 
 
 class StatusOther(Status):
-    def __init__(self, messages=None, count=1, last_occurence=None, detailed_messages=None):
+    def __init__(self, messages=None, count=1, last_occurrence=None, detailed_messages=None):
-        super(StatusOther, self).__init__(count=count, last_occurrence=last_occurence,
+        super(StatusOther, self).__init__(count=count, last_occurrence=last_occurrence,
                                           detailed_messages=detailed_messages)
         if messages is None:
             messages = []

survBotGUI.py

@@ -11,6 +11,8 @@ import os
 import sys
 import traceback
 
+import logging
+
 try:
     from PySide2 import QtGui, QtCore, QtWidgets
 except ImportError:
@@ -41,7 +43,7 @@ try:
     from rest_api.rest_api_utils import get_last_messages, send_message, get_default_params
     sms_funcs = True
 except ImportError:
-    print('Could not load rest_api utils, SMS functionality disabled.')
+    logging.warning('Could not load rest_api utils, SMS functionality disabled.')
     sms_funcs = False
 
 deg_str = '\N{DEGREE SIGN}C'
@@ -54,10 +56,9 @@ class Thread(QtCore.QThread):
     """
     update = QtCore.Signal()
 
-    def __init__(self, parent, runnable, verbosity=0):
+    def __init__(self, parent, runnable):
         super(Thread, self).__init__(parent=parent)
         self.setParent(parent)
-        self.verbosity = verbosity
         self.runnable = runnable
         self.is_active = True
 
@@ -69,11 +70,10 @@ class Thread(QtCore.QThread):
             self.update.emit()
         except Exception as e:
             self.is_active = False
-            print(e)
+            logging.error(e)
-            print(traceback.format_exc())
+            logging.debug(traceback.format_exc())
         finally:
-            if self.verbosity > 0:
+            logging.info(f'Time for Thread execution: {UTCDateTime() - t0}')
-                print(f'Time for Thread execution: {UTCDateTime() - t0}')
 
 
 class MainWindow(QtWidgets.QMainWindow):
@@ -195,7 +195,7 @@ class MainWindow(QtWidgets.QMainWindow):
         station = nwst_id.split('.')[1]
         iccid = get_station_iccid(station)
         if not iccid:
-            print('Could not find iccid for station', nwst_id)
+            logging.info(f'Could not find iccid for station: {nwst_id}')
             return
         sms_widget = ReadSMSWidget(parent=self, iccid=iccid)
         sms_widget.setWindowTitle(f'Recent SMS of station: {nwst_id}')

utils.py

@@ -1,12 +1,24 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
+import logging
+
 import matplotlib
 import numpy as np
+import smtplib
+import os
 
 from obspy import Stream
 
 
+COLORS_DICT = {'FAIL': (195, 29, 14, 255),
+               'NO DATA': (255, 255, 125, 255),
+               'WARN': (250, 192, 63, 255),
+               'OK': (185, 245, 145, 255),
+               'undefined': (240, 240, 240, 255),
+               'disc': (126, 127, 131, 255), }
+
+
 def get_bg_color(check_key, status, dt_thresh=None, hex=False):
     message = status.message
     if check_key == 'last active':
@@ -41,13 +53,9 @@ def get_color(key):
     #                'OK': (173, 255, 133, 255),
     #                'undefined': (230, 230, 230, 255),
     #                'disc': (255, 160, 40, 255),}
-    colors_dict = {'FAIL': (195, 29, 14, 255),
+    if not key in COLORS_DICT.keys():
-                   'NO DATA': (255, 255, 125, 255),
+        key = 'undefined'
-                   'WARN': (250, 192, 63, 255),
+    return COLORS_DICT.get(key)
-                   'OK': (185, 245, 145, 255),
-                   'undefined': (240, 240, 240, 255),
-                   'disc': (126, 127, 131, 255), }
-    return colors_dict.get(key)
 
 
 def get_color_mpl(key):
@@ -82,6 +90,8 @@ def get_mass_color(message):
 def get_temp_color(temp, vmin=-10, vmax=60, cmap='coolwarm'):
     """ Get an rgba temperature value back from specified cmap, linearly interpolated between vmin and vmax. """
     if type(temp) in [str]:
+        if temp in COLORS_DICT.keys():
+            return get_color(temp)
         return get_color('undefined')
     cmap = matplotlib.cm.get_cmap(cmap)
     val = (temp - vmin) / (vmax - vmin)
@@ -165,12 +175,12 @@ def transform_trace(data, transf):
         elif operator_str == '/':
             data = data / val
         else:
-            raise IOError(f'Unknown arithmethic operator string: {operator_str}')
+            raise IOError(f'Unknown arithmetic operator string: {operator_str}')
 
     return data
 
 
-def set_axis_ylabels(fig, parameters, verbosity=0):
+def set_axis_ylabels(fig, parameters):
     """
     Adds channel names to y-axis if defined in parameters.
     """
@@ -178,24 +188,25 @@ def set_axis_ylabels(fig, parameters, verbosity=0):
     if not names: # or not len(st.traces):
         return
     if not len(names) == len(fig.axes):
-        if verbosity:
+        logging.info('Mismatch in axis and label lengths. Not adding plot labels')
-            print('Mismatch in axis and label lengths. Not adding plot labels')
         return
     for channel_name, ax in zip(names, fig.axes):
         if channel_name:
             ax.set_ylabel(channel_name)
 
 
-def set_axis_color(fig, color='0.8'):
+def set_axis_color(fig, color='0.8', shade_color='0.95'):
     """
-    Set all axes of figure to specific color
+    Set all axes (frame) of figure to specific color. Shade every second axis.
     """
-    for ax in fig.axes:
+    for i, ax in enumerate(fig.axes):
         for key in ['bottom', 'top', 'right', 'left']:
             ax.spines[key].set_color(color)
+        if i % 2:
+            ax.set_facecolor(shade_color)
 
 
-def set_axis_yticks(fig, parameters, verbosity=0):
+def set_axis_yticks(fig, parameters):
     """
     Adds channel names to y-axis if defined in parameters.
     """
@@ -203,8 +214,7 @@ def set_axis_yticks(fig, parameters, verbosity=0):
     if not ticks:
         return
     if not len(ticks) == len(fig.axes):
-        if verbosity:
+        logging.info('Mismatch in axis tick and label lengths. Not changing plot ticks.')
-            print('Mismatch in axis tick and label lengths. Not changing plot ticks.')
         return
     for ytick_tripple, ax in zip(ticks, fig.axes):
         if not ytick_tripple:
@@ -216,12 +226,11 @@ def set_axis_yticks(fig, parameters, verbosity=0):
         ax.set_ylim(ymin - 0.33 * step, ymax + 0.33 * step)
 
 
-def plot_axis_thresholds(fig, parameters, verbosity=0):
+def plot_axis_thresholds(fig, parameters):
     """
     Adds channel thresholds (warn, fail) to y-axis if defined in parameters.
     """
-    if verbosity > 0:
+    logging.info('Plotting trace thresholds')
-        print('Plotting trace thresholds')
 
     keys_colors = {'warn': dict(color=0.8 * get_color_mpl('WARN'), linestyle=(0, (5, 10)), alpha=0.5, linewidth=0.7),
                    'fail': dict(color=0.8 * get_color_mpl('FAIL'), linestyle='solid', alpha=0.5, linewidth=0.7)}
@@ -235,6 +244,10 @@ def plot_axis_thresholds(fig, parameters, verbosity=0):
 
 def plot_threshold_lines(fig, channel_threshold_list, parameters, **kwargs):
     for channel_thresholds, ax in zip(channel_threshold_list, fig.axes):
+        if channel_thresholds in ['pb_SOH2', 'pb_SOH3']:
+            annotate_voltage_states(ax, parameters, channel_thresholds)
+            channel_thresholds = get_warn_states_pbox(channel_thresholds, parameters)
+
         if not channel_thresholds:
             continue
 
@@ -244,5 +257,74 @@ def plot_threshold_lines(fig, channel_threshold_list, parameters, **kwargs):
         for warn_thresh in channel_thresholds:
             if isinstance(warn_thresh, str):
                 warn_thresh = parameters.get('THRESHOLDS').get(warn_thresh)
-            if type(warn_thresh in (float, int)):
+            if isinstance(warn_thresh, (float, int)):
                 ax.axhline(warn_thresh, **kwargs)
+
+
+def get_warn_states_pbox(soh_key: str, parameters: dict) -> list:
+    pb_dict = parameters.get('POWBOX').get(soh_key)
+    if not pb_dict:
+        return []
+    return [key for key in pb_dict.keys() if key > 1]
+
+
+def annotate_voltage_states(ax, parameters, pb_key, color='0.75'):
+    for voltage, voltage_dict in parameters.get('POWBOX').get(pb_key).items():
+        if float(voltage) < 1:
+            continue
+        out_string = ''
+        for key, val in voltage_dict.items():
+            if val != 'OK':
+                if out_string:
+                    out_string += ' | '
+                out_string += f'{key}: {val}'
+
+        ax.annotate(out_string, (ax.get_xlim()[-1], voltage), color=color, fontsize='xx-small',
+                    horizontalalignment='right')
+
+def get_credential(source, param):
+    """
+    Retrieve a credential from a Docker secret or environment variable.
+    """
+    if source == 'DOCKER':
+        try:
+            with open('/run/secrets/'+param.lower(), 'r') as f:
+                return f.read().strip()
+        except FileNotFoundError as e:
+            logging.error(f'Could not read from Docker secret at /run/secrets/{param.lower()}')
+            logging.error(e)
+    elif source == 'ENV':
+        try:
+            return os.environ.get(param.upper())
+        except Exception as e:
+            logging.error(f'Could not read from environment variable {param.upper()}')
+            logging.error(e)
+    # return source if no credential was found
+    return source
+
+def connect_to_mail_server(mail_params):
+    """
+    Connect to mail server and return server object.
+    """
+    # get server from parameters
+    server = mail_params.get('mailserver')
+    # get auth_type from parameters
+    auth_type = mail_params.get('auth_type')
+    # set up connection to mail server
+    if auth_type == 'None':
+        s = smtplib.SMTP(server)
+    else:
+        # user and password from parameters, docker secret or environment variable
+        user = get_credential(mail_params.get('user'), 'mail_user')
+        password = get_credential(mail_params.get('password'), 'mail_password')
+        # create secure connection to server
+        if auth_type == 'SSL':
+            s = smtplib.SMTP_SSL(server, mail_params.get('port'))
+        elif auth_type == 'TLS':
+            s = smtplib.SMTP(server, mail_params.get('port'))
+            s.starttls()
+        else:
+            logging.error('Unknown authentication type. Mails can not be sent')
+            return
+        s.login(user, password)
+    return s

write_utils.py

@@ -19,12 +19,13 @@ def get_html_header(refresh_rate=10):
     header = ['<!DOCTYPE html>',
               '<html>',
               '<head>',
-              '  <link rel="stylesheet" media="only screen and (max-width: 400px)" href="mobile.css" />',
+              '  <title>SurvBot status</title>',
-              '  <link rel="stylesheet" media="only screen and (min-width: 401px)" href="desktop.css" />',
+              '  <link rel="stylesheet" media="only screen and (max-width: 400px)" href="mobile.css">',
+              '  <link rel="stylesheet" media="only screen and (min-width: 401px)" href="desktop.css">',
+              f'  <meta http-equiv="refresh" content="{refresh_rate}">',
+              '  <meta charset="utf-8">',
+              '  <meta name="viewport" content="width=device-width, initial-scale=1">',
               '</head>',
-              f'<meta http-equiv="refresh" content="{refresh_rate}" >',
-              '<meta charset="utf-8">',
-              '<meta name="viewport" content="width=device-width, initial-scale=1">',
               '<body>']
     header = _convert_to_textstring(header)
     return header
@@ -86,7 +87,7 @@ def get_html_row(items, html_key='td'):
         html_class = item.get('html_class')
         class_str = f' class="{html_class}"' if html_class else ''
         row_string += 2 * default_space + f'<{html_key}{class_str} bgcolor="{color}" title="{tooltip}"' \
-                      + f'style="color:{font_color}"> {text_str}</{html_key}>\n'
+                      + f'style="color:{font_color}">{text_str}</{html_key}>\n'
     row_string += default_space + '</tr>\n'
     return row_string