Compare commits
1 Commits
feature/do
...
23a2d9b7c9
| Author | SHA1 | Date | |
|---|---|---|---|
| 23a2d9b7c9 |
4
.gitignore
vendored
4
.gitignore
vendored
@@ -213,7 +213,3 @@ flycheck_*.el
|
|||||||
|
|
||||||
/__simulate_fail.json
|
/__simulate_fail.json
|
||||||
/mailing_list.yaml
|
/mailing_list.yaml
|
||||||
|
|
||||||
.vscode/
|
|
||||||
|
|
||||||
*.code-workspace
|
|
||||||
|
|||||||
37
Dockerfile
37
Dockerfile
@@ -1,40 +1,11 @@
|
|||||||
FROM python:3
|
FROM python:3
|
||||||
|
|
||||||
# metadata
|
|
||||||
LABEL maintainer="Kasper D. Fischer <kasper.fischer@rub.de>"
|
|
||||||
LABEL version="0.2-docker"
|
|
||||||
LABEL description="Docker image for the survBot application"
|
|
||||||
|
|
||||||
# install required system packages
|
|
||||||
RUN apt update && apt install -y bind9-host iputils-ping
|
|
||||||
|
|
||||||
# create user and group and home directory
|
|
||||||
RUN groupadd -r survBot && useradd -r -g survBot survBot
|
|
||||||
RUN mkdir -p /home/survBot && chown -R survBot:survBot /home/survBot
|
|
||||||
|
|
||||||
# change working directory
|
|
||||||
WORKDIR /usr/src/app
|
WORKDIR /usr/src/app
|
||||||
|
|
||||||
# install required python packages
|
COPY requirements.txt ./
|
||||||
RUN --mount=type=bind,source=requirements.txt,target=/tmp/requirements.txt \
|
RUN pip install --no-cache-dir -r requirements.txt
|
||||||
pip install --no-cache-dir --requirement /tmp/requirements.txt
|
RUN apt update && apt install -y bind9-host iputils-ping
|
||||||
|
|
||||||
# copy application files
|
COPY . .
|
||||||
COPY survBot.py utils.py write_utils.py LICENSE README.md ./
|
|
||||||
|
|
||||||
# copy configuration files
|
|
||||||
VOLUME /usr/src/app/conf
|
|
||||||
COPY parameters.yaml mailing_list.yam[l] simulate_fail.jso[n] conf/
|
|
||||||
RUN ln -s conf/simulate_fail.json simulate_fail.json
|
|
||||||
|
|
||||||
# copy www files
|
|
||||||
VOLUME /usr/src/app/www
|
|
||||||
COPY logo.pn[g] stylesheets/desktop.css stylesheets/mobile.css www/
|
|
||||||
RUN ln -s www/survBot_out.html www/index.html
|
|
||||||
|
|
||||||
# change ownership of working directory
|
|
||||||
RUN chown -R survBot:survBot /usr/src/app
|
|
||||||
|
|
||||||
# run the application as user survBot
|
|
||||||
USER survBot
|
|
||||||
CMD [ "python", "./survBot.py", "-html", "www", "-parfile", "conf/parameters.yaml" ]
|
CMD [ "python", "./survBot.py", "-html", "www", "-parfile", "conf/parameters.yaml" ]
|
||||||
|
|||||||
55
README.md
55
README.md
@@ -3,7 +3,7 @@
|
|||||||
version: 0.2
|
version: 0.2
|
||||||
|
|
||||||
survBot is a small program used to track station quality channels of DSEBRA stations via PowBox output over SOH channels
|
survBot is a small program used to track station quality channels of DSEBRA stations via PowBox output over SOH channels
|
||||||
by analyzing contents of a Seiscomp data archive.
|
by analysing contents of a Seiscomp3 datapath.
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
|
|
||||||
@@ -40,53 +40,16 @@ The GUI can be loaded via
|
|||||||
python survBotGui.py
|
python survBotGui.py
|
||||||
```
|
```
|
||||||
|
|
||||||
### Docker
|
|
||||||
|
|
||||||
To run the program in a Docker container, first build the image:
|
|
||||||
|
|
||||||
```shell script
|
|
||||||
docker build -t survbot .
|
|
||||||
```
|
|
||||||
|
|
||||||
Then run the container:
|
|
||||||
|
|
||||||
```shell script
|
|
||||||
docker run -v /path/to/conf-dir:/usr/src/app/conf -v /path/to/output:/usr/src/app/www survbot
|
|
||||||
```
|
|
||||||
|
|
||||||
The directory `/path/to/conf-dir` should contain the `parameters.yaml` file, and the directory `/path/to/output` will contain the output HTML files.
|
|
||||||
|
|
||||||
### Configuration of the e-mail server settings
|
|
||||||
|
|
||||||
The e-mail server settings can be configured in the `parameters.yaml` file. The following settings are available:
|
|
||||||
|
|
||||||
* `mailserver`: the address of the mail server
|
|
||||||
* `auth_type`: the authentication type for the mail server (`None`, `SSL`, `TLS`)
|
|
||||||
* `port`: the port of the mail server
|
|
||||||
* `user`: the username for the mail server (if required)
|
|
||||||
* `password`: the password for the mail server (if required)
|
|
||||||
|
|
||||||
The `user` and `password` fields are optional, and can be left empty if the mail server does not require authentication. The `auth_type` field can be set to `None` if no authentication is required, `SSL` if the mail server requires SSL authentication, or `TLS` if the mail server requires TLS authentication. If the `user` or `password` fields are set to `Docker` ore `ENV` the program will try to read the values from the docker secrets `mail_user` and `mail_password` or environment variables `MAIL_USER` and `MAIL_PASSWORD` respectively. Docker secrets are only available in Docker Swarm mode, i.e. if the program is run as a service.
|
|
||||||
|
|
||||||
## Version Changes
|
## Version Changes
|
||||||
|
- surveillance of mass, clock and gaps
|
||||||
### 0.2
|
- individual mailing lists for different stations
|
||||||
|
- html mail with recent status information
|
||||||
* surveillance of mass, clock and gaps
|
- updated web page design
|
||||||
* individual mailing lists for different stations
|
- restructured parameter file
|
||||||
* html mail with recent status information
|
- recognize if PBox is disconnected
|
||||||
* updated web page design
|
|
||||||
* restructured parameter file
|
|
||||||
* recognize if PBox is disconnected
|
|
||||||
|
|
||||||
### 0.2-docker
|
|
||||||
|
|
||||||
* added Dockerfile for easy deployment
|
|
||||||
* added more settings for connection to a mail server
|
|
||||||
|
|
||||||
## Staff
|
## Staff
|
||||||
|
|
||||||
Original author: M.Paffrath (<marcel.paffrath@rub.de>)
|
Original author: M.Paffrath (marcel.paffrath@rub.de)
|
||||||
Contributions: Kasper D. Fischer (<kasper.fischer@rub.de>)
|
|
||||||
|
|
||||||
Jan 2025
|
June 2023
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ logging_level: WARN # set logging level (info, warning, debug)
|
|||||||
track_changes: True # tracks all changes since GUI startup by text highlighting (GUI only)
|
track_changes: True # tracks all changes since GUI startup by text highlighting (GUI only)
|
||||||
warn_count: False # show number of warnings and errors in table
|
warn_count: False # show number of warnings and errors in table
|
||||||
min_sample: 5 # minimum samples for raising Warn/FAIL
|
min_sample: 5 # minimum samples for raising Warn/FAIL
|
||||||
dt_thresh: [300, 1800] # threshold (s) for timing delay colourisation (yellow/red)
|
dt_thresh: [300, 1800] # threshold (s) for timing delay colorization (yellow/red)
|
||||||
html_figures: True # Create html figure directory and links
|
html_figures: True # Create html figure directory and links
|
||||||
reread_parameters: True # reread parameters file (change parameters on runtime, not for itself/GUI refresh/datapath)
|
reread_parameters: True # reread parameters file (change parameters on runtime, not for itself/GUI refresh/datapath)
|
||||||
|
|
||||||
@@ -117,11 +117,6 @@ data_channels: ["HHZ", "HHN", "HHE"]
|
|||||||
|
|
||||||
|
|
||||||
# ---------------------------------------- OPTIONAL PARAMETERS ---------------------------------------------------------
|
# ---------------------------------------- OPTIONAL PARAMETERS ---------------------------------------------------------
|
||||||
# deactivate powbox surveillance for stations (e.g. for solar powered), key: station, value: status message (abbr.)
|
|
||||||
no_pbox_stations: {'GR33': 'SOL', 'GR27B': 'DCN', 'RP01': 'noPBox', 'RP02': 'noPBox', 'RP03': 'noPBox', 'RP04': 'noPBox', 'RP05B': 'noPBox', 'RP06': 'noPBox', 'RP07': 'noPBox', 'RP08': 'noPBox'}
|
|
||||||
|
|
||||||
# deactivate mass position surveillance for stations without connected mass channels (e.g. STS-2 instruments), key: station, value: status message (abbr.)
|
|
||||||
no_mass_stations: {'BIA': 'DCN', 'KNEZ': 'DCN', 'KRUS': 'DCN', 'OHR': 'DCN', 'OTOV': 'DCN', 'SKO': 'DCN', 'STIP': 'DCN', 'VAY': 'DCN', 'ZUPA': 'DCN'}
|
|
||||||
|
|
||||||
# add links to html table with specified key as column and value as relative link, interpretable string parameters:
|
# add links to html table with specified key as column and value as relative link, interpretable string parameters:
|
||||||
# nw (e.g. 1Y), st (e.g. GR01A), nwst_id (e.g. 1Y.GR01A)
|
# nw (e.g. 1Y), st (e.g. GR01A), nwst_id (e.g. 1Y.GR01A)
|
||||||
@@ -135,7 +130,7 @@ add_links:
|
|||||||
add_global_links:
|
add_global_links:
|
||||||
# for example: - {"text": "our homepage", "URL": "https://www.rub.de"}
|
# for example: - {"text": "our homepage", "URL": "https://www.rub.de"}
|
||||||
- {"text": "show recent events on map",
|
- {"text": "show recent events on map",
|
||||||
"URL": "https://fdsnws.geophysik.ruhr-uni-bochum.de/map/?lat=39.5&lon=21&zoom=7&baselayer=osm"}
|
"URL": "https://fdsnws.geophysik.ruhr-uni-bochum.de/map/?lat=39.5&lon=21&zoom=7&baselayer=mapnik"}
|
||||||
|
|
||||||
# html logo at page bottom (path relative to html directory)
|
# html logo at page bottom (path relative to html directory)
|
||||||
html_logo: "logo.png"
|
html_logo: "logo.png"
|
||||||
@@ -145,14 +140,14 @@ EMAIL:
|
|||||||
# specify mail server and credentials
|
# specify mail server and credentials
|
||||||
# port, auth_type, user and password are only required if mailserver is not set to "localhost"
|
# port, auth_type, user and password are only required if mailserver is not set to "localhost"
|
||||||
# user and password can be set to "ENV" or "DOCKER" to read from environment variables or docker secrets
|
# user and password can be set to "ENV" or "DOCKER" to read from environment variables or docker secrets
|
||||||
mailserver: "smtp.example.org" # mail server
|
mailserver: "smtp.rub.de" # mail server
|
||||||
auth_type: "SSL" # mail authentication type, can be "SSL", "TLS" or "None"
|
auth_type: "SSL" # mail authentication type, can be "SSL", "TLS" or "None"
|
||||||
port: 465 # mail port, default 465 for SSL, 587 for TLS
|
port: 465 # mail port, default 465 for SSL, 587 for TLS
|
||||||
user: "DOCKER" # mail user, read from environment variable if set to "ENV" or from docker secret if set to "DOCKER"
|
user: "DOCKER" # mail user, read from environment variable if set to "ENV" or from docker secret if set to "DOCKER"
|
||||||
password: "DOCKER" # mail password, read from environment variable if set to "ENV" or from docker secret if set to "DOCKER"
|
password: "DOCKER" # mail password, read from environment variable if set to "ENV" or from docker secret if set to "DOCKER"
|
||||||
# specify mail recipients, sender and blacklists
|
# specify mail recipients, sender and blacklists
|
||||||
addresses: ["test@example.org"] # list of mail addresses for info mails
|
addresses: ["marcel.paffrath@rub.de", "kasper.fischer@rub.de"] # list of mail addresses for info mails
|
||||||
sender: "<test@example.org>" # mail sender
|
sender: "RUB SeisObs <seisobs@ruhr-uni-bochum.de>" # mail sender
|
||||||
stations_blacklist: [] # do not send emails for specific stations
|
stations_blacklist: [] # do not send emails for specific stations
|
||||||
networks_blacklist: [] # do not send emails for specific network
|
networks_blacklist: [] # do not send emails for specific network
|
||||||
# specify recipients for single stations in a yaml: key = email-address, val = station list (e.g. [1Y.GR01, 1Y.GR02])
|
# specify recipients for single stations in a yaml: key = email-address, val = station list (e.g. [1Y.GR01, 1Y.GR02])
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ th {
|
|||||||
background-color: #17365c;
|
background-color: #17365c;
|
||||||
color: #fff;
|
color: #fff;
|
||||||
border-radius: 3px;
|
border-radius: 3px;
|
||||||
padding: 10px 2px;
|
padding: 10px, 2px;
|
||||||
position: sticky;
|
position: sticky;
|
||||||
top: 0;
|
top: 0;
|
||||||
}
|
}
|
||||||
|
|||||||
34
survBot.py
34
survBot.py
@@ -1,8 +1,8 @@
|
|||||||
#!/usr/bin/env python
|
#!/usr/bin/env python
|
||||||
# -*- coding: utf-8 -*-
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
__version__ = '0.3'
|
__version__ = '0.2-docker'
|
||||||
__author__ = ['Marcel Paffrath <marcel.paffrath@rub.de>', 'Kasper D. Fischer <kasper.fischer@rub.de>']
|
__author__ = 'Marcel Paffrath <marcel.paffrath@rub.de>'
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import io
|
import io
|
||||||
@@ -234,7 +234,7 @@ class SurveillanceBot(object):
|
|||||||
self.gaps = self.dataStream.get_gaps(min_gap=self.parameters['THRESHOLDS'].get('min_gap'))
|
self.gaps = self.dataStream.get_gaps(min_gap=self.parameters['THRESHOLDS'].get('min_gap'))
|
||||||
self.dataStream.merge()
|
self.dataStream.merge()
|
||||||
|
|
||||||
# organize data in dictionary with key for each station
|
# organise data in dictionary with key for each station
|
||||||
for trace in self.dataStream:
|
for trace in self.dataStream:
|
||||||
nwst_id = get_nwst_id(trace)
|
nwst_id = get_nwst_id(trace)
|
||||||
if not nwst_id in self.data.keys():
|
if not nwst_id in self.data.keys():
|
||||||
@@ -351,7 +351,7 @@ class SurveillanceBot(object):
|
|||||||
first_exec = False
|
first_exec = False
|
||||||
|
|
||||||
def console_print(self, itemlist, str_len=21, sep='|', seplen=3):
|
def console_print(self, itemlist, str_len=21, sep='|', seplen=3):
|
||||||
assert len(sep) <= seplen, f'Make sure separator has less than {seplen} characters'
|
assert len(sep) <= seplen, f'Make sure seperator has less than {seplen} characters'
|
||||||
sl = sep.ljust(seplen)
|
sl = sep.ljust(seplen)
|
||||||
sr = sep.rjust(seplen)
|
sr = sep.rjust(seplen)
|
||||||
string = sl
|
string = sl
|
||||||
@@ -595,7 +595,6 @@ class StationQC(object):
|
|||||||
self.status_track = status_track
|
self.status_track = status_track
|
||||||
|
|
||||||
self.powbox_active = self.is_pbox_activated_check()
|
self.powbox_active = self.is_pbox_activated_check()
|
||||||
self.mass_active = self.is_mass_activated_check()
|
|
||||||
|
|
||||||
self.start()
|
self.start()
|
||||||
|
|
||||||
@@ -1081,11 +1080,6 @@ class StationQC(object):
|
|||||||
""" Analyse datalogger mass channels. """
|
""" Analyse datalogger mass channels. """
|
||||||
key = 'mass'
|
key = 'mass'
|
||||||
|
|
||||||
# skip processing if mass is not active
|
|
||||||
if not self.mass_active:
|
|
||||||
self.set_mass_inactive_error(key)
|
|
||||||
return
|
|
||||||
|
|
||||||
# build stream with all channels
|
# build stream with all channels
|
||||||
st = Stream()
|
st = Stream()
|
||||||
for channel in channels:
|
for channel in channels:
|
||||||
@@ -1305,7 +1299,7 @@ class StationQC(object):
|
|||||||
|
|
||||||
# Warn in case of voltage under OK-level (1V)
|
# Warn in case of voltage under OK-level (1V)
|
||||||
if len(under) > 0:
|
if len(under) > 0:
|
||||||
# try calculate number of occurrences from gaps between indices
|
# try calculate number of occurences from gaps between indices
|
||||||
n_occurrences = len(np.where(np.diff(under) > 1)[0]) + 1
|
n_occurrences = len(np.where(np.diff(under) > 1)[0]) + 1
|
||||||
voltage_dict[-1] = under
|
voltage_dict[-1] = under
|
||||||
self.status_other(detailed_message=f'Trace {trace.get_id()}: '
|
self.status_other(detailed_message=f'Trace {trace.get_id()}: '
|
||||||
@@ -1347,12 +1341,6 @@ class StationQC(object):
|
|||||||
msg = self.parameters.get('no_pbox_stations')[self.station]
|
msg = self.parameters.get('no_pbox_stations')[self.station]
|
||||||
self.error(key, detailed_message=f'PowBox not connected', disc=msg)
|
self.error(key, detailed_message=f'PowBox not connected', disc=msg)
|
||||||
|
|
||||||
def is_mass_activated_check(self):
|
|
||||||
return self.station not in self.parameters.get('no_mass_stations', [])
|
|
||||||
|
|
||||||
def set_mass_inactive_error(self, key):
|
|
||||||
msg = self.parameters.get('no_mass_stations')[self.station]
|
|
||||||
self.error(key, detailed_message=f'Mass channels not connected', disc=msg)
|
|
||||||
|
|
||||||
class Status(object):
|
class Status(object):
|
||||||
""" Basic Status class. All status classes are derived from this class."""
|
""" Basic Status class. All status classes are derived from this class."""
|
||||||
@@ -1407,15 +1395,15 @@ class StatusOK(Status):
|
|||||||
|
|
||||||
|
|
||||||
class StatusWarn(Status):
|
class StatusWarn(Status):
|
||||||
def __init__(self, message='WARN', count=1, last_occurrence=None, detailed_messages=None, show_count=False):
|
def __init__(self, message='WARN', count=1, last_occurence=None, detailed_messages=None, show_count=False):
|
||||||
super(StatusWarn, self).__init__(message=message, count=count, last_occurrence=last_occurrence,
|
super(StatusWarn, self).__init__(message=message, count=count, last_occurrence=last_occurence,
|
||||||
detailed_messages=detailed_messages, show_count=show_count)
|
detailed_messages=detailed_messages, show_count=show_count)
|
||||||
self.set_warn()
|
self.set_warn()
|
||||||
|
|
||||||
|
|
||||||
class StatusError(Status):
|
class StatusError(Status):
|
||||||
def __init__(self, message='FAIL', count=1, last_occurrence=None, detailed_messages=None, show_count=False):
|
def __init__(self, message='FAIL', count=1, last_occurence=None, detailed_messages=None, show_count=False):
|
||||||
super(StatusError, self).__init__(message=message, count=count, last_occurrence=last_occurrence,
|
super(StatusError, self).__init__(message=message, count=count, last_occurrence=last_occurence,
|
||||||
detailed_messages=detailed_messages, show_count=show_count)
|
detailed_messages=detailed_messages, show_count=show_count)
|
||||||
self.set_error()
|
self.set_error()
|
||||||
self.default_message = message
|
self.default_message = message
|
||||||
@@ -1432,8 +1420,8 @@ class StatusError(Status):
|
|||||||
|
|
||||||
|
|
||||||
class StatusOther(Status):
|
class StatusOther(Status):
|
||||||
def __init__(self, messages=None, count=1, last_occurrence=None, detailed_messages=None):
|
def __init__(self, messages=None, count=1, last_occurence=None, detailed_messages=None):
|
||||||
super(StatusOther, self).__init__(count=count, last_occurrence=last_occurrence,
|
super(StatusOther, self).__init__(count=count, last_occurrence=last_occurence,
|
||||||
detailed_messages=detailed_messages)
|
detailed_messages=detailed_messages)
|
||||||
if messages is None:
|
if messages is None:
|
||||||
messages = []
|
messages = []
|
||||||
|
|||||||
441
survBotGUI.py
441
survBotGUI.py
@@ -1,441 +0,0 @@
|
|||||||
#!/usr/bin/env python
|
|
||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""
|
|
||||||
GUI overlay for the main survBot to show quality control of different stations specified in parameters.yaml file.
|
|
||||||
"""
|
|
||||||
|
|
||||||
__version__ = '0.1'
|
|
||||||
__author__ = 'Marcel Paffrath'
|
|
||||||
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
import traceback
|
|
||||||
|
|
||||||
import logging
|
|
||||||
|
|
||||||
try:
|
|
||||||
from PySide2 import QtGui, QtCore, QtWidgets
|
|
||||||
except ImportError:
|
|
||||||
try:
|
|
||||||
from PySide6 import QtGui, QtCore, QtWidgets
|
|
||||||
except ImportError:
|
|
||||||
try:
|
|
||||||
from PyQt5 import QtGui, QtCore, QtWidgets
|
|
||||||
except ImportError:
|
|
||||||
raise ImportError('Could import neither of PySide2, PySide6 or PyQt5')
|
|
||||||
|
|
||||||
from matplotlib.figure import Figure
|
|
||||||
|
|
||||||
if QtGui.__package__ in ['PySide2', 'PyQt5', 'PySide6']:
|
|
||||||
from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg as FigureCanvas
|
|
||||||
from matplotlib.backends.backend_qt5agg import NavigationToolbar2QT
|
|
||||||
else:
|
|
||||||
raise Exception('Not implemented')
|
|
||||||
|
|
||||||
from obspy import UTCDateTime
|
|
||||||
|
|
||||||
from survBot import SurveillanceBot
|
|
||||||
from write_utils import *
|
|
||||||
from utils import get_bg_color, modify_stream_for_plot, set_axis_yticks, set_axis_color, plot_axis_thresholds
|
|
||||||
|
|
||||||
try:
|
|
||||||
from rest_api.utils import get_station_iccid
|
|
||||||
from rest_api.rest_api_utils import get_last_messages, send_message, get_default_params
|
|
||||||
sms_funcs = True
|
|
||||||
except ImportError:
|
|
||||||
logging.warning('Could not load rest_api utils, SMS functionality disabled.')
|
|
||||||
sms_funcs = False
|
|
||||||
|
|
||||||
deg_str = '\N{DEGREE SIGN}C'
|
|
||||||
|
|
||||||
|
|
||||||
class Thread(QtCore.QThread):
|
|
||||||
"""
|
|
||||||
A simple thread that runs outside of the main event loop. Executes the function "runnable" and prevents
|
|
||||||
freezing of the GUI. Run method is executed outside main event loop when called with thread.start().
|
|
||||||
"""
|
|
||||||
update = QtCore.Signal()
|
|
||||||
|
|
||||||
def __init__(self, parent, runnable):
|
|
||||||
super(Thread, self).__init__(parent=parent)
|
|
||||||
self.setParent(parent)
|
|
||||||
self.runnable = runnable
|
|
||||||
self.is_active = True
|
|
||||||
|
|
||||||
def run(self):
|
|
||||||
""" Try to run self.runnable and emit update signal, or print Exception if failed. """
|
|
||||||
try:
|
|
||||||
t0 = UTCDateTime()
|
|
||||||
self.runnable()
|
|
||||||
self.update.emit()
|
|
||||||
except Exception as e:
|
|
||||||
self.is_active = False
|
|
||||||
logging.error(e)
|
|
||||||
logging.debug(traceback.format_exc())
|
|
||||||
finally:
|
|
||||||
logging.info(f'Time for Thread execution: {UTCDateTime() - t0}')
|
|
||||||
|
|
||||||
|
|
||||||
class MainWindow(QtWidgets.QMainWindow):
|
|
||||||
def __init__(self, parameters='parameters.yaml'):
|
|
||||||
"""
|
|
||||||
Main window of survBot GUI.
|
|
||||||
:param parameters: Parameters dictionary file (yaml format)
|
|
||||||
"""
|
|
||||||
super(MainWindow, self).__init__()
|
|
||||||
|
|
||||||
# init some attributes
|
|
||||||
self.last_mouse_loc = None
|
|
||||||
self.status_message = ''
|
|
||||||
self.starttime = UTCDateTime()
|
|
||||||
|
|
||||||
# setup main layout of the GUI
|
|
||||||
self.main_layout = QtWidgets.QVBoxLayout()
|
|
||||||
self.centralWidget = QtWidgets.QWidget()
|
|
||||||
self.centralWidget.setLayout(self.main_layout)
|
|
||||||
self.setCentralWidget(self.centralWidget)
|
|
||||||
|
|
||||||
# init new survBot instance, set parameters and refresh
|
|
||||||
self.survBot = SurveillanceBot(parameter_path=parameters)
|
|
||||||
self.parameters = self.survBot.parameters
|
|
||||||
self.refresh_period = self.parameters.get('interval')
|
|
||||||
self.dt_thresh = [int(val) for val in self.parameters.get('dt_thresh')]
|
|
||||||
|
|
||||||
# create thread that is used to update
|
|
||||||
self.thread = Thread(parent=self, runnable=self.survBot.execute_qc)
|
|
||||||
self.thread.update.connect(self.fill_table)
|
|
||||||
|
|
||||||
self.init_table()
|
|
||||||
self.init_buttons()
|
|
||||||
|
|
||||||
# These filters were used to track current mouse position if an event (i.e. mouseclick) is triggered
|
|
||||||
self.table.installEventFilter(self)
|
|
||||||
self.installEventFilter(self)
|
|
||||||
|
|
||||||
# initiate clear_on_refresh flag and set status bar text
|
|
||||||
self.clear_on_refresh = False
|
|
||||||
self.fill_status_bar()
|
|
||||||
|
|
||||||
# start thread that executes qc at first initiation, then activate timer for further thread activation
|
|
||||||
self.thread.start()
|
|
||||||
self.run_refresh_timer()
|
|
||||||
|
|
||||||
def init_table(self):
|
|
||||||
self.table = QtWidgets.QTableWidget()
|
|
||||||
keys = self.survBot.keys
|
|
||||||
station_list = self.survBot.station_list
|
|
||||||
|
|
||||||
self.table.setColumnCount(len(keys))
|
|
||||||
self.table.setRowCount(len(station_list))
|
|
||||||
self.table.setHorizontalHeaderLabels(keys)
|
|
||||||
|
|
||||||
for index, nwst_id in enumerate(station_list):
|
|
||||||
item = QtWidgets.QTableWidgetItem()
|
|
||||||
item.setText(str(nwst_id.rstrip('.')))
|
|
||||||
item.setData(QtCore.Qt.UserRole, nwst_id)
|
|
||||||
self.table.setVerticalHeaderItem(index, item)
|
|
||||||
|
|
||||||
self.main_layout.addWidget(self.table)
|
|
||||||
|
|
||||||
self.table.itemDoubleClicked.connect(self.plot_stream)
|
|
||||||
self.table.setEditTriggers(QtWidgets.QTableWidget.NoEditTriggers)
|
|
||||||
|
|
||||||
if sms_funcs:
|
|
||||||
self.table.verticalHeader().sectionClicked.connect(self.sms_context_menu)
|
|
||||||
|
|
||||||
self.set_stretch()
|
|
||||||
|
|
||||||
def init_buttons(self):
|
|
||||||
if self.parameters.get('track_changes'):
|
|
||||||
button_text = 'Clear track and refresh'
|
|
||||||
else:
|
|
||||||
button_text = 'Refresh'
|
|
||||||
self.clear_button = QtWidgets.QPushButton(button_text)
|
|
||||||
self.clear_button.setToolTip('Reset track changes and refresh table')
|
|
||||||
self.clear_button.clicked.connect(self.refresh)
|
|
||||||
self.main_layout.addWidget(self.clear_button)
|
|
||||||
|
|
||||||
def refresh(self):
|
|
||||||
self.set_clear_on_refresh()
|
|
||||||
self.run_refresh_timer()
|
|
||||||
self.thread.start()
|
|
||||||
|
|
||||||
def run_refresh_timer(self):
|
|
||||||
self.timer = QtCore.QTimer()
|
|
||||||
self.timer.timeout.connect(self.thread.start)
|
|
||||||
self.timer.start(int(self.refresh_period * 1e3))
|
|
||||||
|
|
||||||
def eventFilter(self, object, event):
|
|
||||||
"""
|
|
||||||
An event filter that stores last mouse position if an event is raised by the table. All events are passed
|
|
||||||
to the parent class of the Mainwindow afterwards.
|
|
||||||
"""
|
|
||||||
if hasattr(event, 'pos'):
|
|
||||||
self.last_mouse_loc = event.pos()
|
|
||||||
return super(QtWidgets.QMainWindow, self).eventFilter(object, event)
|
|
||||||
|
|
||||||
def sms_context_menu(self, row_ind):
|
|
||||||
""" Open a context menu when left-clicking vertical header item """
|
|
||||||
header_item = self.table.verticalHeaderItem(row_ind)
|
|
||||||
if not header_item:
|
|
||||||
return
|
|
||||||
nwst_id = header_item.data(QtCore.Qt.UserRole)
|
|
||||||
|
|
||||||
context_menu = QtWidgets.QMenu()
|
|
||||||
read_sms = context_menu.addAction('Get last SMS')
|
|
||||||
send_sms = context_menu.addAction('Send SMS')
|
|
||||||
action = context_menu.exec_(self.mapToGlobal(self.last_mouse_loc))
|
|
||||||
if action == read_sms:
|
|
||||||
self.read_sms(nwst_id)
|
|
||||||
elif action == send_sms:
|
|
||||||
self.send_sms(nwst_id)
|
|
||||||
|
|
||||||
def read_sms(self, nwst_id):
|
|
||||||
""" Read recent SMS over rest_api using whereversim portal """
|
|
||||||
station = nwst_id.split('.')[1]
|
|
||||||
iccid = get_station_iccid(station)
|
|
||||||
if not iccid:
|
|
||||||
logging.info(f'Could not find iccid for station: {nwst_id}')
|
|
||||||
return
|
|
||||||
sms_widget = ReadSMSWidget(parent=self, iccid=iccid)
|
|
||||||
sms_widget.setWindowTitle(f'Recent SMS of station: {nwst_id}')
|
|
||||||
if sms_widget.data:
|
|
||||||
sms_widget.show()
|
|
||||||
else:
|
|
||||||
self.notification('No recent messages found.')
|
|
||||||
|
|
||||||
def send_sms(self, nwst_id):
|
|
||||||
""" Send SMS over rest_api using whereversim portal """
|
|
||||||
station = nwst_id.split('.')[1]
|
|
||||||
iccid = get_station_iccid(station)
|
|
||||||
|
|
||||||
sms_widget = SendSMSWidget(parent=self, iccid=iccid)
|
|
||||||
sms_widget.setWindowTitle(f'Send SMS to station: {nwst_id}')
|
|
||||||
sms_widget.show()
|
|
||||||
|
|
||||||
def set_clear_on_refresh(self):
|
|
||||||
self.clear_on_refresh = True
|
|
||||||
|
|
||||||
def fill_status_bar(self):
|
|
||||||
""" Set status bar text """
|
|
||||||
self.status_message = self.survBot.status_message
|
|
||||||
status_bar = self.statusBar()
|
|
||||||
status_bar.showMessage(self.status_message)
|
|
||||||
|
|
||||||
def fill_table(self):
|
|
||||||
""" Fills the table with most recent information. Executed after execute_qc thread is done or on refresh. """
|
|
||||||
|
|
||||||
# fill status bar first with new time
|
|
||||||
self.fill_status_bar()
|
|
||||||
|
|
||||||
for col_ind, check_key in enumerate(self.survBot.keys):
|
|
||||||
for row_ind, nwst_id in enumerate(self.survBot.station_list):
|
|
||||||
status_dict = self.survBot.analysis_results.get(nwst_id)
|
|
||||||
status = status_dict.get(check_key)
|
|
||||||
message, detailed_message = status.get_status_str()
|
|
||||||
|
|
||||||
dt_thresh = [timedelta(seconds=sec) for sec in self.dt_thresh]
|
|
||||||
bg_color = get_bg_color(check_key, status, dt_thresh)
|
|
||||||
if check_key == 'temp':
|
|
||||||
if not type(message) in [str]:
|
|
||||||
message = str(message) + deg_str
|
|
||||||
|
|
||||||
# Continue if nothing changed
|
|
||||||
text = str(message)
|
|
||||||
cur_item = self.table.item(row_ind, col_ind)
|
|
||||||
if cur_item and text == cur_item.text():
|
|
||||||
if not self.parameters.get('track_changes') or self.clear_on_refresh:
|
|
||||||
# set item to default color/font and continue
|
|
||||||
self.set_font(cur_item)
|
|
||||||
self.set_fg_color(cur_item)
|
|
||||||
continue
|
|
||||||
|
|
||||||
# Create new data item
|
|
||||||
item = QtWidgets.QTableWidgetItem()
|
|
||||||
item.setText(str(message))
|
|
||||||
item.setTextAlignment(QtCore.Qt.AlignCenter)
|
|
||||||
item.setData(QtCore.Qt.UserRole, (nwst_id, check_key))
|
|
||||||
|
|
||||||
# if text changed (known from above) set highlight color/font else (new init) set to default
|
|
||||||
cur_item = self.table.item(row_ind, col_ind)
|
|
||||||
if cur_item and check_key != 'last active':
|
|
||||||
self.set_fg_color(item, (0, 0, 0, 255))
|
|
||||||
self.set_font_bold(item)
|
|
||||||
else:
|
|
||||||
self.set_fg_color(item)
|
|
||||||
self.set_font(item)
|
|
||||||
|
|
||||||
# set item tooltip
|
|
||||||
if detailed_message:
|
|
||||||
item.setToolTip(str(detailed_message))
|
|
||||||
|
|
||||||
# set bg color corresponding to current text (OK/WARN/ERROR etc.)
|
|
||||||
self.set_bg_color(item, bg_color)
|
|
||||||
|
|
||||||
# insert new item
|
|
||||||
self.table.setItem(row_ind, col_ind, item)
|
|
||||||
|
|
||||||
# table filling/refreshing done, set clear_on_refresh to False
|
|
||||||
self.clear_on_refresh = False
|
|
||||||
|
|
||||||
def set_font_bold(self, item):
|
|
||||||
""" Set item font bold """
|
|
||||||
f = item.font()
|
|
||||||
f.setWeight(QtGui.QFont.Bold)
|
|
||||||
item.setFont(f)
|
|
||||||
|
|
||||||
def set_font(self, item):
|
|
||||||
""" Set item font normal """
|
|
||||||
f = item.font()
|
|
||||||
f.setWeight(QtGui.QFont.Normal)
|
|
||||||
item.setFont(f)
|
|
||||||
|
|
||||||
def set_bg_color(self, item, color):
|
|
||||||
""" Set background color of item, color is RGBA tuple """
|
|
||||||
color = QtGui.QColor(*color)
|
|
||||||
item.setBackground(color)
|
|
||||||
|
|
||||||
def set_fg_color(self, item, color=(20, 20, 20, 255)):
|
|
||||||
""" Set foreground (font) color of item, color is RGBA tuple """
|
|
||||||
color = QtGui.QColor(*color)
|
|
||||||
item.setForeground(color)
|
|
||||||
|
|
||||||
def set_stretch(self):
|
|
||||||
hheader = self.table.horizontalHeader()
|
|
||||||
for index in range(hheader.count()):
|
|
||||||
hheader.setSectionResizeMode(index, QtWidgets.QHeaderView.Stretch)
|
|
||||||
vheader = self.table.verticalHeader()
|
|
||||||
for index in range(vheader.count()):
|
|
||||||
vheader.setSectionResizeMode(index, QtWidgets.QHeaderView.Stretch)
|
|
||||||
|
|
||||||
def plot_stream(self, item):
|
|
||||||
nwst_id, check = item.data(QtCore.Qt.UserRole)
|
|
||||||
st = self.survBot.data.get(nwst_id)
|
|
||||||
if st:
|
|
||||||
self.plot_widget = PlotWidget(self)
|
|
||||||
self.plot_widget.setWindowTitle(nwst_id)
|
|
||||||
st = modify_stream_for_plot(st, parameters=self.parameters)
|
|
||||||
st.plot(equal_scale=False, method='full', block=False, fig=self.plot_widget.canvas.fig)
|
|
||||||
# set_axis_ylabels(fig=self.plot_widget.canvas.fig, parameters=self.parameters)
|
|
||||||
set_axis_yticks(fig=self.plot_widget.canvas.fig, parameters=self.parameters)
|
|
||||||
set_axis_color(fig=self.plot_widget.canvas.fig)
|
|
||||||
plot_axis_thresholds(fig=self.plot_widget.canvas.fig, parameters=self.parameters)
|
|
||||||
self.plot_widget.show()
|
|
||||||
|
|
||||||
def notification(self, text):
|
|
||||||
mbox = QtWidgets.QMessageBox()
|
|
||||||
mbox.setWindowTitle('Notification')
|
|
||||||
#mbox.setDetailedText()
|
|
||||||
mbox.setText(text)
|
|
||||||
mbox.exec_()
|
|
||||||
|
|
||||||
def closeEvent(self, event):
|
|
||||||
self.thread.exit()
|
|
||||||
event.accept()
|
|
||||||
|
|
||||||
|
|
||||||
class PlotCanvas(FigureCanvas):
|
|
||||||
def __init__(self, parent=None, width=10, height=8, dpi=100):
|
|
||||||
self.fig = Figure(figsize=(width, height), dpi=dpi)
|
|
||||||
FigureCanvas.__init__(self, self.fig)
|
|
||||||
self.setParent(parent)
|
|
||||||
FigureCanvas.setSizePolicy(self, QtWidgets.QSizePolicy.Expanding, QtWidgets.QSizePolicy.Expanding)
|
|
||||||
FigureCanvas.updateGeometry(self)
|
|
||||||
|
|
||||||
|
|
||||||
class PlotWidget(QtWidgets.QDialog):
|
|
||||||
def __init__(self, *args, **kwargs):
|
|
||||||
QtWidgets.QDialog.__init__(self, *args, **kwargs)
|
|
||||||
self.setLayout(QtWidgets.QVBoxLayout())
|
|
||||||
self.canvas = PlotCanvas(self, width=10, height=8)
|
|
||||||
self.toolbar = NavigationToolbar2QT(self.canvas, self)
|
|
||||||
self.layout().addWidget(self.toolbar)
|
|
||||||
self.layout().addWidget(self.canvas)
|
|
||||||
|
|
||||||
|
|
||||||
class ReadSMSWidget(QtWidgets.QDialog):
|
|
||||||
def __init__(self, iccid, *args, **kwargs):
|
|
||||||
QtWidgets.QDialog.__init__(self, *args, **kwargs)
|
|
||||||
self.setLayout(QtWidgets.QVBoxLayout())
|
|
||||||
self.table = QtWidgets.QTableWidget()
|
|
||||||
self.layout().addWidget(self.table)
|
|
||||||
self.resize(1280, 400)
|
|
||||||
|
|
||||||
self.iccid = iccid
|
|
||||||
self.data = self.print_sms_table()
|
|
||||||
self.set_stretch()
|
|
||||||
|
|
||||||
def print_sms_table(self, n=5, ntextbreak=40):
|
|
||||||
messages = []
|
|
||||||
params = get_default_params(self.iccid)
|
|
||||||
for message in get_last_messages(params, n, only_delivered=False):
|
|
||||||
messages.append(message)
|
|
||||||
if not messages:
|
|
||||||
return
|
|
||||||
# pull dates to front
|
|
||||||
keys = ['dateSent', 'dateModified', 'dateReceived']
|
|
||||||
for item in messages[0].keys():
|
|
||||||
if not item in keys:
|
|
||||||
keys.append(item)
|
|
||||||
self.table.setRowCount(n)
|
|
||||||
self.table.setColumnCount(len(keys))
|
|
||||||
self.table.setHorizontalHeaderLabels(keys)
|
|
||||||
for row_index, message in enumerate(messages):
|
|
||||||
for col_index, key in enumerate(keys):
|
|
||||||
text = message.get(key)
|
|
||||||
if type(text) == str and len(text) > ntextbreak:
|
|
||||||
textlist = list(text)
|
|
||||||
for index in range(ntextbreak, len(text), ntextbreak):
|
|
||||||
textlist.insert(index, '\n')
|
|
||||||
text = ''.join(textlist)
|
|
||||||
item = QtWidgets.QTableWidgetItem()
|
|
||||||
item.setText(str(text))
|
|
||||||
self.table.setItem(row_index, col_index, item)
|
|
||||||
return True
|
|
||||||
|
|
||||||
def set_stretch(self):
|
|
||||||
hheader = self.table.horizontalHeader()
|
|
||||||
nheader = hheader.count()
|
|
||||||
for index in range(nheader):
|
|
||||||
if index < nheader - 1:
|
|
||||||
hheader.setSectionResizeMode(index, QtWidgets.QHeaderView.ResizeToContents)
|
|
||||||
else:
|
|
||||||
hheader.setSectionResizeMode(index, QtWidgets.QHeaderView.Stretch)
|
|
||||||
vheader = self.table.verticalHeader()
|
|
||||||
for index in range(vheader.count()):
|
|
||||||
vheader.setSectionResizeMode(index, QtWidgets.QHeaderView.Stretch)
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
class SendSMSWidget(QtWidgets.QDialog):
|
|
||||||
def __init__(self, iccid, *args, **kwargs):
|
|
||||||
QtWidgets.QDialog.__init__(self, *args, **kwargs)
|
|
||||||
self.main_layout = QtWidgets.QVBoxLayout()
|
|
||||||
self.setLayout(self.main_layout)
|
|
||||||
self.resize(400, 100)
|
|
||||||
|
|
||||||
self.line_edit = QtWidgets.QLineEdit()
|
|
||||||
self.main_layout.addWidget(self.line_edit)
|
|
||||||
|
|
||||||
self.iccid = iccid
|
|
||||||
|
|
||||||
self.buttonBox = QtWidgets.QDialogButtonBox(QtWidgets.QDialogButtonBox.Ok |
|
|
||||||
QtWidgets.QDialogButtonBox.Close)
|
|
||||||
self.main_layout.addWidget(self.buttonBox)
|
|
||||||
self.buttonBox.accepted.connect(self.send_sms)
|
|
||||||
self.buttonBox.rejected.connect(self.reject)
|
|
||||||
|
|
||||||
def send_sms(self):
|
|
||||||
text = self.line_edit.text()
|
|
||||||
params = get_default_params(self.iccid)
|
|
||||||
send_message(params, text)
|
|
||||||
self.close()
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
program_path = sys.path[0]
|
|
||||||
parameters = os.path.join(program_path, 'parameters.yaml')
|
|
||||||
app = QtWidgets.QApplication([])
|
|
||||||
window = MainWindow(parameters=parameters)
|
|
||||||
window.showMaximized()
|
|
||||||
sys.exit(app.exec_())
|
|
||||||
7
utils.py
7
utils.py
@@ -6,7 +6,6 @@ import logging
|
|||||||
import matplotlib
|
import matplotlib
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import smtplib
|
import smtplib
|
||||||
import os
|
|
||||||
|
|
||||||
from obspy import Stream
|
from obspy import Stream
|
||||||
|
|
||||||
@@ -175,7 +174,7 @@ def transform_trace(data, transf):
|
|||||||
elif operator_str == '/':
|
elif operator_str == '/':
|
||||||
data = data / val
|
data = data / val
|
||||||
else:
|
else:
|
||||||
raise IOError(f'Unknown arithmetic operator string: {operator_str}')
|
raise IOError(f'Unknown arithmethic operator string: {operator_str}')
|
||||||
|
|
||||||
return data
|
return data
|
||||||
|
|
||||||
@@ -302,7 +301,7 @@ def get_credential(source, param):
|
|||||||
# return source if no credential was found
|
# return source if no credential was found
|
||||||
return source
|
return source
|
||||||
|
|
||||||
def connect_to_mail_server(mail_params):
|
def connect_to_mail_server(self, mail_params):
|
||||||
"""
|
"""
|
||||||
Connect to mail server and return server object.
|
Connect to mail server and return server object.
|
||||||
"""
|
"""
|
||||||
@@ -326,5 +325,5 @@ def connect_to_mail_server(mail_params):
|
|||||||
else:
|
else:
|
||||||
logging.error('Unknown authentication type. Mails can not be sent')
|
logging.error('Unknown authentication type. Mails can not be sent')
|
||||||
return
|
return
|
||||||
s.login(user, password)
|
s.login(mail_params.get('user'), mail_params.get('password'))
|
||||||
return s
|
return s
|
||||||
|
|||||||
@@ -19,13 +19,12 @@ def get_html_header(refresh_rate=10):
|
|||||||
header = ['<!DOCTYPE html>',
|
header = ['<!DOCTYPE html>',
|
||||||
'<html>',
|
'<html>',
|
||||||
'<head>',
|
'<head>',
|
||||||
' <title>SurvBot status</title>',
|
' <link rel="stylesheet" media="only screen and (max-width: 400px)" href="mobile.css" />',
|
||||||
' <link rel="stylesheet" media="only screen and (max-width: 400px)" href="mobile.css">',
|
' <link rel="stylesheet" media="only screen and (min-width: 401px)" href="desktop.css" />',
|
||||||
' <link rel="stylesheet" media="only screen and (min-width: 401px)" href="desktop.css">',
|
'</head>',
|
||||||
f'<meta http-equiv="refresh" content="{refresh_rate}" >',
|
f'<meta http-equiv="refresh" content="{refresh_rate}" >',
|
||||||
'<meta charset="utf-8">',
|
'<meta charset="utf-8">',
|
||||||
'<meta name="viewport" content="width=device-width, initial-scale=1">',
|
'<meta name="viewport" content="width=device-width, initial-scale=1">',
|
||||||
'</head>',
|
|
||||||
'<body>']
|
'<body>']
|
||||||
header = _convert_to_textstring(header)
|
header = _convert_to_textstring(header)
|
||||||
return header
|
return header
|
||||||
|
|||||||
Reference in New Issue
Block a user