121 lines
6.1 KiB
Markdown
121 lines
6.1 KiB
Markdown
# Requirements for journal-uploader
|
|
|
|
[[_TOC_]]
|
|
|
|
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED",
|
|
"MAY", and "OPTIONAL" in this document are to be interpreted as described in
|
|
[RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119).
|
|
|
|
|
|
## Description
|
|
The journal-uploader has two main functionalities.
|
|
- Take a stream of log messages and filter them depending on their severity
|
|
- Upload journal logs for a specified time when activated through cloud call
|
|
|
|
## Requirements
|
|
### _TOPIC-1_ - Journal Watcher
|
|
#### _SUB-1.1_ - File Monitoring
|
|
- **_REQ-1.1.1_ - Continuous Monitoring:** The tool **_MUST_** continuously monitor a designated directory.
|
|
|
|
#### _SUB-1.2_ - File Detection
|
|
- **_REQ-1.2.1_ - Detection of New Files:** The tool **_MUST_** detect the addition of new files in the monitored directory.
|
|
- **_REQ-1.2.2_ - Avoid Re-processing:** The tool **_MUST NOT_** process files that have already been processed.
|
|
|
|
|
|
### _TOPIC-2_ - Traced Logging
|
|
#### _SUB-2.1_ - File Processing
|
|
- **_REQ-2.1.1_ - Reading Log Messages:** When a new file is processed, each log message **_SHOULD_** be put into a buffer.
|
|
- **_REQ-2.1.2_ - Filtering Log Messages:** The tool will search for messages of a defined priority (Trigger Priority).
|
|
Each message of this priority, as well as all messages before and after, which are inside a defined timespan, **_MUST_**
|
|
get written into a file. Every other message **_SHOULD_** gets dropped.
|
|
- **_REQ-2.1.3_ - No Duplicate Log Messages:** The tool **_SHALL_** make sure that no log entry will be written to the file twice.
|
|
|
|
#### _SUB-2.2_ - Traced Log Rotation
|
|
- **_REQ-2.2.1_ - Rotating Files:** When the size of the current traced log file exceeds a certain threshold,
|
|
it **_MUST_** be closed and a new file **_MUST_** be opened for writing.
|
|
- **_REQ-2.2.2_ - Compression of Rotated Files:** Each traced log file **_MUST_** get compressed after it got rotated.
|
|
- **_REQ-2.2.3_ - Rotating Directory:** When the directory size exceeds a certain threshold, the tool **_MUST_** delete the oldest
|
|
files in the directory, until the size is below the threshold again.
|
|
|
|
|
|
### _TOPIC-3_ - Remote Journal Logging
|
|
#### _SUB-3.1_ - Service Activation
|
|
- **_REQ-3.1.1_ - Cloud Activation:** The remote journal logging **_SHALL_** be startable through a function call from the cloud.
|
|
The api call has the duration and max interval as arguments.
|
|
- **_REQ-3.1.2_ - Duration:** The remote journal logging **_SHOULD_** stay active, until it reaches the specified duration.
|
|
- **_REQ-3.1.3_ - Max Interval:** If no upload was done after the amount of time specified in max interval,
|
|
a log rotation **_SHALL_** be triggered, which will in turn get picked up by the file monitoring.
|
|
- **_REQ-3.1.4_ - Analytics Not Accepted:** If the user has not accepted the usage of their data, the cloud call **_MUST_**
|
|
result in an error.
|
|
|
|
#### _SUB-3.2_ - File Processing
|
|
- **_REQ-3.2.1_ - File Upload:** When a file gets detected, it **_SHOULD_** get uploaded to the cloud.
|
|
- **_REQ-3.2.2_ - No Duplicate Files:** Already processed files **_MUST NOT_** get uploaded again.
|
|
- **_REQ-3.2.3_ - Revoking Analytics:** If the user revokes the usage of their data, the service **_MAY_** continue running
|
|
but **_MUST NOT_** upload any data until the user allows the usage of their data again.
|
|
- **_REQ-3.2.4_ - Duration Expired:** After the specified duration is expired, the service **_SHOULD_** stop uploading files.
|
|
|
|
|
|
### _TOPIC-4_ - Configuration
|
|
- **_CONF-4.1_ - Journal Directory:** Users **_SHOULD_** be able to specify the directory to be monitored for journal files.
|
|
- **_CONF-4.2_ - Output Directory:** Users **_SHOULD_** be able to specify the directory into which the final files will be written.
|
|
- **_CONF-4.3_ - Trigger Priority:** Users **_SHOULD_** be able to specify which priority triggers the filtering.
|
|
- **_CONF-4.4_ - Journal Context:** Users **_SHOULD_** be able to specify how many seconds of context will be added to traced logs when encountering a trigger priority.
|
|
- **_CONF-4.5_ - Max File Size:** Users **_SHOULD_** be able to specify the max file size, at which a file gets rotated.
|
|
- **_CONF-4.6_ - Max Directory Size:** Users **_SHOULD_** be able to specify the max directory size, at which a directory gets rotated.
|
|
- **_CONF-4.7_ - File Monitoring Interval:** Users **_SHOULD_** be able to specify an interval, which **_SHOULD_** change
|
|
how long the tool waits before checking if new files are available.
|
|
|
|
### _TOPIC-5_ - Performance Requirements
|
|
- **_PERF-5.1_ - Efficiency:** The tool **_SHOULD_** efficiently monitor and process files without excessive resource consumption.
|
|
- **_PERF-5.2_ - Interval Delay:** The tool **_SHOULD_** do its work with no more than 10 seconds delay after its interval.
|
|
|
|
### _TOPIC-6_ - Security & Data Protection
|
|
- **_SEC-6.1_ - No Insecure Connection:** The tool **_MUST_** send data only through a secure connection.
|
|
- **_SEC-6.2_ - GDPR compliance:** The tool **_MUST NOT_** upload data if the user has not agreed to share this information.
|
|
|
|
### _TOPIC-7_ - Testing
|
|
- **_TST-7.1_ - Unit Tests:** Comprehensive unit tests **_SHOULD_** be written to cover major functionalities.
|
|
- **_TST-7.2_ - Integration Tests:** Integration tests **_SHOULD_** be conducted to ensure all parts of the tool work together seamlessly.
|
|
|
|
|
|
## Definitions
|
|
- Default Journal Directory: /run/log/journal/<machine_id>
|
|
- Machine ID can be found at /etc/machine-id
|
|
- Default Output Directory: /run/log/filtered-journal
|
|
|
|
## Config Defaults
|
|
- **Journal Directory**
|
|
- Type: Path
|
|
- **Required**: This value **_MUST_** be provided as a start parameter.
|
|
|
|
- **Output Directory**
|
|
- Type: Path
|
|
- **Required**: This value **_MUST_** be provided as a start parameter.
|
|
|
|
- **Trigger Priority**
|
|
- Type: Enum
|
|
- Valid Values: _Emergency, Alert, Critical, Error, Warning, Notice, Info, Debug_
|
|
- Default Value: _Warning_
|
|
|
|
- **Journal Context**
|
|
- Type: Integer
|
|
- Unit: Seconds
|
|
- Default Value: _15_
|
|
|
|
- **Max File Size**
|
|
- Type: Integer
|
|
- Unit: Bytes
|
|
- Default Value: _8388608_ (8 MB)
|
|
|
|
- **Max Directory Size**
|
|
- Type: Integer
|
|
- Unit: Bytes
|
|
- Default Value: _75497472_ (72 MB)
|
|
|
|
- **File Monitoring Interval**
|
|
- Type: Integer
|
|
- Unit: Seconds
|
|
- Default Value: _10_
|
|
|