# Requirements for journal-uploader [[_TOC_]] The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119). ## Description The journal-uploader has two main functionalities. - Take a stream of log messages and filter them depending on their severity - Upload journal logs for a specified time when activated through cloud call ## Requirements ### [_TOPIC-1_] Journal Watcher #### [_SUB-1.1_] File Monitoring - **[_REQ-1.1.1_] Continuous Monitoring:** The tool **_MUST_** continuously monitor a designated directory. #### [_SUB-1.2_] File Detection - **[_REQ-1.2.1_] Detection of New Files:** The tool **_MUST_** detect the addition of new files in the monitored directory. - **[_REQ-1.2.2_] Avoid Re-processing:** The tool **_MUST NOT_** process files that have already been processed. ### [_TOPIC-2_] Traced Logging #### [_SUB-2.1_] File Processing - **[_REQ-2.1.1_] Reading Log Messages:** When a new file is processed, each log message **_SHOULD_** be put into a buffer. - **[_REQ-2.1.2_] Filtering Log Messages:** The tool will search for messages of a defined priority (Trigger Priority). Each message of this priority, as well as all messages before and after, which are inside a defined timespan, **_MUST_** get written into a file. Every other message **_SHOULD_** gets dropped. - **[_REQ-2.1.3_] No Duplicate Log Messages:** The tool **_SHALL_** make sure that no log entry will be written to the file twice. #### [_SUB-2.2_] Traced Log Rotation - **[_REQ-2.2.1_] Rotating Files:** When the size of the current traced log file exceeds a certain threshold, it **_MUST_** be closed and a new file **_MUST_** be opened for writing. - **[_REQ-2.2.2_] Compression of Rotated Files:** Each traced log file **_MUST_** get compressed after it got rotated. - **[_REQ-2.2.3_] Rotating Directory:** When the directory size exceeds a certain threshold, the tool **_MUST_** delete the oldest files in the directory, until the size is below the threshold again. ### [_TOPIC-3_] Remote Journal Logging #### [_SUB-3.1_] Service Activation - **[_REQ-3.1.1_] Cloud Activation:** The remote journal logging **_SHALL_** be startable through a function call from the cloud. The api call has the duration and max interval as arguments. - **[_REQ-3.1.2_] Duration:** The remote journal logging **_SHOULD_** stay active, until it reaches the specified duration. - **[_REQ-3.1.3_] Max Interval:** If no upload was done after the amount of time specified in max interval, a log rotation **_SHALL_** be triggered, which will in turn get picked up by the file monitoring. - **[_REQ-3.1.4_] Analytics Not Accepted:** If the user has not accepted the usage of their data, the cloud call **_MUST_** result in an error. #### [_SUB-3.2_] File Processing - **[_REQ-3.2.1_] File Upload:** When a file gets detected, it **_SHOULD_** get uploaded to the cloud. - **[_REQ-3.2.2_] No Duplicate Files:** Already processed files **_MUST NOT_** get uploaded again. - **[_REQ-3.2.3_] Revoking Analytics:** If the user revokes the usage of their data, the service **_MAY_** continue running but **_MUST NOT_** upload any data until the user allows the usage of their data again. - **[_REQ-3.2.4_] Duration Expired:** After the specified duration is expired, the service **_SHOULD_** stop uploading files. ### [_TOPIC-4_] Configuration - **[_CONF-4.1_] Journal Directory:** Users **_SHOULD_** be able to specify the directory to be monitored for journal files. - **[_CONF-4.2_] Output Directory:** Users **_SHOULD_** be able to specify the directory into which the final files will be written. - **[_CONF-4.3_] Trigger Priority:** Users **_SHOULD_** be able to specify which priority triggers the filtering. - **[_CONF-4.4_] Journal Context:** Users **_SHOULD_** be able to specify how many seconds of context will be added to traced logs when encountering a trigger priority. - **[_CONF-4.5_] Max File Size:** Users **_SHOULD_** be able to specify the max file size, at which a file gets rotated. - **[_CONF-4.6_] Max Directory Size:** Users **_SHOULD_** be able to specify the max directory size, at which a directory gets rotated. - **[_CONF-4.7_] File Monitoring Interval:** Users **_SHOULD_** be able to specify an interval, which **_SHOULD_** change how long the tool waits before checking if new files are available. ### [_TOPIC-5_] Performance Requirements - **[_PERF-5.1_] Efficiency:** The tool **_SHOULD_** efficiently monitor and process files without excessive resource consumption. - **[_PERF-5.2_] Interval Delay:** The tool **_SHOULD_** do its work with no more than 10 seconds delay after its interval. ### [_TOPIC-6_] Security & Data Protection - **[_SEC-6.1_] No Insecure Connection:** The tool **_MUST_** send data only through a secure connection. - **[_SEC-6.2_] GDPR compliance:** The tool **_MUST NOT_** upload data if the user has not agreed to share this information. ### [_TOPIC-7_] Testing - **[_TEST-7.1_] Unit Tests:** Comprehensive unit tests **_SHOULD_** be written to cover major functionalities. - **[_TEST-7.2_] Integration Tests:** Integration tests **_SHOULD_** be conducted to ensure all parts of the tool work together seamlessly. ## Definitions - Default Journal Directory: /run/log/journal/ - Machine ID can be found at /etc/machine-id - Default Output Directory: /run/log/filtered-journal ## Config Defaults - **Journal Directory** - Type: Path - **Required**: This value **_MUST_** be provided as a start parameter. - **Output Directory** - Type: Path - **Required**: This value **_MUST_** be provided as a start parameter. - **Trigger Priority** - Type: Enum - Valid Values: _Emergency, Alert, Critical, Error, Warning, Notice, Info, Debug_ - Default Value: _Warning_ - **Journal Context** - Type: Integer - Unit: Seconds - Default Value: _15_ - **Max File Size** - Type: Integer - Unit: Bytes - Default Value: _8388608_ (8 MB) - **Max Directory Size** - Type: Integer - Unit: Bytes - Default Value: _75497472_ (72 MB) - **File Monitoring Interval** - Type: Integer - Unit: Seconds - Default Value: _10_