IAmHardliner
/
req
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
req/out.md

6.1 KiB
Raw Blame History

Requirements for journal-uploader

[[TOC]]

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Description

The journal-uploader has two main functionalities.

  • Take a stream of log messages and filter them depending on their severity
  • Upload journal logs for a specified time when activated through cloud call

Requirements

TOPIC-1 - Journal Watcher

SUB-1.1 - File Monitoring

  • REQ-1.1.1 - Continuous Monitoring: The tool MUST continuously monitor a designated directory.

SUB-1.2 - File Detection

  • REQ-1.2.1 - Detection of New Files: The tool MUST detect the addition of new files in the monitored directory.
  • REQ-1.2.2 - Avoid Re-processing: The tool MUST NOT process files that have already been processed.

TOPIC-2 - Traced Logging

SUB-2.1 - File Processing

  • REQ-2.1.1 - Reading Log Messages: When a new file is processed, each log message SHOULD be put into a buffer.
  • REQ-2.1.2 - Filtering Log Messages: The tool will search for messages of a defined priority (Trigger Priority). Each message of this priority, as well as all messages before and after, which are inside a defined timespan, MUST get written into a file. Every other message SHOULD gets dropped.
  • REQ-2.1.3 - No Duplicate Log Messages: The tool SHALL make sure that no log entry will be written to the file twice.

SUB-2.2 - Traced Log Rotation

  • REQ-2.2.1 - Rotating Files: When the size of the current traced log file exceeds a certain threshold, it MUST be closed and a new file MUST be opened for writing.
  • REQ-2.2.2 - Compression of Rotated Files: Each traced log file MUST get compressed after it got rotated.
  • REQ-2.2.3 - Rotating Directory: When the directory size exceeds a certain threshold, the tool MUST delete the oldest files in the directory, until the size is below the threshold again.

TOPIC-3 - Remote Journal Logging

SUB-3.1 - Service Activation

  • REQ-3.1.1 - Cloud Activation: The remote journal logging SHALL be startable through a function call from the cloud. The api call has the duration and max interval as arguments.
  • REQ-3.1.2 - Duration: The remote journal logging SHOULD stay active, until it reaches the specified duration.
  • REQ-3.1.3 - Max Interval: If no upload was done after the amount of time specified in max interval, a log rotation SHALL be triggered, which will in turn get picked up by the file monitoring.
  • REQ-3.1.4 - Analytics Not Accepted: If the user has not accepted the usage of their data, the cloud call MUST result in an error.

SUB-3.2 - File Processing

  • REQ-3.2.1 - File Upload: When a file gets detected, it SHOULD get uploaded to the cloud.
  • REQ-3.2.2 - No Duplicate Files: Already processed files MUST NOT get uploaded again.
  • REQ-3.2.3 - Revoking Analytics: If the user revokes the usage of their data, the service MAY continue running but MUST NOT upload any data until the user allows the usage of their data again.
  • REQ-3.2.4 - Duration Expired: After the specified duration is expired, the service SHOULD stop uploading files.

TOPIC-4 - Configuration

  • CONF-4.1 - Journal Directory: Users SHOULD be able to specify the directory to be monitored for journal files.
  • CONF-4.2 - Output Directory: Users SHOULD be able to specify the directory into which the final files will be written.
  • CONF-4.3 - Trigger Priority: Users SHOULD be able to specify which priority triggers the filtering.
  • CONF-4.4 - Journal Context: Users SHOULD be able to specify how many seconds of context will be added to traced logs when encountering a trigger priority.
  • CONF-4.5 - Max File Size: Users SHOULD be able to specify the max file size, at which a file gets rotated.
  • CONF-4.6 - Max Directory Size: Users SHOULD be able to specify the max directory size, at which a directory gets rotated.
  • CONF-4.7 - File Monitoring Interval: Users SHOULD be able to specify an interval, which SHOULD change how long the tool waits before checking if new files are available.

TOPIC-5 - Performance Requirements

  • PERF-5.1 - Efficiency: The tool SHOULD efficiently monitor and process files without excessive resource consumption.
  • PERF-5.2 - Interval Delay: The tool SHOULD do its work with no more than 10 seconds delay after its interval.

TOPIC-6 - Security & Data Protection

  • SEC-6.1 - No Insecure Connection: The tool MUST send data only through a secure connection.
  • SEC-6.2 - GDPR compliance: The tool MUST NOT upload data if the user has not agreed to share this information.

TOPIC-7 - Testing

  • TEST-7.1 - Unit Tests: Comprehensive unit tests SHOULD be written to cover major functionalities.
  • TEST-7.2 - Integration Tests: Integration tests SHOULD be conducted to ensure all parts of the tool work together seamlessly.

Definitions

  • Default Journal Directory: /run/log/journal/<machine_id>
    • Machine ID can be found at /etc/machine-id
  • Default Output Directory: /run/log/filtered-journal

Config Defaults

  • Journal Directory

    • Type: Path
    • Required: This value MUST be provided as a start parameter.

  • Output Directory

    • Type: Path
    • Required: This value MUST be provided as a start parameter.

  • Trigger Priority

    • Type: Enum
    • Valid Values: Emergency, Alert, Critical, Error, Warning, Notice, Info, Debug
    • Default Value: Warning

  • Journal Context

    • Type: Integer
    • Unit: Seconds
    • Default Value: 15

  • Max File Size

    • Type: Integer
    • Unit: Bytes
    • Default Value: 8388608 (8 MB)

  • Max Directory Size

    • Type: Integer
    • Unit: Bytes
    • Default Value: 75497472 (72 MB)

  • File Monitoring Interval

    • Type: Integer
    • Unit: Seconds
    • Default Value: 10