IAmHardliner
/
req
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update

This commit is contained in:
Biedermann Steve 2024-05-08 14:18:03 +02:00
parent e3dceb8e20
commit 5936cdbcad
5 changed files with 275 additions and 28 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
Cargo.toml
View File

@ -2,6 +2,7 @@
name = "req"
version = "0.1.0"
edition = "2021"
default-run = "req"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

8
orig.md
View File

@ -61,23 +61,16 @@ The journal-uploader has two main functionalities.
### 3. Configuration
- **3.1 Configurable Journal Directory:** Users **_SHOULD_** be able to specify the directory to be monitored for
journal files.
- Required: This value **_MUST_** be provided as a start parameter.
- **3.2 Configurable Output Directory:** Users **_SHOULD_** be able to specify the directory into which the final files
will be written.
- Required: This value **_MUST_** be provided as a start parameter.
- **3.3 Configurable Trigger Priority:** Users **_SHOULD_** be able to specify which priority triggers the filtering.
- Default Value: _Warning_
- **3.4 Configurable Journal Context:** Users **_SHOULD_** be able to specify how many seconds of context will be added
to traced logs when encountering a trigger priority.
- Default Value: _10_
- **3.5 Configurable Max File Size:** Users **_SHOULD_** be able to specify the max file size, at which a file gets rotated.
- Default Value: _8388608_ (8 MB)
- **3.6 Configurable Max Directory Size:** Users **_SHOULD_** be able to specify the max directory size, at which a
directory gets rotated.
- Default Value: _75497472_ (72 MB)
- **3.7 Configurable File Monitoring Interval:** Users **_SHOULD_** be able to specify an interval, which **_SHOULD_** change
how long the tool waits before checking if new files are available.
- Default Value: _10_
### 4. Performance Requirements
- **4.1 Efficiency:** The tool **_SHOULD_** efficiently monitor and process files without excessive resource consumption.
@ -98,7 +91,6 @@ The journal-uploader has two main functionalities.
- Default Output Directory: /run/log/filtered-journal
## Config Defaults
- **Journal Directory**
- type: Path
- **Required**: This value **_MUST_** be provided as a start parameter.

151
req.yml
View File

@ -13,6 +13,157 @@ topics:
REQ-1:
name: Continuous Monitoring
description: The tool must continuously monitor a designated directory.
SUB-2:
name: File Detection
requirements:
REQ-1:
name: Detection of New Files
description: The tool must detect the addition of new files in the monitored directory.
REQ-2:
name: Avoid Re-processing
description: The tool must not process files that have already been processed.
SUB-3:
name: File Processing
requirements:
REQ-1:
name: Reading Log Messages
description: When a new file is processed, each log message should be put into a buffer.
REQ-2:
name: Filtering Log Messages
description: |-
The tool will search for messages of a defined priority (Trigger Priority).
Each message of this priority, as well as all messages before and after, which are inside a defined timespan, must
get written into a file. Every other message should gets dropped.
REQ-3:
name: No Duplicate Log Messages
description: The tool shall make sure that no log entry will be written to the file twice.
SUB-4:
name: Traced Log Rotation
requirements:
REQ-1:
name: Rotating Files
description: |-
When the size of the current traced log file exceeds a certain threshold,
it must be closed and a new file must be opened for writing.
REQ-2:
name: Compression of Rotated Files
description: Each traced log file must get compressed after it got rotated.
REQ-3:
name: Rotating Directory
description: |-
When the directory size exceeds a certain threshold, the tool must delete the oldest
files in the directory, until the size is below the threshold again.
FEAT-2:
name: Remote Journal Logging
subtopics:
SUB-1:
name: Service Activation
requirements:
REQ-1:
name: Cloud Activation
description: |-
The remote journal logging **_SHALL_** be startable through a function call from the cloud.
The api call has the duration and max interval as arguments.
REQ-2:
name: Duration
description: The remote journal logging **_SHOULD_** stay active, until it reaches the specified duration.
REQ-3:
name: Max Interval
description: |-
If no upload was done after the amount of time specified in max interval,
a log rotation **_SHALL_** be triggered, which will in turn get picked up by the file monitoring.
REQ-4:
name: Analytics Not Accepted
description: |-
If the user has not accepted the usage of their data, the cloud call **_MUST_**
result in an error.
SUB-2:
name: File Monitoring
requirements:
REQ-1:
name: Continuous Monitoring
description: The tool **_SHOULD_** continuously monitor a designated directory.
SUB-3:
name: File Detection
requirements:
REQ-1:
name: Detection of New Files
description: The tool **_MUST_** detect the addition of new files in the monitored directory.
REQ-2:
name: Avoid Re-processing
description: The tool must not process files that have already been processed.
SUB-4:
name: File Processing
requirements:
REQ-1:
name: File Upload
description: When a file gets detected, it **_SHOULD_** get uploaded to the cloud.
REQ-2:
name: No Duplicate Files
description: Already processed files **_MUST NOT_** get uploaded again.
REQ-3:
name: Revoking Analytics
description: |-
If the user revokes the usage of their data, the service **_MAY_** continue running
but **_MUST NOT_** upload any data until the user allows the usage of their data again.
REQ-4:
name: Duration Expired
description: After the specified duration is expired, the service **_SHOULD_** stop uploading files.
FEAT-3:
name: Configuration
requirements:
REQ-1:
name: Journal Directory
description: Users **_SHOULD_** be able to specify the directory to be monitored for journal files.
REQ-2:
name: Output Directory
description: Users **_SHOULD_** be able to specify the directory into which the final files will be written.
REQ-3:
name: Trigger Priority
description: Users **_SHOULD_** be able to specify which priority triggers the filtering.
REQ-4:
name: Journal Context
description: Users **_SHOULD_** be able to specify how many seconds of context will be added
to traced logs when encountering a trigger priority.
REQ-5:
name: Max File Size
description: Users **_SHOULD_** be able to specify the max file size, at which a file gets rotated.
REQ-6:
name: Max Directory Size
description: Users **_SHOULD_** be able to specify the max directory size, at which a directory gets rotated.
REQ-7:
name: File Monitoring Interval
description: |-
Users **_SHOULD_** be able to specify an interval, which **_SHOULD_** change
how long the tool waits before checking if new files are available.
FEAT-4:
name: Performance Requirements
requirements:
REQ-1:
name: Efficiency
description: The tool **_SHOULD_** efficiently monitor and process files without excessive resource consumption.
REQ-2:
name: Interval Delay
description: The tool **_SHOULD_** do its work with no more than 10 seconds delay after its interval.
FEAT-5:
name: Data Protection
requirements:
REQ-1:
name: No Insecure Connection
description: The tool **_MUST_** send data only through a secure connection.
REQ-2:
name: GDPR compliance
description: The tool **_MUST NOT_** upload data if the user has not agreed to share this information.
FEAT-6:
name: Testing
requirements:
REQ-1:
name: Unit Tests
description: Comprehensive unit tests **_SHOULD_** be written to cover major functionalities.
REQ-2:
name: Integration Tests
description: Integration tests **_SHOULD_** be conducted to ensure all parts of the tool work together seamlessly.
definitions:
- name: Default Journal Directory
value: /run/log/journal/<machine_id>

52
src/lib.rs
View File

@ -30,16 +30,16 @@ where
s.serialize_str(v.trim())
}
#[derive(Deserialize, Serialize)]
#[derive(Debug, Deserialize, Serialize)]
pub struct Requirement {
pub name: String,
#[serde(serialize_with = "my_trim")]
pub description: String,
#[serde(default, skip_serializing_if = "IndexMap::is_empty")]
pub depends: IndexMap<String, String>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub requires: Vec<String>,
}
#[derive(Deserialize, Serialize)]
#[derive(Debug, Deserialize, Serialize)]
pub struct Topic {
pub name: String,
#[serde(default, skip_serializing_if = "IndexMap::is_empty")]
@ -48,7 +48,7 @@ pub struct Topic {
pub subtopics: IndexMap<String, Topic>,
}
#[derive(Deserialize, Serialize)]
#[derive(Debug, Deserialize, Serialize)]
pub struct Definition {
pub name: String,
pub value: String,
@ -56,13 +56,13 @@ pub struct Definition {
pub additional_info: Vec<String>,
}
#[derive(Deserialize, Serialize)]
#[derive(Debug, Deserialize, Serialize)]
pub struct ConfigDefault {
pub name: String,
#[serde(rename = "type")]
pub typ: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub valid_values: Vec<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub valid_values: Option<Vec<String>>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub unit: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
@ -71,7 +71,7 @@ pub struct ConfigDefault {
pub hint: Option<String>,
}
#[derive(Deserialize, Serialize)]
#[derive(Debug, Deserialize, Serialize)]
pub struct Project {
pub name: String,
#[serde(serialize_with = "my_trim")]
@ -101,7 +101,23 @@ The journal-uploader has two main functionalities.
s!("REQ-1") => Requirement {
name: s!("Continuous Monitoring"),
description: s!(r#"The tool must continuously monitor a designated directory."#),
depends: indexmap! {}
requires: vec! [],
}
},
subtopics: indexmap! {}
},
s!("SUB-2") => Topic {
name: s!("File Detection"),
requirements: indexmap! {
s!("REQ-1") => Requirement {
name: s!("Detection of New Files"),
description: s!(r#"The tool must detect the addition of new files in the monitored directory."#),
requires: vec! [],
},
s!("REQ-2") => Requirement {
name: s!("Avoid Re-processing"),
description: s!(r#"The tool must not process files that have already been processed."#),
requires: vec! [],
}
},
subtopics: indexmap! {}
@ -127,7 +143,7 @@ The journal-uploader has two main functionalities.
name: s!("Journal Directory"),
typ: s!("Path"),
unit: None,
valid_values: vec![],
valid_values: None,
default_value: None,
hint: None,
},
@ -135,7 +151,7 @@ The journal-uploader has two main functionalities.
name: s!("Output Directory"),
typ: s!("Path"),
unit: None,
valid_values: vec![],
valid_values: None,
default_value: None,
hint: None,
},
@ -143,7 +159,7 @@ The journal-uploader has two main functionalities.
name: s!("Trigger Priority"),
typ: s!("Enum"),
unit: None,
valid_values: vec![
valid_values: Some(vec![
s!("Emergency"),
s!("Alert"),
s!("Critical"),
@ -152,7 +168,7 @@ The journal-uploader has two main functionalities.
s!("Notice"),
s!("Info"),
s!("Debug"),
],
]),
default_value: Some(s!("Warning")),
hint: None,
},
@ -160,7 +176,7 @@ The journal-uploader has two main functionalities.
name: s!("Journal Context"),
typ: s!("Integer"),
unit: Some(s!("Seconds")),
valid_values: vec![],
valid_values: None,
default_value: Some(s!("15")),
hint: None,
},
@ -168,7 +184,7 @@ The journal-uploader has two main functionalities.
name: s!("Max File Size"),
typ: s!("Integer"),
unit: Some(s!("Bytes")),
valid_values: vec![],
valid_values: None,
default_value: Some(s!("8388608")),
hint: Some(s!("(8 MB)")),
},
@ -176,7 +192,7 @@ The journal-uploader has two main functionalities.
name: s!("Max Directory Size"),
typ: s!("Integer"),
unit: Some(s!("Bytes")),
valid_values: vec![],
valid_values: None,
default_value: Some(s!("75497472")),
hint: Some(s!("(72 MB)")),
},
@ -184,7 +200,7 @@ The journal-uploader has two main functionalities.
name: s!("File Monitoring Interval"),
typ: s!("Integer"),
unit: Some(s!("Seconds")),
valid_values: vec![],
valid_values: None,
default_value: Some(s!("10")),
hint: None,
},

91
src/main.rs
View File

@ -1,8 +1,95 @@
use indexmap::IndexMap;
use req::*;
use stringlit::s;
fn nl() -> String {
s!("")
}
fn add_requirements(output: &mut Vec<String>, requirements: &IndexMap<String, Requirement>) {
for (id, requirement) in requirements {
output.push(format!(
"- **{id} {}:** {}",
requirement.name, requirement.description
));
}
}
fn add_topics(output: &mut Vec<String>, topics: &IndexMap<String, Topic>, level: usize) {
for (id, topic) in topics {
output.push(format!("{} {id} {}", "#".repeat(level), topic.name));
if !topic.requirements.is_empty() {
add_requirements(output, &topic.requirements);
output.push(nl());
}
if !topic.subtopics.is_empty() {
add_topics(output, &topic.subtopics, level + 1);
output.push(nl());
}
}
}
fn main() -> anyhow::Result<()> {
let project = demo_project();
std::fs::write("req.yml", serde_yaml::to_string(&project)?)?;
// std::fs::write("req.yml", serde_yaml::to_string(&project)?)?;
let project: Project = serde_yaml::from_str(&std::fs::read_to_string("req.yml")?)?;
let mut output = vec![
format!("# Requirements for {}", project.name),
nl(),
s!("[[_TOC_]]"),
nl(),
WORD_DESCRIPTION.to_string(),
nl(),
s!("## Description"),
project.description,
nl(),
];
if !project.topics.is_empty() {
output.push(s!("## Requirements"));
add_topics(&mut output, &project.topics, 3);
output.push(nl());
}
if !project.definitions.is_empty() {
output.push(s!("## Definitions"));
for definition in project.definitions {
output.push(format!("- {}: {}", definition.name, definition.value));
for info in definition.additional_info {
output.push(format!(" - {info}"))
}
}
output.push(nl());
}
if !project.config_defaults.is_empty() {
output.push(s!("## Config Defaults"));
for default in project.config_defaults {
output.push(format!("- **{}**", default.name));
output.push(format!(" - Type: {}", default.typ));
if let Some(unit) = default.unit {
output.push(format!(" - Unit: {unit}"));
}
if let Some(valid_values) = default.valid_values {
output.push(format!(" - Valid Values: _{}_", valid_values.join(", ")));
}
if let Some(default_value) = default.default_value {
output.push(format!(
" - Default Value: _{}_{}",
default_value,
default.hint.map(|h| format!(" {h}")).unwrap_or_default()
));
} else {
output.push(format!(
" - **Required**: This value **_MUST_** be provided as a start parameter.{}",
default.hint.map(|h| format!(" {h}")).unwrap_or_default()
));
}
output.push(nl());
}
}
println!("{}", output.join("\n"));
Ok(())
}