From e0f60849a9e0b525c1b4fbf382a26fc773fba7e1 Mon Sep 17 00:00:00 2001 From: Javad Nasrolahi Date: Sat, 1 Aug 2026 14:49:35 +0330 Subject: [PATCH 1/5] pipeline: outputs: arvancloud_cloudlogs: add ArvanCloud CloudLogs documentation Signed-off-by: Javad Nasrolahi --- SUMMARY.md | 1 + pipeline/outputs/arvancloud_cloudlogs.md | 167 +++++++++++++++++++++++ 2 files changed, 168 insertions(+) create mode 100644 pipeline/outputs/arvancloud_cloudlogs.md diff --git a/SUMMARY.md b/SUMMARY.md index c86f24ca7..55d79a4ad 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -180,6 +180,7 @@ * [Amazon Kinesis Data Streams](pipeline/outputs/kinesis.md) * [Amazon S3](pipeline/outputs/s3.md) * [Apache SkyWalking](pipeline/outputs/skywalking.md) + * [ArvanCloud CloudLogs](pipeline/outputs/arvancloud_cloudlogs.md) * [Counter](pipeline/outputs/counter.md) * [Dash0](pipeline/outputs/dash0.md) * [Datadog](pipeline/outputs/datadog.md) diff --git a/pipeline/outputs/arvancloud_cloudlogs.md b/pipeline/outputs/arvancloud_cloudlogs.md new file mode 100644 index 000000000..c38212af9 --- /dev/null +++ b/pipeline/outputs/arvancloud_cloudlogs.md @@ -0,0 +1,167 @@ +--- +description: Send logs to ArvanCloud CloudLogs +--- + +# ArvanCloud CloudLogs + +{% hint style="info" %} +**Supported event types:** `logs` +{% endhint %} + +The _ArvanCloud CloudLogs_ output plugin sends log records to the [ArvanCloud CloudLogs](https://www.arvancloud.ir/en/products/cloud-logs) ingestion API over HTTPS. + +Fluent Bit posts a JSON body to the fixed endpoint `https://napi.arvancloud.ir/logging/v1/entries/write`. Each Fluent Bit record is wrapped in the CloudLogs entry schema. Authentication uses an API key sent in the `Authorization` header as `apikey `. + +## Configuration parameters + +| Key | Description | Default | +| :--- | :--- | :--- | +| `apikey` | Required. API key used for authorization. Fluent Bit sends it as `Authorization: apikey `. | _none_ | +| `gzip` | Enable gzip compression of the HTTP request body. If compression fails, Fluent Bit sends the uncompressed payload. | `false` | +| `include_tag_key` | When enabled, include the original Fluent Bit tag as an extra field on each CloudLogs entry. | `false` | +| `log_type` | Static `logType` value used when `log_type_key` isn't set, or when the record field referenced by `log_type_key` is missing or empty. | `fluentbit` | +| `log_type_key` | Optional [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) that selects a field from the record to use as `logType`. When the field exists and isn't empty, it takes priority over `log_type`. | _none_ | +| `tag_key` | Field name used for the Fluent Bit tag when `include_tag_key` is enabled. | `tag` | +| `timestamp_format` | Optional `strptime`-style format used to parse the value selected by `timestamp_key`. When set, Fluent Bit parses the field and rewrites it as UTC RFC3339 with microseconds. When omitted, the `timestamp_key` value is forwarded as-is. | _none_ | +| `timestamp_key` | Optional [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) that selects a record field to use as the CloudLogs `timestamp`. If unset, missing, empty, or unparseable, Fluent Bit uses the event timestamp. | _none_ | + +The plugin hardcodes the destination host (`napi.arvancloud.ir`), port (`443`), URI (`/logging/v1/entries/write`), and HTTPS. Host, port, and URI aren't configurable. + +## Request payload + +Fluent Bit builds one JSON object per flush: + +```jsonhttps://messages.google.com/web/u/1 +{ + "logs": [ + { + "logType": "fluentbit", + "timestamp": "2024-01-15T10:30:45.000000Z", + "severity": "INFO", + "resource": { + "type": "general" + }, + "payload": { + "key": "value" + } + } + ] +} +``` + +Behavior notes: + +- `payload` contains the original record map unchanged. +- `severity` is always set to `INFO`. +- `resource` is always set to `{"type":"general"}`. +- When `include_tag_key` is enabled, the tag is added as a sibling field of `payload` using `tag_key`. +- `logType` resolution order is `log_type_key` (when present and non-empty), then `log_type`. +- Timestamp resolution: + 1. If `timestamp_key` and `timestamp_format` are set, parse the field and emit UTC RFC3339 with microseconds (for example `2024-01-15T10:30:45.000000Z`). + 2. If `timestamp_key` is set without `timestamp_format`, forward the field value as-is. + 3. Otherwise, or if extraction or parsing fails, use the Fluent Bit event timestamp formatted as UTC RFC3339 with microseconds. + +## HTTP response handling + +| Status | Result | +| :--- | :--- | +| `200`-`205` | Success (`FLB_OK`) | +| `400`, `401`, `403` | Failure without retry (`FLB_ERROR`) | +| `429` | Retry (`FLB_RETRY`) | +| `500` and above | Retry (`FLB_RETRY`) | +| Other HTTP client errors | Failure without retry (`FLB_ERROR`) | +| Connection or transport failure | Retry (`FLB_RETRY`) | + +## Get started + +### Minimal configuration + +`apikey` is the only required option: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + inputs: + - name: dummy + tag: app.logs + + outputs: + - name: arvancloud_cloudlogs + match: '*' + apikey: YOUR_API_KEY_HERE +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[INPUT] + Name dummy + Tag app.logs + +[OUTPUT] + Name arvancloud_cloudlogs + Match * + apikey YOUR_API_KEY_HERE +``` + +{% endtab %} +{% endtabs %} + +### Full configuration example + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + inputs: + - name: dummy + tag: app.logs + dummy: '{"message":"hello","category":"security","ts":"2024-01-15T10:30:45Z"}' + + outputs: + - name: arvancloud_cloudlogs + match: '*' + apikey: YOUR_API_KEY_HERE + log_type: myapp + log_type_key: $category + timestamp_key: $ts + timestamp_format: '%Y-%m-%dT%H:%M:%SZ' + gzip: true + include_tag_key: true + tag_key: fluentbit_tag +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[INPUT] + Name dummy + Tag app.logs + Dummy {"message":"hello","category":"security","ts":"2024-01-15T10:30:45Z"} + +[OUTPUT] + Name arvancloud_cloudlogs + Match * + apikey YOUR_API_KEY_HERE + log_type myapp + log_type_key $category + timestamp_key $ts + timestamp_format %Y-%m-%dT%H:%M:%SZ + gzip true + include_tag_key true + tag_key fluentbit_tag +``` + +{% endtab %} +{% endtabs %} + +With that example, Fluent Bit derives `logType` from `$category` (`security`), normalizes `$ts` to `2024-01-15T10:30:45.000000Z`, compresses the request body with gzip when possible, and includes the tag under `fluentbit_tag`. + +## References + +- [ArvanCloud CloudLogs](https://www.arvancloud.ir/en/products/cloud-logs) From 87627e81a9c52fa32bb9147247c83efcdd92853a Mon Sep 17 00:00:00 2001 From: Javad Nasrolahi Date: Sat, 1 Aug 2026 14:55:20 +0330 Subject: [PATCH 2/5] docs: pipeline: outputs: arvancloud_cloudlogs: fix JSON formatting in documentation Signed-off-by: Javad Nasrolahi --- pipeline/outputs/arvancloud_cloudlogs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/outputs/arvancloud_cloudlogs.md b/pipeline/outputs/arvancloud_cloudlogs.md index c38212af9..1aac3710c 100644 --- a/pipeline/outputs/arvancloud_cloudlogs.md +++ b/pipeline/outputs/arvancloud_cloudlogs.md @@ -31,7 +31,7 @@ The plugin hardcodes the destination host (`napi.arvancloud.ir`), port (`443`), Fluent Bit builds one JSON object per flush: -```jsonhttps://messages.google.com/web/u/1 +```json { "logs": [ { From b966cdee93234bd58434a451f80a09a49ada508a Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Sun, 2 Aug 2026 10:25:58 +0200 Subject: [PATCH 3/5] docs: pipeline: outputs: arvancloud_cloudlogs: resolve Vale style findings - Reword the timestamp_key description to avoid 'unparseable' - Reword the fixed-endpoint paragraph to avoid 'hardcodes' - Change '500 and above' to '500 or greater' in the HTTP status table - Add 'ArvanCloud CloudLogs' to the Vale heading capitalization exceptions - Fixed Title_Case issues in classic config examples Signed-off-by: Eric D. Schabell --- pipeline/outputs/arvancloud_cloudlogs.md | 24 ++++++++++++------------ vale-styles/FluentBit/Headings.yml | 1 + 2 files changed, 13 insertions(+), 12 deletions(-) diff --git a/pipeline/outputs/arvancloud_cloudlogs.md b/pipeline/outputs/arvancloud_cloudlogs.md index 1aac3710c..8699e52b5 100644 --- a/pipeline/outputs/arvancloud_cloudlogs.md +++ b/pipeline/outputs/arvancloud_cloudlogs.md @@ -23,9 +23,9 @@ Fluent Bit posts a JSON body to the fixed endpoint `https://napi.arvancloud.ir/l | `log_type_key` | Optional [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) that selects a field from the record to use as `logType`. When the field exists and isn't empty, it takes priority over `log_type`. | _none_ | | `tag_key` | Field name used for the Fluent Bit tag when `include_tag_key` is enabled. | `tag` | | `timestamp_format` | Optional `strptime`-style format used to parse the value selected by `timestamp_key`. When set, Fluent Bit parses the field and rewrites it as UTC RFC3339 with microseconds. When omitted, the `timestamp_key` value is forwarded as-is. | _none_ | -| `timestamp_key` | Optional [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) that selects a record field to use as the CloudLogs `timestamp`. If unset, missing, empty, or unparseable, Fluent Bit uses the event timestamp. | _none_ | +| `timestamp_key` | Optional [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) that selects a record field to use as the CloudLogs `timestamp`. If the key is unset, or the field is missing, empty, or can't be parsed, Fluent Bit uses the event timestamp. | _none_ | -The plugin hardcodes the destination host (`napi.arvancloud.ir`), port (`443`), URI (`/logging/v1/entries/write`), and HTTPS. Host, port, and URI aren't configurable. +The destination host (`napi.arvancloud.ir`), port (`443`), URI (`/logging/v1/entries/write`), and HTTPS scheme are fixed by the plugin. Host, port, and URI aren't configurable. ## Request payload @@ -68,7 +68,7 @@ Behavior notes: | `200`-`205` | Success (`FLB_OK`) | | `400`, `401`, `403` | Failure without retry (`FLB_ERROR`) | | `429` | Retry (`FLB_RETRY`) | -| `500` and above | Retry (`FLB_RETRY`) | +| `500` or greater | Retry (`FLB_RETRY`) | | Other HTTP client errors | Failure without retry (`FLB_ERROR`) | | Connection or transport failure | Retry (`FLB_RETRY`) | @@ -104,7 +104,7 @@ pipeline: [OUTPUT] Name arvancloud_cloudlogs Match * - apikey YOUR_API_KEY_HERE + Apikey YOUR_API_KEY_HERE ``` {% endtab %} @@ -147,14 +147,14 @@ pipeline: [OUTPUT] Name arvancloud_cloudlogs Match * - apikey YOUR_API_KEY_HERE - log_type myapp - log_type_key $category - timestamp_key $ts - timestamp_format %Y-%m-%dT%H:%M:%SZ - gzip true - include_tag_key true - tag_key fluentbit_tag + Apikey YOUR_API_KEY_HERE + Log_Type myapp + Log_Type_Key $category + Timestamp_Key $ts + Timestamp_Format %Y-%m-%dT%H:%M:%SZ + Gzip true + Include_Tag_Key true + Tag_Key fluentbit_tag ``` {% endtab %} diff --git a/vale-styles/FluentBit/Headings.yml b/vale-styles/FluentBit/Headings.yml index 9b549c4b2..445b1c9fa 100644 --- a/vale-styles/FluentBit/Headings.yml +++ b/vale-styles/FluentBit/Headings.yml @@ -22,6 +22,7 @@ exceptions: - Apache Arrow - API - APIs + - ArvanCloud CloudLogs - AWS - AWS MSK IAM - AWS IAM From 4e37a21ae7bf5e3cbac15f5a32f85fb22a66fb87 Mon Sep 17 00:00:00 2001 From: Javad Nasrolahi Date: Sat, 5 Sep 2026 13:12:55 +0330 Subject: [PATCH 4/5] docs: pipeline: outputs: arvancloud_cloudlogs: update timestamp handling and examples Signed-off-by: Javad Nasrolahi --- pipeline/outputs/arvancloud_cloudlogs.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/pipeline/outputs/arvancloud_cloudlogs.md b/pipeline/outputs/arvancloud_cloudlogs.md index 8699e52b5..9d35e7de4 100644 --- a/pipeline/outputs/arvancloud_cloudlogs.md +++ b/pipeline/outputs/arvancloud_cloudlogs.md @@ -22,8 +22,7 @@ Fluent Bit posts a JSON body to the fixed endpoint `https://napi.arvancloud.ir/l | `log_type` | Static `logType` value used when `log_type_key` isn't set, or when the record field referenced by `log_type_key` is missing or empty. | `fluentbit` | | `log_type_key` | Optional [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) that selects a field from the record to use as `logType`. When the field exists and isn't empty, it takes priority over `log_type`. | _none_ | | `tag_key` | Field name used for the Fluent Bit tag when `include_tag_key` is enabled. | `tag` | -| `timestamp_format` | Optional `strptime`-style format used to parse the value selected by `timestamp_key`. When set, Fluent Bit parses the field and rewrites it as UTC RFC3339 with microseconds. When omitted, the `timestamp_key` value is forwarded as-is. | _none_ | -| `timestamp_key` | Optional [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) that selects a record field to use as the CloudLogs `timestamp`. If the key is unset, or the field is missing, empty, or can't be parsed, Fluent Bit uses the event timestamp. | _none_ | +| `timestamp_key` | Optional [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) that selects a record field to forward as the CloudLogs `timestamp`. The value is sent as-is. If the key is unset, or the field is missing or empty, Fluent Bit uses the event timestamp formatted as UTC RFC3339 with microseconds. | _none_ | The destination host (`napi.arvancloud.ir`), port (`443`), URI (`/logging/v1/entries/write`), and HTTPS scheme are fixed by the plugin. Host, port, and URI aren't configurable. @@ -57,9 +56,10 @@ Behavior notes: - When `include_tag_key` is enabled, the tag is added as a sibling field of `payload` using `tag_key`. - `logType` resolution order is `log_type_key` (when present and non-empty), then `log_type`. - Timestamp resolution: - 1. If `timestamp_key` and `timestamp_format` are set, parse the field and emit UTC RFC3339 with microseconds (for example `2024-01-15T10:30:45.000000Z`). - 2. If `timestamp_key` is set without `timestamp_format`, forward the field value as-is. - 3. Otherwise, or if extraction or parsing fails, use the Fluent Bit event timestamp formatted as UTC RFC3339 with microseconds. + 1. If `timestamp_key` is set and the field exists and isn't empty, forward the field value as-is. + 2. Otherwise, use the Fluent Bit event timestamp formatted as UTC RFC3339 with microseconds (for example `2024-01-15T10:30:45.000000Z`). + +To parse a timestamp string into event time, use a [parser](../parsers/configuring-parser.md). ## HTTP response handling @@ -120,7 +120,7 @@ pipeline: inputs: - name: dummy tag: app.logs - dummy: '{"message":"hello","category":"security","ts":"2024-01-15T10:30:45Z"}' + dummy: '{"message":"hello","category":"security","ts":"2024-01-15T10:30:45.123456+01:00"}' outputs: - name: arvancloud_cloudlogs @@ -129,7 +129,6 @@ pipeline: log_type: myapp log_type_key: $category timestamp_key: $ts - timestamp_format: '%Y-%m-%dT%H:%M:%SZ' gzip: true include_tag_key: true tag_key: fluentbit_tag @@ -142,7 +141,7 @@ pipeline: [INPUT] Name dummy Tag app.logs - Dummy {"message":"hello","category":"security","ts":"2024-01-15T10:30:45Z"} + Dummy {"message":"hello","category":"security","ts":"2024-01-15T10:30:45.123456+03:30"} [OUTPUT] Name arvancloud_cloudlogs @@ -151,7 +150,6 @@ pipeline: Log_Type myapp Log_Type_Key $category Timestamp_Key $ts - Timestamp_Format %Y-%m-%dT%H:%M:%SZ Gzip true Include_Tag_Key true Tag_Key fluentbit_tag @@ -160,7 +158,7 @@ pipeline: {% endtab %} {% endtabs %} -With that example, Fluent Bit derives `logType` from `$category` (`security`), normalizes `$ts` to `2024-01-15T10:30:45.000000Z`, compresses the request body with gzip when possible, and includes the tag under `fluentbit_tag`. +With that example, Fluent Bit derives `logType` from `$category` (`security`), forwards `$ts` as the CloudLogs timestamp, compresses the request body with gzip when possible, and includes the tag under `fluentbit_tag`. ## References From 79b44983845cf0e231abba29b64a595f98817638 Mon Sep 17 00:00:00 2001 From: Javad Nasrolahi Date: Sat, 5 Sep 2026 13:25:23 +0330 Subject: [PATCH 5/5] docs: pipeline: outputs: arvancloud_cloudlogs: update timestamp format in example Signed-off-by: Javad Nasrolahi --- pipeline/outputs/arvancloud_cloudlogs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/outputs/arvancloud_cloudlogs.md b/pipeline/outputs/arvancloud_cloudlogs.md index 9d35e7de4..fda224bbe 100644 --- a/pipeline/outputs/arvancloud_cloudlogs.md +++ b/pipeline/outputs/arvancloud_cloudlogs.md @@ -141,7 +141,7 @@ pipeline: [INPUT] Name dummy Tag app.logs - Dummy {"message":"hello","category":"security","ts":"2024-01-15T10:30:45.123456+03:30"} + Dummy {"message":"hello","category":"security","ts":"2024-01-15T10:30:45.123456+01:00"} [OUTPUT] Name arvancloud_cloudlogs