Repository navigation
feat: made SimpleNS k8s-native - #28
Conversation
Add repository guidelines, update github actions to include lint-core and lint-dashboard, update node version requirement on all packages to 22+ and added .nvmrc for nvm to read node version
feat: update base image for dependency installation to Node.js 22-alpine
refractor dashboard api into api service, make mcp-server to use all the api endpoints as tools, fix dependabot security vulnerabilities, add k8s config files for multi env dev and testing, update bruno api docs and architecture diagram
…v, and master environments - Created docker-compose.dev.yaml for development environment with API server, worker, notification processor, delayed processor, recovery service, and admin dashboard. - Created docker-compose.infra.yaml for shared infrastructure services including MongoDB, Kafka, Redis, Loki, and Grafana. - Created docker-compose.local.yaml for local environment with build context for API server and dashboard. - Created docker-compose.master.yaml for master environment using latest images from GHCR. - Added scripts for switching between environments (switch-env.ps1 and switch-env.sh) with options for rebuilding images and pulling the latest versions.
…oss api and notification processor services
…rity across the codebase
…iguration support
…ion for channel routing
…min authentication - Added dynamic configuration service to manage operational settings with zero-restart updates. - Refactored environment variable handling to utilize dynamic settings for various worker parameters. - Updated system configuration model to support mixed types for dynamic settings. - Enhanced delayed and recovery processors to listen for dynamic configuration changes and adjust intervals accordingly. - Created integration tests for admin authentication routes and settings API. - Added unit tests for dynamic configuration schema validations and functionality.
…hancements - Added NpmAuthService to manage npm authentication tokens and registry configurations. - Introduced new API endpoints for retrieving, saving, and deleting npm auth configurations. - Enhanced plugin installation process to support private packages with authentication. - Implemented validation for SimpleNSProvider to ensure compliance with required methods and manifest structure. - Updated schemas for npm auth configuration and plugin installation payloads. - Added integration and unit tests for the new npm auth service and plugin validation logic.
…s and scopes - Updated API client to handle npm auth configurations with optional scope. - Modified types to include support for multiple npm registries and their statuses. - Enhanced the npm auth service to manage multiple tokens and registry configurations. - Updated controller methods to accommodate new parameters for saving and deleting npm auth configurations. - Improved tests to cover new functionality for multiple registries and scoped tokens.
…im and bulk insert for notifications
…g provider fallback - Introduced `rate_limit_retry_delay_ms` to operational settings for dynamic configuration. - Updated dashboard settings to allow administrators to configure rate limit retry delay. - Enhanced rate limiter to use fixed retry delay instead of staggered delay based on queue position. - Implemented cascading provider fallback mechanism in the unified consumer to handle rate-limited providers. - Added utility function for exponential backoff calculation. - Updated tests to cover new functionality and ensure proper behavior of rate limiting and fallback logic.
- Implemented API endpoints for retrieving and resetting provider rate limits. - Added types and schemas for provider rate limit configurations and statuses. - Enhanced rate limiter logic to support batch querying and telemetry for multiple providers. - Updated Redis key management to include new rate limit metrics. - Added unit tests for the new endpoints and rate limiter functionalities.
…prove environment variable handling - Bump version to 1.0.13 in package.json - Enhance README with updated CLI commands and version compatibility details - Add tests for version comparison and environment variable loading - Refactor env-config to differentiate between legacy and modern SimpleNS versions - Update docker-compose templates to support new versioning scheme - Modify onboarding process to allow skipping plugin configuration for SimpleNS > 1.3.0 - Improve security notices for auto-generated credentials based on version
… management features
- Introduced `api_key_id` field in notification schema for tracking. - Enhanced Redis subscriber service with error handling and reconnection logic. - Added API key schemas and types for creation, response, and usage tracking. - Implemented integration tests for API key management, including CRUD operations and usage tracking. - Updated admin authentication routes to use new API key logic. - Refactored auth middleware to support dynamic API keys and admin JWT authentication. - Added unit tests for JWT utilities and API key usage update functions. - Updated notification controller tests to accommodate new API key features. - Enhanced settings API tests to use admin JWT for authorization. - Updated MCP server tests to reflect changes in tool registration.
…CRET and adjust related logic
- Created health check routes: /api/health, /api/healthz (liveness), and /api/readyz (readiness). - Implemented health check logic in health.controller.js to assess MongoDB and Redis connectivity. - Integrated health probe server for Kubernetes with readiness and liveness checks. - Enhanced delayed and status consumers with health check capabilities, including crash detection and heartbeat monitoring. - Added unit tests for health endpoints and consumer health checks to ensure reliability.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3dd1ebd3a7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| PORT: <number>parseInt(process.env.PORT || "3000"), | ||
| NS_API_KEY: <string>(process.env.NS_API_KEY || ""), | ||
| MAX_BATCH_REQ_LIMIT: <number>parseInt(process.env.MAX_BATCH_REQ_LIMIT || "1000"), | ||
| JWT_SECRET: <string>(process.env.JWT_SECRET || "simplens-default-jwt-secret-key-change-in-production"), | ||
| DELAYED_PARTITION: <number>parseInt(process.env.DELAYED_PARTITION || "1"), |
There was a problem hiding this comment.
Do not sign admin tokens with this checked-in fallback. When JWT_SECRET is omitted, every deployment uses a publicly known HMAC key, allowing an attacker to mint a token with role: "admin" and access every protected endpoint. The supplied environment template leaves this value blank, and the compose configurations also inject the same fallback, so this affects the documented deployment path; fail startup unless a strong secret is provided.
Useful? React with 👍 / 👎.
| logger.warn(`Failed to update usage for API key ${keyDoc.key_id}`, { error: String(err) }); | ||
| }); | ||
|
|
||
| next(); |
There was a problem hiding this comment.
This accepts any active sns_ key and calls next() without attaching a role or scope. server.ts applies the same middleware to provider, plugin, settings, and key-management routes, so a key explicitly created for an external service can call GET /api/providers/:id?include_decrypted=true to obtain provider credentials, or modify operational configuration and install plugins. Require an admin JWT for management routes and reserve dynamic keys for the notification API.
Useful? React with 👍 / 👎.
| // Check if already configured | ||
| const existingDoc = await system_config_model.findOne({ key: ADMIN_CREDENTIALS_KEY }); | ||
| if (existingDoc?.value) { |
There was a problem hiding this comment.
On a fresh deployment this endpoint is public (server.ts mounts the admin-auth router without auth_middleware) and the only guard here is whether the database document already exists. Consequently, the first unauthenticated caller to the exposed API can create the administrator account and permanently lock out the intended operator. Gate setup behind a deployment-provided bootstrap secret, localhost-only binding, or another trusted initialization mechanism.
Useful? React with 👍 / 👎.
| try { | ||
| execSync(`npm install ${specifier}`, { | ||
| cwd: PLUGINS_DIR, | ||
| stdio: 'pipe', |
There was a problem hiding this comment.
execSync executes this interpolated string through a shell, while the install payload only requires a nonempty package string and the version-change endpoint performs no schema validation. A caller with access to plugin management can supply shell metacharacters in the package or version and execute commands in the API container. Use execFileSync/spawnSync with an argument array and validate npm package and version syntax before invoking npm.
Useful? React with 👍 / 👎.
| } else { | ||
| PluginRegistry.unregister(id); |
There was a problem hiding this comment.
Disabling a provider only unregisters it and leaves any channel routing that names it as the default unchanged. PluginRegistry.getDefaultProvider returns undefined for a configured-but-unregistered default instead of considering fallbacks, so disabling the primary from the dashboard causes subsequent notifications for that channel to fail with NO_PROVIDER even when fallback providers are configured. Update the routing atomically to promote a fallback, or make default resolution skip unavailable providers.
Useful? React with 👍 / 👎.
| </div> | ||
| </CardContent> | ||
| </Card> |
There was a problem hiding this comment.
The Recovery & Audits card ends after four inputs, but the new dynamic recovery schema also includes recovery_claim_timeout_ms, cleanup_resolved_alerts_retention_ms, and cleanup_processed_status_outbox_retention_ms. Administrators therefore cannot view or tune those operational fields from the required live settings UI, despite the new dynamic configuration feature. Add inputs and validation hints for all three fields.
AGENTS.md reference: AGENTS.md:L37-L40
Useful? React with 👍 / 👎.
|
|
||
| const elapsedSeconds = (now - lastRefill) / 1000; | ||
| const tokensToAdd = elapsedSeconds * normalizedRate; | ||
|
|
||
| return Math.min(currentTokens + tokensToAdd, config.maxTokens); | ||
| }; | ||
|
|
||
| interface ProviderBasicMeta { |
There was a problem hiding this comment.
ProviderBasicMeta is a newly introduced interface declared directly in src/, contrary to the repository's required type centralization. Move it to src/types/types.ts and import it here so the provider-status contract has a single canonical definition.
AGENTS.md reference: AGENTS.md:L24-L25
Useful? React with 👍 / 👎.
… retention feat(api-keys): introduce apiKeyType for notification and admin keys fix(docker): remove insecure default JWT_SECRET in docker-compose files feat(plugins): enhance plugin management with version change and uninstall validation fix(auth): restrict notification API keys from accessing management routes feat(jwt): implement JWT_SECRET validation for security during server startup feat(api-key): update API key model to include type and enforce validation test(api-keys): add tests for API key management access based on key type test(auth): enhance middleware tests for API key type restrictions test(plugin-fs): add unit tests for npm package name and version validation
This branch includes a lot of changes: