Repository navigation
feat: made simplens k8s native - #30
Conversation
…v, and master environments - Created docker-compose.dev.yaml for development environment with API server, worker, notification processor, delayed processor, recovery service, and admin dashboard. - Created docker-compose.infra.yaml for shared infrastructure services including MongoDB, Kafka, Redis, Loki, and Grafana. - Created docker-compose.local.yaml for local environment with build context for API server and dashboard. - Created docker-compose.master.yaml for master environment using latest images from GHCR. - Added scripts for switching between environments (switch-env.ps1 and switch-env.sh) with options for rebuilding images and pulling the latest versions.
…oss api and notification processor services
…rity across the codebase
…iguration support
…ion for channel routing
…min authentication - Added dynamic configuration service to manage operational settings with zero-restart updates. - Refactored environment variable handling to utilize dynamic settings for various worker parameters. - Updated system configuration model to support mixed types for dynamic settings. - Enhanced delayed and recovery processors to listen for dynamic configuration changes and adjust intervals accordingly. - Created integration tests for admin authentication routes and settings API. - Added unit tests for dynamic configuration schema validations and functionality.
…hancements - Added NpmAuthService to manage npm authentication tokens and registry configurations. - Introduced new API endpoints for retrieving, saving, and deleting npm auth configurations. - Enhanced plugin installation process to support private packages with authentication. - Implemented validation for SimpleNSProvider to ensure compliance with required methods and manifest structure. - Updated schemas for npm auth configuration and plugin installation payloads. - Added integration and unit tests for the new npm auth service and plugin validation logic.
…s and scopes - Updated API client to handle npm auth configurations with optional scope. - Modified types to include support for multiple npm registries and their statuses. - Enhanced the npm auth service to manage multiple tokens and registry configurations. - Updated controller methods to accommodate new parameters for saving and deleting npm auth configurations. - Improved tests to cover new functionality for multiple registries and scoped tokens.
…im and bulk insert for notifications
…g provider fallback - Introduced `rate_limit_retry_delay_ms` to operational settings for dynamic configuration. - Updated dashboard settings to allow administrators to configure rate limit retry delay. - Enhanced rate limiter to use fixed retry delay instead of staggered delay based on queue position. - Implemented cascading provider fallback mechanism in the unified consumer to handle rate-limited providers. - Added utility function for exponential backoff calculation. - Updated tests to cover new functionality and ensure proper behavior of rate limiting and fallback logic.
- Implemented API endpoints for retrieving and resetting provider rate limits. - Added types and schemas for provider rate limit configurations and statuses. - Enhanced rate limiter logic to support batch querying and telemetry for multiple providers. - Updated Redis key management to include new rate limit metrics. - Added unit tests for the new endpoints and rate limiter functionalities.
…prove environment variable handling - Bump version to 1.0.13 in package.json - Enhance README with updated CLI commands and version compatibility details - Add tests for version comparison and environment variable loading - Refactor env-config to differentiate between legacy and modern SimpleNS versions - Update docker-compose templates to support new versioning scheme - Modify onboarding process to allow skipping plugin configuration for SimpleNS > 1.3.0 - Improve security notices for auto-generated credentials based on version
… management features
- Introduced `api_key_id` field in notification schema for tracking. - Enhanced Redis subscriber service with error handling and reconnection logic. - Added API key schemas and types for creation, response, and usage tracking. - Implemented integration tests for API key management, including CRUD operations and usage tracking. - Updated admin authentication routes to use new API key logic. - Refactored auth middleware to support dynamic API keys and admin JWT authentication. - Added unit tests for JWT utilities and API key usage update functions. - Updated notification controller tests to accommodate new API key features. - Enhanced settings API tests to use admin JWT for authorization. - Updated MCP server tests to reflect changes in tool registration.
…CRET and adjust related logic
- Created health check routes: /api/health, /api/healthz (liveness), and /api/readyz (readiness). - Implemented health check logic in health.controller.js to assess MongoDB and Redis connectivity. - Integrated health probe server for Kubernetes with readiness and liveness checks. - Enhanced delayed and status consumers with health check capabilities, including crash detection and heartbeat monitoring. - Added unit tests for health endpoints and consumer health checks to ensure reliability.
… retention feat(api-keys): introduce apiKeyType for notification and admin keys fix(docker): remove insecure default JWT_SECRET in docker-compose files feat(plugins): enhance plugin management with version change and uninstall validation fix(auth): restrict notification API keys from accessing management routes feat(jwt): implement JWT_SECRET validation for security during server startup feat(api-key): update API key model to include type and enforce validation test(api-keys): add tests for API key management access based on key type test(auth): enhance middleware tests for API key type restrictions test(plugin-fs): add unit tests for npm package name and version validation
feat: made SimpleNS k8s-native
…proved configuration
Adhish-Krishna
changed the title
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c160f5e9c8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - simplens-infra | ||
| restart: unless-stopped | ||
| healthcheck: | ||
| test: [ "CMD", "node", "-e", "require('http').get('http://localhost:3000/api/health', (r) => process.exit(r.statusCode === 200 ? 0 : 1)).on('error', () => process.exit(1))" ] |
There was a problem hiding this comment.
With the shipped .env.example default BASE_PATH=/dashboard, the API mounts health at /dashboard/api/health, but this health check (and the equivalent dev/master checks) still probes /api/health. The API container will therefore remain unhealthy, preventing the dependent worker, processor, recovery, and dashboard services from starting under the documented Compose setup.
Useful? React with 👍 / 👎.
| environment: | ||
| NODE_ENV: production | ||
| PORT: ${DASHBOARD_PORT:-3002} | ||
| API_BASE_PATH_URL: http://api:{PORT:-3000}{BASE_PATH:-} |
There was a problem hiding this comment.
This production profile sets API_BASE_PATH_URL, but the dashboard middleware reads API_BASE_URL. Consequently, the master dashboard falls back to the env_file value (http://localhost:3000) or its built-in localhost default; inside the dashboard container that points to itself rather than the api service, so every proxied dashboard API request fails.
Useful? React with 👍 / 👎.
| await Promise.all([ | ||
| system_config_model.findOneAndUpdate( | ||
| { key: PUBLIC_KEY_CONFIG }, | ||
| { value: keyPair.publicKey }, | ||
| { upsert: true, new: true } |
There was a problem hiding this comment.
When two first-time provider operations or API instances call this concurrently, both can observe missing key documents and generate different pairs. The two independent upserts can interleave, leaving the public key from one generated pair and the private key from another; credentials encrypted with either public key then become undecryptable after a restart. Persist the pair atomically or serialize/bootstrap it with a lock.
Useful? React with 👍 / 👎.
| // Find channel from installed plugin manifest | ||
| const pluginDoc = await Plugin.findOne({ name: entry.package }); | ||
| const channel = pluginDoc?.manifest?.channel || 'generic'; | ||
|
|
||
| await Provider.findOneAndUpdate( |
There was a problem hiding this comment.
If npm installation or manifest extraction fails above, migration only logs the failure, then this loop still creates an enabled provider using a generic channel and a plugin record that does not exist. Because future startups see a nonzero provider count, they skip YAML migration entirely, leaving that configured provider permanently unloadable unless an operator manually repairs or deletes it.
Useful? React with 👍 / 👎.
| PluginRegistry.setChannelConfig(routing.channel, { | ||
| default: remainingFallbacks[0] || '', | ||
| fallback: remainingFallbacks.slice(1), | ||
| }); |
There was a problem hiding this comment.
When deleting a provider that is only a fallback, the database update keeps routing.default_provider_id, but this registry update instead promotes the first remaining fallback (or clears the default). The originating process ignores its own sync event, so until a later reload it validates provider-unspecified requests against and routes through a different default than the persisted channel configuration.
Useful? React with 👍 / 👎.
| interface StoredRegistryEntry { | ||
| id: string; | ||
| scope?: string; | ||
| registry_url: string; | ||
| encrypted_token: encrypted_config; |
There was a problem hiding this comment.
Move these local npm-auth interface declarations (including StoredNpmAuthConfig and DecryptedRegistryEntry) into src/types/types.ts and import them here. The repository rule requires every interface and type inside src/ to have that single declaration location, so retaining these definitions splits the mandated type source of truth.
AGENTS.md reference: AGENTS.md:L24-L25
Useful? React with 👍 / 👎.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…PATH for improved flexibility
…mprove service definitions
The changes include: