BE-11: File Uploading and Object Storage Integration

[tecnodeveloper]

Implement file upload functionality for book-related assets (e.g., book images, review media) and store them in an object storage system. The solution should prioritize open-source, actively maintained alternatives to MinIO and include Docker-based orchestration.

User Story

Given a user uploads a file
When the request is processed
Then the file should be stored in object storage and accessible via URL

Goals

• Enable file uploads via API
• Store files in scalable object storage
• Avoid deprecated/unmaintained tools
• Provide a containerized setup using Docker Compose

Research Context

• Investigate object storage solutions

• Avoid MinIO (archived / maintenance mode as of 2026) ([alarik.io][1])

• Evaluate open-source alternatives:

  • Ceph (highly scalable, production-grade)
  • Garage (lightweight, S3-compatible)
  • SeaweedFS (good for large-scale file systems)
  • Alarik (modern, actively developed alternative) ([OpenAlternative][2])

• Compare trade-offs:

  • Complexity vs simplicity
  • Resource usage
  • S3 compatibility

• Document decision (ADR or README section)

Architecture Overview

• API (Node.js + Express)
• File upload middleware
• Object storage service (S3-compatible preferred)
• Docker Compose orchestration

Tasks

File Upload API

• Create file upload endpoint:

  POST /upload
  

• Accept:

  • Book images
  • Review media (optional)

• Validate file type (images/videos only)

• Validate file size limits

• Handle upload errors

Upload Handling

• Integrate file upload middleware:

  • multer (or equivalent)

• Store files temporarily before upload

• Generate unique file names

• Upload files to object storage

• Return file URL in response

Object Storage Integration

• Choose storage provider (after research)

• Ensure S3-compatible API (preferred)

• Configure SDK (e.g., AWS SDK for S3-compatible storage)

• Implement upload service:

  • Upload file
  • Get public/private URL

• Handle failures and retries

Docker Setup

• Create docker-compose.yml

• Add services:

  • API service
  • Object storage service (chosen alternative)

• Configure volumes for persistence

• Configure environment variables

• Ensure services communicate via network

Swagger Documentation

• Document /upload endpoint
• Add file upload schema (multipart/form-data)
• Add response examples

Project Structure

• Add upload-related modules:

  • /services/storage.service.js
  • /middlewares/upload.middleware.js
  • /routes/upload.routes.js

Acceptance Criteria

• File upload endpoint works

• Files stored in object storage

• Returned URL is accessible

• Invalid files rejected (type/size)

• Docker setup runs successfully:

  docker-compose up

• Services communicate correctly

• Swagger supports file upload testing

Testing Steps

• Run project with Docker
• Open Swagger UI
• Upload file via /upload
• Verify response contains file URL
• Check object storage:
  • File exists
• Access file via URL

Definition of Done

• Upload API implemented
• Object storage integrated
• Docker Compose working
• Research documented
• Swagger updated
• Code linted and formatted

Optional (Nice to Have)

• Generate thumbnails for images
• Add signed URLs (private access)
• Add file deletion endpoint
• Organize files by folder (e.g., /books/:id/)

---

Suggested Approach (Practical Tip)

For your level/project:

• Start with Garage (lightweight) or Alarik (modern & active)
• Avoid Ceph initially (too heavy for learning project)

[cyphernetsecurity]

This covers the upload and storage flow well.

In many similar pipelines, once a file passes size and mime-type checks, it’s treated as safe when stored and exposed via URL. That effectively makes the storage layer the trust boundary.

That leaves a gap where files are accepted but not actually validated beyond basic constraints, which can impact downstream services consuming them.

Is there any validation step planned between upload and storage to inspect file contents before exposure?

[tecnodeveloper]

This covers the upload and storage flow well.

In many similar pipelines, once a file passes size and mime-type checks, it’s treated as safe when stored and exposed via URL. That effectively makes the storage layer the trust boundary.

That leaves a gap where files are accepted but not actually validated beyond basic constraints, which can impact downstream services consuming them.

Is there any validation step planned between upload and storage to inspect file contents before exposure?

Good point — currently the implementation focuses on MIME type and size validation, which does leave a gap in content-level verification.

I plan to add an additional validation step before storage, such as:

• verifying file signatures (magic bytes)
• parsing images to ensure they are not corrupted
• optionally integrating a file scanning mechanism

This would ensure that only genuinely valid and safe files are stored and exposed via URL.

[cyphernetsecurity]

That’s a solid direction.

In similar pipelines, once files are stored and exposed via URL, they often get reused across multiple services (processing, rendering, integrations).

At that point, even small inconsistencies in validation can propagate much further than expected.

Are you planning to keep validation strictly at upload time, or also re-validate when files are consumed downstream?

[tecnodeveloper]

Practical Positioning
1. Choose Ceph if u need
- Enterprise-grade storage

• Full S3 semantics
• Multi-region / compliance
• Ops expertise
• Dedicated infra

2. Choose Garage if You want:

• Simplicity
• Low resource usage
• Basic S3 API
• You don’t need advanced S3 features

3. Choose SeaweedFS if You want:

• Massive file counts
• Media / blob-heavy workloads
• You accept:
• Some S3 edge-case limitations

4. Consider Alarik if:
   You’re experimenting / early adopting
   Not for critical production (yet)

[cyphernetsecurity]

Makes sense, the trade offs between those options are pretty clear depending on scale and constraints.

The storage choice definitely sets the foundation, once files start getting reused across different services.