Skip to content

feat(db): introduce new seaweedfs object store usage for e2e - #79

Merged
saswatamcode merged 1 commit into
efficientgo:mainfrom
Saumya40-codes:introduce-seaweedfs-s3-storage
Sep 27, 2026
Merged

saswatamcode merged 1 commit into
efficientgo:mainfrom
Saumya40-codes:introduce-seaweedfs-s3-storage

Conversation

@Saumya40-codes

Copy link
Copy Markdown
Contributor

introduces support for seaweedfs object storage helper which now can be used as a replacement for the minio, implementation I've ported from is this pr thanos-io/thanos#9037 from thanos so it can be used directly via e2e where its needed.

@saswatamcode saswatamcode left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice generally LGTM, some comments!

Comment thread db/db.go Outdated
Comment thread db/db.go Outdated
fmt.Sprintf("-s3.port=%d", ports[AccessPortName]),
"-master.telemetry=false",
"-webdav=false",
"-s3.port.iceberg=0",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given some of the wider community efforts, might be good to make iceberge into an option

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exposed WithSeaweedFSIceberg() so it can be used if needed.

Comment thread db/db.go
}

func NewSeaweedFS(env e2e.Environment, name, bucket string, opts ...Option) *e2emon.InstrumentedRunnable {
o := options{image: "chrislusf/seaweedfs:4.47"}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the official image for it?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Saumya40-codes
Saumya40-codes force-pushed the introduce-seaweedfs-s3-storage branch from ecae0e3 to 2ddb232 Compare September 25, 2026 04:23
@saswatamcode

Copy link
Copy Markdown
Collaborator

@Saumya40-codes can you fix the lint? Will merge then

Signed-off-by: Saumya Shah <saumyabshah90@gmail.com>

refactor a bit make a generalized constant for s3 secret and access key

Signed-off-by: Saumya Shah <saumyabshah90@gmail.com>

defer close idle conn in ready() check and add -master.telemetry=false so non-related to e2e stuff doesnt start

Signed-off-by: Saumya Shah <saumyabshah90@gmail.com>

adress pr review changes, add optional iceberg port expose option

Signed-off-by: Saumya Shah <saumyabshah90@gmail.com>

update deps in examples/thanos and lint changes

Signed-off-by: Saumya Shah <saumyabshah90@gmail.com>
@Saumya40-codes
Saumya40-codes force-pushed the introduce-seaweedfs-s3-storage branch from 2ddb232 to 93acc07 Compare September 25, 2026 17:53
Comment thread db/db.go

ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
exists, err := client.BucketExists(ctx, p.bucket)

@Saumya40-codes Saumya40-codes Sep 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think here another viable way could've been hitting /buckets/<name>/ on the filer port → 200 exists, 404 doesn't (seaweedfs/seaweedfs#11461 (comment)) , though ig right now current one with using S3 API directly works right as well.

@saswatamcode
saswatamcode merged commit a3483d5 into efficientgo:main Sep 27, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants