buckt

Buckt sits inside your Go application and manages media workflows, delegating the actual bytes to whatever backend you point it at. MinIO answers “where should the bytes be stored?” Buckt answers “how should my application manage media?” You can run both: Buckt happily uses MinIO as a backend.

┌─────────────────────────────────────────────────────────────┐
│                      Your Application                       │
└──────────────────────────┬──────────────────────────────────┘
                           │
                  ┌────────▼─────────┐
                  │   buckt.Client   │  ◄── direct API or web client
                  └────────┬─────────┘
            ┌──────────────┼──────────────┐
            │              │              │
       ┌────▼────┐   ┌─────▼─────┐  ┌─────▼──────┐
       │ Folder  │   │   File    │  │   Cache    │
       │ Service │   │  Service  │  │  Manager   │
       └────┬────┘   └─────┬─────┘  └────────────┘
            │              │
       ┌────▼──────────────▼─────┐
       │   Repository (GORM)     │  ◄── SQLite or Postgres
       └────┬────────────────────┘
            │
       ┌────▼─────────────────────────────────────┐
       │            FileBackend                   │
       │  ┌──────┐ ┌─────┐ ┌─────┐ ┌──────┐ ┌──┐  │
       │  │Local │ │ S3  │ │ GCS │ │Azure │ │R2│  │
       │  └──────┘ └─────┘ └─────┘ └──────┘ └──┘  │
       └──────────────────────────────────────────┘

Install

go get github.com/Rhaqim/buckt

Cloud backends and the web UI are separate Go modules, so the core stays lean. Pull in only the SDKs you actually use.

go get github.com/Rhaqim/buckt/cloud/aws     # S3 + Cloudflare R2
go get github.com/Rhaqim/buckt/cloud/gcp     # Google Cloud Storage
go get github.com/Rhaqim/buckt/cloud/azure   # Azure Blob Storage
go get github.com/Rhaqim/buckt/client/web    # Web UI + HTTP API

Quick start

package main

import (
  "log"
  "github.com/Rhaqim/buckt"
)

func main() {
  client, err := buckt.Default()
  if err != nil {
    log.Fatal(err)
  }
  defer client.Close()

  fileID, err := client.UploadFile(
    "user123",    // owner
    "",           // parent folder ID (empty = root)
    "hello.txt",
    "text/plain",
    []byte("hello world"),
  )
  if err != nil {
    log.Fatal(err)
  }

  file, _ := client.GetFile(fileID)
  log.Println(string(file.Data))
}

Files land in ./media, metadata in ./db.sqlite. No extra setup.

Configuration

Functional options. Everything is optional, Default() gives you working defaults.

client, err := buckt.Default(
  buckt.WithLog(buckt.LogConfig{LogTerminal: true, LogFile: "logs"}),
  buckt.WithDB(buckt.Postgres, sqlDB),
  buckt.WithBackend(s3Backend),
  buckt.WithMaxFileSize(buckt.DefaultMaxFileSize),
  buckt.MediaDir("./uploads"),
  buckt.FlatNameSpaces(true),
)

Storage backends

The FileBackend interface lets you swap providers without touching application code.

import (
  "github.com/Rhaqim/buckt"
  "github.com/Rhaqim/buckt/cloud/aws"
)

s3, err := aws.NewBackend(aws.Config{
  AccessKey: "AKIA...",
  SecretKey: "...",
  Region:    "us-east-1",
  Bucket:    "my-bucket",
})
if err := s3.Ping(ctx); err != nil { /* fail fast on bad credentials */ }

client, _ := buckt.Default(buckt.WithBackend(s3))

R2 is S3-compatible, so the cloud/aws backend auto-detects it from the endpoint suffix and switches to path-style addressing. Leave Region empty; it defaults to auto.

r2, err := aws.NewBackend(aws.Config{
  AccessKey: "your-r2-access-key",
  SecretKey: "your-r2-secret",
  Bucket:    "my-bucket",
  Endpoint:  "https://<ACCOUNT_ID>.r2.cloudflarestorage.com",
})

Every cloud backend exposes Ping(ctx). Call it after NewBackend so credential and network problems surface at startup instead of on the first upload.

Live migration

Moving from local disk to S3, or S3 to R2, without downtime. Migration mode dual-writes, reads from both, and lazily migrates forward on read.

              ┌───────────┐    Put   ┌──────────────┐
   Write ────►│  Primary  ├─────────►│  Secondary   │
              │ (current) │          │   (target)   │
              └─────┬─────┘          └──────┬───────┘
                    │                       │
                    │  Get: try primary,    │
                    │  fall back to         │
                    │  secondary if missing │
                    │                       │
                    └─────────► File ◄──────┘

s3, _ := aws.NewBackend(s3Config)

client, _ := buckt.Default(buckt.WithMigration(buckt.MigrationConfig{
  From:        buckt.LocalBackend(), // current source of truth
  To:          s3,                   // target
  Concurrency: 16,                   // default 8
}))

Migration is always forward. The primary is the source of truth and is never overwritten by secondary content.

Dual-write only mirrors new activity. To copy everything that predates the cutover, call MigrateAll and poll:

if err := client.MigrateAll(ctx); err != nil {
  log.Fatal(err) // ErrBackendUnavailable if not built WithMigration
}

for {
  done, total, _ := client.MigrationStatus(ctx)
  log.Printf("migrated %d/%d", done, total)
  if total > 0 && done >= total {
    break
  }
  time.Sleep(time.Second)
}

if failed, _ := client.MigrationFailures(ctx); failed > 0 {
  log.Printf("%d file(s) failed after retries; fix the cause and re-run", failed)
}

MigrateAll is resumable and idempotent: each copied object is recorded in buckt_migration_models, so a restarted migration skips what it already did rather than re-scanning the target. Each in-flight file is buffered in full, so higher concurrency trades memory and provider rate-limit headroom for throughput.

client.BackendName() reports the active backend: "local", "s3", or "local->s3" mid-migration. Once you’re done, drop migration mode:

client, _ := buckt.Default(buckt.WithBackend(s3))

Image derivatives

Generate resized variants from uploads. The built-in processor is pure Go and handles JPEG and PNG with no external dependencies.

client, _ := buckt.Default(
  buckt.WithImageDerivatives(
    buckt.DerivativeSpec{Name: "thumbnail", MaxWidth: 200},
    buckt.DerivativeSpec{Name: "medium", MaxWidth: 800},
  ),
)

_ = client.GenerateDerivatives(fileID)

data, contentType, err := client.GetDerivative(fileID, "thumbnail")

WebP lives in its own module so the core stays dependency-free:

import "github.com/Rhaqim/buckt/imageproc/webp"

client, _ := buckt.Default(
  buckt.WithImageProcessor(webp.New()),
  buckt.WithImageDerivatives(
    buckt.DerivativeSpec{Name: "thumbnail", MaxWidth: 200, Format: "webp"},
  ),
)

Resizing runs inline with GenerateDerivatives. For heavy workloads, call it from an event handler that enqueues to a worker rather than blocking the upload.

Events and scanning

Lifecycle handlers run synchronously after the operation commits. Keep them fast. The intended pattern is enqueue-and-return. A panicking handler is recovered and never fails the originating call.

import "github.com/Rhaqim/buckt/pkg/events"

onEvent := func(ctx context.Context, e events.Event) {
  if e.Type == events.FileUploaded {
    _ = client.GenerateDerivatives(e.FileID)
  }
}

client, _ := buckt.Default(buckt.WithEventHandler(onEvent))

Events fire after the write, so they can’t block an upload. To reject a file before it’s stored, register a scanner. Buckt ships no scanning engine by design. You supply one (ClamAV, VirusTotal, a content-type allowlist).

import "github.com/Rhaqim/buckt/pkg/scan"

scanner := scan.ScannerFunc(func(ctx context.Context, name string, data []byte) error {
  return clamav.Scan(ctx, data) // non-nil rejects the upload
})

client, _ := buckt.Default(buckt.WithUploadScanner(scanner))

_, err := client.UploadFile(userID, "", "invoice.pdf", "application/pdf", data)
if errors.Is(err, buckt.ErrUploadRejected) {
  // err also wraps the scanner's own reason
}

The scanner runs at Buckt’s single upload chokepoint, so every upload path is covered. A caller can’t forget to wire it in per call site.

Dedup, metadata, metrics

With WithDedup(), an upload whose bytes hash-match a file already in the same target folder for the same owner returns the existing file’s ID instead of writing the blob again. Scoped to the folder, so it composes with nested namespaces and never resurrects a trashed duplicate. Off by default.

client, _ := buckt.Default(buckt.WithDedup())

Arbitrary key/value metadata attaches to any file, stored as JSON on the record:

_ = client.SetFileMetadata(fileID, map[string]string{
  "source": "web-ui",
  "album":  "vacation-2026",
})

meta, _ := client.GetFileMetadata(fileID)

Metrics use a pluggable recorder; the built-in collector has no dependencies and costs nothing when unused.

import "github.com/Rhaqim/buckt/pkg/metrics"

collector := metrics.NewCollector()
client, _ := buckt.Default(buckt.WithMetrics(collector))

snap := collector.Snapshot() // map[backend]map[operation]metrics.Stat

Each Stat holds Count, Errors, Bytes and TotalDur. Implement the one-method metrics.Recorder interface to forward to Prometheus, StatsD, or your own sink.

Trash and deletion

The trash folder is a real folder hidden from normal listings, and hierarchy is preserved when items are trashed. Calling DeleteFile on something already in trash hard-deletes it, giving you an “empty trash” mechanism with no extra API.

trash, _ := client.GetTrashFolder("user123")

client.MoveFile(fileID, originalParentID) // restore
client.DeleteFile(fileID)                 // already trashed → hard delete

In non-flat namespace mode, deleted items are physically moved on the backend so paths stay consistent, not just renamed in the database.

Web client

An optional Gin-based HTTP API and Tailwind UI, in its own module.

import (
  "github.com/Rhaqim/buckt"
  web "github.com/Rhaqim/buckt/client/web"
)

func main() {
  bucktClient, _ := buckt.Default()
  defer bucktClient.Close()

  router, _ := web.NewClient(bucktClient)
  router.Run(":8080")
}

The UI gives you breadcrumb navigation, image/video/audio/PDF previews, drag-and-drop moves, inline rename, a folder browser for move targets, and both delete modes.

API reference

// Folders
NewFolder(userID, parentID, name, description string) (string, error)
ListFolders(folderID string) ([]FolderModel, error)
GetFolderWithContent(userID, folderID string) (*FolderModel, error)
GetTrashFolder(userID string) (*FolderModel, error)
MoveFolder(userID, folderID, newParentID string) error
RenameFolder(userID, folderID, newName string) error
DeleteFolder(folderID string) (string, error)                     // → trash
DeleteFolderPermanently(userID, folderID string) (string, error)  // → hard delete

// Files
UploadFile(userID, parentID, name, contentType string, data []byte) (string, error)
UploadFileFromReader(userID, parentID, name, contentType string, r io.Reader) (string, error)
GetFile(fileID string) (*FileModel, error)
GetFileStream(fileID string) (*FileModel, io.ReadCloser, error)
ListFiles(folderID string) ([]FileModel, error)
ListFilesMetadata(folderID string) ([]FileModel, error)
MoveFile(fileID, newParentID string) error
DeleteFile(fileID string) (string, error)              // → trash
DeleteFilePermanently(fileID string) (string, error)   // → hard delete

// Metadata
SetFileMetadata(fileID string, metadata map[string]string) error
GetFileMetadata(fileID string) (map[string]string, error)

// Derivatives
GenerateDerivatives(fileID string) error
GetDerivative(fileID, name string) (data []byte, contentType string, err error)

// Migration (only WithMigration)
MigrateAll(ctx context.Context) error
MigrationStatus(ctx context.Context) (completed, total int64, ok bool)
MigrationFailures(ctx context.Context) (failed int64, ok bool)
BackendName() string

Every method has a *Context variant taking an explicit context.Context.

Branch on failures with errors.Is using the re-exported sentinels:

Security defaults

The repo also runs gitleaks as a pre-commit hook and in CI, so credentials never land in git.