Verbose scrubs secrets from Go logs before they reach disk

Verbose is a Go logging package with one job: keep secrets out of your log files. Every string that passes through its Printf, Println or related calls is scanned against a registry of registered secrets plus a list of known credential formats, and matches are replaced before the line reaches disk.

The project began in 2024 as a solution for a payment and communications platform whose customers' sensitive data only became known at runtime — mid-request, from a database row or a live OAuth exchange. Most logging packages assume you configure redaction once at startup. Verbose assumes the opposite: secrets are registered the instant they become known, from any goroutine, with no restart.

How registration works

Secrets are stored as SHA-512 digests, never plaintext. AddSecret hashes a value and stores the digest together with a replacement string:

err := verbose.AddSecret(verbose.SecretBytes(os.Getenv("DB_PASSWORD")), "[DB_PASSWORD]")

The plaintext is gone once the call returns — what persists in the registry is the 128-hex-character SHA-512 digest, its replacement string and the original byte length. That length matters: sanitizeInput uses it to size a sliding window when scanning log lines, so it never needs the plaintext back.

Because secrets are stored as hashes, they can be imported from an external store without ever touching memory in plain form:

n, err := verbose.ImportSecrets(map[string]int{
    "abc123...128hexchars...": 32,
})

RemoveSecret purges a digest when it is no longer needed. All of this is guarded by read/write mutexes and documented as race-free for concurrent use.

There is one guardrail: SecretMinLength, defaulting to 5 bytes, below which registration fails. Shorter secrets would produce too many false-positive matches during scanning.

Pattern-based cleaning

Beyond registered secrets, Verbose ships a built-in list of KeyType boundary pairs that Scrub uses to replace whole regions with the literal string [CLEANED]. The list covers PEM blocks (RSA, EC, DSA, OPENSSH, CERTIFICATE), PGP messages and signatures, JWTs (eyJhbGci...), GitHub (ghp_) and GitLab (glpat-) tokens, Stripe sk_live_, AWS credentials, Docker auth configs, Kubernetes configs, GCP service accounts, Vault AppRole secrets and Azure connection strings.

Custom patterns can be added at runtime:

verbose.AddKeyType(verbose.KeyType{
    Opening: "BEGIN_MY_SECRET",
    Closing: "END_MY_SECRET",
})

Scrub also runs a Rinse pass that strips ANSI escape codes. The code documentation for AddKeyType states the trade-off plainly: each registered pattern adds one full pass over every log line, so callers should register only what they actually need.

The logging surface

The package exposes two families of functions. Sanitized ones redact before writing:

• Printf / Println / Print, plus explicit aliases Hide / Hidef
• Trace / Tracef, which append a full stack trace
• Return / Returnf, which log and return the message as an error
• To / Tof, which write sanitized output to any caller-supplied *log.Logger

Unsanitized ones bypass redaction entirely: Plain, Raw, Expose (and their format variants), for debugging the sanitizer itself or deliberately logging raw values.

The package-level logger is created once at startup:

err := verbose.NewLogger(verbose.Options{
    Truncate: true,
    Dir:      "/var/log/myapp",
    Name:     "myapp",
    FileMode: 0644,
    DirMode:  0755,
})

NewLogger opens a file under Dir, defaulting to ./logs/verbose.log. Exported functions that write through the package logger call an internal guard() first, so calling them before the logger exists prints to stderr rather than panicking on a nil pointer; registration and key-setting calls such as AddSecret, RemoveSecret and SetKey do not. The error-returning variants Return / TraceReturn return the guard error instead of printing it. A separate SetLogger accepts a custom-built *Logger for callers who want their own io.Writer.

Encryption and stack traces

Two supporting pieces round out the package. SecureBytes wraps a byte slice with AES-GCM authenticated encryption — .Encrypt() turns it to ciphertext in place, .Decrypt() restores it, and SetKey accepts 16-, 24- or 32-byte keys. A default key is generated randomly at init.

The stack-trace functions build on a small StackFrame type that captures file, line, function name and the source line content when readable, walking frames via runtime.Caller.

Performance

Verbose's scan algorithm is intentionally O(n²) in input length: every substring position must be checked against the registry. The README publishes benchmarks measured on an Apple M3 Ultra with -race:

| Input size | ns/op | |---|---| | 0 bytes | 1.4 (zero allocs, fast path) | | 10 bytes | 11,394 | | 80 bytes | 72,649 | | 640 bytes | 1,580,327 |

The empty-input fast path costs nothing because sanitizeInput bails out before any allocation when no secrets are registered or the input is shorter than the shortest secret. For typical log lines under 200 bytes the overhead is described as imperceptible; for inputs over a kilobyte, sanitizeInput's documentation suggests considering whether the whole string needs to pass through.

Setup and testing

The repository ships a Makefile with targets for build, vet, tests (test, test-race, test-repeat), benchmarks and fuzzing. Fuzz targets cover scrub, sanitize, SHA-512 and encrypt paths under FUZZ_TIME. A fuzz_test.go accompanies the package, and per-file test suites accompany the main Go source files — though trace.go has no dedicated top-level trace_test.go.

Verbose is a focused tool with an unusual design bet: hash-based secret storage that keeps plaintext out of memory entirely, plus runtime registration from any goroutine. It is Apache 2.0 licensed at version 0.2.0, requires Go 1.23.4 or later, and installs with go get -u github.com/andreimerlescu/verbose.