Skip to main content
Hashnode
Rashik's blog

Part 13: Protecting the login endpoint: layered rate limiting

Updated 28 min readMarkdown
R
Rashik Ansar Shaik
Part of seriesBuild a production-grade REST API in Go

An attacker doesn't have to break bcrypt if they can simply try a million passwords. We add three more rate limits, each defeating a different attack, and look hard at the one that can be turned against us.

Series: Build a production-grade REST API in Go · Part 13 of 17 Where we are: the authenticated, authorized API from Part 12.

In Part 10 we limited requests per IP address. That's one layer, and the login endpoint needs more, because the threats are specific:

Attack What the attacker does Why per-IP limits aren't enough
Brute force tries many passwords for one account one IP is easy to limit...
Credential stuffing tries leaked email/password pairs from other breaches against your site ...but with a botnet of 10,000 IPs each makes one attempt: every IP is under the limit
Account spam mass-registers accounts same: spread over many IPs
One heavy user a logged-in client hammers the API many people share one IP (office, mobile carrier NAT), so IP is a poor identity for them

So we add layers, each keyed on the right thing:

Layer Key Where Protects against
1. IP (all requests) client IP outermost middleware (Part 10) floods, crude scrapers
2. IP (credential endpoints) client IP /auth/login, /register, /refresh guessing, signup spam, from one source
3. User user id after authentication one logged-in client hogging the service
4. Account (failures only) the email being attacked inside Login distributed brute force of one account

A request passes through them in this order, and any layer can stop it with a 429:

Layer 3: per-user limits

Once we know who the caller is, user id is the fair key. Here is the wiring in main: a small newLimiter helper, two key functions (byIP, byUser), and one limit(...) helper that turns a limiter plus a key function into middleware and counts rejections in metrics:

cmd/rest/main.go (lines 213-255)

	// Rate limiting, in layers (cheapest check first, most specific last):
	//   1. per client IP, every request          (outer net, runs before auth)
	//   2. per client IP, credential endpoints   (strict; stops guessing/signup spam)
	//   3. per authenticated user                (fair quota; people behind one NAT don't share)
	//   4. per account, failed logins only       (inside the login handler)
	trusted, err := ratelimit.ParseCIDRs(cfg.TrustedProxies)
	if err != nil {
		return fmt.Errorf("TRUSTED_PROXIES: %w", err)
	}
	newLimiter := func(rps float64, burst int) *ratelimit.Limiter {
		l := ratelimit.New(rps, burst, 10*time.Minute, 100_000)
		go l.Run(ctx.Done()) // evicts idle keys until shutdown
		return l
	}
	byIP := func(r *http.Request) string { return ratelimit.ClientIP(r, trusted) }
	byUser := func(r *http.Request) string {
		if p, ok := auth.FromContext(r.Context()); ok {
			return "u:" + strconv.Itoa(p.UserID)
		}
		return "" // not authenticated: skip (Authenticate already answered 401)
	}
	limit := func(name string, l *ratelimit.Limiter, key func(*http.Request) string) func(http.Handler) http.Handler {
		return middleware.RateLimit(middleware.RateLimitOptions{
			Limiter: l, Key: key, OnReject: func(*http.Request) { metrics.RateLimited(name)() },
		})
	}

	var limiters []func(http.Handler) http.Handler
	if cfg.RateLimitRPS > 0 {
		limiters = append(limiters, limit("ip", newLimiter(cfg.RateLimitRPS, cfg.RateLimitBurst), byIP))
	}
	var userLimit, authLimit func(http.Handler) http.Handler
	if cfg.UserRateLimitRPS > 0 {
		userLimit = limit("user", newLimiter(cfg.UserRateLimitRPS, cfg.UserRateLimitBurst), byUser)
	}
	if cfg.AuthRateLimitPerMin > 0 {
		authLimit = limit("auth_ip", newLimiter(cfg.AuthRateLimitPerMin/60, cfg.AuthRateLimitBurst), byIP)
	}
	authHandler := handlers.NewAuthHandler(userStore, storage.NewTokenStore(pool), hasher, tm, cfg.RefreshTokenTTL, log, limits)
	if cfg.LoginMaxFailures > 0 {
		failures := newLimiter(1/cfg.LoginFailureRefill.Seconds(), cfg.LoginMaxFailures)
		authHandler.WithLoginGuard(failures, metrics.RateLimited("login_account"))
	}

The crucial detail is where the user limiter runs. It is added after Authenticate:

internal/routes/routes.go (excerpt)

// SetupRoutes registers every endpoint. Reading this file answers "which routes
// are public?": only those registered with `public`.
func SetupRoutes(mux *http.ServeMux, d Deps) {
	userLimit, authLimit := orIdent(d.UserLimit), orIdent(d.AuthLimit)

	public := func(pattern string, h http.HandlerFunc) { mux.Handle(pattern, h) }
	// The order matters: Authenticate first, so UserLimit knows WHO to count and
	// unauthenticated requests (401) never consume anyone's budget.
	protected := func(pattern string, h http.HandlerFunc) {
		mux.Handle(pattern, d.Authenticate(userLimit(h)))
	}

	// Credential endpoints get the strict limiter: they are the ones attackers hammer.
	mux.Handle("POST /auth/register", authLimit(http.HandlerFunc(d.Auth.Register)))
	mux.Handle("POST /auth/login", authLimit(http.HandlerFunc(d.Auth.Login)))
	mux.Handle("POST /auth/refresh", authLimit(http.HandlerFunc(d.Auth.Refresh)))
	public("POST /auth/logout", d.Auth.Logout)
	protected("GET /auth/me", d.Auth.Me)

	protected("GET /users", d.Users.List) // admin only (checked in the handler)
	protected("GET /users/{id}", d.Users.Get)
	protected("PUT /users/{id}", d.Users.Update)
	protected("DELETE /users/{id}", d.Users.Delete)

	protected("GET /orders", d.Orders.List)
	protected("POST /orders", d.Orders.Create)
	protected("GET /orders/{id}", d.Orders.Get)
	protected("DELETE /orders/{id}", d.Orders.Delete)
}

protected := func(pattern string, h http.HandlerFunc) {
    mux.Handle(pattern, d.Authenticate(userLimit(h)))
}

Order matters: Authenticate runs first, so the limiter knows who to count; and unauthenticated requests are answered 401 before they ever reach it, so anonymous junk can never use up a real user's budget. (If the limiter ran first, it would have no user id to key on.)

Notice also that the IP limit (layer 1) and the user limit (layer 3) coexist: IP is the outer net, so set it generously enough to cover everyone behind a shared address; the user limit does the fine-grained work.

Layer 2: a strict limit on the credential endpoints

Login, register and refresh are the endpoints attackers hammer, and a legitimate human uses them rarely. So they get a much smaller budget: 10 per minute with a burst of 5, per IP:

authLimit = limit("auth_ip", newLimiter(cfg.AuthRateLimitPerMin/60, cfg.AuthRateLimitBurst), byIP)

Cheap, effective against the naive attacker, and does nothing against the botnet. For that we need layer 4.

Layer 4: per-account failed-login throttling

If the attacker rotates IPs but always attacks victim@example.com, the thing that stays constant is the account. So we count failures by email, from any IP.

Our token bucket needs a twist. For request limiting we spend a token on every request. For login protection we want to count only failures: a successful login must not burn the budget, or a user who logs in often would lock themselves out. So the limiter gained three methods that use the bucket as a failure counter:

internal/ratelimit/ratelimit.go (excerpt)

// Peek returns how long until key has a token available (0 = not limited).
// It consumes nothing.
func (l *Limiter) Peek(key string) time.Duration {
	now := l.now()
	l.mu.Lock()
	e, found := l.entries[key]
	l.mu.Unlock()
	if !found {
		return 0
	}
	r := e.lim.ReserveN(now, 1)
	d := r.DelayFrom(now)
	r.CancelAt(now)
	return d
}

// Spend consumes one token (records one failure).
func (l *Limiter) Spend(key string) {
	now := l.now()
	if e, ok := l.get(key, now); ok {
		e.lim.AllowN(now, 1)
	}
}

// Reset forgets key entirely (the bucket is full again).
func (l *Limiter) Reset(key string) {
	l.mu.Lock()
	delete(l.entries, key)
	l.mu.Unlock()
}
  • Peek: how long until this key has a token? (consumes nothing)
  • Spend: record one failure.
  • Reset: forget the key (a correct password clears the count).

Now the login handler uses them:

internal/handlers/auth.go (excerpt)

// LoginGuard counts FAILED logins per account (see ratelimit.Limiter Peek/Spend/Reset).
type LoginGuard interface {
	Peek(key string) time.Duration // 0 = allowed, else how long to wait
	Spend(key string)              // record one failure
	Reset(key string)              // successful login: forget failures
}

func (h *AuthHandler) Login(w http.ResponseWriter, r *http.Request) {
	var in loginInput
	if !h.decodeJSON(w, r, &in) {
		return
	}
	email, ok := normaliseEmail(in.Email)
	// Every failure below returns the SAME message and takes similar time, so a
	// caller cannot tell "no such account" from "wrong password".
	invalid := func() { h.writeError(w, r, http.StatusUnauthorized, "invalid credentials") }
	if !ok || len(in.Password) > auth.MaxPasswordLen {
		h.hasher.BurnTime(in.Password)
		invalid()
		return
	}

	// Brute-force protection, keyed by ACCOUNT (the email), not by IP: an attacker
	// rotating through thousands of IPs (a botnet) still hits this one counter.
	// It applies equally to emails that don't exist, so being throttled reveals
	// nothing about which accounts are real. The check comes BEFORE the expensive
	// bcrypt, so a flood of guesses also can't be used to burn our CPU.
	if h.guard != nil {
		if wait := h.guard.Peek(email); wait > 0 {
			if h.onLockout != nil {
				h.onLockout()
			}
			h.log.WarnContext(r.Context(), "login throttled: too many failures for account")
			w.Header().Set("Retry-After", strconv.Itoa(ratelimit.RetryAfterSeconds(wait)))
			h.writeError(w, r, http.StatusTooManyRequests, "too many failed attempts, try again later")
			return
		}
	}
	fail := func() {
		if h.guard != nil {
			h.guard.Spend(email)
		}
		invalid()
	}

	creds, err := h.users.GetCredentials(r.Context(), email)
	if errors.Is(err, storage.ErrNotFound) {
		h.hasher.BurnTime(in.Password)
		fail()
		return
	}
	if err != nil {
		h.internalError(w, r, "login lookup", err)
		return
	}
	if !h.hasher.Verify(creds.PasswordHash, in.Password) {
		h.log.InfoContext(r.Context(), "login failed", "user_id", creds.UserID)
		fail()
		return
	}
	if h.guard != nil {
		h.guard.Reset(email) // a correct password clears the failure count
	}
	user, err := h.users.Get(r.Context(), creds.UserID)
	if err != nil {
		h.storeError(w, r, "login load user", err)
		return
	}
	h.issueTokens(w, r, http.StatusOK, user)
}

Read the flow in Login:

  1. Check first (Peek) before any password work. If the account is throttled we answer 429 with Retry-After immediately, spending no bcrypt time. That means a flood of guesses can't even be used to burn our CPU.
  2. On any failure (fail()): unknown email, wrong password → Spend a token, then answer 401.
  3. On success: Reset.

With a budget of 10 failures refilling at 1 per minute, a botnet gets roughly 10 guesses, then one per minute. Cracking a real password at 1 guess/minute is hopeless.

The subtle bit: unknown emails are throttled too

Look at fail(): it's called for both "no such account" and "wrong password". If only real accounts got locked, the lockout itself would become an oracle: "this email gets throttled, so it must exist." By applying the same counter to non-existent emails, throttling reveals nothing. (This extends the Part 11 lesson: don't leak account existence by message or timing.)

We can see the timing defense working. Bcrypt at cost 12 costs about 180 ms here. Measured against a live server with the limiters switched off:

existing account, wrong password: 0.181 0.180 0.179 0.180 0.180 0.179 s
unknown email (with BurnTime):    0.180 0.179 0.180 0.179 0.184 0.179 s

Indistinguishable. A code path that skipped bcrypt would come back in about 2 ms, and anyone with a stopwatch could enumerate your users.

The trade-off you must understand

This feature can be turned into a weapon against your users. An attacker who knows your email can send ten wrong passwords and keep you throttled, a targeted denial of service on a single account.

There's no free lunch, only judgment. Our mitigations: the budget is generous (10), the throttle is short (one attempt per minute, not a permanent lock), and it heals by itself. Real systems often add one or more of: keying on (account and IP) so the attacker's IP is blocked but you, from your usual IP, are not; requiring a CAPTCHA after N failures instead of refusing outright; "unusual sign-in" email alerts; MFA. We keep the simple, explainable version and tell you plainly what it costs.

Tests: attack it

internal/handlers/handlers_test.go (excerpt)

func TestLoginBruteForceProtection(t *testing.T) {
	e := newEnvWith(t, opts{maxFailures: 3})
	e.register("victim@example.com")
	e.register("other@example.com")
	login := func(email, pw string) (int, string) {
		return e.do("POST", "/auth/login", "", fmt.Sprintf(`{"email":%q,"password":%q}`, email, pw))
	}

	// Three wrong guesses are answered normally...
	for i := 0; i < 3; i++ {
		if got, _ := login("victim@example.com", "wrong-password-x"); got != 401 {
			t.Fatalf("guess %d: %d", i, got)
		}
	}
	// ...then the account is throttled: even the CORRECT password is refused,
	// before any password work happens.
	req, _ := http.NewRequest("POST", e.srv.URL+"/auth/login",
		strings.NewReader(`{"email":"victim@example.com","password":"correct-horse-battery"}`))
	resp, _ := http.DefaultClient.Do(req)
	resp.Body.Close()
	if resp.StatusCode != 429 || resp.Header.Get("Retry-After") == "" {
		t.Fatalf("want 429 + Retry-After, got %d %q", resp.StatusCode, resp.Header.Get("Retry-After"))
	}

	// Per-ACCOUNT: another account is unaffected (and it doesn't matter which IP).
	if got, _ := login("other@example.com", "correct-horse-battery"); got != 200 {
		t.Fatalf("other account: %d", got)
	}
	// Non-existent accounts are throttled identically, so lockout can't be used
	// to discover which emails are registered.
	for i := 0; i < 3; i++ {
		login("ghost@example.com", "wrong-password-x")
	}
	if got, _ := login("ghost@example.com", "wrong-password-x"); got != 429 {
		t.Fatalf("unknown account should be throttled the same way, got %d", got)
	}
}

func TestSuccessfulLoginResetsFailures(t *testing.T) {
	e := newEnvWith(t, opts{maxFailures: 3})
	e.register("ann@example.com")
	for round := 0; round < 3; round++ { // 2 failures + 1 success, repeatedly: never locked
		for i := 0; i < 2; i++ {
			e.do("POST", "/auth/login", "", `{"email":"ann@example.com","password":"wrong-password-x"}`)
		}
		if got, _ := e.do("POST", "/auth/login", "", `{"email":"ann@example.com","password":"correct-horse-battery"}`); got != 200 {
			t.Fatalf("round %d: legit user locked out (%d)", round, got)
		}
	}
}

Both the honest user and the attacker are represented: the first test proves the lockout (including that the correct password is refused while throttled, and that unknown accounts behave identically); the second proves a legitimate user who logs in repeatedly is never locked out.

And the limiter itself, using a fake clock so the test doesn't sleep:

internal/ratelimit/ratelimit_test.go (excerpt)

func TestFailureCounterPeekSpendReset(t *testing.T) {
	l := New(1.0/60, 3, time.Hour, 100) // 3 failures, then one more attempt per minute
	now := fakeClock(l)

	if l.Peek("acct") != 0 {
		t.Fatal("unknown key must not be limited")
	}
	// Peeking any number of times consumes nothing.
	for i := 0; i < 10; i++ {
		if l.Peek("acct") != 0 {
			t.Fatal("peek consumed a token")
		}
	}
	l.Spend("acct")
	l.Spend("acct")
	if l.Peek("acct") != 0 {
		t.Fatal("2 of 3 failures: still allowed")
	}
	l.Spend("acct")
	wait := l.Peek("acct")
	if wait <= 0 || wait > time.Minute {
		t.Fatalf("after 3 failures want a wait up to 1m, got %v", wait)
	}
	*now = now.Add(wait)
	if l.Peek("acct") != 0 {
		t.Fatal("should be allowed again once a token refilled")
	}

	l.Spend("acct")
	l.Reset("acct") // successful login
	if l.Peek("acct") != 0 || l.Len() != 0 {
		t.Fatal("reset must forget the key")
	}
}

Complete code for this part

Everything to type, file by file. Most of the change is in main.go (building the limiters) and the login handler; the limiter itself gains three small methods. Then go build ./....

Change internal/config/config.go

--- a/internal/config/config.go
+++ b/internal/config/config.go
@@ -46,6 +46,20 @@ type Config struct {
 	RateLimitBurst int
 	TrustedProxies string // CIDRs whose X-Forwarded-For we believe, e.g. "10.0.0.0/8"
 
+	// Per-user limit for authenticated requests (0 disables). Keyed by user id, so
+	// people sharing an IP (office, mobile carrier NAT) don't share a budget.
+	UserRateLimitRPS   float64
+	UserRateLimitBurst int
+	// Strict per-IP limit on /auth/login, /auth/register, /auth/refresh
+	// (requests per MINUTE; 0 disables). Slows password guessing and signup spam.
+	AuthRateLimitPerMin float64
+	AuthRateLimitBurst  int
+	// Per-ACCOUNT failed-login budget: after LoginMaxFailures wrong attempts for
+	// one email (from any IP), further attempts wait for a refill of one per
+	// LoginFailureRefill. 0 disables.
+	LoginMaxFailures   int
+	LoginFailureRefill time.Duration
+
 	// Authentication.
 	JWTSecret       string        // HMAC key for access tokens; >= 32 bytes, from a secret store
 	AccessTokenTTL  time.Duration // short: access tokens cannot be revoked
@@ -60,34 +74,40 @@ type Config struct {
 
 func Load() (Config, error) {
 	c := Config{
-		Addr:               env("ADDR", ":7800"),
-		AdminAddr:          env("ADMIN_ADDR", ":9090"),
-		DatabaseURL:        os.Getenv("DATABASE_URL"),
-		DBMaxConns:         10,
-		DBMinConns:         2,
-		DBMaxConnLife:      30 * time.Minute,
-		DBMaxConnIdle:      5 * time.Minute,
-		DBStatementTimeout: 8 * time.Second,
-		ReadHeaderTimeout:  5 * time.Second,
-		ReadTimeout:        10 * time.Second,
-		WriteTimeout:       15 * time.Second,
-		IdleTimeout:        60 * time.Second,
-		ShutdownTimeout:    15 * time.Second,
-		RequestTimeout:     10 * time.Second,
-		MaxBodyBytes:       1 << 20,
-		MaxNameLen:         100,
-		DefaultPageSize:    20,
-		MaxPageSize:        100,
-		ServiceName:        env("SERVICE_NAME", "rest-go"),
-		OTLPEndpoint:       os.Getenv("OTLP_ENDPOINT"),
-		TraceSampleRatio:   1.0,
-		RateLimitRPS:       20,
-		RateLimitBurst:     40,
-		TrustedProxies:     os.Getenv("TRUSTED_PROXIES"),
-		JWTSecret:          os.Getenv("JWT_SECRET"),
-		AccessTokenTTL:     15 * time.Minute,
-		RefreshTokenTTL:    7 * 24 * time.Hour,
-		BcryptCost:         12,
+		Addr:                env("ADDR", ":7800"),
+		AdminAddr:           env("ADMIN_ADDR", ":9090"),
+		DatabaseURL:         os.Getenv("DATABASE_URL"),
+		DBMaxConns:          10,
+		DBMinConns:          2,
+		DBMaxConnLife:       30 * time.Minute,
+		DBMaxConnIdle:       5 * time.Minute,
+		DBStatementTimeout:  8 * time.Second,
+		ReadHeaderTimeout:   5 * time.Second,
+		ReadTimeout:         10 * time.Second,
+		WriteTimeout:        15 * time.Second,
+		IdleTimeout:         60 * time.Second,
+		ShutdownTimeout:     15 * time.Second,
+		RequestTimeout:      10 * time.Second,
+		MaxBodyBytes:        1 << 20,
+		MaxNameLen:          100,
+		DefaultPageSize:     20,
+		MaxPageSize:         100,
+		ServiceName:         env("SERVICE_NAME", "rest-go"),
+		OTLPEndpoint:        os.Getenv("OTLP_ENDPOINT"),
+		TraceSampleRatio:    1.0,
+		UserRateLimitRPS:    30,
+		UserRateLimitBurst:  60,
+		AuthRateLimitPerMin: 10,
+		AuthRateLimitBurst:  5,
+		LoginMaxFailures:    10,
+		LoginFailureRefill:  time.Minute,
+		RateLimitRPS:        20,
+		RateLimitBurst:      40,
+		TrustedProxies:      os.Getenv("TRUSTED_PROXIES"),
+		JWTSecret:           os.Getenv("JWT_SECRET"),
+		AccessTokenTTL:      15 * time.Minute,
+		RefreshTokenTTL:     7 * 24 * time.Hour,
+		BcryptCost:          12,
 	}
 
 	var err error
@@ -107,6 +127,7 @@ func Load() (Config, error) {
 		{"REQUEST_TIMEOUT", &c.RequestTimeout},
 		{"ACCESS_TOKEN_TTL", &c.AccessTokenTTL},
 		{"REFRESH_TOKEN_TTL", &c.RefreshTokenTTL},
+		{"LOGIN_FAILURE_REFILL", &c.LoginFailureRefill},
 	}
 	for _, d := range durations {
 		if v := os.Getenv(d.key); v != "" {
@@ -140,7 +161,7 @@ func Load() (Config, error) {
 	for _, i := range []struct {
 		key string
 		dst *int
-	}{{"DEFAULT_PAGE_SIZE", &c.DefaultPageSize}, {"MAX_PAGE_SIZE", &c.MaxPageSize}, {"BCRYPT_COST", &c.BcryptCost}} {
+	}{{"DEFAULT_PAGE_SIZE", &c.DefaultPageSize}, {"MAX_PAGE_SIZE", &c.MaxPageSize}, {"BCRYPT_COST", &c.BcryptCost}, {"USER_RATE_LIMIT_BURST", &c.UserRateLimitBurst}, {"AUTH_RATE_LIMIT_BURST", &c.AuthRateLimitBurst}, {"LOGIN_MAX_FAILURES", &c.LoginMaxFailures}} {
 		if v := os.Getenv(i.key); v != "" {
 			if *i.dst, err = strconv.Atoi(v); err != nil {
 				return c, fmt.Errorf("%s: %w", i.key, err)
@@ -152,6 +173,16 @@ func Load() (Config, error) {
 			return c, fmt.Errorf("RATE_LIMIT_RPS: %w", err)
 		}
 	}
+	for _, f := range []struct {
+		key string
+		dst *float64
+	}{{"USER_RATE_LIMIT_RPS", &c.UserRateLimitRPS}, {"AUTH_RATE_LIMIT_PER_MIN", &c.AuthRateLimitPerMin}} {
+		if v := os.Getenv(f.key); v != "" {
+			if *f.dst, err = strconv.ParseFloat(v, 64); err != nil {
+				return c, fmt.Errorf("%s: %w", f.key, err)
+			}
+		}
+	}
 	if v := os.Getenv("RATE_LIMIT_BURST"); v != "" {
 		if c.RateLimitBurst, err = strconv.Atoi(v); err != nil {
 			return c, fmt.Errorf("RATE_LIMIT_BURST: %w", err)
@@ -182,6 +213,10 @@ func Load() (Config, error) {
 		return c, fmt.Errorf("BCRYPT_COST must be within [4,16]")
 	case c.AccessTokenTTL <= 0 || c.RefreshTokenTTL <= c.AccessTokenTTL:
 		return c, fmt.Errorf("need 0 < ACCESS_TOKEN_TTL < REFRESH_TOKEN_TTL")
+	case c.UserRateLimitRPS < 0 || (c.UserRateLimitRPS > 0 && c.UserRateLimitBurst < 1),
+		c.AuthRateLimitPerMin < 0 || (c.AuthRateLimitPerMin > 0 && c.AuthRateLimitBurst < 1),
+		c.LoginMaxFailures < 0 || (c.LoginMaxFailures > 0 && c.LoginFailureRefill <= 0):
+		return c, fmt.Errorf("invalid user/auth/login rate-limit settings (rates >= 0; burst and refill must be positive when enabled)")
 	case c.RateLimitRPS < 0 || (c.RateLimitRPS > 0 && c.RateLimitBurst < 1):
 		return c, fmt.Errorf("RATE_LIMIT_RPS must be >= 0 and RATE_LIMIT_BURST >= 1 when enabled")
 	case c.DBMaxConns < 1 || c.DBMinConns < 0 || c.DBMinConns > c.DBMaxConns:

Change internal/handlers/auth.go

--- a/internal/handlers/auth.go
+++ b/internal/handlers/auth.go
@@ -6,11 +6,13 @@ import (
 	"log/slog"
 	"net/http"
 	"net/mail"
+	"strconv"
 	"strings"
 	"time"
 	"unicode/utf8"
 
 	"go-rest/internal/auth"
+	"go-rest/internal/ratelimit"
 	"go-rest/internal/storage"
 )
 
@@ -26,6 +28,13 @@ type TokenStore interface {
 	Revoke(ctx context.Context, hash []byte) error
 }
 
+// LoginGuard counts FAILED logins per account (see ratelimit.Limiter Peek/Spend/Reset).
+type LoginGuard interface {
+	Peek(key string) time.Duration // 0 = allowed, else how long to wait
+	Spend(key string)              // record one failure
+	Reset(key string)              // successful login: forget failures
+}
+
 type AuthHandler struct {
 	base
 	users      AuthUserStore
@@ -33,6 +42,15 @@ type AuthHandler struct {
 	hasher     *auth.Hasher
 	tm         *auth.TokenManager
 	refreshTTL time.Duration
+
+	guard     LoginGuard // optional
+	onLockout func()     // optional: called when an attempt is refused by the guard
+}
+
+// WithLoginGuard enables per-account brute-force protection.
+func (h *AuthHandler) WithLoginGuard(g LoginGuard, onLockout func()) *AuthHandler {
+	h.guard, h.onLockout = g, onLockout
+	return h
 }
 
 func NewAuthHandler(users AuthUserStore, tokens TokenStore, hasher *auth.Hasher, tm *auth.TokenManager,
@@ -129,10 +147,33 @@ func (h *AuthHandler) Login(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
+	// Brute-force protection, keyed by ACCOUNT (the email), not by IP: an attacker
+	// rotating through thousands of IPs (a botnet) still hits this one counter.
+	// It applies equally to emails that don't exist, so being throttled reveals
+	// nothing about which accounts are real. The check comes BEFORE the expensive
+	// bcrypt, so a flood of guesses also can't be used to burn our CPU.
+	if h.guard != nil {
+		if wait := h.guard.Peek(email); wait > 0 {
+			if h.onLockout != nil {
+				h.onLockout()
+			}
+			h.log.WarnContext(r.Context(), "login throttled: too many failures for account")
+			w.Header().Set("Retry-After", strconv.Itoa(ratelimit.RetryAfterSeconds(wait)))
+			h.writeError(w, r, http.StatusTooManyRequests, "too many failed attempts, try again later")
+			return
+		}
+	}
+	fail := func() {
+		if h.guard != nil {
+			h.guard.Spend(email)
+		}
+		invalid()
+	}
+
 	creds, err := h.users.GetCredentials(r.Context(), email)
 	if errors.Is(err, storage.ErrNotFound) {
 		h.hasher.BurnTime(in.Password)
-		invalid()
+		fail()
 		return
 	}
 	if err != nil {
@@ -141,9 +182,12 @@ func (h *AuthHandler) Login(w http.ResponseWriter, r *http.Request) {
 	}
 	if !h.hasher.Verify(creds.PasswordHash, in.Password) {
 		h.log.InfoContext(r.Context(), "login failed", "user_id", creds.UserID)
-		invalid()
+		fail()
 		return
 	}
+	if h.guard != nil {
+		h.guard.Reset(email) // a correct password clears the failure count
+	}
 	user, err := h.users.Get(r.Context(), creds.UserID)
 	if err != nil {
 		h.storeError(w, r, "login load user", err)

Change internal/ratelimit/ratelimit.go

--- a/internal/ratelimit/ratelimit.go
+++ b/internal/ratelimit/ratelimit.go
@@ -47,12 +47,11 @@ func New(rps float64, burst int, ttl time.Duration, maxKeys int) *Limiter {
 func (l *Limiter) Limit() float64 { return float64(l.rate) }
 func (l *Limiter) Burst() int     { return l.burst }
 
-// Allow spends a token for key. If none is available it returns false and how
-// long the caller should wait before retrying.
-func (l *Limiter) Allow(key string) (ok bool, retryAfter time.Duration) {
-	now := l.now()
-
+// get returns the entry for key, creating it if needed. ok=false means the
+// table is full of active keys (the caller should fail open, see below).
+func (l *Limiter) get(key string, now time.Time) (e *entry, ok bool) {
 	l.mu.Lock()
+	defer l.mu.Unlock()
 	e, found := l.entries[key]
 	if !found {
 		if len(l.entries) >= l.max {
@@ -62,15 +61,24 @@ func (l *Limiter) Allow(key string) (ok bool, retryAfter time.Duration) {
 			// Still full of ACTIVE keys (e.g. an attack with many source
 			// addresses). Failing closed would lock out legitimate users and turn
 			// the limiter into the outage; fail open and rely on other layers.
-			l.mu.Unlock()
-			return true, 0
+			return nil, false
 		}
 		e = &entry{lim: rate.NewLimiter(l.rate, l.burst)}
 		l.entries[key] = e
 	}
 	e.lastSeen = now
-	l.mu.Unlock() // rate.Limiter is itself goroutine-safe; don't hold our lock while using it
+	return e, true
+}
 
+// Allow spends a token for key. If none is available it returns false and how
+// long the caller should wait before retrying.
+func (l *Limiter) Allow(key string) (ok bool, retryAfter time.Duration) {
+	now := l.now()
+	e, ok := l.get(key, now)
+	if !ok {
+		return true, 0
+	}
+	// rate.Limiter is itself goroutine-safe, so we don't hold our lock here.
 	// Reserve + Cancel (instead of Allow) lets us learn the wait time without
 	// consuming a token when the request is rejected.
 	r := e.lim.ReserveN(now, 1)
@@ -84,6 +92,42 @@ func (l *Limiter) Allow(key string) (ok bool, retryAfter time.Duration) {
 	return true, 0
 }
 
+// Peek / Spend / Reset use the bucket as a FAILURE counter instead of a request
+// counter: look without consuming, consume only when something bad happened,
+// forget on success. Used for login attempts: successful logins must not eat
+// the budget, otherwise a user who logs in often would lock themselves out.
+
+// Peek returns how long until key has a token available (0 = not limited).
+// It consumes nothing.
+func (l *Limiter) Peek(key string) time.Duration {
+	now := l.now()
+	l.mu.Lock()
+	e, found := l.entries[key]
+	l.mu.Unlock()
+	if !found {
+		return 0
+	}
+	r := e.lim.ReserveN(now, 1)
+	d := r.DelayFrom(now)
+	r.CancelAt(now)
+	return d
+}
+
+// Spend consumes one token (records one failure).
+func (l *Limiter) Spend(key string) {
+	now := l.now()
+	if e, ok := l.get(key, now); ok {
+		e.lim.AllowN(now, 1)
+	}
+}
+
+// Reset forgets key entirely (the bucket is full again).
+func (l *Limiter) Reset(key string) {
+	l.mu.Lock()
+	delete(l.entries, key)
+	l.mu.Unlock()
+}
+
 // Cleanup evicts idle keys. Run it periodically (see Run).
 func (l *Limiter) Cleanup() {
 	l.mu.Lock()

Change internal/routes/routes.go

--- a/internal/routes/routes.go
+++ b/internal/routes/routes.go
@@ -12,17 +12,27 @@ type Deps struct {
 	Users        *handlers.UserHandler
 	Orders       *handlers.OrderHandler
 	Authenticate func(http.Handler) http.Handler
+	// Optional (nil = none):
+	UserLimit func(http.Handler) http.Handler // per-user quota, runs AFTER Authenticate
+	AuthLimit func(http.Handler) http.Handler // strict per-IP quota for login/register/refresh
 }
 
 // SetupRoutes registers every endpoint. Reading this file answers "which routes
 // are public?": only those registered with `public`.
 func SetupRoutes(mux *http.ServeMux, d Deps) {
+	userLimit, authLimit := orIdent(d.UserLimit), orIdent(d.AuthLimit)
+
 	public := func(pattern string, h http.HandlerFunc) { mux.Handle(pattern, h) }
-	protected := func(pattern string, h http.HandlerFunc) { mux.Handle(pattern, d.Authenticate(h)) }
+	// The order matters: Authenticate first, so UserLimit knows WHO to count and
+	// unauthenticated requests (401) never consume anyone's budget.
+	protected := func(pattern string, h http.HandlerFunc) {
+		mux.Handle(pattern, d.Authenticate(userLimit(h)))
+	}
 
-	public("POST /auth/register", d.Auth.Register)
-	public("POST /auth/login", d.Auth.Login)
-	public("POST /auth/refresh", d.Auth.Refresh)
+	// Credential endpoints get the strict limiter: they are the ones attackers hammer.
+	mux.Handle("POST /auth/register", authLimit(http.HandlerFunc(d.Auth.Register)))
+	mux.Handle("POST /auth/login", authLimit(http.HandlerFunc(d.Auth.Login)))
+	mux.Handle("POST /auth/refresh", authLimit(http.HandlerFunc(d.Auth.Refresh)))
 	public("POST /auth/logout", d.Auth.Logout)
 	protected("GET /auth/me", d.Auth.Me)
 
@@ -36,3 +46,10 @@ func SetupRoutes(mux *http.ServeMux, d Deps) {
 	protected("GET /orders/{id}", d.Orders.Get)
 	protected("DELETE /orders/{id}", d.Orders.Delete)
 }
+
+func orIdent(m func(http.Handler) http.Handler) func(http.Handler) http.Handler {
+	if m == nil {
+		return func(h http.Handler) http.Handler { return h }
+	}
+	return m
+}

Change cmd/rest/main.go

--- a/cmd/rest/main.go
+++ b/cmd/rest/main.go
@@ -10,6 +10,7 @@ import (
 	"net/mail"
 	"os"
 	"os/signal"
+	"strconv"
 	"strings"
 	"syscall"
 	"time"
@@ -208,30 +209,61 @@ func serve() error {
 		return err
 	}
 	userStore := storage.NewUserStore(pool)
+
+	// Rate limiting, in layers (cheapest check first, most specific last):
+	//   1. per client IP, every request          (outer net, runs before auth)
+	//   2. per client IP, credential endpoints   (strict; stops guessing/signup spam)
+	//   3. per authenticated user                (fair quota; people behind one NAT don't share)
+	//   4. per account, failed logins only       (inside the login handler)
+	trusted, err := ratelimit.ParseCIDRs(cfg.TrustedProxies)
+	if err != nil {
+		return fmt.Errorf("TRUSTED_PROXIES: %w", err)
+	}
+	newLimiter := func(rps float64, burst int) *ratelimit.Limiter {
+		l := ratelimit.New(rps, burst, 10*time.Minute, 100_000)
+		go l.Run(ctx.Done()) // evicts idle keys until shutdown
+		return l
+	}
+	byIP := func(r *http.Request) string { return ratelimit.ClientIP(r, trusted) }
+	byUser := func(r *http.Request) string {
+		if p, ok := auth.FromContext(r.Context()); ok {
+			return "u:" + strconv.Itoa(p.UserID)
+		}
+		return "" // not authenticated: skip (Authenticate already answered 401)
+	}
+	limit := func(name string, l *ratelimit.Limiter, key func(*http.Request) string) func(http.Handler) http.Handler {
+		return middleware.RateLimit(middleware.RateLimitOptions{
+			Limiter: l, Key: key, OnReject: func(*http.Request) { metrics.RateLimited(name)() },
+		})
+	}
+
+	var limiters []func(http.Handler) http.Handler
+	if cfg.RateLimitRPS > 0 {
+		limiters = append(limiters, limit("ip", newLimiter(cfg.RateLimitRPS, cfg.RateLimitBurst), byIP))
+	}
+	var userLimit, authLimit func(http.Handler) http.Handler
+	if cfg.UserRateLimitRPS > 0 {
+		userLimit = limit("user", newLimiter(cfg.UserRateLimitRPS, cfg.UserRateLimitBurst), byUser)
+	}
+	if cfg.AuthRateLimitPerMin > 0 {
+		authLimit = limit("auth_ip", newLimiter(cfg.AuthRateLimitPerMin/60, cfg.AuthRateLimitBurst), byIP)
+	}
+	authHandler := handlers.NewAuthHandler(userStore, storage.NewTokenStore(pool), hasher, tm, cfg.RefreshTokenTTL, log, limits)
+	if cfg.LoginMaxFailures > 0 {
+		failures := newLimiter(1/cfg.LoginFailureRefill.Seconds(), cfg.LoginMaxFailures)
+		authHandler.WithLoginGuard(failures, metrics.RateLimited("login_account"))
+	}
+
 	mux := http.NewServeMux()
 	routes.SetupRoutes(mux, routes.Deps{
-		Auth:         handlers.NewAuthHandler(userStore, storage.NewTokenStore(pool), hasher, tm, cfg.RefreshTokenTTL, log, limits),
+		Auth:         authHandler,
 		Users:        handlers.NewUserHandler(userStore, log, limits),
 		Orders:       handlers.NewOrderHandler(storage.NewOrderStore(pool), log, limits),
 		Authenticate: middleware.Authenticate(tm, log),
+		UserLimit:    userLimit,
+		AuthLimit:    authLimit,
 	})
 
-	// Rate limiting: one bucket per client IP, applied to every API request.
-	var limiters []func(http.Handler) http.Handler
-	if cfg.RateLimitRPS > 0 {
-		trusted, err := ratelimit.ParseCIDRs(cfg.TrustedProxies)
-		if err != nil {
-			return fmt.Errorf("TRUSTED_PROXIES: %w", err)
-		}
-		ipLimiter := ratelimit.New(cfg.RateLimitRPS, cfg.RateLimitBurst, 10*time.Minute, 100_000)
-		go ipLimiter.Run(ctx.Done()) // evicts idle clients until shutdown
-		limiters = append(limiters, middleware.RateLimit(middleware.RateLimitOptions{
-			Limiter:  ipLimiter,
-			Key:      func(r *http.Request) string { return ratelimit.ClientIP(r, trusted) },
-			OnReject: func(*http.Request) { metrics.RateLimited("ip")() },
-		}))
-	}
-
 	// First listed = outermost. ORDER MATTERS:
 	//  1. otelhttp, RequestID, Timeout all COPY the request (r.WithContext), so
 	//     they go first.

Replace the contents of deploy/prometheus/alerts.yml with

# Alert rules are evaluated by Prometheus. Delivering them (email/Slack/pager)
# needs Alertmanager, which is intentionally not part of this learning stack;
# firing alerts are still visible at http://localhost:9091/alerts.
groups:
  - name: api
    rules:
      - alert: ApiDown
        expr: up{job="api"} == 0
        for: 1m
        labels: { severity: page }
        annotations: { summary: "API target is down" }

      - alert: HighErrorRate           # symptom-based: users see 5xx
        expr: |
          sum(rate(http_requests_total{status=~"5.."}[5m]))
            / clamp_min(sum(rate(http_requests_total[5m])), 1) > 0.05
        for: 5m
        labels: { severity: page }
        annotations: { summary: "More than 5% of requests fail with 5xx" }

      - alert: HighLatencyP95
        expr: histogram_quantile(0.95, sum by (le) (rate(http_request_duration_seconds_bucket[5m]))) > 1
        for: 10m
        labels: { severity: ticket }
        annotations: { summary: "p95 latency above 1s" }

      - alert: DbPoolExhausted         # requests are queueing for a connection
        expr: rate(db_pool_empty_acquire_total[5m]) > 0
        for: 5m
        labels: { severity: ticket }
        annotations: { summary: "DB pool is exhausted; consider raising DB_MAX_CONNS or fixing slow queries" }

      - alert: SustainedRateLimiting   # a client (or attack) is hitting the limit continuously
        expr: sum(rate(rate_limited_requests_total[5m])) > 1
        for: 10m
        labels: { severity: ticket }
        annotations: { summary: "Requests are being rate limited continuously; check for abuse or a too-low limit" }

      - alert: PossibleCredentialStuffing   # accounts are being locked out by failed logins
        expr: sum(rate(rate_limited_requests_total{limiter=~"login_account|auth_ip"}[5m])) > 0.2
        for: 5m
        labels: { severity: page }
        annotations: { summary: "Sustained throttling of login attempts; possible password guessing or credential stuffing" }

Change docker-compose.yml

--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -51,7 +51,14 @@ services:
       # Per-IP limit. Generous locally so `make load` isn't throttled; lower it to see 429s.
       RATE_LIMIT_RPS: ${RATE_LIMIT_RPS:-200}
       RATE_LIMIT_BURST: ${RATE_LIMIT_BURST:-400}
-      # TRUSTED_PROXIES: 10.0.0.0/8   # set when behind a load balancer, or every client shares its IP
+      # Pass-through (null value = take from your shell/.env if set, else use the app default):
+      USER_RATE_LIMIT_RPS:
+      USER_RATE_LIMIT_BURST:
+      AUTH_RATE_LIMIT_PER_MIN:
+      AUTH_RATE_LIMIT_BURST:
+      LOGIN_MAX_FAILURES:
+      LOGIN_FAILURE_REFILL:
+      TRUSTED_PROXIES:              # set e.g. 10.0.0.0/8 when behind a load balancer
     ports:
       - "7800:7800"                          # public API only; admin port 9090 stays internal
     depends_on:

Try it

Lower the limits so the demo is quick (5 failures, then lockout; per-user quota 1/s with burst 4):

AUTH_RATE_LIMIT_PER_MIN=600 AUTH_RATE_LIMIT_BURST=100 LOGIN_MAX_FAILURES=5 \
USER_RATE_LIMIT_RPS=1 USER_RATE_LIMIT_BURST=4 go run ./cmd/rest

# 8 wrong guesses at one account
$ for i in $(seq 1 8); do curl -s -o /dev/null -w "%{http_code} " -XPOST localhost:7800/auth/login \
    -d '{"email":"victim@example.com","password":"guess'$i'-guess-guess"}'; done
401 401 401 401 401 429 429 429

# now even the CORRECT password is refused until a token refills
$ curl -i -XPOST localhost:7800/auth/login -d '{"email":"victim@example.com","password":"correct-horse-battery"}'
HTTP/1.1 429 Too Many Requests
Retry-After: 60
{"error":"too many failed attempts, try again later"}

# a different account is unaffected: this one simply gets "invalid credentials"
$ curl -s -o /dev/null -w "%{http_code}\n" -XPOST localhost:7800/auth/login -d '{"email":"nobody@example.com","password":"..."}'
401

# the per-user limit: 7 quick calls with one user's token
$ for i in $(seq 1 7); do curl -s -o /dev/null -w "%{http_code} " -H "Authorization: Bearer $T" localhost:7800/orders; done
200 200 200 200 429 429 429

And in Prometheus, rate_limited_requests_total is broken down by limiter label (ip, auth_ip, user, login_account). The PossibleCredentialStuffing alert added in this part's alerts.yml uses it: sustained throttling of logins is exactly what an attack looks like from the inside.

Operating it

  • Watch before tightening. A rise in ip throttling after a deploy often means a customer behind a shared NAT, or a missing TRUSTED_PROXIES, not an attack.
  • State is in memory, per instance (see Part 10). With three replicas an attacker gets 3x the failure budget. For exact accounting, move the counters to Redis.
  • Defense in depth, not defense by one trick. The bcrypt cost, the minimum password length, the per-IP and per-account limits, and logging every failed login (with the user id, never the password) work together.

Recap

  • Match the key to the threat: IP, user, account.
  • Count failures for login protection (Peek/Spend/Reset), and check before doing expensive work.
  • Throttle non-existent accounts identically, or the throttle leaks which exist.
  • Put the per-user limiter after authentication.
  • Every lockout mechanism is also a possible denial-of-service lever; know the trade-off and document it.

Exercises

  1. Change the account key to email + ":" + clientIP. What attack does this defeat, and which does it re-open?
  2. Add a response header on the 429 that tells the client how many failed attempts remain before throttling. Is that a good idea security-wise?
  3. Simulate the botnet: write a small Go program that sends one wrong login per goroutine, each with a different X-Forwarded-For (with TRUSTED_PROXIES unset). Which layers stop it, and why?

Comments

Join the discussion

No comments yet. Be the first to comment.

Build a production-grade REST API in Go

Part 14 of 18

A hands-on series: start with under 200 lines of in-memory CRUD and end with Postgres, JWT auth, rate limiting, TLS and a full observability stack.

Up nextPart 14: TLS: encrypting the wire, hot-reloading certificates, and verifying the databaseEvery password and token in this series still crosses the network as plain text. We add HTTPS, learn what a certificate actually proves, rotate certificates without a restart, and encrypt (and verify)

More from this blog