SSRF guard on user-configurable URLs, re-checked on every redirect hop

byob-security.6 httpsecurity

Problem: the moment a config file or flag can supply an endpoint URL (api_url = "..." in a project-scoped config that byob-config.1 happily loads from any directory), a hostile or shared config can steer the CLI's HTTP client at targets the user can reach but the config author shouldn't: localhost admin ports, cloud metadata at 169.254.169.254, RFC 1918 services behind the user's firewall. A CLI run inside CI with cloud credentials makes this a real credential-theft vector, not a theoretical one.

Idea: validate every operator-controlled URL at the boundary — after config load, before the client touches it (byob-input-validation.5's boundary discipline applied to URLs) — and again on every redirect hop:

Scheme allowlist: http/https only.
Address check: if the host is an IP literal, check it directly; otherwise resolve it and check every returned address. Reject loopback, link-local (unicast and multicast — link-local covers the metadata endpoint), private (RFC 1918), unspecified (0.0.0.0), and multicast. net/netip has a predicate for each of those categories — but the predicates don't exhaust special-purpose address space; add explicit checks for CGNAT (100.64.0.0/10) and benchmarking (198.18.0.0/15) ranges if your threat model includes them.
Redirect layer: the initial-URL check alone is defeated by a 302 — a hostile or MITM'd upstream redirects the validated public URL to 127.0.0.1 and the stdlib client follows it. Set client.CheckRedirect to re-run the same validation on each hop's req.URL. A custom CheckRedirect replaces the stdlib default, so re-add the 10-hop cap yourself.
Escape hatch: self-hosted deployments legitimately point at private endpoints. Make the opt-out an explicit per-source config field (allow_private = true), carried to the redirect layer as a context key on the request — the stdlib propagates the request context across redirect hops, so the opt-in follows the request rather than becoming a process-wide off switch. Name the knob in the rejection message so a legitimate user unblocks themselves in one edit.

Scope the guard to operator-controlled URLs. Hardcoded endpoints don't need it, but leave a comment at the guard site saying new configurable sources must route through it — the policy is easy to silently lose when the next URL field lands.

Tradeoffs: validation resolves DNS at check time, and the dial happens later — a DNS-rebinding attacker can pass the check and rebind before connect. Closing that window needs the check inside the dialer (Transport.DialContext wrapping the resolved address), which forces the policy onto every request including hardcoded endpoints; take that trade only if rebinding is in your threat model. ~60 lines total for the boundary + redirect version.

Design

func ValidatePublicHTTPURL(ctx context.Context, urlStr string) error {
    u, err := url.Parse(urlStr)
    if err != nil { return fmt.Errorf("parse url: %w", err) }
    if u.Scheme != "http" && u.Scheme != "https" {
        return fmt.Errorf("scheme %q is not http or https", u.Scheme)
    }
    host := u.Hostname()
    if host == "" { return errors.New("url has no host") }

    if addr, err := netip.ParseAddr(host); err == nil {
        return checkPublicAddr(addr, host) // IP literal
    }
    addrs, err := net.DefaultResolver.LookupNetIP(ctx, "ip", host)
    if err != nil { return fmt.Errorf("resolve %q: %w", host, err) }
    if len(addrs) == 0 { // a successful-but-empty answer must not pass:
        // the dial later re-resolves independently of this check
        return fmt.Errorf("resolve %q: no addresses returned", host)
    }
    for _, a := range addrs {
        if err := checkPublicAddr(a, host); err != nil { return err }
    }
    return nil
}

func checkPublicAddr(addr netip.Addr, host string) error {
    switch {
    case addr.IsLoopback(),
        addr.IsLinkLocalUnicast(), addr.IsLinkLocalMulticast(),
        addr.IsPrivate(), addr.IsUnspecified(), addr.IsMulticast():
        return fmt.Errorf(
            "refusing %q: resolves to non-public %s (set allow_private = true to override)",
            host, addr)
    }
    return nil
}

// Factory's client wiring — redirect hops re-validated:
client := &http.Client{
    Transport: transport,
    CheckRedirect: func(req *http.Request, via []*http.Request) error {
        if len(via) >= 10 { // custom hook replaces the stdlib default; restore its cap
            return errors.New("stopped after 10 redirects")
        }
        if allowPrivateFromContext(req.Context()) {
            return nil // per-source opt-in, carried on the request context
        }
        if err := ValidatePublicHTTPURL(req.Context(), req.URL.String()); err != nil {
            return fmt.Errorf("redirect blocked: %w", err)
        }
        return nil
    },
}