This handler loads config lazily on first request, using sync.Once to make the load thread-safe. If the disk read fails, the initializer panics; the outer handler recovers and returns 500. The team's runbook says: fix the config file, and the next request will re-read it and succeed.

var (
    once   sync.Once
    config *AppConfig
)

func GetConfig() *AppConfig {
    once.Do(func() {
        data, err := os.ReadFile("/etc/app/config.json")
        if err != nil {
            panic(fmt.Sprintf("config load failed: %v", err))
        }
        var c AppConfig
        if err := json.Unmarshal(data, &c); err != nil {
            panic(err)
        }
        config = &c
    })
    return config
}

func handleRequest(w http.ResponseWriter, r *http.Request) {
    defer func() {
        if x := recover(); x != nil {
            log.Printf("recovered: %v", x)
            http.Error(w, "internal error", 500)
        }
    }()
    cfg := GetConfig()
    respond(w, cfg)
}

The first request panics because the config file was missing. Ops fixes it within seconds. Every subsequent request still returns 500 — forever. Restarting the process is the only cure.

The Bug

sync.Once guarantees the function runs at most once — whether it succeeds, panics, or gets killed by runtime.Goexit. The stdlib implementation is roughly:

func (o *Once) doSlow(f func()) {
    o.m.Lock()
    defer o.m.Unlock()
    if o.done.Load() == 0 {
        defer o.done.Store(1)
        f()
    }
}

Notice the ordering: defer o.done.Store(1) is registered before f() runs. When f() panics, the deferred store still executes on the way out, marking the Once as complete. The panic propagates up to your recovery handler — but the Once is now sealed. Every future Do takes the fast path, skips your initializer, and returns immediately. config stays nil for the process lifetime, and respond(w, cfg) nil-derefs into a fresh panic on every request.

This is documented ("If f panics, Do considers it to have returned; future calls of Do return without calling f"), but the guarantee is exactly the opposite of what a caller who recovers wants. sync.Once is designed for initializations that cannot fail — package-level state, sync.Pool factories, atomic wiring. It is not a retry primitive.

The Fix

If initialization can fail, don't use sync.Once. Use a mutex and check the result explicitly:

var (
    mu     sync.Mutex
    config *AppConfig
)

func GetConfig() (*AppConfig, error) {
    mu.Lock()
    defer mu.Unlock()
    if config != nil {
        return config, nil
    }
    data, err := os.ReadFile("/etc/app/config.json")
    if err != nil {
        return nil, fmt.Errorf("config load failed: %w", err)
    }
    var c AppConfig
    if err := json.Unmarshal(data, &c); err != nil {
        return nil, err
    }
    config = &c
    return config, nil
}

Now every failed attempt leaves config == nil, and the next caller retries. For read-heavy workloads, promote to a sync.RWMutex with a fast-path read lock. Go 1.21's sync.OnceValue and sync.OnceValues have the same panic-sealing semantics, so wrap the initializer to return an error rather than panic if you want them to interoperate with retries.