This function reads a log file and counts how many lines contain "ERROR". It looks textbook — open, defer close, scan, count. It passes every unit test against the synthetic fixtures. In production it returns plausible numbers. But the on-call engineer swears there were more errors than this.

package main

import (
    "bufio"
    "os"
    "strings"
)

// CountErrors scans a log file and returns the number of lines
// containing the substring "ERROR".
func CountErrors(path string) (int, error) {
    f, err := os.Open(path)
    if err != nil {
        return 0, err
    }
    defer f.Close()

    count := 0
    scanner := bufio.NewScanner(f)
    for scanner.Scan() {
        if strings.Contains(scanner.Text(), "ERROR") {
            count++
        }
    }
    return count, nil
}

The Bug

bufio.NewScanner ships with a default maximum token size of 64 KiB (bufio.MaxScanTokenSize = 64 * 1024). The instant the scanner encounters a line longer than that — a fat JSON-encoded log entry, a Java stack trace, a base64-blob attachment — Scan() returns false and stashes bufio.ErrTooLong inside scanner.Err().

The for-loop's exit condition can't distinguish "clean EOF" from "I gave up". The code never calls scanner.Err(), so the error is swallowed. Worse, the scanner doesn't just truncate the offending line — it abandons it and every line after it. One oversized line halfway through a 10 GB file means you've counted errors from the first 30% of the file and silently dropped the rest. The function returns (count, nil) with a straight face.

This bug is invisible in tests because fixtures are usually small and uniform. It only bites on real-world data — exactly when you need the tooling to work.

The Fix

Two changes: enlarge the buffer to accommodate realistic line sizes, and always check scanner.Err() after the loop so a truncation failure is surfaced rather than hidden.

func CountErrors(path string) (int, error) {
    f, err := os.Open(path)
    if err != nil {
        return 0, err
    }
    defer f.Close()

    count := 0
    scanner := bufio.NewScanner(f)
    // Allow lines up to 10 MiB; starting buffer is 64 KiB and grows.
    scanner.Buffer(make([]byte, 64*1024), 10*1024*1024)

    for scanner.Scan() {
        if strings.Contains(scanner.Text(), "ERROR") {
            count++
        }
    }
    if err := scanner.Err(); err != nil {
        return count, err  // surface ErrTooLong, I/O errors, etc.
    }
    return count, nil
}

If your input genuinely has no bound on line length, ditch Scanner entirely and use bufio.Reader.ReadString('\n') or ReadBytes, which grow as needed and report partial reads via io.EOF instead of giving up. The Scanner API is optimized for the common case; it trades correctness on pathological input for a clean iterator interface, and that trade is hidden behind a friendly-looking for scanner.Scan().

The deeper lesson: any iterator whose stop condition is a bare bool is hiding an error channel somewhere. In Go, that channel is almost always a separate Err() method, and "I forgot to check it" is one of the most reliable ways to ship a silent data-loss bug.