Glenn Johnson described just how to change optimization flags when compiling X.org, though Joerg Sonnenberger recommends against it.