Does it make sense to rebuild software from sources in a repository with default keys?

Question

linux

A few years ago, when I had an i686 processor, I knew for sure that the binary built on my machine would be at least an iota, but faster than the binary downloaded from the repository and built under i386, even if I did not change a single key.
The question is, is it relevant now? To what extent can two x86_64 processors differ from each other in terms of instruction set? And, accordingly, is it possible, at least theoretically, to win at least something?
And a slightly more general question. What are the really relevant reasons to build software yourself? This is a question, first of all, for those who regularly do this, who are interested in someone else's experience.

Answer 1

I only recompile the kernel, and then:
I don't see much point in rebuilding other packages. There will obviously be an increase (my Haswell can, obviously, have more optimal instructions than the processor a few years ago), but I doubt that the labor costs on the desktop are worth it, the situation may be different for servers.

Answer 2

It makes sense only for those who care more about an iota later than half an hour now.

Answer 3

Is there a Philosophy section here? :)
For me, the actual reason is the lack of a ready-made package. For some acquaintances - the need to assemble somehow in a special way. And for someone - just the lack of personal life.