DaRWIN did kinda just make it up... its a THEORY not a FACTIf people with different eye colors continue to reproduce, why would those colors disappear? Unless there was suddenly some environmental pressure, I don't see why blue eyes would disappear.
Humans are a bit of an anomaly in that we aren't subject to natural selection the same way we used to be. We have vaccines and treatments for various ailments that would have otherwise killed us. So, it's difficult to say what should and should not happen to humans. If someone is born with an undesirable characteristic, often we can mask that.
Would HIV resistance fall into your idea of a small evolutionary change?
Darwin had evidence to support his claims. He didn't just "make it up".