I have these same True Flights- have worn them about 20 times and mine don't look that bad. But I do use a canned airduster and a suede brush on them after each wear. Nothing but casual wear also.
The White stuff is not dust but the suede rubbing off