Honor your word as a Businessman and as a Man and have some integrity.
If things are moving too slowly and you're having second thoughts, email him and let him know you want this completed within the next 48 hours or you're going with another name but still give him the opportunity to move forward with your original offer if he agreed. The end.
PS - Unless you're a mind-reader, maybe the guy didn't even want to sell it for $500 but something came up, or he thought about it and he's actually giving you a break, and now you want it for less? At the end of the day he could be the one with seller's remorse. How about he emails you in the morning saying "On second thought, I want $1000" when you're ready to pay up the $500 he accepted, what's the difference? I bet then you'd be quick to pay up that $500.
Doing the right thing always pays off in the long-run. I've followed-through with deals I had seller's & buyer's remorse only to deal with the same person in the future and cut some nice deals which wouldn't be the case if I had backed out.