How well do you know him? Are you sure he knows who you are?
Don't be afraid of resolving this face to face. That's what you should have done in the first place. (asked him in person)
Also, depending on his cultural background, he may have been taken aback when a woman asked him out.
So what if he declines? Say, "Hey, maybe another time." and then walk away with a smile. That's not going to be the worse thing that will ever happen to you.