@needsadvice,
Well, this is a mess.
Good for you for trying to avoid him, but how long will you be able to do so? I suppose you'll need a chaperone any time/day you're nearby.
My suggestion is to try to channel the attraction and the excitement (it's really just the excitement of something new) to your husband. Maybe suggest some of those things you did out of character with the other guy, whatever they are. E. g. perhaps not stripping down completely, or sex in a room other than the bedroom, etc. Associate some of those behaviors with your husband and I think that'll start to take some of the punch out of what happened with your neighbor.
Also, try a little negative reinforcement with your neighbor, e. g. he brushes up against you, associate that in your mind with something horrible. After all, it actually is something horrible - a threat to your marriage, yes?