@hurtwife1987,
He's taking advantage of the fact that you work nights.
Even if nothing happened at all, and he was a saint the whole evening, he still brought someone into your home without telling you or clearing it with you. That's just plain rude.