Why not just go back to the restaurant for lunch and see what happens? Have your own card with you this time with your email address etc. He might think you are a married lady with kids and it would be improper to call you. Everybody married and/or attached doesn't wear a ring.
That might spare you (and him) some unnecessary embarrassment or awkwardness.
