@Keryxo,
You're allowed to be taken aback when someone springs their feelings on you for the first time.
If you want to see him again, then actively work to set it up. And find other things to do with your time (not that sex isn't lovely, but you should see him in a few more settings before deeper feelings set in - and, frankly, so should have he).
Have fun. Enjoy your time together. If it turns into more, then great. If it doesn't, then he's however many hours away and you won't have to run into him all the time.
I would also caution you, though, that he knew he was leaving and may have tossed out those words because he felt they were expected. Meeting him again and spending out of the sheets time together will help you to see if he was sincere.