He says he loves me but doesn't want a relationship

Details:
So, while working away I was kind of dating this guy that was working with me. It started as friends as I had a boyfriend at the time and then as things went south with him we started hooking up and I broke it off with my guy.

He had only just came out as Bi and just came out of college.
We never spoke about being exclusive at first so he hooked up with other people in our work environment. I told him I didn't want that any and I wanted to be exclusive. Shortly after I got to meet his family (at the time only his sisters knew about him being Bi) and they all loved me.
He said that seeing how much they loved me made him love me more.

Our work contracts came to an end and we knew that it was too difficult to have a relationship but we said we still loved each other and agreed to meet up when we could.
I flew out to New York from Australia not a month later because I wanted to show him that I loved him.
I found out he had already made out with other people and it hurt me so much, almost to the point of me grabbing my bags and getting on the next flight home. We talked it out and he said that it was wrong but he is in a place where he needs to be free. This is the first time he hasn't been in a relationship in his adult life and he is still new to being Bi.
I completely understood and agreed that I we can do whatever we want as long as we don't find out. We said we still love each other.

(I feel it's important to note that when he got home after his contract he told his mother and father that he was Bi and that when they met me we were in a somewhat relationship, it didn't sit well with them and they pulled the Bible thing on him but said they still loved him)

After only three days of being home he went on vacation back to where we were working. We talked everyday (still saying I love you every night).
My friend that happened to be working there informed me that he had slept with one of her friends there. Again absolutely broken I lost it (even thought I had agreed we could sleep with other people and still say we loved each other). It was the finding out part that got me and even more that it had only been a few days since he saw me.
We spoke about it again and he said it's not fair that I found out but I was upset that it had only been a few days since I was sleeping with him.

He still says the same thing, "I love you and you changed my life, I want you in my life but the distance would only make the relationship harder than what it is now and I need to be free for the first time in my life."

I completely understand that I've had many relationships and experienced a lot even though we are the same age ( I didn't go to college so I got a head start on working away a lot). I know that I can't force him to have a relationship and we talk about the "one day" that I feel we both want but I worry that I'm going to be the one that gets hurt again like my last few relationships.

I think I just need advice on if I should keep in contact and just be happy when we see each other and do just like he is doing and hope that he doesn't end up in a relationship between now and then (I KNOW that I won't be as I've expressed that after him I can't do another relationship with someone while I go to work away again as it'll only end the same way).

Should I say that saying "I love you" can't happen anymore?

Should I just forget it all and try to get over him?

I need opinions on this because I can't make much sense of what to do!