Confused with relationship

ok here we go iam 22 years old i have been at a new job 4 about 9 months quite settled and happy. Ive been single 4 a long time since a past one and a near rape inccident left me having serious intimacy and trust issues. Not that iam wanting sympathy i was quite content good friends and life in general was going well. However my team leader at work who i got on with well and have always had a good relationship with started flirting with me. At first i was shocked he had never spoken to me that way but it was no big deal since there are very few women where i work the guys are flirty and this dosent bother me much since it is never serious and all in good fun. He surprised me but i went with it since it was just daft silly flirting nothing serious. He is married so thus i never thought of us like that because although i did like him when i found out he was taken it was instantly wiped from my mind. The flirting got alittle more serious and became a common thing between us he had my number since i have my own horse he had before all this started been coming riding with me but i will say that this was purely a FRIENDSHIP thing and i let anyone ride my horse who asks. He started texting me alot pretty light hearted stuff then he again asked to come riding. No big deal to me everything was fine until it came time to go home. Well long story short we ended up kissing which i stopped asked to go home and he did so. I felt awful i am not that kind of person who gets involved with a married man and i am also not the kind of person who NEEDS to be with someone. I have had 3 relationship my whole life with people who meant alot to me so for me this was completely out of character. These litttle meetings happened a few times more we have never slept with each other just kissed and spent time together. He then said we should stop that this wasent fair to me and sooner or later we would go somewhere there was no going back from. This broke my heart in a odd way even though he was married when he was with me it was easy to lie to myself cause he made me so happy. So work was a nightmare he left me alone and wouldnt speak to me unless he had to and i approached him and said could we at least be friends and civil since we have to get on at work because he is my team leader. So we did and after a couple of weeks we settled into a pleasant and easy going kind of thing but then slowly but surely we both started creeping back into old habits texting and flirting. Once again he has said that we must stop after having a long chat he said he didnt think i was ready 4 a commitment like he wanted. Amd i know for a fact he hasent cheated before and in 16 years of been in and out of relationships iam the only one he has ever been tempted to stray to. Also he wants kids i do not another facture in a future together. But now i dont know what to do i want to be with him i love him yet i know that it is a crazy idea and i dont know if he would leave his wife or not . She has recently found out about us and now i think he is as confused at me what to do. My friends say he is a cheater and i shouldnt even think about it and i am not naive i know this but it is possible to be in a relationship and meet someone new and care for them. Iam not making excuses 4 him i know what he did was wrong and he hurt me but i just dont know what to do. My best friend said he was just after sex yet he stopped it before it went that far and to be honest another couple of weeks and we would have i think so if he just wanted me 4 sex why did he stop this!?!?!? Iam utterly confused i dont know what he wants and i dont know what to do. My friends say leave work get another job but i dont know if that is the answer. Iam lost and depressed i dred work yet i miss him when iam not there he is off now on holiday 4 two weeks which gives me alittle time to clear my head please help what should i do iam lost completely!?!?!?