Really confused by his mixed signals

Firstly please let me apologise for the length of this post. Sorry x..
I'm feeling really confused regarding a guy I'd been seeing..
We went on a date over several years ago and got on really well but we never went on a second date but friended one another on Facebook although never stayed in touch..
Fast forward to July this year and I received a message via Facebook from him saying he seen me about our estate a fair bit over the last several weeks and asking how I was. We chatted via text over the next week, then met up for a walk the following week, and a couple of walks the week after for 2 or 3 hours at a time and just fell into chatting away as though we'd been in touch since our date all those years back and there was a strong physical connection/chemistry between us.. three or four weeks in we slept together. He then had a week off work, and we spent time going walking, cycling, out for a picnic, enjoying a cuppa and a chat and falling into bed together a further few times. He then went quiet on me over the next week, short one word/one sentence replies to texts, I put it down to his long and rather stressful week back at work, as being manager he had a lot to catch up on following his return to work after his hols. We caught up for a couple of cuppa an chats, and walks over the following few weeks, and slept together again. The day after we slept together he invited me out for a walk, but he was really off with me, snappy and on occasion made a few mean comments, it was like he didn't want to be with me. The next day I messaged to see how his day had been at work and he eventually responded about 4 hours later with, 'not ignoring you, **** day x' So I empathised with him and said I was there if he needed a chat. The next evening he messaged me to say, 'another **** day x'. I again empathised and said it sounds like work was really quite tough at the mo. He responded, 'Yep, it's easier to shut myself away and ride my bike', I said I could hear things were difficult and it sounded like he needed some space, and that I was here if he needed a chat.. and I left it there. Towards the end of the same week, I received a message from him wishing me a nice holiday away (I was off on hols for a week the next day) and to please let him know when my two sons and I had arrived safely. I messaged when we had arrived and during the week of my hols he messaged almost every day in the evenings to see how we were/what we'd been up to.
The day after returning home from my holiday, he asked me round for a cuppa. We chatted away about my holiday, his work etc, then he told me he was sorry but that he couldn't be in a relationship with me because he didn't want to hurt me. We decided to remain friends which we were both happy with.
Since then (over the last 3 weeks) he has texted me every few days or so and we've chatted via text, I've not initiated any texts in that time but have responded to his and enjoyed the little bursts of messaging. He invited me over for a cup of tea last weekend and we had a good old chat, but he was also flirting with me, chatting about our previous bouts of lovemaking, what he thinks about/of me, and hugged me tightly before I left to go home. Over this last week he's again been pinging a text every few days, and yesterday he sent me a picture of the box of teabags he'd bought for me of my favourite tea, and this afternoon he invited me round for a cup of tea. I popped by this afternoon and we chatted away about our week, he then turned the conversation onto him setting up a dating profile for me, which I thought was a bit of strange thing to say. This led on to some deep conversation about past relationships mostly mine, and him welling up with tears and apologising for hurting me, which led to me shedding a few tears. We then talked more about this and that and conversation flowed back to relationships and our lovemaking. We agreed we are strongly physically and sexually attracted to one another, and I told him that I can't have the physical attraction without the emotional attraction, I can't just switch that part of me off, it's just how I am. He said he understood and said he struggles so hard to make the emotional connection for fear of the past repeating itself and being afraid of change. There were more tears on my part, as I felt quite emotional at this point (I've a lot of other things going on in my life at the mo - mental illness with my son and my father). He hugged me, told me I'm beautiful just as I am and told me he's there if I need a friend. I left a short while later once I calmed down, and he stood up as I did said come here gorgeous, and pulled me in for another hug and we kissed one another's cheeks. Just before I left he said, 'I'll get the fire pit set up one evening when the nights get a bit darker and we can sit outside and chat with our cuppas'.

I'm not sure if writing all this down helps to explain why I'm confused.. I'd like more, but I'm not sure if he does or doesn't because from my point of view I'm getting mixed messages.. So I guess I'm asking for an outside perspective because my perspective feels somewhat blurred.. What are your thoughts? How do you see this from your perspective? What would you do in my position?

Thanks for listening/reading x