Casual Hookup Sending Mixed Signals

Okay, so I hooked up with this guy. Good time, very casual, fun and uncomplicated. We said we were going to meet up in the future, and me being the straight-forward lady that I am I texted him asking if he wanted to have a repeat. Got no reply and after a few days, I just deleted his number and assumed he wasn't interested anymore or never was, which like no big deal we met once. But then a week later he asks me about meeting up over the weekend sometime, and apologizes for not answering cause he had family troubles. I didn't answer for a few hours and he texted me again saying "Not about it?", to which I responded that I was, but had plans most of the weekend. I asked if I hit him up Sunday/the next week and if he'd be up for it, and again complete silence -_- I texted him a day later saying, "So I'm guessing that's a no?" and more silence. I just don't get this person. I'm not the type to play games, and if we're both adults that want to have sex with each other I don't get how this is so hard. I deleted his number again just cause I don't plan on hearing anything else, but I feel guilty for still wanting to have sex with him (the sex was REALLY good), and normally if it was a dating thing I would have been cut them off, but it doesn't seem as big a deal since we have no ties to one another and it is supposed to casual. Reaching out and not responding is weird to me, though.....

Should I just ignore him if he tries to reach him out again, or would it be worth trying again if he made contact first? Why is he sending weird mixed signals? I know it's completely possible he got absorbed in something again, but I know if I have the chance to have sex it's taking some priority.