Boyfriend in Love with a dead woman and reliving her memory through her daughters. WTF?! lol

Hello, I am 55 and have been seeing a man aged 66 for about a year and a half. I lost a partner about 2 years ago and he lost a partner about four years ago. I thought we had each grieved these relationships and were ready to move when we met. We also understood each other in this way and were willing to give each other the space we need to grieve. For example I have listened to stories about her and he has listened to stories about my former partner. My stories are always about some funny screwed-up experience we had. His are always about how she was God-like, the hardest working, the kindest and the best person ever who was abused and hard done by her whole-life. She wasn't. To hear others tell it she was good-looking but that was about it.

He has always had trouble with boundaries. He's a rescuer so whenever anybody calls (especially the female friends he's had for many years) he goes running. We have had words about this before. So fast forward by a twist of fate we meet his dead partner's daughter (aged 40) at a conference. She lives in a different city and asked if she can keep in touch with me. This is the first time she has seen my partner since her mother died. She and her sister did not keep in touch with him. So she and I keep in touch by text as a some as I am a high level at the conference we were at and she says she would like a mentor. That went well for a while until she and her sister (aged 35) come into town. They called me and asked if my partner would like to visit with their grandmother ( his dead partner's mother) because she is in the hospital. I ask him he says sure and we go to visit. I take my leave, the grandmother has no idea who he is and the youngest daughter starts showing him pictures of her baby. The visit is short and I figure this is just a one time thing and we'll go back to me mentoring the eldest daughter so I am gracious about it. About a week later, they are back in town call him directly by passing me (meaning his phone number changed hands somewhere in there) and they want him to go for lunch with him. He, having no boundaries says sure and invites me. I do not want to sit around for hours and talk about this God-like dead partner so I join them later but by now I am seeing this as a clear over stepping of boundaries. I am also in the dark about how often they are contacting him now that they have his direct number. When I get to the table it is the most depressing lunch ever as they are discussing a)how fantastic his dead partner/their mother is (she was once a drug-addicted hooker and had those daughters taken away) is and b) money - they are crying broke and priming him like a pump as their mother got a free ride off him through their entire relationship. At the end of the lunch he yells "I love you girls" across the parking lot. Keep in mind they haven't contacted him for four years. I remain in the dark as to how often they are contacting him on his cell phone now but I'm beginning to see them as a nuisance. I ask him. He explodes and tells me that they were once a family (the daughters were on their own when he dated her mother and the eldest did not talk to her for years) We lasted for one more week and I kicked him to the curb as on-going contact with these daughters to dredge up memories of their dead mother through him and him through them is not going to happen if he is with me. I told the daughters exactly what I thought as well. There are just too many people showing up and hanging around in this relationship for me including the dead woman. If they were his biological kids absolutely, if it was a one time thing absolutely, if they were 12 years old and orphaned absolutely, if he raised them from birth absolutely, if they had been in touch the whole time and when he and I started seeing each other absolutely, if they did not have a biological dad absolutely, if they had boundaries when he is in a relationship and trying to get on with his life and showed up once in a while, absolutely, if the eldest daughter had stuck to the original agreement we had and called me as she had been doing absolutely -- there are a million scenarios I would have tolerated but this is not one of them. This feels very slippery on the daughters' part and on my former partner's part yet I am being told that I'm the bad guy . In my mind there is no reason on earth for them to have his cell number or to even be contacting him as often as they do. I am ok with being the bad guy in this scenario and doing well. I have said my piece very assertively to everyone (ie: the daughters) involved although my partner and I had a few barn-burners before I walked out and in those I wasn't so assertive. He went deer hunting so we have not been in touch. I sent two possibly three text messages to the daughter and her sister both stating my displeasure and asking them to put themselves in my shoes and think about how they would like me frequently contacting their partners if I was part of his past life if they are even capable and will still mentor her if she wants because I am a woman of my word and stick to my agreements. I am at peace with all my actions and will end the relationship but want to know from objective people: "Does this sound fishy to you or did I overreact?