@harukawaii,
Don't make it about yourself. Make it about your siblings, and about your father. "I see how it's affecting them. I know you have a new life, but we still want to be in yours."
And, BTW, offer to really be a part of his life. You said he lives nearby. Well, then visit if you can, or call! Not to complain, but to see how he's doing. Try to get along with your stepmother, and with any step siblings you might have.
Am I asking you to do about 75 - 90% of the heavy lifting? Yes. Sometimes the younger person has to be the grown up in the relationship. This is one of those times. And it is time for you to make it less about how he hurt you (even though he did) and more about others.