I'm responding from the other side. I lived with a man for 20 years and I had a child; he did not. We each had our own bank accounts and we opened a joint one for the monthly expenses, which we split 50-50. As for the mortgage payments, we each paid according to our incomes, ie. if he made 2x as much, he paid 2x as much. That way, we each had the same percentage left of our salaries.
I agree with whoever it was that suggested this might be a control issue, not that I know the man or you. But to suggest that you should put your whole paycheque into ONE account, and that you should be responsible for his 3 kids' expenses, treats, whatever is not right. They are not your kids, so why should you pay for them?
I agree with you and don't think you're being selfish in the least.
Keep us posted!