@peggysue123,
If he is the father to either or both of the other woman's kids (paternity test FTW), then either or both children are half-siblings to your own children. Your children are entitled to a relationship with them. You do not have to love this although you should allow it.
The rest of it is troubling. If you want out, then go. You'll probably get a lot of what you want in any divorce settlement, seeing as your husband is the one who strayed, he doesn't care for the children well, smokes pots (even if it's legal in your state, it will be seen as being just as irresponsible as drinking in front of them), and you are a wage earner whereas he seems to just be a freeloader.
I do hope you won't make it hard for your children to see their father. Don't consider it as conceding anything to him; consider it as doing the right thing for your children.
BTW, I'm not a fan of snooping or breaking into a phone. Already that should have told you it was over. But that pales in comparison to the rest of it, plus you have plenty of corroboration.
I hope your children can have a decent relationship with their father. I'm sorry this is happening.