@Linkat,
While your brother is in the hospital, he should have a social worker as part of his treatment team. That person should be very involved with his discharge planning--where he will go for his follow-up psych care, where he will live, etc.---and your family, at least your mother and nurse brother, should meet with that person ASAP if they have not already done so.
Your mother can refuse to take your brother back into her home, in which case the social worker would be obligated to find an alternative living arrangement for him, even a temporary one, because they can't just discharge him to the street. Beside group homes, there are all sorts of supported independent living programs, day hospitals, and other arrangements that might be available and suitable for your brother. That's why the family needs to discuss this with a social worker who is familiar with your brother's psychiatric condition, and with your brother as well. Since your brother is now taking medication, is his functioning/thinking any less paranoid? Could he now participate in his discharge planning?
I think the best thing your family could do now would be to lean on the hospital staff responsible for your brother's current care and enlist their assistance in both short-term and long-term planning for both his psych treatment and living situation so that your family doesn't have to continue to struggle with this on their own.
Hopefully, they will have your brother on some meds that alleviate his paranoid, and possibly depressive, symptoms without causing him unpleasant side effects--that might increase the probability he will continue to take the meds after his discharge. However, medication non-compliance is a major problem with the mentally ill, and they will often discontinue their medication because they feel better and don't think they need it any more. Unless they are posing an immediate threat to self or others, medication cannot be forced on a person. The best approach is, therefore, to do everything possible to support voluntary compliance with medication--and the treating psychiatrist plays a big role in that. Your brother has to be helped to understand and control his illness, with medication, so that it doesn't wind up controlling him. And he might be quite capable of understanding that, particularly with the ongoing help of a therapist that he trusts..
I really hope your family--particularly your brother--will be able to get needed assistance and support--and intervention-- from the hospital staff before he is discharged from the hospital. Among other things, they might help him apply for disability if his psychiatric problems interfere with his ability to work. If disability is granted, it will also provide him with Medicare health insurance coverage.