I agree with the others you would need to talk with a lawyer and definitely keep track of what you have done so far.
A part from that - have you talked with your ex? Does she have any sort of custody at this point? Also the positive it seems she wants you involved in the child's life as she lets you have her 2-4 days a week. How is that relationship working as sharing the child? You might have an advocate in your ex - in that it seems she sees the positive in having you in the child's life. Does it make sense the two of you work with a lawyer together to get some sort of shared custody?
If money is a problem there is pro bono lawyers willing to help low income people as well.