I'd say learning any foreign language is truly hard for most of us, including Chinese English learners keen to speak English well just like native speakers. And it takes a lot of doing for you to get to know not just Chinese but Chinese people if you fancy getting the hang of it. I'd try mingling with Chinese people living in your city if I were you, and I would even take the trouble to pay a visit to China with a view to creating an environment for someone like you to learn Chinese.
Guess what? China has scores of foreigners speaking perfect Chinese; they have been living in China for many years. And some of them even have gotten a chance to show up in some top-rated Chinese TV shows as guests or even co-anchors, who are just gung-ho about Chinese language studies.
What I'm saying is if they can pull it off, then you can make it too. Just buck up and hop to it. You will come through if you strive to learn it with gusto; besides, it doesn't hurt to learn Chinese now, thanks to assorted business opportunities proffered by Chinese powerhouses or juggernauts looking to bolster their business operations around the world.