If your late payment was reported to credit agencies, yes, it will negatively impact your scores. And being over your limit itself may do so as well.
But over time your scores can recover if you show responsible credit history.
The key things:
1) No negative reports of late payments of failure to pay.
2) Balances well below the limits. If you are even close to your overall credit card limit it will negatively impact your score.
3) Time. You need to show a history of this credit behavior for even higher scores.
Basically what they are looking for is to see if you are overextending yourself, the number of credit cards you have may itself be a negative though canceling them can make your balance to limit ratio look worse.
The bottom line is to stop spending on the cards and start paying them off. This makes sense for reasons other than your credit score. You already got hit with a late fee, just be more thrifty and get ahead of the spend. It will pay off for you in many ways, the least of which is your credit score.