How to find whether there is a significant difference between these data sets

Hello! It's been a while since I took my introductory statistics class, and I think that the situation I've found myself in is a little bit beyond what I learned there anyway.
I want to find whether there is a significant difference between the means of two sets of data. Each data point in the data sets is the mean value of a smaller data set, but each of these smaller data sets have a different sample size. I have the sample sizes of the smaller data sets, but not the standard deviations. How should I go about this?
Edit: Also, the overall distribution of the data doesn't seem to be normal (it is for one of the data sets, but not for the other).