Sure you did and it is only 'natural' that as a teenaged boy you would have enjoyed looking at a beautiful woman. For my part, I would rather see a man in the same setting!
You're also a decade younger than me, because I was in my late 20s when the movie came out. However, as a woman, I genuinely enjoyed seeing Colin Firth in the sort of glasses . . . but with a better haircut . . . that were popular among intellectual men, whether they worked in media or higher ed . . . around the time I came of age, sexually. Of course, I admired his performance in A Single Man, but, the sight of a handsome man in those early 60s glasses was more exciting than the tasteful nude scenes in the water.