ghosts

-" 'It is simplicity itself,' said I. 'When you bared your arm to draw that fish into the boat I saw that J. A. had been tattooed in the bend of the elbow. The letters were still legible, but it was perfectly clear from their blurred appearance, and from the staining of the skin round them, that efforts had been made to obliterate them. It was obvious, then, that those initials had once been very familiar to you, and that you had afterwards wished to forget them.'

" 'What an eye you have!' he cried with a sigh of relief. 'It is just as you say. But we won't talk of it. Of all ghosts the ghosts of our old loves are the worst. Come into the billiard-room and have a quiet cigar.'

=================
My interpretation:
-In our lives we have come across people who are now dead, so ghosts. Among these people are those we have loved, but now when I think of them, they bring back unpleasant memories, so worst.

Do you agree?