Byron's "To Ianthe"

The following is the first five lines of the first stanza of Byron's "To Ianthe."

Not in those climes where I have late been straying,
Though Beauty long hath there been matchless deem'd;
Not in those visions to the heart displaying
Forms which it sighs but to have only dream'd,
Hath aught like thee in truth or fancy seem'd;

It's hard for me to understand what the poet means in the third and fourth lines. I think "Not" in the first and third lines deny the fifth line, so it seems to me that the third and fourth lines go like this:

In those visions displaying forms (which it sighs but to have only dreamed) to the heart.

'displaying forms... to the heart' is similar to 'showing forms... to the heart', like "He showed the book to me."
'which it sighs but to have only dreamed' is a relative clause, modifying 'forms'.
'which it sighs' is originally 'the heart sighs (for) forms', omitting the preposition 'for'. This phrasal verb 'sigh for' is similar to 'long for'.
And 'but to have only dreamed' is what I can't get. When I say "I have no choice but to go to school," 'but' means 'except'. If so, then 'but to have only dreamed' can be paraphrased like this: except that I have only dreamed (of them[the forms]'.

Am I on the right track? Can anyone "grammatically" tell me what Byron is trying to say?