@akafakt,
Do you mean the other page likes your page, or that you like another page?
If the other page likes your page, you cannot change that.
If it is your like, you have to go to the other page and "unlike" it.
From your description it appears that it is the other page liking your page. That stays.