Thomas, upstate NY, especially around West Point is so quaint, it will become a favorite spot of yours.
You're preaching to the choir, to the extent that one voice can be called a choir. I have already fallen in love with the Hudson Valley. Three little events were enough.
1) driving across it over George Washington Bridge, where both the look South and the look North is amazing. To the South, you're looking at Manhattan's skyscrapers; to the North, you're looking into a rural landscape. The contrast between those two looks from the same point couldn't be greater.
2) driving across the Hudson on interstate 287 over the Tapan Zee bridge. (Yes, I do a lot of driving lately.) At this latitude, the valley widens to about a mile, maybe even a little more, and looks more like an ocean bay than a river. This view is equally amazing.
3) watching a PBS documentary about Pete Seeger, who in the 70s and 80s invested a lot of activist work into cleaning the river up. The footage suggests that the Hudson Valley is equally stunning between George Washington Bridge and Tapan Zee bridge.
So yes, I'll definitely check out the Hudson Valley. I'll also pay a visit to New Hope, PA.