looking through the window

If you look at the window, you are directing your attention at the window itself - the glass, the frame, etc. To see something on the other side of a hole or a window or lens, you look through it.

thanks