I'm not the worlds biggest sports fan, but my son played soccer and was in little league and of course I went to all the games. I've been enjoying the World Cup, although I can't say I'm very knowledgable about the fine points. Also my husband played very competitive softball for many years. I loved watching him play, he was a natural. He's also nuts about college basketball, ergo I've become a fan.
Even if you can't do color commentary while watching, I think it's amazing to watch athletes compete in surprising ways. I think everybody can appreciate a skilled play by skilled player.
I did notice that you've been interested in the games, I have been as well. I also understand that some folks find it boring, so if they do, they shouldn't watch. I knew so little about basketball when I first met my husband, my early characterization was that whoever started first would win the game. But I gradually learned to appreciate the skills involved. I still can't call traveling, but my husband usually jumps up, screams "TRAVELING" and makes that signal with his hands you see the coaches make. That's my tip off that traveling has occurred.