@hingehead,
A2K doesn't cache anything with the gravatars. They cache it themselves and your browsers can cache it, and between the two there are some long delays and can potentially see it go back and forth between them.
Most of the times I saw people saying they saw the wrong avatar, clearing the browser cache would solve it. Other times they just need to wait for gravatar. One hint is when they say that they see the right on on the profile and the wrong one on posts, that usually means the post one is in their cache, but their profile (which they visit less often) isn't.
In any case, gravatars aren't affected by A2K much at all, we just link to them and the rest is between their servers and your computer.