dedmedved- I read a novel earlier this year that included graphics. Its narrator was a boy who had autism, who drew maps and pictures to help explain the text, which consisted of explanations of his daily routine,etc.
I thought they were an asset to the story as they added to the authenticity of this boy's particular state of mind. I thought they also added to the humor.
One of my favorite parts of the book in fact had to do with the fact that this boy had trouble "reading" peoples' faces. He used his own little smiley faces to illustrate it - but I'll use emoticons. He said, "I can understand what it means when peoples' faces look like this:

and I can understand what it means when people's faces look like this:

, or this:

- but I don't understand what this means:

, so I asked my teacher Siobhan. She told me not to worry about it, because she could never understand what

meant either."
It sounds like a map would fit in with your general theme. I know I always appreciate having a sense of place set or described as thoroughly as possible when I'm reading a novel.
I have no idea about how that would affect marketing or sales though.
*Sorry - I just reread your post and realized that the pirate map was ony hypothetical. I guess I would say, if your graphic enhanced the story and its insertion fit the personality characteristics of your narrator - in other words if there was a compelling reason that helped propel the story or cement a theme, I think it might work.