He has nothing on you. He also has no authority. Cops, if called, would have smelled liquor on his breath, too.
If this loser tries to blackmail you, go to the cops and tell them what you told us. Possession/use of weed, in a lot (but not all) of the US is a misdemeanor if it's charged at all. Rape is a felony. You may not think so, but you are in a far better position than he is.
If it concerns you, contact a criminal lawyer in your jurisdiction and ask what you can do/tell them your story. Be prepared to pay an hourly rate for the privilege. But at least you'll know, and will have someone to turn to in case things get ugly.
And FYI, while individual police officers and other persons in authority might take bribes and threaten blackmail, that is not our culture. Connections or not (and I think he was bullshitting you anyway, and preying on your ignorance), he can't just call the cops and have them waltz in and arrest you unless there is a reasonable suspicion that you've done something wrong.