How should a non-expert read an AI-detection report?

Like pretty much any form of communication, it depends on your audience.

Most workers aren't going to read an extra report/caveats unless that stuff is right there, in their faces (they often don't have the time to hunt for anything). As a result, they may end up believing the AI more than results would indicate they should.

Hence, you're probably going to be stuck with having to throw a bunch of caveats into a report. Which, you're right, could eventually render a report meaningless.

What about using colors and shapes to get some of your point across? Start with, say, three graphs. Maybe the first is for overall review signal. Go with red, amber, green for negative, neutral, positive, just like a traffic light (use a shape like a traffic light to bring the point home). Decide on the % for each color, e. g. 75-100% positive = green; 45-74% = amber; 0-44% = red or whatever you prefer.

Maybe make the next one an arrow pointing downward in black next to an arrow pointing upward in white, to show false negative or false positive chances, respectively. But since these probably aren't perfectly in opposition to one another, you may need a different design. I don't know.

And then maybe make the third one a bar chart showing for sentences of X length (5 words, let's say), the AI level was 68%, for sentences of 6 words, it was 12% or whatever.

You may be able to handle some of the interpretive issues at the start by just asking people what their purpose is in getting this information. e. g. they open the utility and the first thing it asks is: Will this report be used for school, work, legal, or disciplinary decisions?

And then have the utility answer with something like: There are limits on using this information for this purpose. Here they are: ___. Do you still wish to use this utility? Y/N.

This could be tailored depending on intended usage. You could show this with a workflow diagram/flow chart. And maybe even note if the utility is being used against advice.

I wouldn't put people through the third degree before letting them use the utility, so you're probably limited to no more than 3 - 5 preliminary questions before people would stop using it. So, decide what you really need and make those questions count.

Finally, I'd recommend the utility showing sample length and document type no matter what it was used for. That could be with the results. This way, if a document length of under, say, 20 pages, with a 75% chance of a false positive might be an indicator that the sample size is too small to be trusted.

I’d put the main caveat right next to the score. That’s the part people will look at first, so it should be impossible to miss that the score is not proof. I’d also want the report to show why it gave that result in simple terms.
The sample length and document type should also be shown, because a short email and a long essay are not the same thing. And if the result is less reliable for any reason, that should be shown clearly.
Then you can have a longer explanation lower down, but the basic warning should be beside the score. Otherwise, people may just see the number and misuse it.