distribution comparison

What is the data type of the tests? If they are binomial (pos/neg) then the process is a straight forward comparison of two binomial variables. Are they quantitative? If they are continuous variables then you can standardize the results around a standard normal curve or do some other transformation that allows for direct comparison. If you don't want to make any assumptions about the shape of the distributions then you're looking at a non-parametric test. Again, depending on the data type, there are different non-parametric options.

Hi. Thanks for the interest on my post.
The data are continuous variables. Each of the two tests yield two clusters of result, that we assume are the negative cluster and the positive cluster. There is almost no overlap in the distribution of the positive and negative data. And this for each of the 2 tests we want to compare.

Start with a scatterplot of the results with one test on one axis and the other test on the other axis. This will give you a visual idea if elevated values of one test correlate to elevated values of the other test.
If you have clear separation of elevated results on both tests then it's easier to assume you're comparing a pos and neg population on two tests.

Can you tell me what the tests are testing for?

Oh, at this point you don't have to transform or standardize the data. The scales of the two axes do not need to be the same. You're just looking to see if you can separate the results into two subgroups based on elevated results on two tests.

Good morning,
Here are more details about the actual project. The tests are urea breath test to assess the presence of a bacteria (Helicobacter pylori) in the stomach responsible for common type of gastritis. It is an old test. Both tests have been validated previously with the gold standard, which is biopsy from gastroscopy, slightly more invasive than breathing in a tube ;-) We are in the preliminary phase and we only have 180 data entries in one of the test.

Test1 : C14 urea breath test: measurements of radioactive counts over background in the sample: positive if higher than the cutoff value, non-diagnostic if too close and negative if really low value.
Test2: C13 urea breath test: the new test also validate against biopsies specimen. Positive or negative results given by deflection of a beam of near-infrared light. Almost no value near the cutoff.

The question is thus to assess if the test2 give less indeterminate or near-the-cut-off results than the previous test1.Of the 180 entries we have of test2, there is no overlap of data and a gap of results around the cut-off. From experience, the results of test1 were sometime near the cut-off and we used to repeat tests that were considered indeterminate.

In the preparation of the design of this study, I was looking for the appropriate statistical test to assess the difference in separation of the clusters of negative and positive of both tests. The scatter plot of the results is a great idea to visually assess the distribution of the date. Should I test if the positive cluster and negative cluster each follow a Gaussian distribution before computing the difference in mean +/- sd of each cluster? Is there a more global way of testing the results?

Thanks

Perfect! You DO have two binomial outcomes! The grey-zone results are considered negative. If you have really want to focus on the grey-zone (we used to do this in viral testing of blood donors that didn't have an associated confirmatory assay except we called our grey-zone samples positive to keep them out of the blood supply) then you can focus purely on the range near the cutoff. Establish the grey-zone of interest - S/CO of 0.9 - 1.1, for example, and then simply count the number of samples that fall in the grey-zone.

You're doing both assays on every sample, right? You'll have a very straight forward analysis by McNemar's/Kappa if it's the same sample on both tests, or a chi-square/Exact test if you have independent sample sets - one per assay.

As to the best way to correlate all of the data, use the scatterplot approach for a visual and then see if you can fit a best fit regression line (linear or nonlinear) to the numerical results.

As for significance testing on the separation of the positive (or negative) samples from the cutoff, you'll want to treat each subgroup separately. Convert each outcome to an S/CO value (which accomplishes the standardization across assays that you need) and then do a t-test (or nonparametric alternative, if there's no reasonable way to say that the s/co values follow a normal distribution) on the means. The t-test (or rank sum test) will take both the mean and sd into account. If you have both results on each sample do a paired-t, if it's independent groups you'll want to do a two-sample t. Do this separately for both the pos and neg samples.

This should be done on S/CO values to on a scale of 0-(higher of the two upper limits of output of the assays). Unless you're doing an RIA on one of them, most assays have an upper limit.


edit: oops, you are doing an RIA on one of them. Is there an upper limit on the counts?

Hi,
Thanks a lot by the way: this helps!
No, we have independant sample sets: the previous test went backorder and we had to change to test2. These are sample sets retreived in a historical prospective design, all comers, no exclusion criterias.
There are no published gray zone. By experience we used to define one on test1, but there is still no grey zone define for test2. May be I could collect a lot of data and find a way to fit a double bell curve on these cluster of data to extrapolate the intersection or overlap of the given curves: is this feasable?

Hi again, I'm trying to follow your thinking, but since I'm not perfectly fluent in english, I am not sure to understand the meaning of the abbreviation S/CO you're using Thanks

S/CO = sample value divided by the cutoff value.

If your sample reads 1000 and your cutoff is 900 then you have an S/CO of 1.11 (positive). If your sample reads 450 then the S/CO is 0.50. Any result >= 1.0 equals positive and any sample < 1.0 is negative. It's perfectly fine if you establish your own grey zone for the new test. You're just trying to compare the proportion of sample with results at or near the cutoff. Just apply the same grey zone to both assays.

Ok, independent samples. That helps. It's likely that you have access to a large amount of historical data. You can generate percentages around the cutoff (both positive and negative) on the historical data. You don't need equal sample sizes, but you want to make sure that you have enough data from your new assay to have enough power to make the comparison to the old assay. If you give me a feel for the prevalence of a grey-zone result in the old assay I can help you identify the sample size required for the new one.

Yes one is RIA and the other is infrared spectrometry. I do not know if there is an upper-limit. They positive clusters seems to be clustered around a central mean value, but there are clearly some outlier that are much more away from the central cluster. That is for test2 preliminary data set we have so far.

Do you see how converting all of your results to S/CO and Pos or Neg will allow you to do two comparisons? You'll be able to assess the reactive rates (% pos on the test population) and distribution of the signals around the cutoff. Converting both of them to a value that is a ratio vs its own cutoff standardizes the data so that you can look at them side-by-side on the same scale.

Another thing you can use is the performance characteristics in the kit package inserts. There should be sections for both sensitivity and specificity in both inserts. These #s are important when comparing two assays. If one has high specificity vs sensitivity (no false positives) and the other has high sensitivity vs specificity (no false negatives) then you already have an identified difference between the two assays.

Thanks!
I will do exactly as you suggest. First we will collect the data and I'll try to perform your proposed analysis. I WILL get back to this post to update it in a few weeks ;-) Thanks again. It was enlightening !

Hello again.
We now have actual data to work with! As suggested the data were transformed in S/CO in each test categories C13 and C14 breath tests.
I have some question regarding the way to analyze the difference in distribution between the positive and negative clusters. For example,

For the C13 breath test, we have 454 patients (min: -0.54, max: 21.81). There is 335 negative results (mean 0,103 +/- 0,084) and 119 positive results (mean 5,902 +/- 2,960).

The negative results are mostly clustered over a very small area of the overall results range, but the positive results spread more. Is there a way to test if the overall distribution follows an exponential or other scale? It is difficult to chart the negative and positive results cluster on the same graph to illustrate the main hypothesis, that there is a greater difference between the pos and neg results in C13 than in C14. Perhaps I should only plot the neg results of C13 and neg results of C14 together in a given graph and find a way to analyses the strength of the distance away from the cutoff.

My ultimate goal would be to create and indeterminate result categories of patients that have result too close to the cutoff and that need a repeat test.

Thanks for any help!