# Test to determine if percentage increase between two groups different?

Discussion in 'Scientific Statistics Math' started by Brett, Apr 6, 2007.

1. ### BrettGuest

Hi all:

I have a question that I have been unable to answer from my statistics
reference books.

My problem is that I have two groups of consumers, an active group and
a control group. The active group was exposed to an advertisement at a
point in time and their purchase decisions for a product were tracked
weekly before and after that point. The control group was not exposed
over the same course of time. The control group was assigned a
"before" and "after" period corresponding to the same time span for
the active group.

The vast majority of both groups made no purchases either before or
after the exposure. I have categorized the number of persons in both
groups who made at least one purchase in the before and after periods
as "purchasers".

The number of purchasers (>=1 purchase) in the after period was 33%
greater than in the before period in the active group and 24% greater
in the control group. A Wilcoxon test shows that the before-after
change was significant within each group.

So, my question: Is there a test to determine whether the 33% increase
in the active group is statistically different from the 24% increase
have an effect, given that there was also increases in purchasing in
the control group over the same period?

I can find plenty of tests that address the before and after within
each group (paired samples) and before/after between groups
separately, but nothing that seemingly addresses before and after
simultaneously between the two groups.

TIA for any ideas or suggestions!

Brett

Brett, Apr 6, 2007

2. ### Brett MagillGuest

Have a look at the McNemar test for paired proportions. From a quick
search:

http://www.medcalc.be/manual/mcnemartest2.php

Brett Magill, Apr 6, 2007

3. ### BrettGuest

Thanks, Brett. I actually use the MedCalc software as well.

I had looked at using the McNemar test and it seemed to me that it
didn't handle what I'm looking to do. In the MedCalc example given,
the test looks at a single group now and later that had a disease or
not:

Disease Later
Disease Now Yes No
Yes a b
No c d

Casting my problem into the format above, "Yes" would correspond to a
purchaser, no to a non-purchaser (so the sum of purchaser and non-
purchaser adds to the overall group size).

However, what I'm asking seems to involve *two* 2x2 tables, since
there are two groups (active and control) who are composed of
purchasers and non-purchasers at two points in time:

Active Purchased After
Purchased Before Yes No
Yes a b
No c d

Control Purchased After
Purchased Before Yes No
Yes e f
No g h

As I understand the McNemar, you need to include both the purchasers
and non-purchasers in order to calculate the proportion of purchasers
to total at the before/after points. So that leaves me with the 8 data
points if I want to compare control/active, before/after, purchase/non-
purchase. In other words, I can't set up the problem like this:

Purchasers After
Purchasers Before Control Active
Control a b
Active c d

since the controls and actives aren't paired observations. All the
pairing reflects the behavior within each group before/after.

Hope I haven't confused the issue too much, but just wanted to set
forth my understanding of why I thought the McNemar wouldn't work for
my application.

Thanks,

Brett

Brett, Apr 7, 2007
4. ### John KaneGuest

I'm not a statistician but it sound like you have a categorical
repeated measures analyis of variance. I think something like CATMOD
in SAS might be a good software approach.

John Kane, Apr 7, 2007
5. ### Richard UlrichGuest

You can look at Post by itself, and compare means of number
of purchases, or counts for purchase/no-purchase. That is not bad
when there is very little Pre variance -- and thus, very little
correlation.

For the distributions as you describe them, it sounds as if the
hypothesis is not about great numbers of purchases, so you might
want to trim the counts to (0,1) or (0,1,2)

There are 2 general approaches for change scores.
a) Compare crude changes: Do a t-test for group A vs. B on the
change scores.

b) Compare regressed changes: Do a oneway ANCOVA on the
counts Post, using Pre as covariate.

All of these are prone to give faulty inference if the two groups
differ at Pre. If they differ at Pre, there is no sure answer unless
the LESS group becomes the MORE group.

Richard Ulrich, Apr 7, 2007
6. ### BrettGuest

Thanks, I appreciate all of the suggestions. As I thought, this isn't
necessarily a problem with a straightforward application of a standard
test.

I have previously looked at comparing the mean of the differences
between before and after purchases between the active and control
groups. Per Rich's last point, one problem is that the control group
isn't well matched to the active group (the active group is more
likely to have purchases than the controls both before and after the
ad exposure, so on this measure, there is a significant difference
between controls and actives). So I think one solution is to go back
and better match the controls to the actives on pre-exposure
purchases, then compare the before and after delta.

Thanks,

Brett

Brett, Apr 9, 2007