Test to determine if percentage increase between two groups different?

Discussion in 'Scientific Statistics Math' started by Brett, Apr 6, 2007.

  1. Brett

    Brett Guest

    Hi all:

    I have a question that I have been unable to answer from my statistics
    reference books.

    My problem is that I have two groups of consumers, an active group and
    a control group. The active group was exposed to an advertisement at a
    point in time and their purchase decisions for a product were tracked
    weekly before and after that point. The control group was not exposed
    to the advertisement, but their purchase decisions were also tracked
    over the same course of time. The control group was assigned a
    "before" and "after" period corresponding to the same time span for
    the active group.

    The vast majority of both groups made no purchases either before or
    after the exposure. I have categorized the number of persons in both
    groups who made at least one purchase in the before and after periods
    as "purchasers".

    The number of purchasers (>=1 purchase) in the after period was 33%
    greater than in the before period in the active group and 24% greater
    in the control group. A Wilcoxon test shows that the before-after
    change was significant within each group.

    So, my question: Is there a test to determine whether the 33% increase
    in the active group is statistically different from the 24% increase
    in the control group? In other words, did the advertisement actually
    have an effect, given that there was also increases in purchasing in
    the control group over the same period?

    I can find plenty of tests that address the before and after within
    each group (paired samples) and before/after between groups
    separately, but nothing that seemingly addresses before and after
    simultaneously between the two groups.

    TIA for any ideas or suggestions!

    Brett
     
    Brett, Apr 6, 2007
    #1
    1. Advertisements

  2. Brett

    Brett Magill Guest

    Have a look at the McNemar test for paired proportions. From a quick
    search:

    http://www.medcalc.be/manual/mcnemartest2.php
     
    Brett Magill, Apr 6, 2007
    #2
    1. Advertisements

  3. Brett

    Brett Guest

    Thanks, Brett. I actually use the MedCalc software as well.

    I had looked at using the McNemar test and it seemed to me that it
    didn't handle what I'm looking to do. In the MedCalc example given,
    the test looks at a single group now and later that had a disease or
    not:

    Disease Later
    Disease Now Yes No
    Yes a b
    No c d

    Casting my problem into the format above, "Yes" would correspond to a
    purchaser, no to a non-purchaser (so the sum of purchaser and non-
    purchaser adds to the overall group size).

    However, what I'm asking seems to involve *two* 2x2 tables, since
    there are two groups (active and control) who are composed of
    purchasers and non-purchasers at two points in time:

    Active Purchased After
    Purchased Before Yes No
    Yes a b
    No c d

    Control Purchased After
    Purchased Before Yes No
    Yes e f
    No g h

    As I understand the McNemar, you need to include both the purchasers
    and non-purchasers in order to calculate the proportion of purchasers
    to total at the before/after points. So that leaves me with the 8 data
    points if I want to compare control/active, before/after, purchase/non-
    purchase. In other words, I can't set up the problem like this:

    Purchasers After
    Purchasers Before Control Active
    Control a b
    Active c d

    since the controls and actives aren't paired observations. All the
    pairing reflects the behavior within each group before/after.

    Hope I haven't confused the issue too much, but just wanted to set
    forth my understanding of why I thought the McNemar wouldn't work for
    my application.

    Thanks,

    Brett
     
    Brett, Apr 7, 2007
    #3
  4. Brett

    John Kane Guest

    I'm not a statistician but it sound like you have a categorical
    repeated measures analyis of variance. I think something like CATMOD
    in SAS might be a good software approach.
     
    John Kane, Apr 7, 2007
    #4
  5. You can look at Post by itself, and compare means of number
    of purchases, or counts for purchase/no-purchase. That is not bad
    when there is very little Pre variance -- and thus, very little
    correlation.

    For the distributions as you describe them, it sounds as if the
    hypothesis is not about great numbers of purchases, so you might
    want to trim the counts to (0,1) or (0,1,2)

    There are 2 general approaches for change scores.
    a) Compare crude changes: Do a t-test for group A vs. B on the
    change scores.

    b) Compare regressed changes: Do a oneway ANCOVA on the
    counts Post, using Pre as covariate.

    All of these are prone to give faulty inference if the two groups
    differ at Pre. If they differ at Pre, there is no sure answer unless
    the LESS group becomes the MORE group.
     
    Richard Ulrich, Apr 7, 2007
    #5
  6. Brett

    Brett Guest

    Thanks, I appreciate all of the suggestions. As I thought, this isn't
    necessarily a problem with a straightforward application of a standard
    test.

    I have previously looked at comparing the mean of the differences
    between before and after purchases between the active and control
    groups. Per Rich's last point, one problem is that the control group
    isn't well matched to the active group (the active group is more
    likely to have purchases than the controls both before and after the
    ad exposure, so on this measure, there is a significant difference
    between controls and actives). So I think one solution is to go back
    and better match the controls to the actives on pre-exposure
    purchases, then compare the before and after delta.

    Thanks,

    Brett
     
    Brett, Apr 9, 2007
    #6
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.