t-tests of adjusted means after an anova with within- and between-subjects factors?

hi there,
does anyone know how to compare group means that have been adjusted for
a covariate? i would like to run t-tests to interpret an interaction in
my 2x2-design (1 within-factor with two levels, 1 between-factor with
two levels, 1 covariate, 1 dv).
My first approach was to adjust the individual raw values in the dv
using a regression (covariate-dv)and then to calculate the means and to
compare them, but it didn't work. now i have no idea what to
do....somebody please help!!!
martin

 

You may have to use MANOVA rather than GLM. I don't have an example for
a split-plot design like yours, but here's one for a two-way design with
both factors between-subjects.

MANOVA
y BY a(1 2) b(1 3)
/NOPRINT PARAM(ESTIM)
/METHOD=UNIQUE
/ERROR within+residual
/DESIGN= a, b W a(1), b W a(2).

This will give simple main effects of B at each level of A. I expect
you can do something similar for your design. MANOVA is not available
via the pull-down dialogs, so you'll have to look it up in the help
files and write your own code.

 

thanks for the fast answer! i will try using manova, although it will
take some time because i'm not that familiar with the spss-syntax yet;
unfortunately i've always worked with the pull-down dialogs another
problem is that i'd like to do all possible comparisons with the
adjusted means (group1: condition1 vs. condition2, group2: condition1 vs
condition2, condition1: group1 vs. group2, condition2: group1 vs. group2).
isnt' there a way to build new variables by reconstructing the
adjustment of the dv made by the covariate and then run normal t-tests
(for dependent & independent samples) with these variables? a colleague
told me that saving the residuals of the regressions: covariate (iv) -->
dv and then comparing the means of the residuals would do the job. yet
another colleague told me to use the betas of the above mentioned
regressions to build "adjusted variables" with a calculation like this:
ADJUSTED INDIVIDUAL SCORE DV = INDIVIDUAL SCORE DV - (BETA * INDIVIDUAL
SCORE COVARIATE). by now i'm totally confused since both approaches
didn't work, but i would really prefer to use some kind of "adjusted
variables" and compare their means, because i never worked with the spss
syntax so far. do you know any easier ways for an spss-beginner like me?
thanks again,
martin
ps: maybe it is helpful when i describe my study in more detail: 2
groups (defined by extreme motivational aptitudes), each group was
exposed to 2 conditions (2 different instructional methods), each group
reported their emotions (dv) in each condition, self-concept as a
covariate to control for differences in emotions due to different
ability-related beliefs

 

Hi

Without considering the covariate, the following would illustrate
MANOVA simple effects analyses for mixed (split-plot) designs.

manova y1 y2 BY a(1 2) /WSF = B(2)
/wsd = b /design = mwithin a(1) mwithin a(2).

manova y1 y2 BY a(1 2) /WSF = B(2)
/wsd = mwithin b(1) mwithin b(2) /design = a.

I'm not certain with these examples, but I have noticed
occasionally designs where you cannot in fact include all of the
called-for effects (e.g., an effect for each of the 3 dfs in the
above examples).

Best wishes
Jim

============================================================================
James M. Clark (204) 786-9757
Department of Psychology (204) 774-4134 Fax
University of Winnipeg 4L05D
Winnipeg, Manitoba R3B 2E9
CANADA http://www.uwinnipeg.ca/~clark
============================================================================

 

---- snip -----

Just a quick question: Do you mean you that based on scores from a
motivational aptitude scale, you sorted people into two groups (i.e.,
group 1 scored higher than some cutoff, and group 2 lower than the same
or a different cutoff)? If so, you might consider using the actual
scores as another covariate rather than as a basis for sorting folks
into extreme groups. Here is a summary of some posts on that issue from
a year or two ago:

http://core.ecu.edu/psyc/wuenschk/StatHelp/Dichot-Not.doc

 

i heard about the problems of median splits and the like; in fact, i
used upper and lower tertiles to form my groups. however, this has been
done for quite important theoretical reasons (the interaction i wanted
to test - an aptitude-treatment-interaction - is only supposed for
extreme aptitudes rather than for the whole range of the aptitude) so i
decided intentionally to accept less statistical power and variance. of
course i could form the groups by using TWO cutoffs (motivation AND
self-concept) and integrate no covariate at all into the model, but that
would lead to a really serious sample-size problem . i had no
problems performing the necessary computations so far - i just can't
handle the post-hoc interpretation of my interaction, because i don't
know how to compare the 4 adjusted means. i think that you can compare
adjusted means with t-tests, because i found an article (in german,
unfortunately) with a design similar to mine (a training study; they
used the pretest-scores as a covariate) but i have no idea how to do
this. what do you think about the idea of my colleagues (building new
variables by reconstruction of the covariance adjustment -->
dv-variables with "adjusted individual scores" --> compare the means of
these variables with t-tests)? is something like this possible at all?
thanks,
martin

 

i heard about the problems of median splits and the like; in fact, i
used upper and lower tertiles to form my groups. however, this has been
done for quite important theoretical reasons (the interaction i wanted
to test - an aptitude-treatment-interaction - is only supposed for
extreme aptitudes rather than for the whole range of the aptitude) so i
decided intentionally to accept less statistical power and variance. of
course i could form the groups by using TWO cutoffs (motivation AND
self-concept) and integrate no covariate at all into the model, but that
would lead to a really serious sample-size problem . i had no problems
performing the necessary computations so far - i just can't handle the
post-hoc interpretation of my interaction, because i don't know how to
compare the 4 adjusted means. i think that you can compare adjusted
means with t-tests, because i found an article (in german,
unfortunately) with a design similar to mine (a training study; they
used the pretest-scores as a covariate) but i have no idea how to do
this. what do you think about the idea of my colleagues (building new
variables by reconstruction of the covariance adjustment -->
dv-variables with "adjusted individual scores" --> compare the means of
these variables with t-tests)? is something like this possible at all?
thanks,
martin

 

i heard about the problems of median splits and the like; in fact, i
used upper and lower tertiles to form my groups. however, this has been
done for quite important theoretical reasons (the interaction i wanted
to test - an aptitude-treatment-interaction - is only supposed for
extreme aptitudes rather than for the whole range of the aptitude) so i
decided intentionally to accept less statistical power and variance. of
course i could form the groups by using TWO cutoffs (motivation AND
self-concept) and integrate no covariate at all into the model, but that
would lead to a really serious sample-size problem . i had no problems
performing the necessary computations so far - i just can't handle the
post-hoc interpretation of my interaction, because i don't know how to
compare the 4 adjusted means. i think that you can compare adjusted
means with t-tests, because i found an article (in german,
unfortunately) with a design similar to mine (a training study; they
used the pretest-scores as a covariate) but i have no idea how to do
this. what do you think about the idea of my colleagues (building new
variables by reconstruction of the covariance adjustment -->
dv-variables with "adjusted individual scores" --> compare the means of
these variables with t-tests)? is something like this possible at all?
thanks,
martin

 

Did I misread this thread? if there is a 2*2 design, there is only one
"difference of differences", i.e., 1 df. the t is the square root of F.
sketch out the results.
use the DV on the Y-axis (vertical).
Put the within subject group factor on the X-axis (horizontal).
Plot the 4 adjusted means.
draw a line segment between each pair of repeats (one line for each
level of the between group factor).

If the F for interaction is significant the lines depart from parallel
to a degree that is not compatible with random variation.

Seriously consider leaving the between subjects factor continuous and
use a "regression approach". remember to center the between IV and
"covariate" before creating the interaction terms. bends are created by
using power terms square for 1 bend cube for 2 bends.



see:
Cohen, Jacob, et al (2003) Applied multiple regression/correlation
analysis for the behavioral sciences, third edition. Lawrence Erlbaum
Associates, Mahwah, NJ.
ISBN 0-8058-2223-2
LoC HA31.3 .A67.2003

Art

Social Research Consultants
University Park, MD USA
(301) 864-5570

 

hello art,
first let me thank you for your interest in my problems with spss. i
think i somehow solved my problem now - i assessed my interaction by
disentangling my split-plot-design, i.e. computing 1 single-factor
within-subjects anova for each group and one between-subjects anova
(with the covariate) for each treatment condition. i found this approach
in: Keselman, H.J. & Keselman, J.C. (1993). Analysis of repeated
measures. In L.K. Edwards (Ed.), Applied analysis of variance in
behavioral science. New York: Dekker. of course, i adjusted alpha for
the number of comparisons.
a remark to your suggestion concerning using a regression approach: in
fact i first considered to do that, and consulted the book of cohen you
recommended. however, i really want to compare extreme groups only,
because my ATI is only supposed for extreme scores on the aptitude. as i
understand it, the biggest problems with median splits, quartiles, and
tertiles (which i used) concern a) that you can not be sure that the
treshold or cut-of you used is the "real" drawing line between the
groups and b) that you lose a lot of statistical power, i.e. a R of .44
tuns into .30 when you dichotomize the sample. well, in my case,
concerning a), i had no other choice due to theretical considerations,
and b) only makes the test of my assumptions more conservative. but i
have to admit that i'm not that much into statistics (you should have
noticed that by now ), and maybe this kind of arguing is complete
nonsense. at the moment i`m reading Waller & Meehl, 1998, Multivariate
taxometric procedures: Distinguishing types from continua. Thousand
Oaks, CA.: Sage Publications, i heard from a colleague they discuss when
and how to dichotomize continous variables. again, thank you very much
for your helpful remarks; just in case you want to look at my design in
more detail, i attached the spss-procedure and the main results
(including a graphic display of the interaction, just as you
recommended). best regards,
martin

 

"why i transformed a continous variable", part II: as i already said, my
study is concerned with aptitude-treatment-interaction-research (ATI),
which has been "invented" by cronbach & snow ("aptitudes and
instructional methods: a handbook of research on interactions", 1977).
here's what they had to say about extreme group designs and what led me
to use one: "Measuring an aptitude, dropping cases from the middle of
the distribution, and randomly dividing cases in each of the tails to
form treatment groups produces a comparatively powerful design." (p.59).
And: "A design treating upper and lower thirds of the population, or
upper and lower quarters, is appreciably more powerful than a study with
the same N distributed over the full aptitude range (...). With more
extreme selection of cases, the efficiency of the design is even
greater" (p. 60).