Item Discrimination Values

Hello everyone,

I have been assigned to work on a 3 point likert scale by my professor
and I have been doing the reliability analyses for a while now. The
next thing I should do is to calculate the item discrimination values
and their significance levels, which I could not find how to.

I have to use the upper and lower 27% technique to calculate the item
discrimination values and significance levels.

Could you please tell me how I should do that?

Thanks a lot.

 

"Likert scales", in the narrow description, are what you get as
an additive scale made up of a number of symmetrical "agreement"
items. It is probably most common to have 5 anchor points for
each item; three is unusual.

The main sort of reliability that is possible from just using the
items themselves is internal consistency (for instance, Cronbach's
alpha). You might also look at the homogeneity of the means,
but that is a different consideration than simple "reliability".

When I think of "discrimination" as an aspect of reliability, I
think mainly of discriminative validity -- which requires some
sort of criterion, which you do not mention. But I also do not
know what you are referring to as the "upper and lower 27%
technique" so I wonder if you assuming that items do have
different means, and have some technique in mind for
"discriminating" extremes, in relation to the total score.
Presumably, that is a method described by *someone* -- Do
you have a reference?

Do you have a text? (Read it!) Does this fall under IRT,
or some other standard name?

 

Hi, thanks for the reply,

The 27% thing that I mentioned is written here
http://www.spsstools.net/Syntax/ItemAnalysis/UsingSPSSforItemAnalysis.pdf.
And I have seen that used in many papers, especially by an educational
sciences professor at my university (who unfortunately only sells
information unless you take a course directly from him), that is why I
feel like I need to do it.

In the link it is present as "The Index of Discrimination", I need to
calculate that.

Thanks again,

 

The paper describes it. It is the difference in raw percentage
between the fraction of right answers for (in the paper) the top third
versus the bottom third of the test-takers. The context for this part
of the paper seems to be the examination of multiple-choice answers,
with Right vs. Wrong. I suppose that educators may find this useful,
especailly if most of the test answers are supposed to be equally
difficult.

However, it does seem to me that it would be awkward and not very
profitable to use it for Likert-type scales, which is what you said
that you have. Neither extreme is a "right" answer, so you would
probably need to look at both.

As the paper says about that index, "you can get these results
using SPSS." I presume that you would start by forming some
three groups from the Total score, and then you could aggregate
to count the number of particular answers for each item, and so on.

The Cronbach alpha and the corrected item-total correlation are
much more appropriate for scales. Those are available from
the Reliability procedure.


I noticed a couple of things about this paper. It has a copyright
date of 1998, and it says,

SPSS Inc. is a multinational software company that delivers
“Statistical Product and Service Solutions".
- I really don't remember anyone suggesting that the SPSS
initials stood for that. It started out as (IIRC) Statistical
Package for the Social Sciences, and then it became initials-only.
Did that alternate expansion of SPSS ever get used much? Is it
something I've forgotten?


The paper itself talks about "point-biserial correlation" instead of
just saying "Pearson" or "product-moment" r. I didn't think that
anybody talked about "point-biserial" after we got well into the
computer era, and it was no longer necessary to point one's
research assistant to the proper, easiest hand-formula to use.
- That gave me a somewhat negative opiinion of the paper
before I had barely begun reading it.

 

Hello again,

Well, I guess you are right, and I found the Croanbach Alpha and Item-
total Correlations (for each factor) as you suggested (And this is
confirmed by many different papers using Likert-scale).

On the other hand, I have also come across with something again for
reliability, called "item-remainder correlation, (rir) in short". Do
you think I should calculate this? If yes, do you happen to know how
to do that on SPSS since I failed to find any direct calculation?

Thanks for all the help once again,

 

Hello again, and thanks for all the help.

I found the croanbach's alpha and item-total correlations as you
suggested, which is also confirmed by many different papers.

Meanwhile, I have come across with 2 different values which are used
for reliability, the first one of which is called "item-remainder" or
rir in short, and the second one "Rulon" coefficient. Do you think I
should also calculate these? And do you happen to know how if yes,
since I failed to find any direct calculation?

Thanks again,

 

People have inferred all sorts of meanigs for SPSS, which started (before my time) as Statistical Package for the Social Sciences. I've heard, besides Statistical Product and Service Solutions,

Superior Performing Statistical Software
Statistical Package for Scaring Students
Self Propelled Semi Submersible (shows up a lot in drug interdiction)
St Paul's Secondary School
Surfers Paradise State School

among many others,
not to mention the unlammented PASW.

Only the first was ever official except, briefly, for PASW but it was abandoned long ago. Now it's just SPSS. It stands for, well, SPSS, and all the products now have a multi-part name such as IBM SPSS Statistics, IBM SPSS Modeler, ...

Now, does anybody know what the numbers stood for in the name of the IBM 1401 (my first computer)?

 

If you just computed a correlation between Item and Total, it
would be inflated by the fact that the item is part of the Total.

That is why the Reliabilitiy procedure provides the "corrected
item-total" correlation. It is the correlation of an item with the
remainder of the items. (Isn't this information in the Help file?)

I've never seen anybody offer a split-half reliability (Rulon)
since Cronbach's alpha (which is a generalization of it) became
available from stat-packs.

Googling on < Rulon reliability > found two versions of the
formula in the first 10 hits, though they may read obscurely
since they are not simple computation formulas. But you don't
want them, anyway.

 

Then it means I have completed the scale construction analyses and I
can move on to apply the questionnaire to my sample.

Thanks for all the help, Mr Ulrich, I can can hand in a good research
homework thanks to you, I appreciate that.

 

Depending upon who is doing the spin:
Smart People Selling Software.
Silly People Selling Software.