Talk:Rating scale

From Wikipedia, the free encyclopedia

[edit] POV

The article currently is about why commonly useed rating scales like IMDB's are crap. It needs to add the following:

• Advantages of IMDB-type scales (ease-of-use,...?)
• A survey of "good" rating scales that have been used in real-life
• A better separation of general-theoretical material from specific examples; the current Background -- Online dichotomy is only partially thus.

The article should ideally be structured someting like:

1. A list of theoretical criteria a rating scale needs to satisy to be reliable
2. Practical difficulties arising in attempting to satisfy one such criteria, or several such together
3. How close in practice can a system which fails to observe a particular criterion come to being "reliable"? How can you even measure this?
4. Examples of different real-world systems: which criteria they satisfy and don't; who thinks they are reliable and who doesn't.

jnestorius^(talk) 21:20, 25 August 2006 (UTC)

I don't have any problems with most of what you say, but these issues don't seem to have anything with NPOV. Did you read the references cited or refer to the polytomous Rasch model article? The theoretical criteria are spelled out in detail in both. Examples of good rating scales would be useful, so why not add them? I'll try to do so if I have time. You state there should be a focus on 'practical difficulties' but you don't explain why. It is worth mentioning such issues, but there is no need for a section. On your third point, it is already explained that it is impossible to get measures of reliability with single ratings. So not sure what you mean. Cheers Holon 01:13, 26 August 2006 (UTC)

I should mention that I have intended all along to bring in examples of rating scales from the literature that have strong psychometric properties -- just a matter of time. This will hopefully balance article more. Please keep in mind that NPOV does not preclude stating definite positions. It only implies that counterpositions should be stated if they exist. If you think there are sources that indicate there are quality online rating scales, perhaps you can refer to these? Holon 01:16, 26 August 2006 (UTC)

Hmmm. I take your point about POV. Allow me to clarify: I am not a potential collaborating editor on this topic, about which I know little (actually, only what I have read here on this page). I have no interest in going deeper into the technical details. I am simply a reader who has read the page and feels they have not understood as much as they might. I got to this page via a wikilink from Internet Movie Database. The information on this page offered me a useful and informative critique of the deficiencies of IMDB's rating system. But reading it provoked other questions, such as what better systems might be available (whether implementable on a website or otherwise). Perhaps all I really need is a better orientation of these concepts with those the other Category:Psychometrics articles. A greater density of interlinking between this and those might help. Maybe even an article series? I realise you may have other priorities, here or in the Real World. I'm just offering some friendly feedback. Cheers. jnestorius^(talk) 04:51, 26 August 2006 (UTC)

Sure, I appreciate the feedback. You're right, the article needs more explanation of what makes for good rating scales. I think it will seem a much more balanced article if there is a tangible example of the use of rating scales. Fair enough regarding contributing, but you shouldn't need to know the area to get the key points, so I'll try to sharpen and perhaps you can give further feedback. Holon 05:35, 26 August 2006 (UTC)

I would love to see the article with a more mathematical approach. Working on scale for my company[1] we discovered that when rating people tend to be far from the average (2.5), the average is more close to 3. And when it comes to entertainment it's higher ... Netflix has an average of 3.3. That's why we used a scale with words.88.160.187.150 11:08, 4 December 2006 (UTC)