Wikipedia talk:Scientific peer review/Archive 2

From Wikipedia, the free encyclopedia

This is an archive of past discussions. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.

From the other end

The past six days of brainstorming here (yes, this discussion has been going on for almost a week) have focused on "da board". I think it would be nice to discuss a little about the actual reviews for a change. My proposal: we pick a guinea pig article and try to carry out a review process in vitro. Karol 22:15, 15 March 2006 (UTC)

It's somewhat hard for a board to review an article when that board hasn't been selected. Once it has, though, I wouldn't mind a guinea pig article. Also, we could have all the nominees review an article (or two or three—the solar astrophysicist might not be a good reviewer of, say, an obscure microorganism). That way, we could see them in action and work out the system at the same time.--HereToHelp (talk • contribs) 22:35, 15 March 2006 (UTC)

I was rather thinking about reviewing an article without a board, or with everyone on the board if you like. Karol 22:38, 15 March 2006 (UTC)

Indeed. There need be no paralysis for lack of a board, since not having a board in no way prevents good reviews from being carried out. Try it and see. Go find an article, and see if it can be competently reviewed without the existence of a board. I'll bet you find it can be. -Splash^talk 23:23, 15 March 2006 (UTC)

Alright: Science.--HereToHelp (talk • contribs) 23:30, 15 March 2006 (UTC)

We have our first request, then. Fixed the template along the way, too. Karol 23:44, 15 March 2006 (UTC)

Science is really not a good example. This article defines what science is. It hardly contains any science itself, so it does not need experts to review it. It could be done by WP:PR. --Bduke 23:46, 15 March 2006 (UTC)

I agree. So this is our first problem: how to deal with bad requests. Karol 23:53, 15 March 2006 (UTC)

How to deal with it? The Board discusses it briefly and decides to put it to WP:PR and deletes it from the WP:SPR process. One reason for having a Board. I'm not suggesting we put Science to WP:PR now. --Bduke 23:59, 15 March 2006 (UTC)

This article would benefit from peer review much more than it will from scientific peer review at this point: the layout is all over the place! - Samsara (talk • contribs) 00:16, 16 March 2006 (UTC)

OK, well, there's also a glaring error pertaining to physics in the subsection titled "philosophy of science". It’s exactly the kind of error a college kid might make; it implies that atoms are "mostly empty space"; they most certainly are not. Now what? I can fix this error, but so what? How does the board help here? Or let me rephrase: bduke says "It hardly contains any science itself, so it does not need experts to review it." -- yet it contains an error that non-physicists will almost certainly overlook. ... which is part of the problem with WP articles. linas 00:09, 16 March 2006 (UTC)

OK, I missed there was some science there, but I did say "Hardly any .." and I was making a general point. Some requests should be moved to WP:PR. This project should deal with articles that contain a lot of science that needs reviewing. --Bduke 00:20, 16 March 2006 (UTC)

Well, I don't follow the logic, and I don't understand the "general point". Here's an article, about science, with an error. Why shouldn't the science review board concern itself with errors in the article on science? To me, the "general point" is that most of the WP articles on broad topics, such as gravitation, mathematics, physics are weak, poorly structured, and often contain dubious claims. Its not like anyone is proposing that we reivew supergravity, which is an obscure topic far off the pop mainstream. linas 02:26, 16 March 2006 (UTC)

Linas, gravitation was just refactored. I am interested in your comments on it, please put them on talk:gravitation. Much appreciated, --Ancheta Wis 02:41, 16 March 2006 (UTC)

But I am proposing that we review articles like supergravity. In fact I thought that was the main point. It is this kind of article that needs expert review. How are we know that this "obscure topic" has been written by someone who knows what he/she is talking about. It could be badly out and we would not know. However, the reviewers that "Nature" might ask to look at it would know and WP would get a hammering and quite right. I do however agree that articles such as physics and gravitation are our concern. My general point was that we are not dealing with absolutely any article that might have something scientific in it. Some proposals for review should be passed on to WP:PR. I thought science was one of them. I could have been wrong, but it does not invalidate the general point. The decision would be made by the Board, not one person. --Bduke 02:44, 16 March 2006 (UTC)

Ah, well, then this is another critical point which this group, as a whole, has not come to any agreement on. There are three classes of science articles on WP:

Basic science. Articles in basic science attract many anonymous authors who add poor, low-quality material on a daily basis. These articles are painful to read, and yet, since they're the basic intro articles, are the ones that are the most widely read. They are painful to keep on one's watchlist: one must engage almost daily with some confused author who thinks this means that. These pages cause major wikistress for all participants.

Advanced science, the more obscure topics which are not accessible to most readers. For example, supergravity. I'm not concerned about the accuracy of these pages, mostly because WP itself already provides a rather good de-facto peer-review system. The people who can read and write these pages are almost all uniformly educated, and keep these pages honest. There may be occasional honest mistakes, but these get caught an fixed. There aren't any newbies dorking with these pages. The current WP process works well for these articles, a layer of "peer review" won't change anything here.

Crank science. These are the original-research, POV-pushing extravaganza turd-balls that are wrong from top to bottom. My current favorite: Bios theory. I'm not sure, but am deeply suspicious of Emergy Synthesis and its brethren. These need true scientific peer review and possibly censure.

The point is that the middle group is in good shape. Its the first and the third that need focus. linas 15:49, 16 March 2006 (UTC)

Linas, Junk science can be dealt with by existing processes. I hear quite clearly that the Scientific peers would like to get to the Science in the articles, and to review that. The rest of it, laymen can compile and organize. I agree that a Science peer review would definitely add value to a Basic article, as that would be a high-traffic page. And a Scientific peer might even endorse a page that might look suspicious to a layman, at first glance. It would be a real service to the encyclopedia that a Scientific peer recommend categories for an article, such as category:pseudoscience, or category:protoscience, or category:theories of gravitation, for example. --Ancheta Wis 17:29, 16 March 2006 (UTC)

I agree about the problems with Crank Science but I think they all fall into the area of articles that are highly disputed. I raised these earlier and there appeared to be a consensus, which I agree with much as I would like to sort out this mess, to keep away from them. That discussion is now on the archive page. I agree about the Basis Sciences, but I doubt that a proper expert review will convince the editor on oxidation that it is not an acid-base phenomenon! We can try though. --Bduke 21:00, 16 March 2006 (UTC)

Alternative suggestions for guinea pig review

Structure of the Earth

If you want to wet your teeth on something more technical, but not generally controversial, how about Structure of the Earth. It is a substantially technical article with only a small number of references, mostly to other websites, and few interesting problems like the percent compositions not equally 100%. Dragons flight 01:47, 16 March 2006 (UTC)

Science

Well, my review of science is already in. - Samsara (talk • contribs) 02:22, 16 March 2006 (UTC)

I don't see anything wrong with using science as a place to start. We found an error, and that probably means there are more. let the experts fact check it. Let those with less expertise do stuff like structure and organization. We need to get the word out that we welcome reviews from non-board members—aka regular peer review.--HereToHelp (talk • contribs) 03:16, 16 March 2006 (UTC)

Supergravity

I was wondering about a trial of supergravity, not a trial of a review but more a trial of whether we could find a reviewer. Looking at the article raised some new issues. JarahE, who has nominated for the Board, could, from his nomination, do the review. However he has edited the article considerably in the last three days. Charles Mathews had an edit on it earlier, although it is only a modification to a category. Linas, who raised this article, has extensively edited it. Surely it would not do to have, as a reviewer, someone who had helped to write the article. So it might be difficult to find someone. This raises another issue. It might be best if the nomination of an article was only put on the Project page first, leaving the tag on the article's talk page to be added when we know we can review it. Maybe it does not matter. I do not know.

I think the process would be this. The Board or perhaps one person (this role might rotate) concludes that the expertise to review this article is not available among the Board members and reviewers list. S/he puts on this talk page:-

Article [[XXX]] has been nominated.
There appears to be no sufficient expertise on the Board or among the registered reviewers to review this article. Is this correct or does someone think they can review it?

If nobody responds after a couple of days, they add:-

Can anyone on the Board recommend a suitable reviewer either from within the Wikipedia community or from outside.
We seek a volunteer to review this article. If you think you have the expertise to review it, please indicate below giving a brief summary of your expertise and qualifications.

There might be a gap of a day or so between the two points above being added. If this gets a nominee to review the article, the Board consults and if they find the reviewer is expert enough, they ask them to do the review. Later the Board looks at the review and helps to put together a suitable report.

I think this is where the Board is useful. There is someone with a responsibility to try to find a reviewer. Without this, it is much less likely that someone will try to find one. --Bduke 21:48, 16 March 2006 (UTC)

I still think that articles such as "supergravity" are the wrong things for this board to focus on. For example, WP mathematics has 13 thousand articles on math; I'd guess that 90% are in the "supergravity" category, requiring true domain experts. This board will choke on the volume. By contrast, WP math *does* have a semi-workable review/triage process; its rather informal, but quite effective. We pile onto Wikipedia:WikiProject Mathematics/Current activity, and whack on what we find/what we know about. Other projects could very well benefit by erecting similar infrastructure. By contrast, I am starting to feel more and more strongly that this board, and this process, needs to focus on reviewing, for scientific accuracy, the 1% to 5% of WP science articles that are on general science topics. linas 00:36, 17 March 2006 (UTC)

(p.s. the "current activity" page has the counter of all math articles: 13297 as of this instant).linas 00:38, 17 March 2006 (UTC)

Comment on nominees

While I understand Samsara's point, could I suggest that we leave comments until there is an actual call to comment and vote? This candidate might withdraw. I think general comment on this talk page is OK, but let's leave the Project page for now. --Bduke 00:29, 16 March 2006 (UTC)

I hadn't seen that it had been commented on here. - Samsara (talk • contribs) 01:07, 16 March 2006 (UTC)

Subpages

HereToHelp has just altered the recommended headers for sub pages with this comment - (If we're not templating the subpages but rather linking them (sounds okay to me) we should use bigger headers.). The original proposal did have the sub page "templated" i.e in {{ }} brackets, but this was changed to [[ ]]. I let it go at the time, because I thought we could come back to it later. Now it seems is later. I think we should template in the sub-pages, in which case we should put the headers back to something like they were before. I grant you that this is a small technical point, but we need to address it sometime. Any thoughts? --Bduke 03:32, 16 March 2006 (UTC)

I had assumed it was going to be templated, like is common practice everywhere else in the Wikipedia namespace. It will make for a long TOC, though. - Samsara (talk • contribs) 09:42, 16 March 2006 (UTC)

Look at WP:PR. The page is HUGE. This is debilitating, see Wikipedia:Article size. I think it is behooving to have the pages linked to. Furthermore, it reduces the need to use miniscule header sizes and leaves us more room for sub-headers. For WP:PR, this is not an issue. But for us, we need to break down the subpages further into board and non-board comments, comments by user, and reccomendations. I think it's best to link the pages.--HereToHelp (talk • contribs) 23:19, 16 March 2006 (UTC)

You may be right, but WP:PR has 70 pages included into it. OK, they are smaller than our review pages are going to be, but we will not have anywhere near 70. Also the templated in stuff does not count to the page size. It does not warn us that it is too big, but I agree that it is large and takes a while to load. AfD is probably longer. My proposal is to leave the current page as it is for a few days. I will then edit it to include it in the page and we leave it like that for a few days. Then we can see what people think. Common practice is to template them in and this has the advantage that they all appear in the contents and readers can see the whole picture better. However, this may be offset by size. Lets just wait, have a look at both and then decide. --Bduke 23:32, 16 March 2006 (UTC)

Alright. My biggest complaint was with all the subheadings. Think: two equals signs for "Requests", three for the article name, four for "Board reviews" and "Non-board reviews", and maybe five for the individual people. With [[ ]]s, you never get past === ===.--HereToHelp (talk • contribs) 00:48, 17 March 2006 (UTC)

SPR Method and Model

Just wondering...if SPR stands for "scientific" peer review, should it then be reflexive and apply scientific method (aim, hypothesis, equipment, Results, Discussion, Conclusion, Bibliography, etc.) to itself? Or does SPR have immunity from scientific method, and if so then how is the word "scientific" defined in the SPR acronym? Is there a "science" of SPR? Should SPR demonstrate the property "science" which the peer reviewer lays claims to?

Let's try an example - Samsara's current SPR of the Science article.

Title

A general model of scientific peer review in the open source user-defined environment.

Abstract

Open source user-defined scientific peer review (SPR) is an emergent property of the Wikipedia internet technology. Initial steps are taken to produce a model of SPR with the aim of SPR formalization. Two definitions of 'science' were identified as needing analysis and universally agreed definition. Results are pending.

Aim

To define the term "science" in such a way that it will be achieve maximal agreement among the world's population of scientists.

Hypothesis

Scientific method says

"Normally hypotheses have the form of a mathematical model."

Hypothesis says

"The hypothetical-deductive method demands falsifiable hypotheses, framed in such a manner that the scientific community can prove them false (usually by observation). (Note that, if confirmed, the hypothesis is not necessarily proven, but remains provisional.)"

Method

Samsara's SPR of the Science article was analyzed through the Wikipedia collaborative online method. In particular, two definitions of science were analyzed.

Equipment

Wikipedia portal, computer, internet access.

Analysis

Two definitions of science have been obtained throughout the process of SPR.

"science is a system of knowledge acquisition"

"science is an exercise in model selection"

Results

Pending...

Cost Benefit Analysis

Time: ?!

$: ?!

Time/$ efficiency = ?!

Energy: ?! joules

Energy efficiency: ?!

Exergy: ?!

Embodied energy: ?! embodied joules

EROEI: ?1

Conclusions

Firstly, it is difficult to make a formal decision of which definition of science has general acceptance by the world's scientific community.
Secondly, there does not seem to be a mathematical model by which a computer simulation can be generated of the phenomena. Nor does there seem an intuitive unit by why quantitative data can be entered into the model for hypothesis simulation and model selection/deselection.
Third, my head is going under trying to think this through.

Recommendations

Further development of a mathematical model & unit of inquiry.
A proforma for SPR that follows scientific method.
A holiday. :)

Bibliography

Science SPR

Sholto Maud 04:45, 16 March 2006 (UTC)

Need advice on a template name

I was going to nominate Classical theories of gravitation but it currently sits in a category with 100's of members. The category:pages needing expert attention does not currently separate out the Natural science articles. But there is currently only 'Template:expert' and 'Template:Technical (expert)' for the remediation markup. So, what would everyone like to see:

'Template: Science (expert)'

perhaps? That would invite expert attention to the article, before or perhaps in the process of a review. What other templates might come to mind? --Ancheta Wis 11:30, 16 March 2006 (UTC)

Shouldn't there rather be one for each of the major fields? Biology, chemistry, ... Also, do we have any data on whether anyone actually navigates by category to specifically fix those articles? Might need advertising. - Samsara (talk • contribs) 12:53, 16 March 2006 (UTC)

This article is in need of attention from an expert on Biology.
Please help recruit one, or improve this page yourself if you can.

I propose that we create 1 template for each named science (Biology for example), and invoke it with {{subst:Expert|Biology}} for example --Ancheta Wis 13:29, 16 March 2006 (UTC)

This article is in need of attention from a biology expert.
Please help recruit one, or improve this page yourself if you can.

Samsara (talk • contribs) 13:32, 16 March 2006 (UTC)

OK, here is the usage.

{{subst:expertAstronomy}}

{{subst:expertBiology}}

{{subst:expertChemistry}}

{{subst:expertGeology}}

{{subst:expertPhysics}}

One of these notices, when placed in an article, ought to feed right to the cleanup team who can then route the names of the respective articles to an appropriate Peer review pool.

It is possible, and in my opinion preferable, as it makes for a more pleasant reading experience, to place this notice on the Talk page of the article, which will then appear properly alphabetized in the category listings. --Ancheta Wis 13:58, 16 March 2006 (UTC)

Splash, I have also created the analogous expertScience expertTechnology and expertHistory templates. Others may wish to create Med. and Law templates, but those would need disclaimers. --14:07, 16 March 2006 (UTC)

It should be possible to coordinate this project with the entries in Wikipedia:Pages needing attention/Mathematical and Natural Sciences

The templates can have an embedded category in them to facilitate searches. Example: category:Biology page needing Scientific peer review --Ancheta Wis 15:46, 16 March 2006 (UTC)

Would a single template do? Called like:-

{{subst:expert|Chemistry}}

We would then not have to keep writing new ones. --Bduke 21:07, 16 March 2006 (UTC)

Mistake

I think that the idea of a scientific board is great (one of the best idea so far), but I disagree with one point : it seems to me that creating a board with people of diverse origins is a bad idea. I would prefer several boards specialized on one topic (astronomy, biology, chemistry). I think that the community is large enough so that we can find seven high-level scientist in every field. The problem is see right now is that I don't know how an astronom will be able to make any correction on the content of a biology article (for example). So, the goal of the committee would fail. What do you think about creating several committees ? Poppypetty 13:11, 16 March 2006 (UTC)

Benjamin Franklin had the same idea when he built the public library of Philadelphia - when others asked him to include their city, he replied, 'why don't you build the library for your own city?'

Let's not overstate the value of multiple expert input into each article; that's what the contributors should supply. If at least one expert in each area is on the board, that's fine: the rest will all be scientists, and so much of critiquing a scientific text is concerned with scientific method and scientific language, in which all board members should have expertise.

Having said that, I think your idea may be prophetic. If this board works well, perhaps it will evolve to subdivide itself naturally. Let's try to get the process right with this overarching board first, don't you think? Tony 14:11, 16 March 2006 (UTC)

Some of the Nominated board members are NOT experts

Being an expert in the Wikipedia Manual of Style or in editing articles does NOT make anyone an expert in any scientific fields. I fully realize that my next statement may offend some people, but I would be remiss not to say it. Anyone that is still an undergraduate, a post-graduate or a post-doc in any university is NOT per se a scientific expert no matter how brilliant a student he or she may be. Anyone without at least 10 years of real world experience in a specific field is NOT a scientific expert. mbeychok 23:33, 16 March 2006 (UTC)

One undergrad may be nice, but on the whole I agree. You are welcome to voice your opinions during the vote.--HereToHelp (talk • contribs) 23:41, 16 March 2006 (UTC)

How fantastically arbitrary. I always find it dumb when people equate knowledge/wisdom/expertise to time, as if one naturally flows from the other. What someone has actually achieved (including credentials) is much better gauge of ability then some fixed number of years. Expertise comes in many shapes and sizes. Someone who had published on the "Antimicrobial properties of tungsten alloys for fighting Athlete's foot", might well be the world's expert on that topic even if a grad student or similar. Does that make them an expert on either biology or metallurgy, probably not, but then every scientific expert is going to be limited (in lesser or greater degrees) in what they are expert in. Dragons flight 00:09, 17 March 2006 (UTC)

In theoretical particle physics we (we, the people of Earth) don't have anyone with real world experience, except maybe for some crackpots. I guess that means we don't have any experts. Hopefully we can still be represented on the board. JarahE 00:17, 17 March 2006 (UTC)

I believe that any intelligent person can carry out a scientific review of a wikipedia article given suitable review articles and access to primary literature. Whether this is efficient is another matter, but I think your objection falls through.

I recommend you read my review of the guinea pig article and come back when you've found a fault with it. If you can get two other regular contributors to this discussion to agree with you, I will immediately withdraw my self-nomination. But attacking good faith nominations without providing substance is absolutely NOT ON. - Samsara (talk • contribs) 00:28, 17 March 2006 (UTC)

If my nomination is going to cause a problem, then I shall withdraw. I agree, being a student does not make me a expert but the idea behind my nomination was to be a 2nd opinion, not an expert but someone who can understand what your talking about. I think its important to have a 2nd party (in all forms of review) but physics being physics is not understandable by all, so someone who can understand physics, but who is not an expert, ie, a physics student would be an idea. Above was by User:Eevo 11:53, 17 March 2006.

In the spirit of good faith and politeness, let's not make anyone (except some anon) withdraw. If they are deemed unqualified, they'll lose the election (just becuase we have less than 12 nominees doesn't mean we have to take them all). However, if you want to withdraw, nobody's stopping you.--HereToHelp (talk • contribs) 00:58, 17 March 2006 (UTC)

STOP. The question of who is qualified to review what hinges very much on what is to be reviewed (which is not settled). Are we reviewing "basic science" articles, or are we reviewing "supergravity" articles? First, I believe this group should focus on the "basic science" articles. Second, I think that mbeychok is mostly right; (and that JarahE misinterpreted a bit). In my experience, the self-righteous "I have a bachlors degree and read a lot about science" -- these people are the source of the problem, not the solution. They get their info from pop-lit sources, such as New Scientist, which has a track record for publishing questionable and wrong and bad science, as long as its sensational and has a "wow" factor. Then these people show up on WP and try to add this crap to WP articles, and they’re convinced its true because they read about it, or that's what their college prof once told them. This is one of the main routes for bad science creeping into WP: well-meaning, but in-expert, self-imagined smarty-pants. We need real pro's doing this. For example, the statement that "atoms are mostly empty space": I'm guessing I heard this several times in college. It’s probably even in textbooks. One might even be able to find a quote from Wolfgang Pauli saying something similar. However, it took me a while before I realized that this statement is utter hogwash, and, unless you have a bit of "real world" experience, statements like that will drift by un-noticed. mbeychok is simply being hardheaded, because, if you want results, you're going to have to be hard-headed, and not wishy-washy. linas 01:02, 17 March 2006 (UTC)

I don’t see a problem with undergraduates becoming part of the Scientific peer review. What important here is scientific knowledge and availability of resources. If you have that than by all means I think you should be part of this peer review. If the standard for editing Wikipedia was 10 years of experience in a certain field than Wikipedia would be dead. Tutmosis 01:05, 17 March 2006 (UTC)

As the nominee who meets mbeychok's criteria in spades, it being 42 years since I did my D Phil and I worked in academia until 2002 with an Honorary Reserarch position since then, I want to say that I think his comment is quite silly in the Wikipedia context and I do not appreciate being shouted at (Capital letter, NOT). To Eevo, I say that being a student is not the issue. Your lack of experience on Wikipedia may be. --Bduke 01:09, 17 March 2006 (UTC)

I partially agree with Linas. But while degrees and publications can be documented and corroborated, experience cannot (it's always harder to do it right!) The applications are deceptive. Very few, if any, provide any hard fast proof of their credentials. I'd vote for just about everyone (and those I would vote against, only because they are new and need more time here). By all means, let's get qualified people on the board. The problem is, those qualified are hard to come by and harder to weed out from the others.--HereToHelp (talk • contribs) 01:11, 17 March 2006 (UTC)

Lack of experience would not be a problem, under the name Evo, I have about 10 edits, under no name, i have about 800 edits (about 200 in physics/astro).Evo

In my original comment, I noted that I was cognizant of the fact that my comments would offend ... and evidently it did. I have no intention of starting a prolonged debate. But I do want reply to some of your comments ... and then I shall cease my participation in this discussion.

Dragons flight: I don't see why you felt the need to use the word "dumb" because my thoughts don't coincide with yours. Whether you like it or not, knowledge/wisdom/expertise are indeed related to time. None of us were born with an inherited BS or PhD degree. It took time to acquire them. And don't put words in my mouth ... I did not state any "fixed number of years". I stated "at least 10 years" meaning it could be any number of years beyond that.

Jarah E: Let us assume that a well-known theoretical physicist, working in that field for many years, were to contribute an article on that subject to the Wikipedia. Do you truly believe that a third year student working on a Bachelor of Science degree would be qualified to critique the science in his contribution?

No. My point was very narrow, and I stand by it. I was objecting to the 10 years in the real world criterion for board members. The well-known theoretical physicist probably doesn't satisfy this either, really theoretical physicists don't get out of academia much and getting out more wouldn't make them better reviewers. It's not a good criterion in my field. I did not object, for example, to requiring a PhD for reviewers. Also I object to the distinction, in the same message, between postdocs in and out of academia. Personally I have a postdoc simultaneously at a a university and at a multinational corporation, I don't feel there's any difference between the two. (Responding to a different comment:) As for my ability to review the supergravity article which I've rewritten, actually right now I'm not very satisfied with it, it doesn't give you any idea what supergravity is, but I don't think that people should be reviewing there own articles.JarahE 17:37, 20 March 2006 (UTC)

Samsara: I completely disagree with your belief that any intelligent person can carry out a scientific review of a Wikipedia article given suitable review articles and access to primary literature. That simply is not true. I guess we will have to simply agree to disagree.

All others: I am not asking anyone to withdraw their self-nomination. I simply believe that until this project includes a set of definitions as to what it takes to qualify as a member of the scientific review board, it is doomed to failure. A board made up of unqualified experts will simply drive well-qualified expert contributors away from the board. I recently had the experience of asking a scientist at NOAA with a world-wide reputation in his field to contribute an article. His response was that he could not do so if "just anyone" could revise his contribution. In my opinion, that is what will happen over and over again if unqualified experts are reviewing scientific articles. mbeychok 01:26, 17 March 2006 (UTC)

To Turmosis. Nobody is suggesting that you need 10 years of experience to edit Wikipedia. The question is about expert reviews of articles about science. First year students make great Wikipedians. My experience suggests they would get the science wrong quite often. To Linas. I think we have to cover both basic science and advanced science. If we only do the former, we do not need this project. We just need a mechanism on the various WikiProject Science pages to send people over to WP:PR to look at the science and you can put the articles there that need attention. Of course in one sense the whole discussion is not resolvable. I suspect the WP:SPR will try to deal with any article that gets nominated. That might be a weakness, but whatever guidelines we agree on now are likely to be ignored. --Bduke 01:27, 17 March 2006 (UTC)

Bduke: Isn’t the original question about who is qualified to be a expert reviewer? Tutmosis 01:36, 17 March 2006 (UTC)

I just realized that there really isn’t a point to this discussion. All the people are still in the nomination stage and I am sure the community will make the right decision for who to vote for. I don’t think mbeychok has anything to worry about since I am sure everyone understands that only people with scientific knowledge are suitable for this. Tutmosis 01:46, 17 March 2006 (UTC)

Indeed, it is the first topic (how's that for poetic justice?) I say we finish this: get an official qualifying statement written, now, and focus on the fact that we want only the best. If that means a board of 5, so be it. 5g of concentrated acid can be more potent than 12g of acid water.--HereToHelp (talk • contribs) 01:48, 17 March 2006 (UTC)

Sorry to pain in the ass, that depends on how you use the acid. Eevo

You may be more qualified in science than I thought...but you still have yet to learn to even sign your posts. (helpful link) --HereToHelp (talk • contribs) 02:33, 17 March 2006 (UTC)

im trying to get the firefox entension to do that for me, she be done soon, but until then Eevo 02:50, 17 March 2006 (UTC)

PLEASE let's not invest too much in this election. It's the process of conducting an experiment in a new structure on WP for improving articles that is much more important than any individual. Whoever misses out this time can stand again, and who knows, by then, there might be more 'boards' with narrower specialisations.

I can see that nominees who miss out this time may drop the bucket and be inclined not to participate at all (that would be my immediate impulse if I were standing and lost). For this reason, I wonder whether people might consider that all nominees who miss out be automatically offered the position of 'Special science contributor', or an equivalent. They represent talent that we cannot afford to lose. Tony 02:41, 17 March 2006 (UTC)

On second thought, perhaps not 'all', which might encourage nominations solely for that purpose; but the new board might give serious consideration to motivating those who didn't succeed. Tony 02:46, 17 March 2006 (UTC)

Yes, you're right. I have problems with a few of the nominees not for their science capabilities but for their Wikipedia ones (see my above comment).--HereToHelp (talk • contribs) 02:47, 17 March 2006 (UTC)

I think what makes someone good for reviewing articles is:

having a nose for good sources
using many sources
being able to admit they were wrong

I think the credentials matter-- but they only go so far.[1] Everyone that has spent some time reading peer-reviewed scientific literature has come across papers that have mistakes and others that are completely wrong.

What really matters, in my opinion, is the above three points and a track record that demonstrates that. You can have people that do things for a hobby and are really darn good at 'em. Einstein did a lot of his best work when he was working at the patent office. Srinivasa Ramanujan basically had no credentials whatsoever... but made great contributions to his area.

While I agree that a lot of crap comes from people with a bit of knowledge and a big ego, we should think about how we can design a process that lets the talented contribute, revise and review. The key, I think, is not selecting the people with the best credentials-- it is selecting the best people. Nephron 23:40, 18 March 2006 (UTC)

In case the absurdity of the statement at the start of this section is not manifest to everybody, let me point out that in theoretical physics, most of the business of peer reviewing for scientific journals is done by postdocs. That is because: they have the time to familiarize themselves with the recent literature, which professors (with large teaching and administrative responsibilities) do not. Of course, this leads to a certain shortsightedness, where results more than a few years old are sometimes overlooked, but Mbeychok's statement, along with being fantastically arbitrary, is at odds with the facts. If you require that only tenured professors can review from Wikipedia, I imagine you'll end up with a rather small board. –Joke 02:39, 19 March 2006 (UTC)

Scientific community

A wiki depends on a set of close-knit relationships (meaning concord among its members) in support of some project, such as an article, or an intellectual achievement. A scientific community is knit from similar interests. A group of scientific peers, by definition, is a group of equals (in some sense). When a peer review occurs, the participants coöperate (as defined among themselves). The coöperation is the project.

But principled actions can be respected.

It is not cause for celebration when we lose a contributing/producing member of any community. I would like to voice my regret at the circumstances which have prompted Samsara's resignation from Wikipedia and furthermore thank him for his contributions in our behalf. --Ancheta Wis 09:35, 17 March 2006 (UTC)

Indeed, it is not cause for any kind of celebration. I have no idea however why Samsara has done this. We had a reasonable exchange of views on our talk pages not too long before he left. He expressed a bit of frustration but nothing more. I hope someone can get him back. --Bduke 11:14, 17 March 2006 (UTC)

This kind of situation is why I (and also people above) suggested a formal 'expert/authority' weighting system. It is also why I have tried to talk about the model and science of SPR, else we seem to leave the scientific community open to internal emotional turmoil, which is not scientific... or is it? I think it is worth repeating my question: Does SPR need to be scientific in the sense that it should follow scientific method? (or am I talking to myself here? :) ) Sholto Maud 12:45, 17 March 2006 (UTC)

I think using the scientific method here is overly bureaucratic. I have no idea why Samsara left, and I too regret the loss. If, at the very minimum, we knew why...--HereToHelp (talk • contribs) 12:48, 17 March 2006 (UTC)

Sci. meth. is bureaucratic. But also is a SPR "board". Is there a scientific measure of "bureaucraticness" by which we might compare? (intended in good spirit :) )Sholto Maud 22:28, 17 March 2006 (UTC)

However, if you want to withdraw, nobody's stopping you. posted by HereToHelp 00:58, 17 March 2006 (UTC)

I hope this speaks for itself.

Samsara (talk • contribs) 13:18, 17 March 2006 (UTC)

Good editors leaving isn't any fun at all. I'd feel pretty crummy if I were told something to that effect. It says implicitly that I'm not important here, and that my ideas and thoughts and feelings don't count. That's how it could be interpreted, which is also why it is a good idea to generally steer clear of those sorts of comments. --HappyCamper 19:11, 17 March 2006 (UTC)

Sometimes I would like to have somebody arround heretohelp and to bring science to a higher standard. Lets see how it works and than see if we need improvement! Wikipedia always had a goal but the way to this final stage the users have to manage! I will suggest one or two articles to the SPR and than see how the board (notvoted) is working. If this test is not working we have to think about something.--Stone 13:29, 17 March 2006 (UTC)

An interesting scientific experiment! aim: to test the SPR board for operationality, hypothesis: is falsified if SPR board 'not working' (how is this defined, how do we measure this?), method: etc. etc. Cool! :) (P.S. Yes come back Samsara. I like your conception of science. Let's try and make a model of it :) -- ultimately I reckon that this process we are engaged here in at the moment is not just about reducing the error in the Wikipedia articles but also about reducing the error (be it emotional, linguistic, etc) in ourselves and our models of the way the world works, which is what Samsara calls science, and is why I like the definition) -- It also seems that we are caught in something of a verification-falsification loop where there is no objective measure for valuing our SPR/PR/User contributions and breaking us out of the loop... Sholto Maud 22:28, 17 March 2006 (UTC)

Sigh. I have no idea why Samsara decided to leave. The discussion here is vigorous, but hardly as heated as some WP conversations, and certainly never descended into vileness. The topic is important, there are a lot of alternative views, and the goal of this debate is to have all participants come to a common understanding of what the issues are, and how to work them. Samsara, please reconsider. You were a voice of reason, and if this debate soured you... I'm sorry. 22:21, 17 March 2006 (UTC)

Agree, I've seen worse. Are suggesting that "However, if you want to withdraw, nobody's stopping you." caused Samsara to leave? I meant, as looking back at the original would show, withdraw from the board and I do not think I targeted Samsara. There is no reason that I would shoo a person away from Wikipedia other than if they are a complete vandal (the good faith editor who messes stuff up is a tough call but Samsara was better than that). So please, if you're reading this (lurking in geek speak), consider coming back.--HereToHelp (talk • contribs) 23:02, 17 March 2006 (UTC)

Refocus, Regroup

I believe the discussion and the experiment has identified several issues that are at this moment unresolved. I'll try to recap here. I'd like discussion to work around getting consensus on these.

Generalist vs. Specialist articles

What type of article should this project focus on? Generalist and overview articles, such as Science, etc. or narrow, specialist topics? Almost all of the rest of the debate on this issue hinges on the answer to this question, including what the structure and organization will be, who the participants will be, what their qualifications might be, etc.

Generalist articles:

Have a much larger readership than the specialist articles, by many orders of magnitude.
Are more likely to contain minor errors, because they are edited more often by non-specialists.
Are far fewer in number, and are less likely to overwhelm a central committee.

Specialist articles:

Make up the vast, overwhelming amount of content of WP.
Require (highly) specialized knowledge to review.
Require a review structure that can scale to meet both the need for specialist knowledge, and handle the large number of articles.

Proposal

Set up two very distinct systems to deal with these two very different classes of articles.

For the Generalist articles, the current suggested board structure might work. The board would be responsible for reviewing content, as well as interfacing with the general WP peer review process. In particular, we've identified a "bug" in he current WP peer review process: it does not recruit or solicit help from qualified Wikiprojects when conducting peer review. There is no mechanism to, for example, contact the Biology Wikiproject when an article about biology is nominated for good article status. This is wrong, and needs fixing.

For the Specialist articles, I suggest setting up a decentralized network of specialist boards: e.g. a board of string theorists who would be responsible for reviewing the content of string theory articles, etc. Each such board would be "licensed", and issued an imprimatur which they could stamp on reviewed articles: for example, "This article has been reviewed for scientific accuracy by the Ichthyologists Association of Wikipedia". These smaller groups would hash out their own procedures for vetting an article, which might vary from discipline to discipline. The pedigree of participants would also vary from discipline to discipline: there are 500 articles in Category:Elementary mathematics which could all be reviewed by the "Middle and High School Teachers Association of Wikipedia": clearly this does not require a PhD in physics, but does require experience in pedagogy. The only real question is whether we have enough participants and enough interest to get the specialist boards going. Its not clear that we do.

Can this work, then? linas 23:09, 17 March 2006 (UTC)

If you look back, I first suggested a pool of specialists, sub-divided by specialty, which the board could call upon. The specialists would examine it for factual errors and the board would make sure it fit in in a broader sense: wasn't a duplicate article; uses correct style and templates, is linked to by the general article (and well the reviewed article links to it), is categorized properly, and also scan it for any general inaccuracies. We'd have to expend a lot of time and energy to get that pool assembled though—but once we've got that in place, we could progress quickly and accurately through many articles. Then, we'd have a template saying "this article has had a SPR by the board and specialists in the field of x. The recommendation was to y. There archive can be found here."--HereToHelp (talk • contribs) 23:22, 17 March 2006 (UTC)

Yes, well, fine, but its time to start coming to a consensus? Or am I rushing things? The only person who nominated themselves for the board, and actually performed the trial review at Wikipedia:Scientific peer review/Scienceis Samsara, who has quit WP. At some point Wikipedia:Scientific peer review will need to be changed to reflect consensus on organization, procedure and policy. linas 23:53, 17 March 2006 (UTC)

When different people arrive at the same point from different locations and by different methods it suggests there is merit in the location.

Proposed amendment: I'd like to support the proposal and supplement it with a further longer term proposal to develop a system whereby Wikipedia can give the same academic reward points for contributions by experts which are already given for contributions to peer reviewed journals. I imagine that there will need to be a Wikicode system developed along with the vice-chancellor committees for allotting the academic publication/review points correctly. Sholto Maud 01:32, 18 March 2006 (UTC)

Sholto Maud, do you really seriously think Vice Chancellors (or equivalent) in any country will consider work for Wikipedia as being worthy of points for promotion? Dream on. It is not going to happen any time soon. --Bduke 07:51, 18 March 2006 (UTC)

Its a nice dream. But I'm left wondering, if Vice Chancellors (or equivalent) in any country will not consider work for Wikipedia as being worthy of points for promotion, then it seems to me that Wikipedia will never formally be an authoritative source of information in academic circles. 58.104.14.222 08:10, 18 March 2006 (UTC)

Bduke, perhaps #Category phrases below might suggest some mechanisms in the spirit of Sholto Maud's proposal. --Ancheta Wis 12:23, 18 March 2006 (UTC)

On general issues, there is really so much new material here while I have been watching a Rowing Regatta all day, that I think I have to sleep on it all before commenting. However, I will say one thing. I think Linas' division between generalist and specialist is too sharp. We do not jump from science to supergravity in one leap. There are rather a lot of articles that school students and undergraduates think they know enough about to edit, sometime with good results and sometimes with very bad results. In chemistry, for example, there are rather narrow articles such as Redox or VSEPR theory, which are 1st year undergraduate topics. These often contain errors and they need a reasonable expert to fix at least some of them. We really have a gradual change from general to specialist here. --Bduke 07:51, 18 March 2006 (UTC)

Kim just added a great page at Wikipedia:Peer reviewers - assuming this builds up, we don't need to worry what sorts of topics we'll review. Just the ones that have lots of authorities around... --HappyCamper 19:40, 18 March 2006 (UTC)

A new direction

Many of the issues surrounding the board / reviewers, specialist v. generalist, and other concerns are rooted in a framework of peer review that I think the community might well be better off without. So let me propose an alternative that could address many of the same issues regarding article improvement and maintenance of scientific topics. Technically, what follows could be created in addition to SPR, but I am inclined to believe it might be a better/more effective first step.

My radical suggestion is as follows: How about forming a "Scientific Contributors Noticeboard", where people could come to ask for assistance in the maintenance and improvement of technical topics. It could be broken out into disciplines, and those with appropriate technical training could be encouraged to watch the pages and contribute. To first order, what we need is a way to get scientific expertise to those articles that need it, and I believe that creating such a noticeboard would do so while being less bogged down in the formality of peer review.

This proposal obviously doesn't address the concern that our articles are never checked by what the outside world might view as authority, and so we might still need something like SPR, but I think the Nature study shows our content is already doing pretty well without it. As a first step, I think it makes more sense to try and build the scientific community within Wikipedia by providing a common forum for discussion of the scientific work within Wikipedia. Dragons flight 00:22, 18 March 2006 (UTC)

I like it ! linas 01:24, 18 March 2006 (UTC)

Interesting. This hsould work in conjunction with SPR.--HereToHelp (talk • contribs) 02:37, 18 March 2006 (UTC)

Ditto. No reason why we can't have two complementary programs on Wikipedia. Might be a very good thing to have. --HappyCamper 02:45, 18 March 2006 (UTC)

The key is motivating expert contributors to review scientific articles. The Board might have done that, although I concede that there are problems in creating a hierarchy. I don't see how a noticeboard alone will motivate the required WPians to put in the required effort; there's not sufficient reward or recognition. Tony 02:47, 18 March 2006 (UTC)

The key is that we wait, and let the Magic of the Wiki take over - the idea is that people should be intrinsically motivated enough to help out. The noticeboard simply needs to be made free and available, and it will gradually grow into something useful. Experts will come along if they see something inviting, interesting, and useful. I use the Wikipedia:Reference Desk as an example here - I will be bold and say that the participants on the reference desk are one of the strongest subgroups on Wikipedia - it took at least a few years to build up, but now it is a self sustaining forum where people can exchange questions and ideas. --HappyCamper 02:57, 18 March 2006 (UTC)

A Reservation

I've contributed to a fairly wide variety of Wikipedia articles, so I've had some experience with the wide variety of levels of expertise that may be applied to controversial articles. When I'm dealing with writing of people who are writing at or below my own level of expertise, it's usually fairly easy for me to see where there are obvious mistakes or problems, and it is usually fairly easy to find definitive sources that can be cited to establish whatever the current "wisdom of the field" may be. When I am dealing with articles that are out of my field and even a little over my head, then it may be perfectly obvious to me when the article is poorly written. When I have attempted to get these articles straightened out to the point that they at least contain comprehensible sentences I have encountered difficulties because when I have sought help in making clarifications from people who claim expertise (1) many of them have been satisfied with the existing formulations because (it seems to me) they believe that they understand what the formulations were intended to convey, (2) sometimes two "experts" will support opposing positions, and (3) I have been unable to determine who to believe because they stand on the authority of their own superior qualifications and offer no citations or even explications.

Some of the articles in Wikipedia, e.g., storm cellars could be badly written and inaccurate and it probably would not make a great deal of difference. Other articles are on the cutting edge of science, discuss topics where it would be a great service (especially to young people) to be both correct and also very clear, and most likely are controversial in the sense that scientific development has not ironed out all the problems and contending explanations exist.

In the generation of Bohr, Heisenberg, Schrödinger, Dirac, et al., who could have refereed a Wikipedia article on quantum theory? What would have been the correct way out if a physics professor Wikipedian from North Dakota had written an article on Dirac's theory and another physics professor from New Delhi had ripped it up? Who among us would know whether the Indian physicist was to be believed or whether the U.S. physicist had it right? If neither of their versions of such an article had any obvious flaws of logic or English-language presentation, would we go to a Venezuelan physicist to decide between the two of them? Or would we try to get some assessment of each writer/editor's merit in the field and then, if still necessary, find somebody whose qualifications were so much better than theirs that it would be reasonable to expect that s/he could sort things out and put matters that are in doubt in their proper perspective?

In the early 1900s we could have probably asked any of the several physicists mentioned above to "vet" an article, and any one of them would have treated the contributions made by "rivals" to the field in a fair and objective way. In the area of spiders I know of a few people who do not ordinarily write articles but who have international reputations in the field and who are willing to answer the occasional "dumb" question. I'm not sure whether somebody like George Greenstein would do that kind of thing for articles on astrophysics, or whether Brian Greene would straighten us out on quantum physics, but what we really need are people of that caliber if we really want an "imprimatur" on an advanced article. P0M 02:51, 18 March 2006 (UTC)

I'm sorry, but I don't quite understand how the example you brought up relates to what it is you are saying. Could you please explain a bit more please? --HappyCamper 19:42, 18 March 2006 (UTC)

To be blunt, it is easy for someone to claim expertise. When two or more people claim "high credentials" and disagree, or when somebody claims "high credentials" and just happens to be wrong, much mischief can be done.

Somebody claimed that Britannica articles are not written by experts. I don't know whether they have fallen from their previous high standards, but when I was in graduate school I happened to hear about the experience of one of my professors who had been asked to write a specific article, which was to appear with his name and academic position at the end of it to signify that it wasn't put together by some clerk with a paste pot and scissors. For his services, he received a copy of that issue of the encyclopedia. They probably didn't get Einstein to write the article on relativity theory, but they might have asked somebody like Robert H. Brehme to do it. Brehme and Sears wrote the Addison-Wesley Introduction to the Theory of Relativity, so they were at near the top of their field as physicists who both researched and taught physics. I doubt that we have many people with qualifications on their level in any field. Sears was, at that time, Appleton Professor of Physics at Dartmouth, and he had earlier been a Professor of physics at MIT, and Sears was an Associate Professor at Wake Forest University. The editors of Addison-Wesley at that time had a "stable" of extremely lucid, and doubtless correct, physics textbooks, which is just to say that they knew what they were looking for when they set out to add a relativity text to bring their line up to date.

The important thing is to have people overseeing a writing project who are high enough above the fray that they can be fair to all concerned, spot oversights, certify correctness, etc.

If we have people of such high qualifications, I wonder what the nature of their contributions to existing articles has been. Why would they now hang back and later come in to judge an article? And if they contribute to the writing and then certify their own article, where is that at? Even in the case of the abovementioned relativity text, my guess is that Brehme did most of the work and Sears vetted it. Afterwards, the publisher might have had an unrelated physicist such as Rolf Nevanlinna or Claude Kacser give it a pre-publication review. One way or another, it seems important to have articles on especially significant topics get oversight from people with recognized credentials in their fields. P0M 02:46, 19 March 2006 (UTC)

It is a totally valid concern and reservation. You're right - this is a problem, but in my mind, it's also a problem that is probably beyond the scope of the Wiki to solve. Any system set up is vulnerable to misuse or whatever negative connotations we choose assign. What we do have on our side, is that it is operating on a Wiki - which means that in principle, changes can take place as fast as the flaws are found. In my heart, whatever results from all these posts is at best probably an imperfect peer review system for Wikipedia - but at least it is one to build from. It is sure to have flaws that we cannot anticipate yet, but they are also ones that can be easily rectified. I have a feeling that very few people actually know about Wikipedia - not everyone is connected to the internet, and experts especially of the older generation are not quite so exposed to Wikipedia. (Oh, and no need to be conscientious about being direct - it is both tiring and draining to write with euphemisms on Wikipedia and wade through the "wikipolitics" - personally, I'd rather sacrifice these for clarity and transparency. It can be a bit tiring though, so generally I throw in some nice pictures for good measure. It substitutes for eye contact and smiling on the internet. lol...) --HappyCamper 03:59, 19 March 2006 (UTC)

On Samsara's departure...

I decided to do some digging, and I want to share my finds. I really wanted to know how and why this Wikipedian could turn into this Wikipedian. I found that he was discontent with the running of the peer review and verification process: see this and this. Ultimately—and you don't know how bad I feel about this—I think he felt that we didn't want him here: I told him he was welcome to withdraw if he so desired. Sure enough, he did with the edit summary: don't expect me to support this project in any way in future. After saying a few other goodbyes, he simply left. It would be one thing to withdraw his nomination—though his peer review of Science looked great—but I never said I would support him leaving. I would never fully support anyone leaving besides vandals. I don't know whether this is cumulative or some sudden epiphany, or even a misunderstanding, but give Wikipedia another chance. Samsara, please forgive this samsara and join us.--HereToHelp (talk • contribs) 02:52, 18 March 2006 (UTC)

You've done your best to explain your perspective, and that's all you can ask of yourself. Don't let this get to you too much. Hopefully Samsara will read this, and will decide to come back. If not, then we can just hope that this project becomes more welcoming for Samsara in the future. --HappyCamper 03:05, 18 March 2006 (UTC)

Thanks. Let's try to move on without him—there isn't much of a choice.--HereToHelp (talk • contribs) 12:12, 18 March 2006 (UTC)

Samsara also indicates that they're nearing completion on a PhD. This is often a stressful time in one's life. I'm chalking up the wikivacation to the stress of having a prof remind you that you better get cracking or else. linas 17:14, 19 March 2006 (UTC)

Yeah, that makes sense.--Here T oHelp 17:17, 19 March 2006 (UTC)

Some ideas

I have been following things from the sideline in the last days, and I have not read everything (especially earlier stuff), so maybe I am going to say some very stupid things.

I have the feeling there is a kind of mixing going on. Normally, you have the editorial board, and the outside reviewers. The editorial board can make decisions, but they are at the level of suitability for the journal, or they turn down articles that are blatantly incorrect. In general, they do not do the peer reviews themselves (although depends on the field).

So, why not set it up with this in mind. Have a small editorial board (2, 3 people to start with ), and a longer list of people who list themselves with their credentials and field of expertise. The editorial board should consist of people who can do the initial reading (it is at least written well, and understandable (at least the basic section) to lay people?) and can request people from the list to do the review. They can (in special cases) also go outside Wikipedia, and ask specialists for their comments (aka Nature).

Ok, enough rambling, shoot it down... :-) KimvdLinde 03:00, 18 March 2006 (UTC)

Rambling is very good! The more ideas the better. This is a massive brainstorming session. Ideas are going to be thrown out, revised, edited, et cetera...this is a healthy thing! --HappyCamper 03:09, 18 March 2006 (UTC)

Do you mean like #Refocus, Regroup?--HereToHelp (talk • contribs) 12:38, 18 March 2006 (UTC)

No, that is a different division line that can be drawn, but this would leave specialis-generalist issues in the middle, and something for the editorial board members to look at and to match with potential reviewers. KimvdLinde 17:47, 18 March 2006 (UTC)

IF we have a board, I like having only 2 people on it to start with. What was that saying? I'm mincing words here. 1 monk can carry water from the well. 2 monks can carry water from the well. 3 or more monks can't. But again, I'm not sure what the board would do...what could they do that other editors could not? --HappyCamper 19:31, 18 March 2006 (UTC)

Just keeping track of stuff, matching articles with reviewers, have a look at pages as mediation cabal, along that line. They make also the first rough selection, if an article is really bad written, you do not want to bother a reviewer to go over it in detail. Those will be rejected for peer review beforehand on editorial grounds. Peer review is an intense job. A real full-length article can take me up to 1 to 2 working days, depending on how much I have to check, and I have a paper in review (by others) that will take easily a full week to be reviewed. So, you want to be sure it is worth the time and effort of the reviewers before you ask them. I personally think 2, 3 people for the board, with a good general understanding of science, would be sufficient to start with. Depending on how it evolves, it can be extended.KimvdLinde 19:42, 18 March 2006 (UTC)

So essentially, the board just acts as a sort of "gating function" which regulates which sorts of articles can be peer reviewed or not? I suppose it makes sense, but the article itself should be quite self evident whether it needs peer review or not. If I were to review an article, I expect it to have a list of references somewhere. The first thing I would do is to go find every single one of them, and independently verify that the facts cited in the article accurately represent the original source. Of course, this might be lots of extra work to do, but I do this regularly anyway, and it is sort of fun. --HappyCamper 19:48, 18 March 2006 (UTC)

The "editorial board" concept was discussed up top, and many/most academics seemed to cotton on to it, as that is the structure they're used to. But then the discussion sank on whether one even needed to have expert credentials for reviewing certain types of articles. I'm now of the opinion that a light-weight, minimal process "noticeboard" might be the way to start. linas 17:21, 19 March 2006 (UTC)

From what I have been reading, the problem was that two things got linked, editorial board members with peer reviewing experts. KimvdLinde 17:23, 19 March 2006 (UTC)

Category phrases

Chris Hillman brought this up to me. There are some templates (for example Template:expertBiology) which are intended to tag an article as being in category:Articles needing review by a biology expert. The template will include a category. Might we at least settle on a phrase for the category, which can then be placed in the templates? I moot ..."Articles needing review by"... . Suggestions, please? These categories could then serve as organizing lists of Biology articles needing review, under review, already reviewed, Good articles in Biology, etc.

As currently implemented, the generated text can then be modified to suit the article. If an article had been reviewed, then the category could be updated as well, to reflect status. For example, ..."Article reviewed by"... on the Talk page. Thus, after a review of an article, WMC might see fit alter its category to category:Articles reviewed by William M. Connolley. Or perhaps category:Very Good articles reviewed by William M. Connolley.

If a reviewer were to amass enough of these reviews, as tagged in a category of his/her own, this might even constitute an objective list of the reviews by the reviewer on Wikipedia, and possibly even worthy of the notice of a Vice Chancellor. --Ancheta Wis 12:03, 18 March 2006 (UTC)

I admit this requires administration. For example, there would need to be a page with a list of all the Science peers, containing links of the form category:Articles reviewed by William M. Connolley. But that would be a Good Thing if there were thousands of these reviews to administer. 12:03, 18 March 2006 (UTC)

This also implies that a reviewer use his name, William M. Connolley, in order to get credit for the review. --Ancheta Wis 12:33, 18 March 2006 (UTC)

That would also answer the objection of the NOAA scientist, who wanted assurances that his work would remain whole. His review could be signed, giving him the right to keep his review intact and free of vandalism. --Ancheta Wis 12:36, 18 March 2006 (UTC)

Hm...I don't think I like having those categories around very much. I think it would be better for a user to say on their talk page that they have reviewed articles X, Y, Z. In each of the articles X, Y and Z, there would be a tag that says to the effect "This article has been reviewed by the science notice board on Wikipedia. The reviewers and their comments can be found on page W. This is current as of date ABCD". It would be so much cleaner, and not expose Wikipedia articles to "ownership" - if Steven Hawking reviewed our article on black holes, and the article was subsequently labeled as such, would it be a deterrent for others to not contribute? It might not, but we don't want that issue to arise to begin with I think.

Now, for the other concern. What exactly does this NOAA scientist mean by his work "remaining whole"? I don't know of any user on Wikipedia who has contributed article content and subsequently earned the "right" to have it stay "intact" and "free of vandalism". Don't all articles and content behave like this already on Wikipedia? I mean, malicious edits are removed, vandalism is removed...if the reviewers' comments are completely modified to misrepresent the original sentiments, it's also reverted, and also followed with a comment on the talk page...in what sense does "remaining whole" mean? Copyrighted and completely not editable by others? In that case, that might be something which fundamentally contrasts with the aims of Wikipedia. Having partially protected content means that it is not easy (in fact, nearly impossible) to distribute it worldwide and freely. Content generated here is in some sense motivated by altruism.

It is probably our burden to show that the system on Wikipedia (despite all its successes and shortcomings) can also work for those who are skeptical of it, and favor more traditional methods of information dissemination and commentary. The fact is, people need to recognize that the technology with which Wikipedia runs on enables a plethora of new approaches to science and collaboration which were previously not available nor as accessible to the general public. It is our burden to show that this can work, and win the skeptics over!

Everything on Wikipedia is essentially released under the GFDL - even the vandalism edits - even the comments on talk pages, like this one. But even in such a seemingly chaotic environment, things still hold their integrity - people know not to misrepresent or alter other people's comments, and "good Wikipedians" know to give the courtesy of alerting others of any substantial changes to content, and giving the original author a chance to respond, or modify their wording, or keep the original text. All done in good faith, and all done with positive dialogue. "Bad Wikipedians" who don't respect process here simply don't survive. They're blocked, sidelined, and simply weeded out by the community.

Sometimes, I think we really need to create work shops or some sort to inform people on how to contribute to Wikipedia, targeted to these people. It would be so much easier to speak to someone in person and tell them what this is all about, and address their concerns on the spot. Maybe even at the next Wikimania conference...hm...well, maybe that might be just the thing I might work on instead. So much stuff that one can do around here! Isn't this wonderful? I have typed this response maybe 5 or 6 times now, so it tells me that I'm very excited to address these concerns, but at the same time, it is too vague for me to really help out and answer precisely what needs to be addressed, and probably my excitement and imagination of what really needs to be address is completely colouring the response here. Apologies in advance but hopefully it would be useful to whoever is browsing around reading this. --HappyCamper 19:21, 18 March 2006 (UTC)

I do not think there is any objection in archiving all peer reviews at a central place (with or without a name), nicely listed etc. These pages are defacto immune, and the discussion can be done over the copied and pasted version at the talk page of the article. I would not be in favour or many new categories, but favour a simple and easy to manage system, as many who are now contributing to this discussion might leave wikipedia for whatever reason in the future. KimvdLinde 19:29, 18 March 2006 (UTC)

I agree with Kim. Simple is good. As I said above (somewhere!) the categorization is something that we can probably do without. --HappyCamper 19:34, 18 March 2006 (UTC)

A list of potential reviewers?

Maybe we should start to make a list of people who are wiling to do peer reviews Wikipedia:Peer reviewers, and list them selves to a list with the short credentials, along the line of:

KimvdLinde. Post-doc, evolutionary biology. Expertise: Community ecology, biodiversity, quantitative genetics, life-history evolution, adaptation, morphometrics.

This list can be made regardless of the outcome of this discussion here. KimvdLinde 17:44, 18 March 2006 (UTC)

As a complement to this suggestion I propose that Reviewers always sign, and own, their Reviews. That way they might contribute an article (anonymously, per Wikipedia policy) and simultaneously Review their article with commentary about it, and sign it and date it (~~~~).

Then the wiki action could take over with edits on the article and perhaps Reviews of that article by others; a contributor might simultaneously comment on changes to the article in the signed Review.

Others might also comment on the Talk page of the article (as public space), with parallel Reviews of their own, which they might sign (in Reviewer space, to coin a phrase).

This might allow critical commentary with a form of ownership (the ownership of the reviewer's Review page for an article), allowing publication of credentials along with a Review. --Ancheta Wis 18:35, 18 March 2006 (UTC)

This option would have allowed the distinguished researcher, mentioned in the archive, a way to retain some form of ownership for that researcher's contribution to Wikipedia. --Ancheta Wis 18:35, 18 March 2006 (UTC)

My suggestion would to have the reviews at a central place, and let people copy and past it for commenting to the talk page of the article (cf the Nature reviews). In wikipedia, nobody owns a page, even pages with reviews. However, protection from editing (cf result pages of article for deletion proposals) is logical. It should be up to the reviewer whether they sign it or not, but in the later case, there is a need for an editorial board that has the oversight. KimvdLinde 18:50, 18 March 2006 (UTC)

The archived AfD pages are not protected. I think there are other alternatives which do not require protection - archiving the page history is one example - the page history itself will contain the authoritative version - we simply need to provide a permanent link to it. Subsequent edits and vandalism can then be discarded. We can also mark the review very much like how closed AfDs and RfAs are done. --HappyCamper 19:37, 18 March 2006 (UTC)

If the consensus is that the review pages are not change, cf AfD pages, changes can be reversed. KimvdLinde 19:52, 18 March 2006 (UTC)

Let's just adopt what we have from AfD then? My comment above was perhaps a bit out of place and a bit pedantic. Others? --HappyCamper 21:42, 18 March 2006 (UTC)

Created page: Wikipedia:Peer reviewers

I created the page Wikipedia:Peer reviewers, so if you are available to do peer reviews at request, list yourself. --KimvdLinde 18:59, 18 March 2006 (UTC)

Ooh...wonderful! --HappyCamper 19:38, 18 March 2006 (UTC)

Confused

So many new ideas and I do not have time to address them. I think we need someone, preferably Adam who started this all off, to try to write a summary of the current consensus with space for people to agree or disagree that each item is consensus.

I am about to strike out (note strike out, not delete) my nomination for membership of the Board. I recommend that all other nominees do likewise. It is not clear whether there is going to be a Board. If there is, it is not clear what it would do. It is not clear what the Project is going to do. For example, the above section on Wikipedia:Peer reviewers is possibly a great idea, but it seems to me to be setting up an alternative peer review of scientific articles system before we have decided whether we want this one. I am not clear any more that I want to be on any Board that arises. It all depends on what it is supposed to do and in what sort of project. --Bduke 22:44, 18 March 2006 (UTC)

As I indicated above, I did not follow the discussion from the beginning, and I just came with some ideas. The page I created can be deleted if needed, it is just an idea. KimvdLinde 22:48, 18 March 2006 (UTC)

I was not getting at your idea. If we accept the broad proposal originally set out here by Adam, we are going to need something like your page, but its presence now just adds to the confusion (or at least mine) and highlights that we are actually further away from agreeing than we were a week ago. I need to be out all day today, but I will try to outline my concerns and views in more detail sometime in the next 24 hours. --Bduke 23:08, 18 March 2006 (UTC)

Ok. KimvdLinde 23:11, 18 March 2006 (UTC)

I've been very busy with revision of the last few days so have been absent from discussion. I'll try to find some time later this evening and attempt to summarise our consensus. --Oldak Quill 18:26, 19 March 2006 (UTC)

What is the task of a board member?

Ok, I am confused also. What is the task of a board member? KimvdLinde 23:46, 18 March 2006 (UTC)

So we see one of the problems of multiple conversations occurring across the same page. We don't necessarily know who is talking to whom and the ideas are not synchronizing in a linear conversation. Here is a linearized version by one person who has filtered the concepts thru his own brain; please do not take it as policy. I took the liberty of embedding my understanding of the rôle of a board member into a larger context.

This page was first formulated by User:OldakQuill ~~User:HereToHelp~~, whom I believe was the one to announce its link on the Wikipedia:Community Portal.

Actually, I saw the link on the CBB and jumped aboard then.--Here T oHelp 02:03, 19 March 2006 (UTC)
Scientific peer review (SPR) is, at minimum, designed to improve the quality of a Wikipedia article in the natural sciences, per User:Karol Langner and User:Bduke's concept.
The editors of Wikipedia may, at their discretion, request Wikipedia:Peer review (PR) on any article. But currently, Peer review is an adjunct to the Featured Articles Candidacy process, with no other superstructure. But per an observation made by User:Linas, PR is 'broken' from the perspective of Scientific peer review ~~(as a timestamp or versionID or other bookkeeping device is not attached to the extant PR).~~ (as PR fails to recruit domain experts, and/or notify the relevant WikiProjects which can supply such expertise).
Specialist Articles in Natural Science are among the interests of contributors like Bduke, who is part of the Chemistry Wikiproject.
General Articles are among the interests of contributors like Linas, who wish to improve the level of the General Articles to an authoritative level.
Scientific Wikipedia:Peer review might also be performed independently of the SPR process which is under construction on this page as noted by User:Splash and others.
This page, Wikipedia talk:Scientific peer review has the same objective; the thread began with a discussion of the credentials needed to convince the general public that practicing academic scientists (Bduke, and numerous others) are (or were) part of the pool of reviewers. The discussion then began to fork in several threads at this point.
1. If independent SPR were to ignore credentials, then all the actions could be taken as if a board of editors or panel of reviewers did not also exist. (Ala User:Splash and others)
2. If a board of credentialed editors could not review an article, then a call could be made to the pools of credentialed reviewers.
3. If the SPR were to follow the tenets of scientific method, then all of the actions of SPR could potentially be marshalled at a more formal level. (Ala User:Sholto Maud)
4. A pool of credentialed or accomplished editors available via Wikipedia:Peer reviewers who can be asked by the board to conduct a Scientific Peer Review and are selected based on expertise, quality of previous reviews, credentials etc. Reviews are stored centrally. Separation of reviewer and page editors, reviewers remain outsiders. (ala User:KimvdLinde)

Please feel free to edit, add to, delete or rearrange this summary if I have misspoken, especially other Users whom I may have neglected, please. --Ancheta Wis 01:15, 19 March 2006 (UTC)

Ok, after reading this, and let it digest a bit, I see three problems:

What is the purpose of this page? The current peer review is indeed broken, is more a request for comments on content and editing.
What is the scope of this page? All articles, science articles, natural sciences?
How to achieve this? Informal, formal? Board of specialists that covers all fields, editorial board that is facilitating, no board at all? Specialists, generalists, credentials?

Is this a correct summary of the discussion points? KimvdLinde 01:48, 19 March 2006 (UTC)

One more: 4) will the process scale to accommodate all disciplines? (WPMath has 13K articles alone). linas 17:39, 19 March 2006 (UTC)

Looks okay to me. As for that third point, we need to strike a balance between throughness and consistency and being overly bureaucratic.--Here T oHelp 02:07, 19 March 2006 (UTC)

Just to vent my ideas. I think a board should be small (start with 2, 3 people, if needed, members can be added). They have the first look at the submitted articles for readability (unreadable, no review), editwarring (no review), unresolved conflicts (no review) and read the article to get a general impression whether this is a decent and understandable article (if not, no review). If ok, they list it as accepted for review, and they contact potential reviewers. If they can not find a knowledgeble reviewer (because there are non at the list for the exotic subject), the article goes back without review. If the knowledgeble reviewers accept, they list it as in review. When the reviews are done, they store the reviews and label it as reviewed (date). At every stage, they put a notification at the talk page of the article about the stage it is in and where to find the reviews. After that, the page editors can start 'fighting' about it, or they can come back with a rebuttal, after which the reviewer can repond as well. Reviewers can opt to remain annonymious (via e-mail, only board members know) or choose to be known.

This avoid having a all knowladgeble board with all diciplines and all specialisations. It is low level maintenance, no categories or so. Board members do not need to be specialists, but rather good editors with a good general sence for science and good reading skills (and a commitment to Wikipedia). In time, we could start experimenting with asking outside reviewers (ala the Nature review). It also avoids overasking the reviewers, because doing a detailed review of a 100kb/10,000 word page takes considerable time, and should not be taken to lightly (professionally, it takes me several hours to upto 2 days to do a good review).KimvdLinde 02:30, 19 March 2006 (UTC)

WP featured music project: useful ideas?

Dear contributors

I wonder whether user Tufkat's recent innovation, the Wikipedia:Featured_Music_Project, provides ideas that might be useful in achieving our goal of raising the standards of scientific articles. Tony 06:55, 19 March 2006 (UTC)

Philosophy of SPR

This might be interesting, but sorry, No, I for one do not want you to continue and do not think it helps matters.

(Removed philosophizing for now). Sholto Maud 20:50, 19 March 2006 (UTC)

Review for science in Wikipedia is exactly the same as for other areas. Are we reporting correctly what scientists are saying? Is what we write verified from sources or at the very least verifiable? Is it NPOV? The only difference is that it might need an expert to understand what the scientists are saying. There is no need for philosophy of any kind about the editing process or the review process. We are just writing an encyclopedia. --Bduke 09:33, 19 March 2006 (UTC)

I do not seek to dicourage or criticise. I wish to point out that the aim of producing reliable error-free knowledge through a human-oriented editing process, implies a philosophical system and moral that attributes value to error-free knowledge, and "high quality" information. Moreover you assume that verificationism is valid, thereby placing the process of writing an encyclopedia within the school of philosophy of science known as logical positivism:

"Logical positivism holds that philosophy should aspire to the same sort of rigor as science. Philosophy should provide strict criteria for judging sentences true, false and meaningless.".

It seems that SPR process of writing an encyclopedia involves provision of strict criteria for judging sentences true, false and meaningless.Sholto Maud 21:09, 19 March 2006 (UTC) Sholto Maud 02:43, 20 March 2006 (UTC)

You are fundamentally misunderstanding the wikipedia process. Verifiability is about whether there is a reputable source for the information. See Wikipedia:Verifiability. It does mean that the information is true in some absolute philosophical sense, although of course we should use sources that we have reason to believe are reliable. We are not "judging sentences true, false and meaningless" in the sense that you use it. Look at Wikipedia:Neutral point of view and Wikipedia:Scientific point of view.

Ok perhaps I do misunderstand. If you are concerned to correct my point of view on Wikipedia rules please comment on my thoughts.

Re: verifiability:

I'm concerned that we do not have a reliable system for establishing what qualifies as a "reliable source". If we had such a system then SPR would seem to be a doddle, and could be automated.
I'm not convinced that "We are not "judging sentences true, false and meaningless"". Sokal's hoax threw into question the role of peer review in a supposedly "reliable source". According to the Wikipedia entry "Sokal's stated point wasn't to see if the editors could detect fraud, it was to see if they could detect meaningless nonsense". So if this is right, Sokal understands that the function of peer review of "scientific" articles is to "detect meaningless nonsense". Sholto Maud 02:43, 20 March 2006 (UTC)

I would add that your suggestions elsewhere on this page that the review process should follow the scientific method are also misplaced.

You may be right, and that I should place them elswhere. My suggestions were motivated by the term, "scientific peer review". If this SPR project is not "scientific", and does not follow any scientific method then I would like to suggest that SPR is an innapropriate name. It should be called something more like, "peer review of scientific articles" (PRSA). Sholto Maud 02:26, 20 March 2006 (UTC)

I am reviewing an article right now for a Royal Society of Chemistry journal. I am not asked to follow the scientific method. I am asked to address a series of questions. There is no scientific way to review. It is a human process with flaws. An attempt here to develop one would be original research and that is not allowed on Wikipedia.

But again, isn't the process that this SPR situation is concerned with here all OR - because there is no Wikipedia system for establishing the epistemological reliability of article content? No one here, including you and me, has cited any verifiable reputable sources on SPR. And Wikipedia itself doesn't quailfy as a reliable source on SPR or PR (yet) because it relies on outside sources for its reliability ... so I'm not sure what the objection is. Sholto Maud 02:26, 20 March 2006 (UTC)

See Wikipedia:No original research. I note also that you defined scientific method as "aim, hypothesis, equipment, Results, Discussion, Conclusion, Bibliography, etc.". It is not. That is just one way (very common, yes, but not the only way) to write up scientific reports and academic papers.

I agree that the sci. meth. I've given is "one" method of sci. But I haven't claimed that it is the only one, and you have contradicted yourself. It is the standard method that reputable sources use to report true (and false) results and data from scientific experiments. It is a way of establishing repeatability: "A measurement may be said to be repeatable when [some] variation is smaller than some agreed limit." Verifiability is a form of repeatability - i.e. anyone can repeat the process of finding a "reliable" source. On this basis it seems to me that we are actually talking about a "scientific" peer review process, however we are unsure about the "measurement procedure", in this context. Sholto Maud 02:26, 20 March 2006 (UTC)

I thought at the time that you were attempting to add a light note to the debate, but it seems you were serious. --Bduke 21:34, 19 March 2006 (UTC)

Actually because of the paradox noted on my User page, my suggestions shoudl sound both serious and light - if that is possible. :) Sholto Maud 02:26, 20 March 2006 (UTC)

You and I (with I think wikipedia on my side) are at loggerheads. Do not confuse peer review of academic journal articles with the review of science articles proposed here. Read how WP uses the term "verify". It is about sources (not OR) and not about repeatability. Your search for a measurement procedure for articles is misplaced. Actually neither review process uses such a thing. Understand the difference between WP:NPOV and WP:SPOV. Any experienced scientist can judge whether an article in their own field is a good summary of what the standard texts and review articles are saying. You are trying to make it more difficult than it is. If there is a disagreement in the texts or articles, then all points of view should be given in a NPOV. We are not trying to resolve truth. This is an encyclopedia. --Bduke 03:07, 20 March 2006 (UTC)

Verify

Ok Bduke lets see where we get with this (maybe I need to be beaten into submission):

"arguments that have already been published by a reputable publisher".

Take an example. The scientific article, maximum_power_theorem, has one cited source: H.W. Jackson (1959) Introduction to Electronic Circuits, Prentice-Hall. Which I put there so it does not count. What is left are six external links.

1. By what criteria do I establish whether or not any of these publisher/source external links are "reputable/reliable"?

2. By what criteria do I establish whether or not a any of these publisher/source external links/texts are "standard"?

I.e. how do I verify these sources?Sholto Maud 04:20, 20 March 2006 (UTC)

No offense, but that seems like an odd example as none of those appear to be "sources", but external references added after the body of the article was written. --Limegreen 04:54, 20 March 2006 (UTC)

None taken. So then does the 'original' article have no sources? If it has no sources what does our pseudoSPR do with the article? Sholto Maud 05:16, 20 March 2006 (UTC)

There is a tendency, and a bad one, to think that since WP is an internet encyclopedia that all references should be internet references. It may well be that this article is not well referenced. However, we also look at verifiable. Is, for example, the material in the article well covered by the book you added as a reference? If so you confirmed, perhaps indirectly, that the material is verified. Do the external links actually support what is said? What kind of links are they? Some web links are more plausable than others. If they do support it are they saying the same thing as standard texts. If the facts are not verifiable, then someone is likely to say so. This is a wiki afterall. --Bduke 05:26, 20 March 2006 (UTC)

Ahem. Now lets take another example. Let's look at the foundational concept of most of natural science; i.e. power. The scientific article, Power_(physics). No sources.... Sholto Maud 05:29, 20 March 2006 (UTC)

If I understand your argument, wouldn't you be better off with a page that was extensively referenced (ie lots of inline references/footnote type things), because the concern is whether those references are reputable sources, or have I completely misunderstoo this argument?--Limegreen 05:38, 20 March 2006 (UTC)

In short yes, I think so. I use these articles because of their importance to science, and because they flout the general Wikipedia principle of Verfiability. Verfiability is, apparently, an important part of the imagined SPR process. An important part of verfiability is the provision of reliable sources. I would like to see a formal statement about how one establishes the reliability of a source - I think Bduke is giving us some inroads. It would seem that extensive referencing would make one "better off". But in the important article on power, the science editors seem to have the inverse point of view; i.e. no referencing makes one better off, and thus the knowledge-content of the article more reliable with no references. Remember it is verifiability, not truth that we are concerned with here. Sholto Maud 06:17, 20 March 2006 (UTC)

To Sholto Maud Several points:-

The crunch comes if someone adds a "disputed" tag to the article. Then everything has to be referenced.
I always take external links to be both more information and a source of at least some of the material in the article. So is the material in the links? I have not looked but I guess the answer is "No".
A review therefore should demand that more references be added and that they be precise for different sections, unless you can say cite three textbooks that all contain this material. It is not the sort of article that needs to cite the primary literature.
At the same time, the reviewer looks to see whether this material is the standard stuff that appears in text books.
Is this so difficult? --Bduke 05:43, 20 March 2006 (UTC)

But wait a second,

"One of the keys to writing good encyclopedia articles is to understand that they should refer only to facts, assertions, theories, ideas, claims, opinions, and arguments that have already been published by a reputable publisher. The goal of Wikipedia is to become a complete and reliable encyclopedia, so editors should cite reliable sources so that their edits may be verified by readers and other editors"

"Verified" in this context means that reader2 and/or other editor2, can repeat the process of going to a source to check the accuracy/reliability of editor1's works. Thus repeatability is inherent in verify. In the power article I am unable to repeat the process of going to a source. Thus the article is unverifiable and fails to comply to Wikipedia rules. This, by the way, is the same meaning of "unverifiable" as used in logical positivism. Sholto Maud 06:06, 20 March 2006 (UTC)

The repeatablility of going to a source is a bit different from repeatability of an experiment. However, what you says does mean it is unverifiable (unless the stuff is in the sources I mentioned - the book you say you added and the external links). However, it does not mean it is unveferifiable. Just go to a standard text and see whether the article is supported by the text. Add that text as a reference and the article is verified. Simple. Lots of articles do not comply with Wikipedia rules, but most of them are easily fixed. That is my last post for 15 hours or so. --Bduke 06:15, 20 March 2006 (UTC)

Thanks User:Bduke! Have a well-earned break. Sholto Maud 06:49, 20 March 2006 (UTC)

Fatal errors

The concept of verification appears to be a fundamental feature of the imagined SPR process. Representing the popular opinion of Wikipedia users, User:Bduke seems to agree that the Wikipedia concept of verification is the same as that used by logical positivists. A difficulty arises here because the logical positivist definition of verifiability defers to the correspondence theory of truth: "The correspondence theory of truth states that something is rendered true by the existence of a fact with corresponding elements and a similar structure". And this would appear to be what we do when conducting a scientific peer review. That is, we check whether a Wikipedia article is true to the facts of the reliable sources. But then this makes the Wikipedia philosophy of verification non-standard, because it is convcerned with "verifiability, not truth". Hence the Wikipeda theory of verification is Original Research. This would appear to qualify as a --FATAL ERROR--

Furthermore the Wikipedia theory of verifiability does not cite any reliable sources in defining the concept, thereby failing it's own conditions --FATAL ERROR-- Sholto Maud 09:50, 20 March 2006 (UTC)

I would caution about this line of reasoning. For example it uses reification which is commonly held to be a logical fallacy. --Ancheta Wis 09:56, 20 March 2006 (UTC)

Could you please elaborate. What is reified? (by the way the Reification article cites no reliable sources thereby transgressing the verification rule, but so the the verification rule so I suppose that's ok). Sholto Maud 09:59, 20 March 2006 (UTC)

We need to be careful when attributing policy to 'Wikipedia'.

We need to get SPR running before there can be such a thing as WikiScience. People outside WP are starting to talk "perpetually updated papers with a thousand authors". --Ancheta Wis 11:01, 20 March 2006 (UTC)

SPR Method/algorithm

I feel motivated to make a statement of the SPR process as given by User:Bduke above.

1. Check: Is article content supported by book references?

1.1 If so then print: "book reference material is verified".

1.2 If not then print: "book reference material is not verified. SPRbot demands correct book references be provided or article will be excluded in 7 days."

2. Check: Is article content supported by external link references?

2.1 If so then print: "web reference material is verified".

2.2 If not then print: "web reference material is not verified. SPRbot demands correct web references be provided or article will be excluded in 7 days."

3. Check link type: Is web link "edu", "gov" (any other categories?).

3.1 If so then print: "external link type is X ".

3.2 Print: "Plausability of link type X is Y":

---!!!SEMANTIC ERROR: "Plausability" variable Y is undefined!!!---

4.0 Check: If 2.1, then check external link references == standard texts.

---!!!SEMANTIC ERROR: "standard texts" variable is undefined!!!---

4.1. If so then print: "external link references supported by standard texts".

4.2. If not then print: "external link references are not supported by standard texts.

SPRbot demands the article is supported by standard texts or will be excluded in 7 days."

5.0 Check: article type.

---!!!SEMANTIC ERROR: "article type" variable is undefined!!!---

6.0 Check: is article type required to cite primairy literature?

---!!!SEMANTIC ERROR: "article type requirement" variable is undefined!!!---

6.1. If so then print: "article required to cite primairy literature".

6.2. If not then print: "article not required to cite primairy literature".

7.0 Check: is article content == text book content?

---!!!SEMANTIC ERROR: "article content" variable is undefined!!!--- ---!!!SEMANTIC ERROR: "text book content" variable is undefined!!!---

7.1. If so then print: "article material is the standard stuff that appears in text books".

7.2. If not then print: "article content is non-standard. SPRbot requires article material be standard stuff or be excluded in 7 days".

Not so difficult. However this hasn't formalised the criteria by which to evaluate the reliability of a source. Sholto Maud 06:49, 20 March 2006 (UTC)

No disrespect, but reading this debate reminds me of why I gave up studying philosophy and logic, and went over to the dark-side. What passes for a sound source in one area will not be the same as another area. Presumably part of what the reviewers would do is decide whether the article and its sources match what could reasonably be expected for that area. Power (physics) represents an area for the most part that is not rapidly evolving, and thus, a textbook would likely be a good source. In fact, that it has got to the stage it has with no referencing shows just how uncontentious it is. If it were rapidly evolving, cutting edge science, something in quantum physics, or the neural mechanisms underlying ADHD, textbook material is likely to be stale. Thus, in most disciplines, cutting edge science will require journal articles or similar. Within each area, some will be more reputable, and some studies will have more robust science. A person who works in that area will be best able to judge.

OK. So perhaps I'm revealing that I work in something more approximating the 'social' sciences, but ``it's the vibe" (The Castle (film)).--Limegreen 11:07, 20 March 2006 (UTC)

Categories: Wikipedia talk archives | Scientific peer review