Misplaced Pages

I have not given the information you are presenting much review, and have not looked at it at all in context, but generally, the idea is to exclude primary sources and cite secondary sources. MEDRS sources typically will cite and be derived from non-MEDRS sources. For issues of toxicology, what happens is that someone reports small-scale research which is not MEDRS compliant. No conclusions can be drawn from this research alone. However, when an organization like the EPA or FDA cites those research findings, then because of that their credibility increases. That may or may not make the government paper a MEDRS source, but one cannot be certain without examining the document because there are a range of quality among government sources. If the government gives an opinion or has a conclusion then that is worth mentioning only by virtue of it being from a major stakeholder and that has nothing to do with MEDRS at all, although often major organizations will back their statements with research which would also confirm it applicable under MEDRS. Is that a start toward helping? Blue Rasberry (talk) 15:20, 27 February 2014 (UTC)

Yeah, I think that's good. It just seems troubling to me that in a lot of ways the MEDRS rules seem designed mostly to address issues of efficacy (where most subjects are expected to show an effect of exposure) and not so much toxicity (where a serious adverse effect in 0.1% of exposed persons is significant, but difficult to detect via the usual RCT gold standard). Because of the lower frequency of events in toxicology and because of ethical considerations, toxicology knowledge is much more likely than efficacy knowledge to be based on lower quality evidence such as in vitro studies, animal studies, and case reports. Furthermore (and possibly because it is lower quality data), my experience is that reviews of tox data are fewer and farther between than efficacy reviews in most cases.

So I find myself worrying that MEDRS, if systematically enforced (its often not), would systematically bias Misplaced Pages toward an unrealistically high benefit/risk ratio of drugs, ag chemicals, etc. So maybe this is a philosphical concern more than an interpretation one. Would you delete any or all of the specific items above if only primary references could be found? Formerly 98 (talk) 15:53, 27 February 2014 (UTC)

I totally support adding language to MEDRS specific to toxicity as per my previous bellyaching on this. I was invited to create an essay for consideration which I haven't done yet. my bad. i very much appreciate Formerly bringing this to everyone's attention, as it remains a problem. Jytdog (talk) 16:28, 27 February 2014 (UTC)

We want sources that take into account all the primary sources and give them due weight. This is often a regulatory agency or a toxicology textbook. Both of these would be good sources. We do not want to have people selecting their favorite primary sources as than they could write anything they want as there are nearly always sources that go both ways. Doc James (talk · contribs · email) (if I write on your page reply on mine) 22:46, 27 February 2014 (UTC)

Reviews by from regulatory agencies are still reviews. Reviews of case studies are still reviews.

Some regulatory documents are primary, but you may WP:USEPRIMARY. It'd be silly to demand a systematic review on whether the FDA approved a drug or not: you can perfectly well cite a primary source, such as the FDA's announcement of their decision for this non-biomedical fact. WhatamIdoing (talk) 01:01, 6 March 2014 (UTC)

"Primary sources should generally not be used for medical content"

I propose we add the following line to WP:MEDRS to clarify things for people. Doc James (talk · contribs · email) (if I write on your page reply on mine) 19:27, 27 February 2014 (UTC)

SUPPORT::I think it would cause a few problems (including exacerbating the ones in the section above to some extent), but would solve more than it caused.

I am a little new here to be weighing in on policy, but my impressions as a somewhat new guy and former industry guy are as follows:

• I see a lot of Admins and senior editors telling people that a secondary source is required for medical information, but when I read the MEDRS text as it currently stands, it only seems to recommend secondary sources, not require them. It would be good for the Admins and the guidance to be very clearly saying the same thing. At a minimum, pick one or the other but speak with one voice.

• Requiring secondary references as the source of all medical information would be a very powerful way of keeping a cap on those who want to use Misplaced Pages to WP:RGW or to "enlighten" the mis-informed mainstream experts. It might aggravate some of the issues I refer to in the section above, but I think it would be an overall improvement.

• On the other hand, as I noted above, I'm worried that the more or less blanket prohibition on results from animal studies and in vitro testing is not reflective of how toxicology is practiced in industry or in the regulatory world. I won't repeat what I wrote above, but I'd like to see the restrictions on toxicology results loosened in a few specific situations. The most important of these would be carcinogenicity and reproductive toxicology testing. Most of what we know about the carcinogenicity and repro tox of most substances is derived from animal studies and in vitro genotox studies. I'd like to see these data allowed provided they are taken from secondary references. Since case reports are frequently the basis for regulatory action in the case of rare but severe side effects, I think there should be some point at which they are allowed in, though the threshold should be high. Perhaps it is arbitrary, but the threshold I would suggest is that if a nationally competent regulatory authority takes action on the basis of AE case reports, the agency memo documenting that regulatory action and the reasoning behind it should be considered a MEDRS compliant source.

I've suggested to Jytdog that we contact the Society for Toxicology to find an expert interested in helping us set guidelines for toxicology reliable sources. I'd be interested to hear what people think of that. Formerly 98 (talk) 21:10, 27 February 2014 (UTC)

Positions of national bodies is deemed to be a high quality source. So are major medical textbooks. If these comment on the animal studies and invitro testing than so should we. (keeping in mind due weight of course) Doc James (talk · contribs · email) (if I write on your page reply on mine) 22:41, 27 February 2014 (UTC)

It would be nice to get this language clarified in the MEDRS document, as it currently seems to read "generally omit in vitro and animal data" or at least "don't imply that it is relevant in peoople". Overall the language appears non-clearcut, which may simply represent the fact that it is a compromise document with many fathers/mothers. For drugs, this information will usually be available from the FDA approval summary, available through the drugs@fda website. Formerly 98 (talk) 01:30, 28 February 2014 (UTC)

Yes agree that bit needs fixing too. If this data is emphasized in high quality secondary sources we should have it too. Much tox research is based on the precautionary principle and extrapolation from animals as the trials is people would be unethical. Doc James (talk · contribs · email) (if I write on your page reply on mine) 02:39, 28 February 2014 (UTC)

I think that the guideline is already sufficiently clear, in its Definitions, Basic advice, Use up-to-date evidence and Choosing sources sections, that primary sources generally should not be used for medical content, though it does note exceptions. I don't think that Jmh649 (Doc James)'s proposal is going to have the type of benefit that Formerly 98 is suggesting, but we can certainly try it. Flyer22 (talk) 21:56, 27 February 2014 (UTC)

SUPPORT. a lot. Jytdog (talk) 21:57, 27 February 2014 (UTC)

Jytdog, I think the reason some of us see a disconnect is that of the 4500 words of the MEDRS article, nearly 20% (and mainly in the first third of the guideline) describe when and how it is OK to use primary references. At a minimum its confusing. Maybe its just 'undue weight". : > ) Formerly 98 (talk) 22:26, 27 February 2014 (UTC)

i think you are right about that. there have been loads of loads of discussions on this Talk page about primary sources. and it will be hard to get consensus to ban them outright. Your comment about WEIGHT is awesome; I agree it should be toned down a lot. But there is no explicit sentence like the one that DocJames proposed and it would be great to get it in. It can be a separate proposal to reduce the weight on primary sources... maybe it would make sense to have a new section called "on primary sources" and consolidate and rationalize it. btw, with respect to what you wrote above, about tox... especially for tox we want only secondary sources. otherwise we open the door for every stupid paper where some basic researcher dumps a bunch of chemical right on cells and kills and them and goes "look look X is toxic!!" argh. so i deadly oppose (!) opening the door to primary sources for tox. i want it closed more tightly. the regulatory action itself becomes the source we cite, btw! Jytdog (talk) 22:36, 27 February 2014 (UTC)

Comment - This was closed too fast. What about primary sources on human trials? Oppose as is. - Sidelight12 04:44, 5 March 2014 (UTC) *rationale 05:03, 5 March 2014 (UTC)

Yes primary source on human trials should not be used. Many primary human trials go unpublished as the results are negative. The positive ones are published. A proper review article will take into account both the published and unpublished results. Doc James (talk · contribs · email) (if I write on your page reply on mine) 04:51, 5 March 2014 (UTC)

Also the category of human trials can span an enormous range of trial design and execution quality and therefore result quality and statistical power. Zad68 05:09, 5 March 2014 (UTC)

This is a drastic proposed change, I think we should get more opinions. Certainly, abstract human primary studies, especially where there is open-ended interpretation, can be prohibited. I'm uncertain if everything falls under that category. - Sidelight12 05:57, 5 March 2014 (UTC)

It is not a drastic change. This is how this guideline is currently interpreted. This is simply clarifying how most people interpret the guideline right now. Doc James (talk · contribs · email) (if I write on your page reply on mine) 06:10, 5 March 2014 (UTC)

Hey User:Sidelight12 - the spirit of MEDRS (and[REDACTED] in general) is that we provide content that is reliable. As MEDRS already says, in the health arena publications get released and the press goes gaga quite often; we need to rely on secondary sources to tell us how big of a deal the work and conclusions in the primary source are - we don't decide ourselves nor do we follow popular media. We take the long view guided by reviews. So, so important and fundamental. I generally find that when people are pushing content based on primary sources into articles, they are grinding an ax and/or treating[REDACTED] like it is "science news" instead of an encyclopedia; neither are good. Jytdog (talk) 15:01, 5 March 2014 (UTC)

It is not drastic, it's a plain-language restatement of the current consensus regarding the application of WP:MEDRS. Sidelight removed the sentence, the sentence should be restored to the guideline. Zad68 15:13, 5 March 2014 (UTC)

Support this and even stronger wording if support can be developed for it. Zad68 05:07, 5 March 2014 (UTC)

Support as this is basically codifying what we already do with regards to use of primary sources Yobol (talk) 15:40, 5 March 2014 (UTC)

It has been open a week with substantial support. Thus added the text in question again. Doc James (talk · contribs · email) (if I write on your page reply on mine) 18:40, 5 March 2014 (UTC)

• Three more supports here: Doc James (talk · contribs · email) (if I write on your page reply on mine) 23:24, 5 March 2014 (UTC)

• Support. This looks like a straightforward clarification. Axl ¤ 00:27, 6 March 2014 (UTC)

Question about a journal

Hey all, I am looking to get some wider feedback on a journal. The journal is Acupuncture In Medicine. Briefly, AiM is owned by the British Medical Acupuncture Society and published by BMJ; it is MEDLINE-indexed, and peer-reviewed. Its stated aim is "to promote the scientific understanding of acupuncture and related treatments by publishing scientific investigations of their effectiveness and modes of action as well as articles on their use in health services and clinical practice....The Western understanding of neurophysiology and anatomy is used to interpret the effects of acupuncture." Arguments are being made on the German acupuncture trials Talk page (here and here) that the journal itself is an unreliable source under Misplaced Pages:FRINGE#Independent_sources and, most relevant to this Talk page, under Misplaced Pages:MEDRS#Use_independent_sources. The claim is that all articles related to acupuncture from this journal should be considered unreliable sources, because the journal is not independent of the field of acupuncture. All the other usual issues would apply (what is the specific content being supported, and is the specific article used as a source a review, etc). It is the wider question I am coming with, however, as it is a conversation stopper. I can see how reasonable arguments can be made on either side of the question, but my sense is that the claim is invalid. Input is welcome! Input grounded as carefully on policies and guidelines as possible (not wikilawyered, just grounded) is very welcome. Jytdog (talk) 13:25, 5 March 2014 (UTC)

The editorial board is clearly made up of only acupuncture believers, and they are promoting an editorial bent which is clearly promotional of acupuncture in a very seedy way. This list is particularly damning. They are preferentially recruiting acupuncturists and pseudoscience/altmed-supporters of various sorts. Notice how, for example, Edzard Ernst is prominent in his absence on that list as are Bed Goldacre, David Colquhoun, and Steven Novella. The journal is systematically excluding WP:FRIND. jps (talk) 13:44, 5 March 2014 (UTC)

I'm not so concerned about the absence of Ben Goldacre on the editorial board (he plays pretty fast and loose with the facts himself at times), but on first pass roughly 9 of the first 10 clinical trials I found published in this journal gave positive results. This is a dramatically higher rate than is found in the mainstream literature. I don't know how exactly we exclude this journal however given that it is Medline indexed. Formerly 98 (talk) 14:12, 5 March 2014 (UTC)

Curiosity: with which facts has Ben Goldacre played fast and loose? jps (talk) 18:22, 5 March 2014 (UTC)

A journal owned by acupuncturists whose editorial control is by acupuncturists cannot possibly be what WP:FRINGE and WP:MEDRS has in mind regarding independent sourcing for an objective look at acupuncture. I would suggest that articles from this journal are reliable for what acupuncturists think but are probably not reliable as an independent, objective source to see if it is effective. As always, there is an "out" here; if the journal article in question has been positively cited in the medical literature, this would go a long way to establishing it as reliable for MEDRS purposes, regardless of where it is published. If it has not garnered much support in the medical literature, I would avoid its use and use only those sources clearly independent of the topic. Without knowing the background, if the only journal articles stating that acupuncture work are coming from acupuncture owned and edited journals, that is a massive red flag.

I would think an analogous situation would be the journal Homeopathy (See WP:PARITY), which although MEDLINE indexed, is specifically disallowed by WP:FRINGE, for what I assume is the exact same reasons. Yobol (talk) 15:37, 5 March 2014 (UTC)

You can't condemn the entire journal out of hand merely because it has an editorial position. This is on par with saying that (your choice: Fox News or The New York Times) should always be rejected as a source. Shall we reject all specialist journals, unless we can prove that the editorial board has regular fights? I bet you'll find that the editorial board of NEJM has only people holding MDs. Anyone can see that this is obviously "biased" against DOs and nurses, and promotes MDs. Do you think you'll find many neurologists on the editorial board of JAMA Psychiatry? I don't. How about endocrinologists, to make sure that thyroid disorders that mimic psychiatric conditions are given fair attention? I'm guessing they've got zero. Shall we toss them out, too, since we've just proven that their boards are "biased" and "non-independent"?

When we talk about the need for an WP:Independent source, we are talking about whether that the author or publisher is likely to directly benefit from it. Publishing information about how you are the best _____ in the world or that people with this condition should pay you for treatment is not independent. Publishing what you did in a clinical trial, with no reason to believe that you'll benefit from this, is sufficiently independent.

Or, to put it another way, whatever the game, whatever the rules, the rules are the same for both sides. If it's "independent" for a surgery journal with a bunch of surgeons on its board to let a surgeon publish what happened to his surgery patients, then it's "independent" for an acupuncture journal with a bunch of acupuncturists on its board to let an acupuncturist publish what happened to his acupuncture patients. SPOV was rejected. We don't apply different rules to subjects that editors decide are "scientific" or "not".

Additionally, nearly any source can be reliable for some statements, per the FAQ (which you'll find towards the top of WT:V). WhatamIdoing (talk) 01:39, 6 March 2014 (UTC)

The publisher of this journal is in the business of promoting acupuncture for essentially any and all ailments. Acupuncture, I would remind you, is a non-protected term. Anyone can call themselves an acupuncturist. What surgery journal are you referencing which excludes articles that show evidence that surgery for certain conditions may not be advisable? Most surgery journals allow for non-surgeons to review clinical studies, for example. Do you see the problem? jps (talk) 03:30, 6 March 2014 (UTC)

thanks, WAID. this is pretty close to how i see it. this is such an interesting question to me. under your (our) reasoning, however, any group of fringe-y practitioners can set up a journal and even have critical reviews in it that are perfectly MEDRS compliant. right? would you please address that? am very interested in your thoughts on the hard part :) Jytdog (talk) 02:23, 6 March 2014 (UTC)

@WAID: Not sure if you're replying to me or not, but I did say specifically say what it would be reliable for, in my opinion. Regardless, let's take this a step back and use a more apt analogy. Let's say a group of orthopedic surgeons say they came up with a "revolutionary" new toe surgery that cures low back pain. The technique is so lucrative that they teach a select group of other surgeons who promote this across the country. There is no medical/physiological reason for such a surgery to work, and studies conducted by outside groups find negative results. This group then sets up a journal specifically dedicated to this surgical technique, the editorial board is comprised of surgeons who perform this technique and surprisingly, the results in this journal are markedly more positive than the studies done by, and published in, journals not affiliated with this technique. I would suggest to you that this journal has a clear conflict of interest with regards to the technique, and is not an independent source to objectively discuss the effectiveness of this technique, though it is reliable for what proponents of the technique believe. Is it your suggestion that we should take such a journal at face value? When you have a fringe group, publishing a fringe journal specifically about the fringe topic they are advocating, they are probably not independent of the topic, no matter what degree they hold. If Misplaced Pages's rules say that we need to treat this particular journal as equally valid as other journals, then there is something systemically wrong with Misplaced Pages's rules. Yobol (talk) 02:49, 6 March 2014 (UTC)

User:Yobol, I love what you wrote there - that faces that prong of the dilemma headon. OK now I am going to put the hard part of the question to you. It seems entirely plausible (to me at least) that there are scientists who (for whatever reason) want to test acupuncture interventions to see if they work and who do so rigorously and publish good science, and others who might write rigorous reviews. Where do they publish that, but for journals that at least aspire (as per the mission statement) to be solidly scientific, like the one I posted about? There are hardcore skeptics among us who will cut the legs out from an under an effort to use such a source - calling the journal FRINGE stops the conversation; and Misplaced Pages loses what may be good sources that way (and good sources seem to be hard to find in acupuncture). This is why I am on the fence and brought the question. Maybe this is a case where the fifth pillar comes in, but I am very uncomfortable doing that in alt med articles that are fringey and where we need to reason closely based on PAG. do you see what i mean? thx Jytdog (talk) 03:04, 6 March 2014 (UTC)

Let me be clear, I do not want suggest that there is any intentional wrong-doing on any part of any of these journals. I suspect that the vast, vast majority of these journals, editors, authors have nothing but the best of intentions to do good science; however we all know how easy it is to have unintentional biases, and how many different ways biases can creep into any work, despite the best of intentions. I also do understand your point; we all want to follow WP rules, be as fair to all sources as possible. However, I think WP rules already give us guidance here; we use independent sources (both in fringe and in medical topics), and I would not consider these are not independent sources. There is no need to invoke WP:IAR here. There are enough studies about acupuncture in non-acupuncture journals that we do not need to use the dubious ones. Yobol (talk) 03:25, 6 March 2014 (UTC)

Yobol nails it. The problem is that they just exclude independent review. That's inimical to our rules about independent sources being necessary. jps (talk) 03:27, 6 March 2014 (UTC)

One important barometer for determining the notability and level of acceptance of fringe ideas related to science, history or other academic pursuits is the presence or absence of peer-reviewed research on the subject. While a lack of peer-reviewed sources does not automatically mean that the subject should be excluded from Misplaced Pages, there must be adequate reliable sources to allow the subject to be covered in sufficient detail without engaging in original research. Care should be taken with journals that exist mainly to promote a particular viewpoint. Journals that are not peer reviewed by the wider academic community should not be considered reliable, except to show the views of the groups represented by those journals. Peer review is an important feature of reliable sources that discuss scientific, historical or other academic ideas, but it is not the same as acceptance by the scientific community. It is important that original hypotheses that have gone through peer review do not get presented in Misplaced Pages as representing scientific consensus or fact. Articles about fringe theories sourced solely from a single primary source (even when it is peer reviewed) may be excluded from Misplaced Pages on notability grounds.