Clinical trials are very expensive to run. The only people financially interested in running them are the pharmaceutical companies trying to make new products available.
Replication is hard enough to achieve in hard sciences where replicating a study might cost tens of thousands of dollars. Finding the money to replicate or refute a clinical trial for millions of dollars is very difficult.
@Scaevolus so very true. I use to work in this field. I was the IT director for a small pharmaceutical company that produced prescription creams and ointments and worked with a nearby university to work on new drugs.
My role was to develop the custom software that ran on laptops we gave to hospitals involved in the clinical trial. In the drug manufacturing world, anytime computer software is used in a clinical trial or in the tracking of drug production, you have to have the software validated.
Software validation is basically a process by which you hire an outside company to go over what your software does. Similar to an ISO certification. The folks & companies you hire are very clueless... They know enough buzz words to ask good questions but they often over look things.
Anyway, long story short, the whole process of just collecting data is riddled with security problems. None of this, as far as I know, is exposed to the public either. So if buggy software is, somehow, corrupting test results, there is not feedback loop to correct or discard the data. Companies like the one I worked for at the time, would throw out the data as a policy and start over. Hence, very expensive situation and very crucial to make sure your systems do the right thing the first time.
This was in the late 90s and in one situation I was in charge of some software that we were developing that was expected run past Y2K. Back then there was a lot of hoopla about the Y2K century roll over and what it would do to computer systems.
The company we hired validated our software for Y2K use cases but they often over looked basic things like data encryption, plain text storage of user passwords, etc...
I imagine the validation companies have gotten better over the years but in the end I learned that software validation is nothing more than a big CYA tactic. It's a paper trail to try and claim we did our due diligence if the shit were to hit the fan.
I'm also in the medical device industry, though we don't farm our V&V work out. The bugs are supposed to be caught in the verification phase; validation is all about making sure that the software is functioning correctly from a user perspective? Are you just conflating the terms here for the sake of simplicity?
Also, I don't think people do pure replications. People generally do replication with a twist, which is essentially a new branched experiment. Rather than confirm one specific part of a theory more than once, it's better to confirm various wings or implications of a theory.
If all the steps are made available in the paper to reproduce the experiment I would consider it science. Just because people don't replicate the study because of other factors like time or money shouldn't take away from the first experiment.
Well, technically they are replicating the study over 50+ different hospitals until they reach some confidence interval on the safety data. If you get fraud in a small fraction of them, it should not (and if you read the actual testimony, does not) impact the net result.
If you need 50 experiments to actually show that the effect exists, then even a small fraction of fraudulent experiments can heavily skew the result when looking at confidence intervals.
Think of selfhealing as a coin toss with 50/50 chance and a medication increasing this chance:
a) Medication increases healing chances to 80% -> Only few experiments needed.
b) Medication increases healing chances to 55% -> Many experiments needed to prove that the effect exists.
c) Medication is not effective but 5% fraud is introduced so that data shows 55% healing chances -> The same as in b) but with only "a little" fraud involoved.
A major flaw I see in most medications studies is that the market prices for the experimental drug or placebo are not included in the study.
My hypothesis is that a placebo described as costing $1000 per dose will produce a greater effect than a placebo described as costing $10 per dose, even if the patients are not asked to pay that cost.
Could it be that those drugs that barely outperform a gratis placebo are only producing that effect because the act of charging more for them makes them better placebos?
If that's the case, the fraud would only be needed to eliminate evidence of harm to get approval for a bogus remedy. All the effectiveness would come from magnified placebo effect.
This is an area where some kind of crowd-sourcing seems like it ought to work. The facts on side effects and morality exist once a drug is out in the wild, but we don't have any way of collecting it, curating it, or analyzing it.
Yet odds are that most of the people taking it (for any value of "it") have smartphones and could be prospectively matched with people not taking it (I'm an enemy of case-control studies for lots of reasons, but this may be an area where a well-designed one would work.)
Replication is hard enough to achieve in hard sciences where replicating a study might cost tens of thousands of dollars. Finding the money to replicate or refute a clinical trial for millions of dollars is very difficult.