How much can we trust science?

[ReluctantMathematician]

If you are unaware science is considered to be in a state of crisis. Replication is one of the foundations of science, but scientists are having problems reproducing results.

Here is an article about it, but you can also Google the replication crisis if you want more information.

In this survey of 1500 scientist they found

More than 70% of researchers have tried and failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments. Those are some of the telling figures that emerged from Nature's survey of 1,576 researchers who took a brief online questionnaire on reproducibility in research. [...] Data on how much of the scientific literature is reproducible are rare and generally bleak. The best-known analyses, from psychology1 and cancer biology2, found rates of around 40% and 10%, respectively. Our survey respondents were more optimistic: 73% said that they think that at least half of the papers in their field can be trusted, with physicists and chemists generally showing the most confidence.

1,500 scientists lift the lid on reproducibility


There have been a number of studies on this replication crisis, so feel free to investigate more if you like.

Given the current state of the replication crisis, how much can we trust science?

 

[PoetPhilosopher]

A balance is needed of skepticism and trust in my opinion but where that balance lies is debatable.

 

[bobhikes]

Given the current state of the replication crisis, how much can we trust science?

You can trust it as much as you use it.

 

[Brickjectivity]

If you are unaware science is considered to be in a state of crisis. Replication is one of the foundations of science, but scientists are having problems reproducing results.

Here is an article about it, but you can also Google the replication crisis if you want more information.

In this survey of 1500 scientist they found



1,500 scientists lift the lid on reproducibility


There have been a number of studies on this replication crisis, so feel free to investigate more if you like.

Given the current state of the replication crisis, how much can we trust science?

This fellow on youtube makes Applied Science videos and in this particular one he struggles to follow directions in Scientific American to make his own electron gun. He points out several flaws in the instructions, but he manages to overcome them.

These are the kinds of problems you might encounter when trying to reproduce something that someone else has done. First, they have to want their results to be reproducible. Secondly they have to describe their experimental process well enough. It may surprise but being good at research doesn't necessarily make one good at explaining things.

I guess overall my thought is that Science has always been a very long term process, and so sometimes results don't get verified in which case they fall to the wayside. They may be forgotten. On the other hand a successful experiment encourages other to try, so its better than nothing.

 

[ReluctantMathematician]

A balance is needed of skepticism and trust in my opinion but where that balance lies is debatable.

I think we need greater awareness and education of scientific methods in the general public. The public just trusts that scientists do what they should be doing, but scientists are failing on many points, often just to get published. Now that is not entirely the fault of the scientific community, as unless they produce interesting results then they don't get published at all. If it is not interesting then it doesn't sell and no one wants to read a scientific study that failed to produce results. However, if the public had a deeper understanding of scientific methods then they may be better able to spot inappropriate methods, and perhaps they would grow to appreciate good scientific work even when it doesn't produce flashy results.

 

[Twilight Hue]

If you are unaware science is considered to be in a state of crisis. Replication is one of the foundations of science, but scientists are having problems reproducing results.

Here is an article about it, but you can also Google the replication crisis if you want more information.

In this survey of 1500 scientist they found



1,500 scientists lift the lid on reproducibility


There have been a number of studies on this replication crisis, so feel free to investigate more if you like.

Given the current state of the replication crisis, how much can we trust science?

Trust science as much as its grant money.

 

[syo]

how much can we trust science?

A little. Most science is stupidity. But some of it is ok.

 

[Shadow Wolf]

If you are unaware science is considered to be in a state of crisis. Replication is one of the foundations of science, but scientists are having problems reproducing results.

Here is an article about it, but you can also Google the replication crisis if you want more information.

In this survey of 1500 scientist they found



1,500 scientists lift the lid on reproducibility


There have been a number of studies on this replication crisis, so feel free to investigate more if you like.

Given the current state of the replication crisis, how much can we trust science?

How can failure to replicate results be a crisis? It's not! It is a testament to the power of science and the mandate for replication that lets us know it works when it's applied. (unfortunately money from all levels has also greately tainted to the process, from publications to professor's jobs to doctor's offices).
This just means the system is working, and it does go to show we are only humans. We are silly apes who think we are very smart. This shows even our brightest and best are prone to making mistakes, having bias, and just getting things wrong.
Now, if we have a very high rate of replication, then we truly have a very serious problem and the system has failed. Science lives and breaths to be disproven. It's lost it's purpose and usefulness if things aren't getting proven wrong.

 

[cladking]

A great deal of "science" now isn't really science at all.

I call it "soup of the day science" or "Look and See Science".

There are many reasons for it but the root of the problem is usually metaphysical or definitional. When all scientists agree and have no experiment you can be pretty sure they are wrong. Modern humans are best at circular reasoning and proving they are right and were right even before they began studying the subject.

 

[Polymath257]

Well, any subject that uses a p<.05 for significance (as opposed to something more reasonable like p<.0005) will ultimately have a lot of problems with replication. Unfortunately, that includes a lot that is usually thought of as being 'science'.

You can trust those areas of science that *do* have replication and higher standards for relevance.

As an example, particle physics requires at least a five standard deviation signal to be considered a discovery. Anything less than that is, perhaps, interesting, but it is *expected* that many such things will be false signals.

The replication crisis, such as it exists, is primarily because of very poor statistical standards for relevance of poor research.

 

[Quintessence]

Well, any subject that uses a p<.05 for significance (as opposed to something more reasonable like p<.0005) will ultimately have a lot of problems with replication. Unfortunately, that includes a lot that is usually thought of as being 'science'.

A p-value that low isn't reasonable for many fields of study. It's not reasonable for basically anything outside of mathematics, physics, and chemistry where you can strictly control conditions of the experiments. It definitely would not work for biology. That's not to say we never see significance values that low, but it is substantially harder to get because of the nature of what's being studied.

To my mind, the real problem is the general public (and occasionally scientists) not understanding a study's limitations.

 

[Mock Turtle]

A great deal of "science" now isn't really science at all.

I call it "soup of the day science" or "Look and See Science".

There are many reasons for it but the root of the problem is usually metaphysical or definitional. When all scientists agree and have no experiment you can be pretty sure they are wrong. Modern humans are best at circular reasoning and proving they are right and were right even before they began studying the subject.

Opening a can of worms here but - the alternative is?

 

[ReluctantMathematician]

Well, any subject that uses a p<.05 for significance (as opposed to something more reasonable like p<.0005) will ultimately have a lot of problems with replication. Unfortunately, that includes a lot that is usually thought of as being 'science'.

You can trust those areas of science that *do* have replication and higher standards for relevance.

As an example, particle physics requires at least a five standard deviation signal to be considered a discovery. Anything less than that is, perhaps, interesting, but it is *expected* that many such things will be false signals.

The replication crisis, such as it exists, is primarily because of very poor statistical standards for relevance of poor research.

What works in particle physics does not necessarily work for everything else. And for some distributions being considered if you used a .0005 cutoff you'd start racking up type two errors instead of type one errors, then all you did was trade one error in for another. There are methods for trying to find an optimal alpha level if you must use one, but really you don't need a significance level at all. You can simply interpret the p-value for what it is, a conditional probability.

 

[ChristineM]

If you are unaware science is considered to be in a state of crisis. Replication is one of the foundations of science, but scientists are having problems reproducing results.

Here is an article about it, but you can also Google the replication crisis if you want more information.

In this survey of 1500 scientist they found



1,500 scientists lift the lid on reproducibility


There have been a number of studies on this replication crisis, so feel free to investigate more if you like.

Given the current state of the replication crisis, how much can we trust science?

My view is bad science isn't science, if results cannot be replicated then the hypothesis should be either ditched or modified and retested until it can be replicated.

How much should science be trusted. Considering everyone here uses several proven scientific principals to post then i would say they trust it well enough.

Considering the vast majority of people over the age of 30 are only here because of science i would say it's not only trustworthy but of great benefit.

Of course there are those sucking a considerable bunch of sour grapes for science and are happily posting on RF in their dotage.

 

[Native]

. . . how much can we trust science?

It depend on which scientific branch you´re thinking of, but the cosmological and astrophysical part of science is mostly "gravitational science fiction" filled with all kinds of dark this and that ghosts which is added as pure hindsight bias additions ad hoc.

This "science" is completely untrustworty and in deep crisis.

 

[Wandering Monk]

Even with its flaws, I don't know of any other method for measuring and comprehending reality. Do you?

 

[cladking]

Opening a can of worms here but - the alternative is?

Even with its flaws, I don't know of any other method for measuring and comprehending reality. Do you?

There are potentially alternatives but this isn't my point.

My point is that we have a poor perspective for understanding. While we must use reason and experiment we must remember that only these are effective and that our knowledge is still very tiny compared to what there is to know.

We could figure out how animals invent their wonders and adapt that to our use.


For us "understanding" is a state of mind and people easily get comfortable with very little of it. We mistake technology for theory and comfort for understanding.

 

[Polymath257]

A p-value that low isn't reasonable for many fields of study. It's not reasonable for basically anything outside of mathematics, physics, and chemistry where you can strictly control conditions of the experiments. It definitely would not work for biology. That's not to say we never see significance values that low, but it is substantially harder to get because of the nature of what's being studied.

To my mind, the real problem is the general public (and occasionally scientists) not understanding a study's limitations.

I understand that. But this is why those areas simply shouldn't command the same respect for their conclusions that physics and chemistry do. A smaller p value means that your conclusions are more likely to be valid. But a p<.05 means that *way* too many studies will draw false conclusions. This ultimately erodes the confidence in science as a whole.

 

[Polymath257]

What works in particle physics does not necessarily work for everything else. And for some distributions being considered if you used a .0005 cutoff you'd start racking up type two errors instead of type one errors, then all you did was trade one error in for another. There are methods for trying to find an optimal alpha level if you must use one, but really you don't need a significance level at all. You can simply interpret the p-value for what it is, a conditional probability.

Understood and agreed. But at this point, the biggest problem is having p<.05 as a goal in too many areas of study. Even bringing that down to p<.01 or p<.005 would immensely help.

And, of course, there is the issue of p-hacking, which is a different and disturbing aspect of some areas of modern science.

 

[gnostic]

If you are unaware science is considered to be in a state of crisis. Replication is one of the foundations of science, but scientists are having problems reproducing results.

Here is an article about it, but you can also Google the replication crisis if you want more information.

In this survey of 1500 scientist they found



1,500 scientists lift the lid on reproducibility


There have been a number of studies on this replication crisis, so feel free to investigate more if you like.

Given the current state of the replication crisis, how much can we trust science?

I don’t see any crisis in science.

If the evidence are against the proposal models (hypotheses), then the problems are with the models, not the evidence.

A weak or poor models should be discarded if refuted.

And if there are no evidence whatsoever, then the model doesn’t even qualify as being “hypothesis”, because the model is unfalsifiable and untestable.

Such unfalsifiable models should be burned, because they are toxic.

Intelligent Design and Irreducible Complexity fit the bill of being unfalsifiable...they are pseudoscience.