How much can we trust science?

[Quintessence]

I understand that. But this is why those areas simply shouldn't command the same respect for their conclusions that physics and chemistry do. A smaller p value means that your conclusions are more likely to be valid. But a p<.05 means that *way* too many studies will draw false conclusions. This ultimately erodes the confidence in science as a whole.

I don't agree, but to each their own. The sort of pathetic backbiting that p-value elitism would create among scientists would be incredibly toxic and undermine science significantly worse because it would be a slap in the face to pretty much every scientist who isn't a physicist or a chemist. I mean, these days we mostly just joke about that, but the joke would stop being funny.

 

[metis]

The words "scientist" and "trust" are not synonymous as we simply cross-check everything and everybody-- that's the nature of us beasts. Portray it as a weakness I guess, but it's one of the most endearing things about science as we struggle to try and find what the truth may be regardless of who said what.

The alternative is blind obedience, and we ain't into that.

 

[MNoBody]

If you are unaware science is considered to be in a state of crisis. Replication is one of the foundations of science, but scientists are having problems reproducing results.

Here is an article about it, but you can also Google the replication crisis if you want more information.

In this survey of 1500 scientist they found



1,500 scientists lift the lid on reproducibility


There have been a number of studies on this replication crisis, so feel free to investigate more if you like.

Given the current state of the replication crisis, how much can we trust science?

as much as any map of the unknown can be trusted I suppose.

 

[Augustus]

Given the current state of the replication crisis, how much can we trust science?

Depends which science.

Physics, chemistry, etc: quite a lot

Psychology, neuroscience, medicine, and social sciences: a great deal less.

In certain fields, it's basically a coin flip whether published research is right of wrong and in some of these you can make at least a reasonable argument that they have a net negative impact on knowledge.

Knowledge comes from both the understanding of correct information and the avoidance of 'anti-knowledge': false information.

How can failure to replicate results be a crisis? It's not! It is a testament to the power of science and the mandate for replication that lets us know it works when it's applied.

Publishing false information based on flawed studies that informs everything from healthcare, to government policy, to workplace management, to economics, etc. is certainly a problem.

Not only can it waste large sums of money and impact quality of life, it can literally be a matter of life and death.

Not only that, but it damages the credibility of the more reliable sciences which is problematic for all kinds of reasons that you are no doubt aware of.

As well as being one of the best tools we have for understanding the world we live in, science is also one of the major sources of incorrect information too.

 

[Augustus]

And, of course, there is the issue of p-hacking, which is a different and disturbing aspect of some areas of modern science.

This tool illustrates p-hacking pretty well:

Hack Your Way To Scientific Glory

You’re a social scientist with a hunch: The U.S. economy is affected by whether Republicans or Democrats are in office. Try to show that a connection exists, using real data going back to 1948. For your results to be publishable in an academic journal, you’ll need to prove that they are “statistically significant” by achieving a low enough p-value.

FiveThrityEight p-hacking tool

Just randomly play around with the parameters and you can find a 'statistically significant' result about whether the right or the left are 'objectively' better!

In your face other side! Science FTW!

 

[Augustus]

The words "scientist" and "trust" are not synonymous

While that may be true, calling something scientific certainly gives it a hell of a lot more credibility in the mind of the average person.

 

[ReluctantMathematician]

Understood and agreed. But at this point, the biggest problem is having p<.05 as a goal in too many areas of study. Even bringing that down to p<.01 or p<.005 would immensely help.

And, of course, there is the issue of p-hacking, which is a different and disturbing aspect of some areas of modern science.

I understand that. But this is why those areas simply shouldn't command the same respect for their conclusions that physics and chemistry do. A smaller p value means that your conclusions are more likely to be valid. But a p<.05 means that *way* too many studies will draw false conclusions. This ultimately erodes the confidence in science as a whole.

You claim to understand, but it is clear that you don't. This .0005 alpha level that you are preaching about in particle physics likely has more to do with the variance of the targeted population and the sample size than this higher standard you are boosting about.

In Null Hypothesis Testing, the power of the test is the probability of correctly rejecting a false null. While alpha (the significance level) is the probability of rejecting the null. Now think about this logically, if you decrease alpha then you decrease your power. You decrease your probability of rejecting a false null hypothesis, therefore generating errors.

Also, power is affected by your variance and your sample size, so you can't just use an arbitrary alpha for every population distribution and expect fewer errors. Perhaps you are not aware of this but not every population in the world has the same variance.

Further, a smaller p-value simply provides more evidence against the null, it does not mean that your conclusions are more valid. The p-value is simply the probability of the observed or more extreme if the null is true, so the smaller it is the less likely we consider the null, but p-values don't create facts, they provide evidence against the null assumption. Also, p-values are impacted by sample size, if you have a large enough sample size you'll always get a small p-value.

 

[cladking]

In certain fields, it's basically a coin flip whether published research is right of wrong and in some of these you can make at least a reasonable argument that they have a net negative impact on knowledge.

Actually there are fields (such as Egyptology) where everything published or Peer reviewed is probably wrong. At least there is rational reason to believe everything concerning the so called old kingdom is wrong. I believe this applies to much of archaeology and anthropology as well. Most of "sociology" depends on assumptions that can not be shown and all social sciences are founded ideas that can not be shown experimentally.

"Truth" has become largely convention which supports a status quo which is unsupportable and unsustainable.

 

[Shadow Wolf]

medicine

I don't see that as a problem with science, at all. Rather, in all fields, it's a problem with people, and especially with greed and pressures and mandates to get published. It gets a lot of junk studies out there, because someone had to get published but they had nothing so they play with the numbers until they find something significant enough. It plants corporate doctors to publish deliberately-contradicting article so the corporation (such as the NFL) can protect themselves. It looks at suffering and says "I'm going to profit from people's misery with this drug" and self-publishes it's own self-funded studies that says the drug doesn't work and then lie and say it does anyways (Former Elli Lilly executive/Governor of Indiana/Current head Purdue person Mitch Daniels did this with Prozac).
We also made both malicious and well-intentioned mistakes in the past, which has skewed the present (such as no tests on women or children).
And of course we have the legal version of that shady dude in alley giving dope samples. They are called drug/pharmacy reps. They are often young, attractive, bubbly women, often cheerleaders, and they are sent to market their drugs to doctors. Sometimes with gifts (ranging from free lunch to more lucrative resort stays). This is intentional, it is deliberate and by design, and it is not always or inherently illegal.

 

[Shadow Wolf]

I also know one of these legal drug pushers. She quit driving her own personal car (a newer then Camry) because her employer gave her better. Technically for business, but she gave me her car when my Cougar when kaput because she didn't need, use, or miss it.
That should say a lot about how profitable her position is to her legal drug cartel.

 

[Native]

A weak or poor models should be discarded if refuted.

When the observations of the starry motion around a galactic center was discovered in 1932, this refuted Newtons law of celestial motion.

Why isn´t Newtons law discarded long time ago then?

 

[Native]

Actually there are fields (such as Egyptology) where everything published or Peer reviewed is probably wrong.

Not only probably but surely wrong as historic authors and present scholars are downgrading the ancient knowledge of the ancient known part of cosmos, thus interpreting and mistaking the Sun to have created everything in our galaxy and this misconception and misinterpretation can be read all over in books and encyclopedia.

 

[stvdv]

Given the current state of the replication crisis, how much can we trust science?

I trust my conscience + Common Sense
I use science within my own trust

 

[Polymath257]

When the observations of the starry motion around a galactic center was discovered in 1932, this refuted Newtons law of celestial motion.

Why isn´t Newtons law discarded long time ago then?

Because it is a very good approximation. It was actually shown to be wrong well before that, by the way.

 

[Polymath257]

Not only probably but surely wrong as historic authors and present scholars are downgrading the ancient knowledge of the ancient known part of cosmos, thus interpreting and mistaking the Sun to have created everything in our galaxy and this misconception and misinterpretation can be read all over in books and encyclopedia.

Show me one book that claims the sun created everything in the galaxy.

 

[ReluctantMathematician]

I understand that. But this is why those areas simply shouldn't command the same respect for their conclusions that physics and chemistry do. A smaller p value means that your conclusions are more likely to be valid. But a p<.05 means that *way* too many studies will draw false conclusions. This ultimately erodes the confidence in science as a whole.

Also, I am sorry to tell you this but significance level testing is not the current gold standard, it is more like the tin standard. Strength of evidence interpretations for the p-value is the new gold standard and significance level testing is starting to be abandoned by statisticians, some universities, and some branches of science.

You really should not be preaching on about this stuff as if you are an authority when you are not. That lacks intellectual integrity and it just misleads those who don't know better.

 

[nPeace]

If you are unaware science is considered to be in a state of crisis. Replication is one of the foundations of science, but scientists are having problems reproducing results.

Here is an article about it, but you can also Google the replication crisis if you want more information.

In this survey of 1500 scientist they found



1,500 scientists lift the lid on reproducibility


There have been a number of studies on this replication crisis, so feel free to investigate more if you like.

Given the current state of the replication crisis, how much can we trust science?

It seems to be becoming more and more problematic, and less and less reliable, as more and more people seem to be losing faith in the apparent double talk.

People usually look for answers that are reliable, and when they find themselves being swung from one ladder to the next, they soon make a bee line for the exit... just as they do when the clergy give them the same kind of double talk... The pews quickly become empty.

Carl Sagan wrote of arguments from authority:
One of the great commandments of science is, "Mistrust arguments from authority." ... Too many such arguments have proved too painfully wrong. Authorities must prove their contentions like everybody else.

What do we see today?
Cosmic inflation debate bleeds into popular science media

Spoiler: Excerpt

When three physicists published “Cosmic inflation theory faces challenges” earlier this year in Scientific American, the article itself constituted a challenge. Its subhead declared, “The latest astrophysical measurements, combined with theoretical problems, cast doubt on the long-cherished inflationary theory of the early cosmos and suggest we need new ideas.” The trio’s aggressive reappraisal of a scientific consensus inspired an energetic rebuttal, also in Scientific American, from 33 prominent physicists, including four Nobel laureates.

33 Top Physicists Including Hawking Defend the Theory of Cosmic Inflation

Spoiler: Excerpt

"The latest measurements of the cosmic microwave background (CMB), the universe's oldest light, raise concerns about the inflationary theory of the cosmos—the idea that space expanded exponentially in the first moments of time. Inflation typically produces a different pattern of temperature variation in the CMB (although it can be made to predict almost any outcome). It would also generate primordial gravitational waves, which have not been found. The data suggest cosmologists should reassess this favored paradigm and consider new ideas about how the universe began".

[Three physicists - Anna Ijjas, the John A. Wheeler postdoctoral fellow at the Princeton Center for Theoretical Science; Paul Steinhardt, Albert Einstein Professor in Science at Princeton University, director of the Princeton Center for Theoretical Science, and an original architect of cosmic inflation theory; and Abraham Loeb, Harvard University’s astronomy chair, founding director of Harvard’s Black Hole Initiative, and director of the Institute for Theory and Computation at the Harvard–Smithsonian Center for Astrophysics] also went on to say that other scientists are taking a new avenue to reinforce the idea of cosmic inflation by discarding empirical science and adapting a more theoretical approach on the matter.

"Some scientists accept that inflation is untestable but refuse to abandon it. They have proposed that, instead, science must change by discarding one of its defining properties: empirical testability. This notion has triggered a roller coaster of discussions about the nature of science and its possible redefinition, promoting the idea of some kind of non-empirical science".

Moreover, they claim that the inflation theory is untestable because of its flexible nature.

"Individually and collectively, these features make inflation so flexible that no experiment can ever disprove it".

That's just one among many, and I haven't mentioned the ongoing arguments concerning evolution by natural selection, speciation, etc.

When some people see that, and they see the answers to their questions making seesaw motions, they are not sure they can trust those "pastors" holding their "Bibles", so those pews start to empty too.

I remember there was an article I read some time back - Has Science Replaced the Bible?
After mentioning some of the things science has accomplished, the first article says...
However, the question must be asked: Has science learned enough about the natural world to be able to draw definitive conclusions? The simple answer is no. Science has made tremendous progress, but many scientists recognize that there are still many unknowns and perhaps unknowables. “We will never get to the bottom of things,” said physicist and Nobel laureate Steven Weinberg on understanding nature. Professor Martin Rees, Astronomer Royal of Great Britain, wrote: “There may be things that humans will never understand.” The truth is that much of nature, from the tiny cell to the vast universe, still remains beyond the grasp of modern science.

The following article goes into discussing the fact that Science Has Its Limitations.
I think that's where the problem lies in science today. Hubris - man claiming to be able to know everything, thus misleading the public with this false "quest".
It will backfire, because people are not the stupid apes they seem to want people to believe they are.

 

[gnostic]

Actually there are fields (such as Egyptology) where everything published or Peer reviewed is probably wrong. At least there is rational reason to believe everything concerning the so called old kingdom is wrong. I believe this applies to much of archaeology and anthropology as well. Most of "sociology" depends on assumptions that can not be shown and all social sciences are founded ideas that can not be shown experimentally.

"Truth" has become largely convention which supports a status quo which is unsupportable and unsustainable.

Not only probably but surely wrong as historic authors and present scholars are downgrading the ancient knowledge of the ancient known part of cosmos, thus interpreting and mistaking the Sun to have created everything in our galaxy and this misconception and misinterpretation can be read all over in books and encyclopedia.

As I have told you in the past threads, there aren’t any Peer Review for Egyptology, because like anthropology, archaeology, history, there are also non-scientific fields, like translation, linguistics and literature which would all fall under Humanities, which play a large crucial roles in archaeology and history. Humanities also encompass fine arts, liberal arts, politics, laws, etc.

Some of Humanities studies do cross over to Social Sciences, but all subjects of Social Sciences (eg psychology, sociology, anthropology, archaeology, political science, economics, etc), are considered soft sciences, so they don’t require Scientific Method and Peer Review in the ways Natural Sciences (this encompassed all branches of Physical Sciences + Life Sciences) do use Scientific Method and Peer Review.

Translations of ancient texts is called philology, which is part of linguistics. Linguistics falls under Humanities, not under Social Sciences, and most not under Natural Sciences. So philology isn’t scientific, so it do not require to meet the same criteria of Natural Sciences - Scientific Method and Peer Review.

That you keep saying Egyptology use Peer Review, is simply wrong, because philology, as well as arts, are non-scientific subjects, which don’t require it to be peer reviewed.

 

[ReluctantMathematician]

It seems to be becoming more and more problematic, and less and less reliable, as more and more people seem to be losing faith in the apparent double talk.

People usually look for answers that are reliable, and when they find themselves being swung from one ladder to the next, they soon make a bee line for the exit... just as they do when the clergy give them the same kind of double talk... The pews quickly become empty.

Carl Sagan wrote of arguments from authority:
One of the great commandments of science is, "Mistrust arguments from authority." ... Too many such arguments have proved too painfully wrong. Authorities must prove their contentions like everybody else.

What do we see today?
Cosmic inflation debate bleeds into popular science media

Spoiler: Excerpt

When three physicists published “Cosmic inflation theory faces challenges” earlier this year in Scientific American, the article itself constituted a challenge. Its subhead declared, “The latest astrophysical measurements, combined with theoretical problems, cast doubt on the long-cherished inflationary theory of the early cosmos and suggest we need new ideas.” The trio’s aggressive reappraisal of a scientific consensus inspired an energetic rebuttal, also in Scientific American, from 33 prominent physicists, including four Nobel laureates.

33 Top Physicists Including Hawking Defend the Theory of Cosmic Inflation

Spoiler: Excerpt

"The latest measurements of the cosmic microwave background (CMB), the universe's oldest light, raise concerns about the inflationary theory of the cosmos—the idea that space expanded exponentially in the first moments of time. Inflation typically produces a different pattern of temperature variation in the CMB (although it can be made to predict almost any outcome). It would also generate primordial gravitational waves, which have not been found. The data suggest cosmologists should reassess this favored paradigm and consider new ideas about how the universe began".

[Three physicists - Anna Ijjas, the John A. Wheeler postdoctoral fellow at the Princeton Center for Theoretical Science; Paul Steinhardt, Albert Einstein Professor in Science at Princeton University, director of the Princeton Center for Theoretical Science, and an original architect of cosmic inflation theory; and Abraham Loeb, Harvard University’s astronomy chair, founding director of Harvard’s Black Hole Initiative, and director of the Institute for Theory and Computation at the Harvard–Smithsonian Center for Astrophysics] also went on to say that other scientists are taking a new avenue to reinforce the idea of cosmic inflation by discarding empirical science and adapting a more theoretical approach on the matter.

"Some scientists accept that inflation is untestable but refuse to abandon it. They have proposed that, instead, science must change by discarding one of its defining properties: empirical testability. This notion has triggered a roller coaster of discussions about the nature of science and its possible redefinition, promoting the idea of some kind of non-empirical science".

Moreover, they claim that the inflation theory is untestable because of its flexible nature.

"Individually and collectively, these features make inflation so flexible that no experiment can ever disprove it".

That's just one among many, and I haven't mentioned the ongoing arguments concerning evolution by natural selection, speciation, etc.

When some people see that, and they see the answers to their questions making seesaw motions, they are not sure they can trust those "pastors" holding their "Bibles", so those pews start to empty too.

I remember there was an article I read some time back - Has Science Replaced the Bible?
After mentioning some of the things science has accomplished, the first article says...
However, the question must be asked: Has science learned enough about the natural world to be able to draw definitive conclusions? The simple answer is no. Science has made tremendous progress, but many scientists recognize that there are still many unknowns and perhaps unknowables. “We will never get to the bottom of things,” said physicist and Nobel laureate Steven Weinberg on understanding nature. Professor Martin Rees, Astronomer Royal of Great Britain, wrote: “There may be things that humans will never understand.” The truth is that much of nature, from the tiny cell to the vast universe, still remains beyond the grasp of modern science.

The following article goes into discussing the fact that Science Has Its Limitations.
I think that's where the problem lies in science today. Hubris - man claiming to be able to know everything, thus misleading the public with this false "quest".
It will backfire, because people are not the stupid apes they seem to want people to believe they are.

Religion is what claims to know everything, apparently magically with no supporting evidence. Despite its problems, I trust science over religion.

 

[nPeace]

Religion is what claims to know everything, apparently magically with no supporting evidence. Despite its problems, I trust science over religion.

Religion does not claim to know everything. Where did you get that from?
Nor does religion claim a theory of everything.

I accept both science and religion. However, when it comes to truth, I follow the evidence where it leads.
Science is not without limits, and unproven conclusions, same as religion, but I see no need to pit one against the other.

Note - When I say science, I mean science, not what is called science, although untested, unobserved.