How much faith do you have in science?

[youknowme]

Statistics are facts. How those facts relate to matters of claims of cause and effect is something else altogether.

It seems to me that you are identifying a very real problem, but entirely misattributing its causes and conceivable solutions. We are not supposed to have "faith in science". If it requires faith, then it is probably not science at all.

The article that you are quoting is superb, by the way.

But the matter at hand here is not even close to an excess of "faith in science", but rather one of neglect of the needs of rigor of method and expression. To a very large extent, that is not even a flaw of the scientific community, but rather of the social and political environment, which goes out of its way to misunderstand and misrepresent scientific findings.

There is a reason why Feynman called that "Cargo Cult Science".

Do not confuse statistics the science with statistics a value.

 

[youknowme]

Statistics, the science, is about making probabilistic inferences of unknown parameters under a set of assumptions, that does not establish scientific fact, and it is different than "statistics" a calculated value from obtained data.

 

[youknowme]

Indeed, you can not conclude that any model is true out of statistical tests alone. You can become pretty certain, but even that is unlikely and in practice almost unheard of.

Then again, one of the reasons why it is so rare is because it is not meant to happen in the first place. Statistical models are great investigative tools, but almost demonstrably unsuitable for showing actual causation or causal relation.

For that you have to build an actual theoretical model and test it. You have to attain falsifiability. Your model should be capable of supporting claims that may be objectively tested for accuracy of prediction.

Sure, that can be challenging at the best of times, and arguably impossible in human and biological fields. We definitely should remind ourselves often that statistical correlation does not imply causation at all. Above all, we should watch for and reign in our natural tendencies for pursuing certainty at the flimsiest of excuses.

One of the two types of studies you can do with statistics (the science) is to make cause and effect inferences. There are only two types of studies in statistics, observational and experiments. Then there are two types of models, predictive models (not looking for cause and effect relations on this one) and explanation models. In some cases we are very much modeling to make cause and effect inferences and the science has been designed to do just that (realizing these are still probabilistic inferences).

Now you can test predictive models in a more reliable fashion than explanation models; you just test how close it is to predicting actual values, and in some cases this will work with the other model as well, but not always. We need to consider statistics as just evidence, and not as facts, because that is what it actually is, and then we need to take that evidence and consider it alongside other evidence. We cannot rely on a one statistical study and we cannot rely on one approach, but to hit deadlines that is what is happening. So I agree with you in a sense, that we need to take our model and find some why to actually test it; however, your claim that we can't make cause and effects inference with statistics is just not correct, under the right conditions you can.

 

[youknowme]

Let's clear a few things up here:

There are descriptive statistics, these are the raw values your calculated from your sample. They are mathematically true. Not auguring otherwise.

However,

There are also inferential statistics, these are what we are using to make probabilistic inferences about the characteristics of the population we got our sample from. THESE ARE NOT FACTS, they are evidence. It is very important to understand this difference.

Now there is also something we call the "scope of inference":

If you have random sampling you can make generalized inferences from your sample to the larger population and it will be support by a probability model.

If you have random assignment (that is where you randomly assign treatment effects) you can make cause and effect inferences and it will be supported by a probability model.

 

[beenherebeforeagain]

It works this way in biological sciences too, in many cases.

It's likely that different fields have different standards. There really wasn't a push for "statistical significance" in the field I trained in. That was something that my mentors were insistent upon dispelling, including the statistician I was working with at the time. In conservation biology, whether or not something is statistically significant is often irrelevant when we're dealing with topics that have both practical and policy implications. That's not to say the stats aren't important, but they are often not the driving factor in how you write up your research.

Also, I'd be remiss not to post up one of my favorite webcomics again, because it's relevant to the issue (the problem is not so much the science itself as the interpretation thereof):

Yeah, that looks like what I was used to...

 

[Augustus]

We are not supposed to have "faith in science". If it requires faith, then it is probably not science at all.

I have to disagree with this. Science isn't simply some normative, textbook concept that describes a high ideal for discovering knowledge, it is a real world phenomenon that exists in a very human society and is used as a basis for all kinds of decisions and actions.

As nobody can independently verify everything themselves, we rely on the belief that 'the system' will moderate itself.

We thus need to have faith that scientific knowledge, and therefore scientific expertise, is based on a solid, accurate foundation.

When politicians and bureaucrats rely on studies to inform policies, we need to have faith these studies are meaningful. When doctors rely on studies to treat our illnesses we need to have faith these are not based on flawed science (which, unfortunately, they often are). When scientist engage in genetic engineering we need to have faith that they understand what they are doing and understand the risks that such actions entail.

But the matter at hand here is not even close to an excess of "faith in science", but rather one of neglect of the needs of rigor of method and expression. To a very large extent, that is not even a flaw of the scientific community, but rather of the social and political environment, which goes out of its way to misunderstand and misrepresent scientific findings.

While socio-political actors do misrepresent science for ideological reasons, a lot of the problems are inherent to practice within the sciences.

It's not simply science good, politics bad, a lot of the science is bad on its own merits.

 

[youknowme]

I have to disagree with this. Science isn't simply some normative, textbook concept that describes a high ideal for discovering knowledge, it is a real world phenomenon that exists in a very human society and is used as a basis for all kinds of decisions and actions.

As nobody can independently verify everything themselves, we rely on the belief that 'the system' will moderate itself.

We thus need to have faith that scientific knowledge, and therefore scientific expertise, is based on a solid, accurate foundation.

When politicians and bureaucrats rely on studies to inform policies, we need to have faith these studies are meaningful. When doctors rely on studies to treat our illnesses we need to have faith these are not based on flawed science (which, unfortunately, they often are). When scientist engage in genetic engineering we need to have faith that they understand what they are doing and understand the risks that such actions entail.




While socio-political actors do misrepresent science for ideological reasons, a lot of the problems are inherent to practice within the sciences.

It's not simply science good, politics bad, a lot of the science is bad on its own merits.

I think part of the problem here is that using probability in the fashion that statistics does is still relatively new compared to some of the older and more established methods of science. The concept of statistics is not that new, but it was limited in the past because we lacked computers that could do such intense calculations. However, now that computational power is cheap it has unlocked a new world of data analysis.

My hope is that these issues are just growing pains, because despite our problems, the use of probability theory in science is an incredibly powerful tool.

 

[MonkeyFire]

Every faith and intuition must be validated by science before you know the truth.

 

[youknowme]

Every faith and intuition must be validated by science before you know the truth.

I think it is a bit reckless to think of science as some type of truth validation, personally, I view science as evidence.

 

[MonkeyFire]

I think it is a bit reckless to think of science as some type of truth validation, personally, I view science as evidence.

Science requires truth at all what so ever.

 

[tas8831]

Some consider science to be in a state of crisis and this in part due to the fact that they have been using the p-value in a way it was never meant to be used and to get published they push for "statistical significance" regardless of if it actually means anything worthwhile.

While I agree that the p-value is misused, just curious who considers science to be in a state of crisis?
In my area of research, i am happy to say that we do not employ such things, we have several other statistical tests to employ.

Think of all the studies you see in the news day in and day out: Do you really think science happens that fast?

Not sure what you mean - that we see lots of studies being (usually incorrectly) discussed in media is likely due to the fact that lots and lots and lots of people are involved in research and of the thousands of new bits of published research that come out on any given day, a few dozen get picked up by the various news outlets doesn't seem like it is out of line. And these generally are the results of months of work.

I see several people on these forums chucking statistics at each other, as if they are facts, and a few of them have even insisted that statistics are facts, so my question is: How much faith do you have in science?

If we are to use such loaded terminology on a forum like this, I have a great deal of faith in science as a general endeavor. I have far more faith - again, as used (i think) in the context of your question - in science than I do in 'other ways of knowing.'

 

[LuisDantas]

Do not confuse statistics the science with statistics a value.

I do not.

 

[LuisDantas]

I have to disagree with this. Science isn't simply some normative, textbook concept that describes a high ideal for discovering knowledge, it is a real world phenomenon that exists in a very human society and is used as a basis for all kinds of decisions and actions.

As nobody can independently verify everything themselves, we rely on the belief that 'the system' will moderate itself.

We thus need to have faith that scientific knowledge, and therefore scientific expertise, is based on a solid, accurate foundation.

When politicians and bureaucrats rely on studies to inform policies, we need to have faith these studies are meaningful. When doctors rely on studies to treat our illnesses we need to have faith these are not based on flawed science (which, unfortunately, they often are). When scientist engage in genetic engineering we need to have faith that they understand what they are doing and understand the risks that such actions entail.




While socio-political actors do misrepresent science for ideological reasons, a lot of the problems are inherent to practice within the sciences.

It's not simply science good, politics bad, a lot of the science is bad on its own merits.

Unsurprisingly, it looks like we will have to agree to disagree.

 

[LuisDantas]

Statistics, the science, is about making probabilistic inferences of unknown parameters under a set of assumptions, that does not establish scientific fact, and it is different than "statistics" a calculated value from obtained data.

Indeed.

It also involves a lot of responsibility for the information, how it is collected, how it is handled, and how it is presented.

All of those are often neglected, not too rarely in rather gross ways. It is a bit surprising that it happens so often; it is not like there is a lot of effort put into making it appear respectable or convincing.

 

[Cooky]

There is no "faith" in science.

 

[QuestioningMind]

Significance magazine - Cargo-cult statistics and scientific crisis | Significance magazine

Here is the ASA's (American Statistical Association) statement on p-values:

https://amstat.tandfonline.com/doi/abs/10.1080/00031305.2016.1154108#.XFefHlVKiCh


Here is a very simple short video on the p-value to give you a basic idea what a p-value is:

Some consider science to be in a state of crisis and this in part due to the fact that they have been using the p-value in a way it was never meant to be used and to get published they push for "statistical significance" regardless of if it actually means anything worthwhile.

Think of all the studies you see in the news day in and day out: Do you really think science happens that fast? I see several people on these forums chucking statistics at each other, as if they are facts, and a few of them have even insisted that statistics are facts, so my question is: How much faith do you have in science?

The scientific method has demonstrated itself to be by far the best method we've ever found for determining how the universe works. that's still the case. The issues that you mention come from individual scientists with an agenda and far more often in my opinion, the media.

A perfect example of this is how so many people claim that back in the 70's science declared that we were about to enter a new ice age. The reality is that a single group of scientists published a single study that suggested a new ice age was coming and prior to having the results peer reviewed Time Magazine ran a front page article declaring SCIENCE SAYS A NEW ICE AGE IS COMING! and media throughout the world starting saying the same thing. Of course when the study WAS reviewed and found by the scientific community to have significant flaws, the news media didn't bother to mention the correction.

 

[Augustus]

The scientific method has demonstrated itself to be by far the best method we've ever found for determining how the universe works. that's still the case.

Some scientific methods have demonstrated themselves to be by far the best methods we've ever found for determining how some parts of the universe works.

For other parts, they're not doing so grand...

The issues that you mention come from individual scientists with an agenda and far more often in my opinion, the media.

When certain scientific disciplines (neuroscience, psychology, etc.) have 30-60% error rates in publication that is often passing peer-review, blaming the problems on individual scientists and the media seems a bit misguided.

In many areas, the problems are systemic and result from what is considered 'scientific best practice'.

 

[youknowme]

Indeed.

It also involves a lot of responsibility for the information, how it is collected, how it is handled, and how it is presented.

All of those are often neglected, not too rarely in rather gross ways. It is a bit surprising that it happens so often; it is not like there is a lot of effort put into making it appear respectable or convincing.

And how it is read, which I think the general public lacks the needed know-how to do correctly.

 

[beenherebeforeagain]

The scientific method has demonstrated itself to be by far the best method we've ever found for determining how the universe works. that's still the case. The issues that you mention come from individual scientists with an agenda and far more often in my opinion, the media.

A perfect example of this is how so many people claim that back in the 70's science declared that we were about to enter a new ice age. The reality is that a single group of scientists published a single study that suggested a new ice age was coming and prior to having the results peer reviewed Time Magazine ran a front page article declaring SCIENCE SAYS A NEW ICE AGE IS COMING! and media throughout the world starting saying the same thing. Of course when the study WAS reviewed and found by the scientific community to have significant flaws, the news media didn't bother to mention the correction.

you greatly oversimplify what was going on in climate research at the time, and since.

 

[QuestioningMind]

Some scientific methods have demonstrated themselves to be by far the best methods we've ever found for determining how some parts of the universe works.

For other parts, they're not doing so grand...



When certain scientific disciplines (neuroscience, psychology, etc.) have 30-60% error rates in publication that is often passing peer-review, blaming the problems on individual scientists and the media seems a bit misguided.

In many areas, the problems are systemic and result from what is considered 'scientific best practice'.

Some scientific methods have demonstrated themselves to be by far the best methods we've ever found for determining how some parts of the universe works.

There is only ONE scientific method. Some scientists may employ it better than others, but there remains just the one scientific method.

When certain scientific disciplines (neuroscience, psychology, etc.) have 30-60% error rates in publication that is often passing peer-review, blaming the problems on individual scientists and the media seems a bit misguided.

Who do you think does the peer review, if not individual scientists? It is individual scientists who are passing these studies that you claim have large error rates, not the scientific method. And how would you know that there is a 30-60% error rate, unless other scientist who properly used the scientific method hadn't evaluated the evidence in order to reach that conclusion?