Monday, July 24, 2017

Supporting the Redefinition of Statistical Significance

Recently an article entitled `Redefining Statistical Significance' (RSS) has been made available. In this piece a diverse bunch of authors (including four philosophers of science - represent) put forward an argument with the thesis: ``[f]or fields where the threshold for defining statistical significance for new discoveries is P<0.05, we propose a change to P<0.005.'' In this very brief note I just want to state my support for the broad principle behind this proposal and make explicit an aspect of their reasoning that is hinted at in RSS but which I think is especially worth holding clear in our minds.

RSS argues that, basically, rejecting the null at P<0.05 represents (by Bayesian standards) very weak evidence against the null and in favour of the hypothesis under test, and further than its communal acceptance as the standard significance level for discovery predictably and actually leads to unacceptably many false-positive discoveries. P<0.005 taken as the norm would go some way towards solving both these problems, and the authors emphasise most especially that it would bring false positive levels down to within what they deem to be more acceptable levels. RSS doesn't claim originality for these points, and is a short and very readable paper; I recommend checking it out.

The authors then have a section replying to objections. They note that they do not think that changing the significance level communally required for discovery claims is a cure-all, and deploy a number of brief but very interesting arguments against the counter-claim that the losses in terms of false-negatives would outweigh the gains in avoiding false positives. This is all interesting stuff, but the point at which I wish to state my broad agreement comes when they consider the objection that ``The appropriate threshold for statistical significance should be different for different research communities.'' Here their response is to say that they agree in principle that different communities facing different sorts of puzzles ought use different norms for discovery claims, but note that many communities have settled on the idea that given the sort of claims they are considering and tests they can do  P<0.05 is an appropriate standard for discovery claims. They are addressing those communities in particular with their proposal, so are addressing communities which have already come to agree that they should share a standard for discovery claims.

My one small contribution here, then, is in following up on this point. They briefly note in their reply to this objection that -- `it is helpful for consumers of research to have a consistent benchmark.' I think this point deserves elaboration and emphasis, and it is why I feel that, although I do not feel sufficiently expert to comment on the specific proposal they made, the broad contours of their argument are right. Why, after all, do we actually have to agree on a communal standard for what counts as an appropriate significance level for `claims of discovery of new effects' at all? Couldn't we leave that to the discretion of individual researchers? Or maybe foster for some time a diversity of standards across journals and let a kind of Millian intellectual marketplace do its work? To put it philosophically, why have something rather than nothing here?

I take it that a lot of what the communal standard is doing is providing a bench mark for those not able to make expert or highly-informed personal assessment of the claims and evidence to know that the hypothesis in question is confirmed to the standards of those who are able to make expert or highly informed assessments. These consumers of the research are those for whom the consistent benchmark helps. Especially for the kind of social scientific fields which have in fact adopted this benchmark, a pressing methodological consideration has to be that non-scientists or folk not able to assess statistical claims, and more pointedly people with policy or culturally influential positions, will consume the research, and take actions based on what they believe to be reliable, or at least take action on the grounds of what convinces them. The trade off between Type 1 and Type 2 errors, then, must be made with it in mind that there is an audience of non-experts to the claims made in this field, and an audience who will shape actions and lives and self-perceptions (in part) upon the results these fields put out. As a scientific community we must therefore decide what we think of our own work can be vouchsafed to these observers, or validated to the standard this cultural responsibility entails.

In theory, of course, we could still leave this up to individuals or allow for a diversity of standards among journals. But I think awareness of the scientific community's public role tends to speak against that. Such diversity, I'd wager, would either result in a cacophonic public discourse on science in which the media and commentators constantly reported results, then their failure to replicate, and then their replication once more (as well as contrary results, their failure to replicate...). This because the diversity of standards led to non-experts picking who to believe randomly among folk with different standards, or according to who they judged to have the flashiest smile, or whichever university PR department reached out to them last, or factionally choosing their favourite sources. Or, it would result in silence, as gradually scientific results come to be seen as too unreliable, too divided among themselves, to be worth paying much attention to at all. If you think that scientifically acquired information can make a positive difference to public discourse, either of these seem like bad outcomes. (The somewhat self-promoting Du Bois scholar nerd in me can't resist pointing out that Du Bois brought similar considerations to bear in responding to widespread failures of social scientific research in his day.) In fact, I think this epistemic environment makes a conservative attitude sensible, and speak in favour of adopting a very low tolerance for false-positives. This because is much harder to correct misinformation once it is out there than it is to defer announcing until we are more confident, and the very act of correction may induce the same loss of trust worry mentioned before. This means that in addition to elaborating upon RSS' reply to an objection, and without feeling competent to quite judge whether P<0.005 in particular is the right standard, I also think the overall direction of change advocated by RSS is the right one, relative to where we are now.

Saturday, July 1, 2017

Decolonise Philosophy!

The following thoughts, prompted by this article, will (I suspect) almost all be super obvious to anybody who has been thinking about decolonising philosophy for an extended period of time. But my audience is largely composed of people, methinks, who do not regularly think about such things.

Lots of people would agree with the slogan ``We ought decolonise philosophy!'' but, philosophy being what it is, the meaning of the slogan is highly contentious. I'll work with one account thereof, based on this and related papers by Kwasi Wiredu, but bear in mind that it's not the only account of what it would take to decolonise philosophy that is out there. I think this particular account makes my point very stark, but something essentially similar to what I say would go if I had worked with some other prominent accounts. Wiredu begins by saying that what it means to decolonise African philosophy would be  ``divesting African philosophical thinking of all undue influences emanating from our colonial past.'' This is then cashed out in terms of taking conscious control of the concepts deployed in philosophical reasoning, as well as the substantive positions covered and the questions asked, by means of subjecting them to critique via cross cultural comparisons. The idea, basically, is to try and ferret out aspects of philosophical thinking now going on in Africa which can't earn their keep on their own merits but rather persist simply because the colonialists imposed them during their occupations -- and to ferret them out by using the fact that indigenous languages, conceptual schemes, and thought traditions have resources that can make incongruences stark by means of comparison, undermine false claims to necessity by evincing in practice alternate ways of going on, or may occupy regions of logical or conceptual space that the colonists never bothered to explore.

So, for instance, Wiredu argues that there are certain puzzles about existence or the nature of capital-b-Being which simply cannot arise if you are to formulate your thoughts in certain West African languages. In the essay linked, for instance, he argues that the notion of creation-ex-nihilo which has caused so much debate in philosophy of religion is nigh-on-incomprehensible if one tries to discuss it in his native Akan language. It is not that he thinks this therefore proves that those questions of Being are pseudo-puzzles, or that creation-ex-nihilo is impossible, but rather simply that it would be a colonial attitude to simply assume that this difference must be due to an expressive fault with the West African languages rather than a tendency to produce misleading linguistic confusions in the European traditions which concentrate on those puzzles and on the basis of nothing more than this assumption work to import the European concept. If it is a genuine improvement on the indigenous conceptual scheme that must be argued for. Further, having realised the incongruity, and not uncritically accepting the Western mode as just obviously superior, one can see whether and how thinking with the concept derived from one's own linguistic tradition would ramify through philosophical issues -- and in this paper he concludes, for instance, that attempts to harmonise or synthesise religions indigenous to Ghana and Christianity are probably not as coherent as some claim, but are relying on equivocation at key moments. In his own work he has applied this method to a number of other problems to interesting effect -- to give some of the provocative examples, he concludes that Descartes' cogito would fairly immediately have been seen to be an invalid argument had Descrates attempted to formulate it in a West African language, or on other occasions that the correspondence theory of truth is a tautology in an Akan language.

The point, then, is not simply to reject everything associated with the colonialists. (As he says, the emphasis in his initial definition of decolonising philosophy should be on the word `undue' before `influences'.) Rather, the point is to ensure that the tools we think with are up to task, and to use the availability of alternative tools as a means of facilitating test and comparison. So whether or not you one ultimately ends up accepting the problems-in-Western-languages as genuine or pseudo-problems, the decolonised philosophy is that which has used the conceptual resources and intellectual traditions of the former colonised nation to put itself in a position to consciously decide whether or not its inherited problems are worth pursuing in light of consideration of a fuller range of facts, rather than uncritically (or without due consideration of the facts adduced by considering the thought of the colonised) accepting the concepts, problems, and solution space given to it by Western tradition.

Before drawing out my intended moral, some comments on Wiredu's account as an account of decolonising philosophy. I think it does a pretty good job of rationalising a lot of what people tend to actually do under the aegis of decolonising the field (I usually see people try and change: (i) what is taught, and (ii) who does the teaching), since it is basically an attempt to leverage cognitive diversity in a way that tends to align with the various reform efforts now going on. Wiredu would, I think, also be of the opinion that this is a contribution to the broader project of decolonisation -- since the historic task of former colonies at this moment is to deal with the legacy of colonialism by taking the reigns of history and no longer simply having Western modes of life and government imposed, but rather consciously weighing the colonists mode of life against the indigenous tradition and attempting to forge a synthesis that allows for the best of both as far as is possible. That is to say, Wiredu's account of what African nations should be up to during periods of post-colonial modernisation looks a lot like his account of what African philosophy should be up to. He might therefore think that each can reinforce the other. If you do not agree with Wiredu on what broader cultural and political decolonisation means, I think one could reasonably fault this as failing to properly contribute to the broader decolonial project. I am not sure what I think the broader decolonial project will or should amount to, so I am agnostic on this point.

Ok here's the thing that Wiredu's account makes especially for me: I have never seen an account of decolonising philosophy that does not make it seem like it is just a generally desirable thing to do. I can understand why it is of especially pressing importance in departments in former colonies. But the thing Wiredu described just sounds like a corollary of enlightenment, assuming you don't a priori limit the capacity for interesting thought or concept creation to Westerners. (This would have to be a corollary of some version of the enlightenment that did not share the patronising assumptions of many actors within the actual historical enlightenment! Enlightenment itself is, of course, famously a concept subject to much critique.) Everyone, citizen or descendent of former (or presently) colonised nation or not, should want to decolonise philosophy in Wiredu's sense. Could we still claim our mantle as true philosophers if we, as a matter of policy, uncritically made use of our inherited concepts? Can we really vow to just set aside pertinent information about the limits or oversights of our own conceptual scheme, or are we so sure that there is no pertinent information to be drawn from the kind of cross-cultural comparisons which examination of various world traditions makes possible? In addition to Wiredu's aforementioned work, I am reading David Wong's Natural Moralities at the moment, which through its comparative approach between Confucian and western liberal ethics seems to be another proof in practice of the possibility of drawing pertinent information for philosophical puzzles from this -- and, by Wiredu's lights, is an instance of decolonising philosophy.

The only people I see talk about decolonising philosophy tend to be people from former colonies or right-on-lefties in the West. But when I read accounts of what decolonising philosophy would amount to it seems like anybody committed to the enlightenment ideals held by most philosophers should likewise find themselves engaged in full sympathy with this activity.