The Knowledge Factory Crisis: A different, anthropological way to view universities

Nothing we humans do lives up to its own mythology. We are fallible, social, competitive, acquisitive, our understanding is incomplete, and we have competing interests to address, in our lives and as a society.  I posted yesterday about universities as 'knowledge factories, reacting to a BBC radio program that discussed what is happening in universities, when research findings seem unrepeatable.

That program, and my discussion of what is going on at universities, took the generally expressed view of what universities are supposed to be, and examined how that is working.  The discussion concerned technical aspects that related to the nature of scientific information universities address or develop.  That is, in this context, their 'purpose' for being.  How well do they live up to what they are 'supposed' to be?

Many of my points in the post were about the nature of faculty jobs are these days, and the way in which pressures lead to the over-claiming of findings, and so on.  I made some suggestions that, in principle, could help science live up to its ideal.

Here in this post, however, I want to challenge what I have said about this.  Instead, I want to take a somewhat distanced viewpoint, looking at universities from the outside, in a standard kind of viewpoint that anthropologists take, rather than simply accepting universities' own assessments of what they are about.

Doing poorly by their ideal standard
My post noted ways in which universities have become not just a 'knowledge factory', but more crass business factories, as making money blatantly increasingly over-rides their legitimate--or at least, stated--role as idea and talent engines for society.  Here's a story from a few years ago about that, that is still cogent.  The fiscal pursuit discussed in this post is part of the phenomenon.  As universities are run more and more as businesses, which happens even in state universities, they become more exclusive, belying their original objective which (as in the land-grant public universities) was to make higher education available to everyone.  In addition to becoming money makers themselves, academia has become a boon for student-loan bankers, too.

But this is a criticism of university-based science, and expressed as it relates to how universities are structured.  That structure, even in science, leads to problems of science.  One might think that something so fundamentally wrong would be easy to see and to correct.  But perhaps not, because universities are not isolated from society--they are of society, and therein lies some deep truth.

Excelling hugely as viewed anthropologically
If you stop examining how universities compare to their ideals, or to what most people would tell you universities were for, and instead look at them as parts of society, a rather different picture emerges.

Universities are a huge economic engine of society.  They garner their very large incomes from various sources: visitors to their football and basketball stadiums, students whose borrowed money pays tuition, and agencies private and public that pour in money for research.  Whether or not they are living up to some ideal function or nature, they are a major and rather independent part of our economy.

Their employees, from their wildly paid presidents, down to the building custodians, span every segment of society.  The money universities garner pays their salaries, and buys all sorts of things on the open commercial economy, thereby keeping many other people gainfully employed.  Their activities (such as the major breakthrough discoveries they announce almost daily) generate material and hence income for the media industries, print and electronic, which in turn helps feed those industries and their relevant commercial influences (such as customers, television sales, and more).

Human society is a collective way for we human organisms to extract our living from Nature.  We compete as individuals in doing this, and that leads to hierarchies.  Overall, over time, societies have evolved such that these structures extract ever more resources and energy.  Via various cultural ideologies we are able to keep things going smoothly enough, at least internally, so as not to disrupt this extractive activity.

Religion, ownership hierarchies, imperialism, military, and other groups have self-justifications that make people feel they belong.  This contributes to building pyramids--whether they be literal, or figurative such as religions, universities, armies, political entities, social classes, or companies.  Often the justification is religious--nobility by divine right, conquest as manifest destiny, and so on.  That not one of these resulting societal structures lives up to its own ideology has long been noted.  Why should we expect universities to be any different?  These are the cultural ways people organize themselves to extract resources for themselves.

Universities are parasites on society, very hierarchical with obscenely overpaid nobles at the top?  They show no limits on the trephining they do on those who depend on them, such as graduating students with life-burdening debt?  They churn through those who come to them for whom they claim to 'provide' the good things in life?  Of course!  Like it or not, by promising membership and a better life, they are just like religions or political classes or corporations!

Institutions may be so caught up in their belief systems that they don't adapt to the times or competitors, or they may change their actions (if not always their self-description).  If they don't adapt they eventually crumble and are replaced by new entities with new justifications to gain popular appeal or acceptance.  However, fear not, because relative to their actual (as opposed to symbolic) role in societies, universities are doing very well: at present, they very clearly show their adaptability.

In this anthropological sense, universities are doing exceedingly well, far better than ever before, churning resources and money over far faster than ever before.  Grumps (like us) may point out the failings of lacking to live up to our own purported principles--but how is that different from any other engine of society?

In that anthropological sense, whether educating people 'properly' or not, whether claiming more discoveries that stand up to scrutiny, universities are doing very, very, very well.  And that, not the purported reason that an institution exists, is the measure of how and why societal institutions persist or expand.  Hypocrisy and self-justification, or even self-mythology, are always part of social organization. A long-standing anthropological technique for understanding distinguishes what are called emics, from etics: what people say they do, from what they actually do.

Yes, there will have to be some shrinkage with demographic changes, and fewer students attending college, but that doesn't change the fact that, by material measures, universities are incredibly successful parts of society.

What about the intended material aspect of the knowledge factory--knowledge?
But there is another important side to all of this, which takes us back to science itself, which I think is actually important, even if it is naive or pointless to crab at the hypocrisies of science that are explicable in deep societal terms.

This has to do with knowledge itself, and with science on its own terms and goals.  It relates to what could, at least in principle, advance the science itself (assuming such changes could happen without first threatening science's and scientists' and universities' assets).  That will be the subject of our next post.

The 'knowledge factory'

This post reflects much that is in the science news, in particular our current culture's romance with data (or, to be more market-savvy about it, Big Data).  I was led to write this after listening to a BBC Radio program, The Inquiry, an ongoing series of discussions of current topics.  This particular episode is titled Is The Knowledge Factory Broken?

Replicability: a problem and a symptom
The answer is pretty clearly yes.  One of the clearest bits of evidence is the now widespread recognition that too many scientific results, even those published in 'major' journals, are not replicable.  When even the same lab tries to reproduce previous results, they often fail.  The biggest recent noise on this has been in the social, psychological, and biomedical sciences, but The Inquiry suggests that chemistry and physics also have this problem.  If this is true, the bottom line is that we really do have a general problem!

But what is the nature of the problem?  If the world out there actually exists and is the result of physical properties of Nature, then properly done studies that aim to describe that world should mostly be replicable.  I say 'mostly' because measurement and other wholly innocent errors may lead to some false conclusion.  Surprise findings that are the luck of the draw, just innocent flukes, draw headlines and are selectively accepted by the top journals.  Properly applied, statistical methods are designed to account for these sorts of things.  Even then, in what is very well known as the 'winner's curse', there will always be flukes that survive the test, are touted by the major journals, but pass into history unrepeated (and often unrepentant).

This, however, is just the tip of the bad-luck iceberg.  Non-reproducibility is so much more widespread that what we face is more a symptom of underlying issues in the nature of the scientific enterprise itself today than an easily fixable problem.  The best fix is to own up to the underlying problem, and address it.

Is it rats, or scientists who are in the treadmill?
Scientists today are in a rat-race, self-developed and self-driven, out of insatiability for resources, ever-newer technology, faculty salaries, hungry universities....and this system can be arguably said to inhibit better ideas.  One can liken the problem to the famous skit in a candy factory, on the old TV show I Love Lucy.  That is how it feels to many of those in academic science today.

This Inquiry episode about the broken knowledge factory tells it like it is....almost.  Despite concluding that science is "sending careers down research dead-ends, wasting talent and massive resources, misleading all of us", in my view, this is not critical enough.  The program suggests what I think are plain-vanilla, clearly manipulable 'solutions.  They suggest researchers should post their actual data and computer program code in public view so their claims could be scrutinized, that researchers should have better statistical training, and that we should stop publishing just flashy findings.  In my view, this doesn't stress the root and branch reform of the research system that is really necessary.

Indeed, some of this is being done already.  But the deeper practical realities are that scientific reports are typically very densely detailed, investigators can make weaknesses hard to spot (this can be done inadvertently, or sometimes intentionally as authors try to make their findings dramatically worthy of a major journal--and here I'm not referring to the relatively rare actual fraud).

A deeper reality is that everyone is far too busy on what amounts to a research treadmill. The tsunami of papers and their online supporting documentation is far too overwhelming, and other investigators, including readers, reviewers and even co-authors are far too busy with their own research to give adequate scrutiny to work they review. The reality is that open-publishing of raw data and computer code etc. will not generally be very useful, given the extent of the problem.

Science, like any system, will always be imperfect because it's run by us fallible humans.  But things can be reformed, at least, by clearing the money and job-security incentives out of the system--really digging out what the problem is.  How we can support research better, to get better research, when it certainly requires resources, is not so simple, but is what should be addressed, and seriously.

We've made some of these points before, but with apology, they really do bear stressing and repeating.  Appropriate measures should include:

     (1) Stop paying faculty salaries on grants (have the universities who employ them, pay them);

     (2) Stop using manipulable score- or impact-factor counting of papers or other counting-based items to evaluate faculty performance, and try instead to evaluate work in terms of better measures of quality rather than quantity;

     (3) Stop evaluators considering grants secured when evaluating faculty members;

     (4) Place limits on money, numbers of projects, students or post-docs, and even a seniority cap, for any individual investigator;

     (5) Reduce university overhead costs, including the bevy of administrators, to reduce the incentive for securing grants by any means;

     (6) Hold researchers seriously accountable, in some way, for their published work in terms of its reproducibility or claims made for its 'transformative' nature.

     (7) Grants should be smaller in amount, but more numerous (helping more investigators) and for longer terms, so one doesn't have to start scrambling for the next grant just after having received the current one.

     (8) Every faculty position whose responsibilities include research should come with at least adequate baseline working funds, not limited to start-up funds.

     (9)  Faculty should be rewarded for doing good research that does not require external funding but does address an important problem.

     (10)  Reduce the number of graduate students, at least until the overpopulation ebbs as people retire, or, at least, remove such number-counts from faculty performance evaluation.

Well, these are snarky perhaps and repetitive bleats.  But real reform, beyond symbolic band-aids, is never easy, because so many people's lives depend on the system, one we've been building over more than a half-century to what it is today (some authors saw this coming decades ago and wrote with warnings). It can't be changed overnight, but it can be changed, and it can be done humanely.

The Inquiry program reflects things now more often being openly acknowledged. Collectively, we can work to form a more cooperative, substantial world of science.  I think we all know what the problems are.  The public deserves better.  We deserve better!

PS.  P.S.:  In a next post, I'll consider a more 'anthropological' way of viewing what is happening to our purported 'knowledge factory'.

Even deeper, in regard to the science itself, and underlying many of these issues are aspects of the modes of thought and the tools of inference in science.  These have to do with fundamental epistemological issues, and the very basic assumptions of scientific reasoning.  They involve ideas about whether the universe is actually universal, or is parametric, or its phenomena replicable.  We've discussed aspects of these many times, but will add some relevant thoughts in the near future.