# The Post-Truth Society is a Failure to Teach Probability

Many commentators have suggested that recent unexpected events, like the continuing support of US voters for Donald Trump, or victory of the UK Brexit campaign, should be attributed to a new era of human reasoning, where truth is no longer important. Ironically, the real truth is much more subtle, and considerably more damning of the very media who peddle the post-truth myth.

Probabilistically, we may quantify our trust in expert opinion by measuring the chances that some event happens, given that experts have predicted it. Mathematically we write

$\displaystyle P(\textrm{trust}) = P(A \textrm{ happens } | \textrm{ experts predict } A)$

Now the media advocate that we can estimate this directly. Just think about all the times that experts have made a prediction, and work out how often they got it right. Although this is mathematically correct, it isn’t of much practical use. After all, who wanders round reciting all the predictions that experts have ever made? The times we tend to remember predictions are when they were wrong!

This problem is inherent in the traditional frequentist interpretation of probability, taught in schools. This says that probabilities can be calculated empirically by repeating an experiment many times, and looking at the outcomes. While this works fine for tossing coins, it doesn’t intuitively extend to more complicated situations.

So when the media (and to some extent experts themselves) encourage people to make the estimate directly, we’re bound to make mistakes. In particular, in many cases where we remember predictions, they were exactly wrong! This artificially bumps up the importance of outlier events like the financial crash of 2010. The result is that we put a very low value on our trust in the experts.

Fortunately, there’s a better way of looking at the problem, which is sadly unknown to the vast majority of people. It goes by the name of Bayes’ Theorem. Despite the technical name, it represents a far easier way to quantify uncertainty. What’s more, it lies closer to our intuition that probability should measure how much we believe in something.

Put simply Bayes’ theorem says that our belief that $A$ will happen, given supporting evidence from experts, is the chances that experts predicted $A$ given that it happens, multiplied by our prior belief in $A$. (Strictly speaking, there’s another factor involved, but we won’t need it). In maths we can write

$\displaystyle P(\textrm{trust}) \propto P(\textrm{experts predicted } A \ | \ A \textrm{ happens}) \times P(A \textrm{ happens})$

This makes sense – we should update our prior expectations, based on the likelihood that $A$ happens. Of course, some people are so entrenched in their view as to never change their minds. But there are often many “floating voters” who might adopt just such a strategy in forming an opinion, if encouraged to do so! The undecided segment of the population played an important role in the Brexit vote, and may yet become a crucial factor in the US election.

So what’s the advantage of this formula over the frequentist interpretation? Because it sits closer to our intuition, it’s much easier for us to make a good guess. The first term in the product asks us to consider all the times that events have happened, and determine the proportion of the time that experts correctly predicted them. Notice that this reverses the perspective we had in the original direct formula!

This change of viewpoint is great news for our intuition. Most of us can think of various economic or political events that have happened over our lifetime. And while some of these were unexpected (like the financial crash), many others were not – witness the long periods of economic stability and the foregone conclusions of many elections over the past 20 years. So we are likely to say that when an event happens, experts have often predicted it first.

Let’s look at this mathematically. Rather than calculating the probability of trust, which is a slightly wishy-washy concept, it’s better to compare our trust and our doubt. We do this automatically in everyday life, as a sanity check. If we have niggling doubts, it’s probably because we’ve overestimated our trust, and vice versa. In equation form we’ll determine

$\displaystyle \textrm{trust factor} = \frac{ P(\textrm{trust}) }{ P (\textrm{doubt}) }$

If this is (say) bigger than $3$, we should be happy to trust the experts, since it far outweighs any doubts we might have. Let’s suppose that prior to the expert opinion, we have no strong view as to whether $A$ will happen or not. In other words

$\displaystyle P(A \textrm{ happens}) = P(A \textrm{ doesn't happen}) = 0.5$

Then using Bayes we find that our trust factor is a so-called Bayes factor, namely

$\displaystyle \textrm{trust factor} = \frac{P(\textrm{experts predicted } A \ | \ A \textrm{ happens})}{P(\textrm{experts predicted } A \ | \ A \textrm{ doesn't happen})}$

We’ve already argued that the term is the numerator is plausibly large. It is also sensible to think that the term in the denominator is relatively small. We’d all agree that major events not happening is rather common. And of all the events that don’t happen, experts don’t often tend to say they will. Of course, there are some doomsayers who are frequently forecasting disaster, but they’re mostly on the periphery of expert opinion.

So if the numerator is large and the denominator is small, we can conclude that our trust factor is quite large indeed. It’s not unreasonable to suspect it’ll be at least greater than $3$. With the right intuitive tools, we’ve arrived at a more reasonable level of trust in our experts. Sadly, such arguments are few and far between in a media hell-bent on a “keep it simple, stupid” attitude, and expert spokespeople convinced that “dumbing down” is the only way to communicate. Ironically, this is more alienating and less intuitive than Bayesian logic!

The post-truth society is a myth, created by a media who themselves are confused about probability. More accurately, we are currently living in a pre-inference society. Bayesian estimation has never been adequately taught in schools, and for many years this was no great loss. But in the modern world, with the ubiquitous availability of data, it is imperative that we provide people with adequate logical tools for inferring reasonable decisions.

P.S. For balance, here’s why Bayes isn’t always best!

# Collabor8 – A New Type of Conference

Next month, I’m running a day-long conference here at QMUL. The meeting is intended to give early career researchers the chance to seek possible collaborations. Despite living in this globalised age, all too often PhD students and postdocs are restricted to working with faculty members in their current institution. This is no surprise – at the conferences and meetings where networking opportunities arise, we’re usually talking about completed work, rather than discussing new problems.

We’re shaking up the status quo by asking our participants to speak about ongoing research, and in particular to outline roadblocks where they need input from theorists with different expertise. What’s more, we’re throwing together random teams for speed collaboration sessions on the issues presented, getting the ball rolling for possible acknowledgements and group projects. We’re extremely fortunate to have the inspirational Fernando Alday as our guest speaker, a serial collaborator himself.

The final novelty of this conference comes in digital form. The conference website doubles as a social network, making it easy to keep track of your connections and maintain interactions after the meeting. We hope to generate good content on the site during the day, where some participants will be invited to act as scribes and note down any interesting ideas that arise. This way, there’ll be a valuable and evolving database of ideas ready for future collaborations to draw on.

Over to you! If you’re doing a PhD or a postdoc in the UK, or you know someone who is, send them a link to the website

http://www.collabor8research.org

If you’re further afield, feel free to follow developments from afar. In the long term we’re hoping to roll out the social network to other conferences and institutions – watch this space!

If you know someone who works in academia, chances are they’ve told you that research takes time. A lot of time, that is. But does it have to?

It’s 3:15pm on an overcast Wednesday afternoon. A group of PhD students, postdocs and senior academics sit down to discuss their latest paper. They’re “just finishing” the write-up, almost ready to submit to a journal. This phase has already been in motion for months, and could well take another week or two.

Of course, this sounds monumentally inefficient to corporate ears. In a world where time is money, three months of tweaking and wrangling could not be tolerated. So why is it acceptable in academia? Because nobody is in charge! Many universities suffer from a middle management leadership vacuum; the combined result of lack of training and unwise promotions.

It is ironic that renowned bastions of learning have fallen so far behind their industrial counterparts when it comes to research efficiency. When you consider that lecturers need no teacher training, supervisors no management expertise, and interviewers no subconscious bias training, the problem becomes less surprising. No wonder academia is leading the way on gender inequality.

The solution – a cultural shake-up. Universities must offer more teaching-only posts, breaking the vicious cycle which sees disgruntled researchers forced to lecture badly, and excellent teachers forced out from lack of research output. Senior management should mandate leadership training for group leaders and supervisors, empowering them to manage effectively and motivate their students. Doctoral candidates, for that matter, might also benefit from a course in followership, the latest business fashion. Perhaps most importantly, higher education needs to stop hiring, firing and promoting based purely on research brilliance, with no regard for leadership, teamwork and communication skills.

Conveniently, higher education has ready made role-models in industrial research organisations. Bell Labs is a good example. Not long ago this once famous institution was in the doldrums, even forced to shut down for a period. But under the inspirational leadership of Marcus Weldon, the laboratory is undergoing a renaissance. Undoubtedly much of this progress is built on Marcus’ clear strategic goals and emphasis on well-organised collaboration.

Universities might even find inspiration closer to home. Engineering departments worldwide are developing ever-closer ties to industry, with beneficial effects on research culture. From machine learning to aerospace, corporate backing provides not only money but also business sense and career development. These links advantage researchers with some client-facing facets, not the stuffy chalk-covered supermind of yesteryear. That doesn’t mean there isn’t a place for pure research – far from it. But insular positions ought to be exceptional, rather than the norm.

At the Scopus Early Career Researcher Awards a few weeks ago, Elsevier CEO Ron Mobed rightly bemoaned the loss of young research talent from academia. The threefold frustrations of poor job security, hackneyed management and desultory training hung unspoken in the air. If universities, journals and learned societies are serious about tackling this problem they’ll need a revolution. It’s time for the 21st century university. Let’s get down to the business of research.

# (Not) How to Write your First Paper

18 months ago I embarked on a PhD, fresh-faced and enthusiastic. I was confident I could learn enough to understand research at the coal-face. But when faced with the prospect of producing an original paper, I was frankly terrified. How on earth do you turn vague ideas into concrete results?

In retrospect, my naive brain was crying out for an algorithm. Nowadays we’re so trained to jump through examination hoops that the prospect of an open-ended project terrifies many. Here’s the bad news – there’s no well-marked footpath leading from academic interest to completed write-up.

So far, so depressing. Or is it? After all, a PhD is meant to be a voyage of discovery. Sure, if I put you on a mountain-top with no map you’d likely be rather scared. But there’s also something exhilarating about striking out into the unknown. Particularly, that is, if you’re armed with a few orienteering skills.

I’m about to finish my first paper. I can’t and won’t tell you how to write one! Instead, here’s a few items for your kit-bag on the uncharted mountain of research. With these in hand, you’ll be well-placed to forge your own route towards the final draft.

Any good rambler takes a compass. Your supervisor is your primary resource for checking direction. Use them!

It took me several months (and one overheard moan) to start asking for pointers. Nowadays, if I’m completely lost for more than a few hours, I’ll seek guidance.

You start off without a map. It’s tempting to start with unrecorded cursory reconnaissance, just to get the lie of the land. Although initially speedy, you have to be super-disciplined lest you go too far and can’t retrace your steps. You’d be better off making notes as you go. Typesetting languages and subversion repositories can help you keep track of where you are.

Your notes will eventually become your paper – hence their value! But there’s a balance to be struck. It’s tempting to spend many hours on pretty formatting for content which ends up in Appendix J. If in doubt, consult your compass.

Some of them have written papers before. All of them have made research mistakes. Mutual support is almost as important in a PhD programme as on a polar expedition! But remember that research isn’t a race. If your colleague has produced three papers in the time it’s taken you to write one, that probably says more about their subfield and style of work than your relative ability.

4. Confidence

Aka love of failure. If you sit on top of the mountain and never move then you’ll certainly stay away from dangerous edges. But you’ll also never get anywhere! You will fail much more than you succeed during your PhD. Every time you pursue an idea which doesn’t work, you are honing in on the route which will.

In this respect, research is much like sport – positive psychology is vital. Bottling up frustration is typically unhelpful. You’d be much better off channelling that into…

You can’t write a paper 24/7 and stay sane. Thankfully a PhD is full of other activities that provide mental and emotional respite. My most productive days have coincided with seminars, teaching commitments and meetings. You should go to these, especially if you’re feeling bereft of motivation.

And your non-paper pursuits needn’t be limited to the academic sphere. A regular social hobby, like sports, music or debating, can provide a much needed sense of achievement. Many PhDs I know also practice some form of religion, spirituality or meditation. Time especially set aside for mental rest will pay dividends later.

6. Literature

No, I don’t mean related papers in your field (though these are important). I’ve found fiction, particularly that with intrigue and character development, hugely helpful when I’m struggling to cross an impasse. Perhaps surprisingly, some books aimed at startups are also worth a read. A typical startup faces corporate research problems akin to academic difficulties.

Finally, remember that research is by definition iterative! You cannot expect your journey to end within a month. As you chart the territory around you, try to enjoy the freedom of exploring. Who knows, you might just spot a fascinating detour that leads directly to an unexpected paper.

My thanks to Dr. Inger Mewburn and her wonderful Thesis Whisperer blog for inspiring this post.

# Bad Science: Thomson-Reuters Publishes Flawed Ranking of Hottest Research

Thomson-Reuters has reportedly published their yearly analysis of the hottest trends in science research. Increasingly, governments and funding organisations use such documents to identify strategic priorities. So it’s profoundly disturbing that their conclusions are based on shoddy methodology and bad science!

The researchers first split recent papers into 10 broad areas, of which Physics was one. And then the problems began. According to the official document

Research fronts assigned to each of the
10 areas were ranked by total citations and the top 10 percent of the fronts in each area were extracted.

Already the authors have fallen into two fallacies. First, they have failed to normalise for the size of the field. Many fields (like Higgs phenomenology) will necessarily generate large quantities of citations due to their high visibility and current funding. Of course, this doesn’t mean that we’ve cracked naturalness all of a sudden!

Second their analysis is far too coarse-grained. Physics contains many disciplines, with vastly different publication rates and average numbers of citations. Phenomenologists publish swiftly and regularly, while theorists have longer papers with slower turnover. Experimentalists often fall somewhere in the middle. Clearly the Thomson-Reuters methodology favours phenomenology over all else.

But wait, the next paragraph seems to address these concerns. To some extent they “cherry pick” the hottest research fronts to account for these issues. According to the report

Due to the different characteristics and citation behaviors in various disciplines, some fronts are much smaller than others in terms of number of core and citing papers.

Excellent, I hear you say – tell me more! But here comes more bad news. It seems there’s no information on how this cherry picking was done! There’s no mention of experts consulted in each field. No mathematical detail about how vastly different disciplines were fairly compared. Thomson-Reuters have decided that all the reader deserves is a vague placatory paragraph.

And it gets worse. It turns out that the scientific analysis wasn’t performed by a balanced international committee. It was handed off to a single country – China. Who knows, perhaps they were the lowest bidder? Of course, I couldn’t possibly comment. But it seems strange to me to pick a country famed for its grossly flawed approach to scientific funding.

Governments need to fund science based on quality and promise, not merely quantity. Thomson-Reuters simplistic analysis is bad science at its very worst. It seems to offer intelligent information but  in fact is misleading, flawed and ultimately dangerous.

# Science and Faith – The Arts of the Unknown

I spent this morning singing a Sunday service at St. George’s Church in Borough. An odd occupation for a scientist perhaps, especially given the high profile of several atheist researchers! Yet a large number of scientists see no contradiction between faith and science. In fact, my Christian faith is only deepened by my fascination with the natural world.

Picture a scientist. Chances are you’ve already got in your mind a geeky, rational person, calibrating a precise experiment or poring over a dry mathematical formula! As with any stereotype, it has it’s merits. But it misses a vital quality in research – imagination.

To succeed as a scientist, you must be creative above all else. It’s no use just learning experimental techniques or memorising formulae. Every new idea must necessarily start off as a fantasy. Great painters are not merely lauded for their 10,000 hours of practice with a paintbrush. It is their capacity to conceive and relay vivid scenes which ensures their place in history. And so it is with science.

So why are scientists seen as cold and calculating and exact, rather than passionate and original? The problem lies in education. While young children are encouraged to express themselves in Literacy, Numeracy is all too often a trudge through tedious and predictable sums. In “arts” subjects, questions are a magical tool enabling discussion, debate and opinion. In “sciences” they merely distinguish right from wrong.

After 15 years of schooling, no wonder the stereotype is embedded! As a teenager, I very nearly ditched the sciences in favour of subjects where expression was free and original arguments rewarded. I’m eternally thankful to my teachers, parents and bookshelf for convincing me that the curriculum was utterly unrepresentative of real science.

So what’s to be done. For any budding scientists out there, your best bet is to read some books. Not your school textbooks – chances are they are dull as ditchwater and require no creative input at all. I mean books written by real life mathematicians, physicists, biologists… These will give you an insight into the imagination that drives research, the contentious debates and the lively exchanges of ideas.

You might not understand everything, but that’s the whole point – science is about the unknown, just as much as art or faith. It is exactly this point which we must evangelise again and again. Perhaps then fewer people will write negative reviews criticising science for being complex, poetic and beautiful.

As a wider society, we can take action too! We must demand better science teaching from a young age. Curricula should emphasise problem solving over knowledge, ideas over techniques and originality over regurgitation. This is already the mantra for many traditionally “artistic” discplines. It must be the battle cry for scientists also!

A better approach to science would democratize opportunity for the next generation. No longer will the relative creativity of girls be arbitrarily punished – an approach which can only discourage women from entering science in the long run. No longer will there be a tech skills gap threatening to undermine the thriving software industry. The UK has a uniquely privileged scientific pedigree. For future equality, economy and diversity, we must use it.

In the service this morning Fr Jonathan Sedgwick talked of the danger of applying scientific laws to the world at large. The concepts of “cause and effect” and “zero sum games” may well work in vacuo, but they are artificial and burdensome when applied to interpersonal relationships. Quite right – as Christians we must question these human rules, and look for a divine inspiration to guide our lives!

But this is also precisely what we must do as scientists. A good scientist always questions their models, constantly listening for the voice of intuition. For science – like our own existence – is ever changing. And it’s our job to search for the way, the truth and the life.

My thanks to Margaret Widdess, who prepared me for confirmation two years ago at St. Catharine’s College, Cambridge and with whom I first talked deeply about the infinity of science and faith.

# Why does feedback hurt sometimes?

Research is hard. And not for the reasons you might expect! Sure, my daily life involves equations which look impenetrable to the layman. But by the time you’ve spent years studying them, they aren’t so terrifying!

The real difficulty in research is psychological. The natural state for a scientist is failure – most ideas simply do not succeed! Developing the resilience, maturity and sheer bloody mindedness to just keep on plugging away is a vital but tough skill.

This letter, written by an experienced academic to her PhD student is a wonderfully candid account of the minefield of academic criticism, both professional and personal. What’s more, it lays bare some important coping strategies – I certainly wish I’d read it before embarking on my PhD.

Above all, this letter is an admission of humanity. As researchers, we face huge challenges in our careers. But the very personal process of responding to them is precisely what makes us better scientists, and perhaps even improves us as people.

This letter was written by an experienced academic at ANU to her PhD student, who had just presented his research to a review panel and was still licking her wounds.

The student sent it to me and I thought it was a great response I asked the academic in question, and the student who received it, if I could publish it. I wish all of us could have such nuanced and thoughtfu feedback during the PhD. I hope you enjoy it.

A letter to…My PhD student after her upgradeWell you did it. You got your upgrade. But from the look on your face I could tell you thought it was a hollow victory. The professors did their job and put the boot in. I remember seeing that look in the mirror after my own viva. Why does a win in academia always have the sting of defeat?

Yeah, it’s a…

View original post 1,120 more words

# Bibliographies and The arXiv

I’m currently writing up my first paper! The hope is that my collaborators and I will release the paper in the next couple of months. When we do, it’ll go on the arXiv – a publically accessible preprint server.

This open-access policy is adopted pretty much universally throughout mathematics and theoretical physics. I think it’s extremely good for science to be freely accessible to all. There’s still a place for journals, allowing research to be ranked by quality and rigorously peer reviewed. But the arXiv is vital in maintaining the pace of research, particularly in hot topic areas.

Every piece of work on the arXiv gets its own unique identifier. I rather like codes, so I tend to remember these numbers for my favourite papers. Just typing the number into Google search immediately takes you to the relevant document.

My current paper draft is peppered with arXiv numbers referring to important papers we need to cite. When we come to making a bibliography I’ll need to convert these into a standard form. Technically this involves making a BibTex file, and referring to it in my typesetting program.

I thought this would take ages, but it turns out that there’s an online Easter Egg solving the problem in a flash. Inspire HEP is a database of physics papers, providing all the metadata you could ever need including ready formatted BibTex. And it even has a feature which automatically generates a bibliography for you – check it out!

If you’re writing up your first paper and this tip helped you out, do drop me a line in the comments! And to the curators of arXiv and Inspire HEP – a huge thank you from me and the whole physics community.

# Why Should Undergraduates Attend Classes?

I’ve just finished teaching classes for the Quantum Mechanics B course here at Queen Mary. It’s been an enjoyable few weeks, watching the students grapple with bra-ket notation, spherical harmonics and the Stern-Gerlach experiment. All in all, I’ve found it rather rewarding.

A perennial bugbear is that many undergraduates don’t turn up for the classes. Although nominally compulsory, the university rarely imposes any sanctions for lack of attendance. It’s natural to wonder whether there’s any benefit in running the sessions!

I decided to do some elementary analysis to determine any correlation between class attendance and performance. Fortunately, the results were favourable – students who come to class tend to do better than those who don’t. And what’s more, the gap widens as the term goes on!

I’d like to conclude that this effect is due to the usefulness of my teaching, of course. But my scientific brain doesn’t allow such an easy deduction. After all, correlation does not imply causation! To say anything more we’d need a control study, which is unlikely to happen any time soon!

Still, I can now tell my undergraduate students that they’re more likely to succeed if they come to class. Flawed logic aside, surely that means I’m doing something right?

# Elsevier Journals – The QMUL Figure

A few weeks ago I reblogged Tim Gowers’ post about the cost of Elsevier journals. I noticed that my own institution (QMUL) had deflected his Freedom of Information request. Curious to learn more, I did some digging.

It turns out that QMUL paid a total of £454,422.44 to Elsevier for the academic year 2013/14. Interestingly this is more than Exeter and York, who also joined the Russell Group recently. However it’s still much cheaper than the bill Cambridge, UCL, Imperial or Edinburgh face.

Unfortunately QMUL weren’t able to provide any further breakdown of the figures. Apparently this information isn’t available to the university, which seems like a very odd way of doing business. I think it likely that the vast majority of the cost is the subscription fee.

I should point out that QMUL and Cambridge certainly have differentiated access to Elsevier journals. For example QMUL Library does not have access to Science Direct papers before the early 1990s. Cambridge University Library has universal access to this material.

However with all the smoke and mirrors in this story, it’s impossible to turn this anecdotal evidence into an accurate account of Elsevier’s charging policy. There’s clearly a need for much greater transparency.

Below is a transcript of the email I received from the QMUL FOI Department. I owe a debt of gratitude for their help.

Dear Edward Hughes

Thank you for your email of 25th April requesting information about spend on Elsevier journals at Queen Mary University of London.

The total amount paid to Elsevier for 2013/14 was £545,306.93 (inclusive of £90,884.49 VAT). We do not hold any further break down.

If you are dissatisfied with this response, you may ask QMUL to conduct a review of this decision.  To do this, please contact the College in writing (including by fax, letter or email), describe the original request, explain your grounds for dissatisfaction, and include an address for correspondence.  You have 40 working days from receipt of this communication to submit a review request.  When the review process has been completed, if you are still dissatisfied, you may ask the Information Commissioner to intervene. Please see www.ico.org.uk for details.

Yours sincerely

Paul Smallcombe

Records & Information Compliance Manager