Category Archives: Bad science


Nearly every example of faulty reasoning that has been published is accompanied by the phrase `of course’ or its equivalent.

Donald Knuth


Tagged ,

New paper: “Top Tips to Make Your Research Irreproducible”

It is an unfortunate convention of science that research should pretend to be reproducible; we have noticed (and contributed to) a number of manifestos, guides and top tips on how to make research reproducible, but we have seen very little published on how to make research irreproducible.

Irreproducibility is the default setting for all of science, and irreproducible research is particularly common across the computational sciences (for example, here and here). The study of making your work irreproducible without reviewers complaining is a much neglected area; we feel therefore that by encapsulating our top tips on irreproducibility, we will be filling a much-needed gap in the domain literature. By following our tips, you can ensure that if your work is wrong, nobody will be able to check it; if it is correct, you can make everyone else do disproportionately more work than you to build upon it. Our top tips will also help you salve the conscience of certain reviewers still bound by the fussy conventionality of reproducibility, enabling them to enthusiastically recommend acceptance of your irreproducible work. In either case you are the beneficiary.

  1. Think “Big Picture”. People are interested in the science, not the experimental setup, so don’t describe it.
  2. Be abstract. Pseudo-code is a great way of communicating ideas quickly and clearly while giving readers no chance to understand the subtle implementation details that actually make it work.
  3. Short and sweet. Any limitations of your methods or proofs will be obvious to the careful reader, so there is no need to waste space on making them explicit.
  4. The deficit model. You’re the expert in the domain, only you can define what algorithms and data to run experiments with.
  5. Don’t share. Doing so only makes it easier for other people to scoop your research ideas, understand how your code actually works instead of why you say it does, or worst of all to understand that your code doesn’t work at all.

Read the full version of our high-impact paper on arXiv.

Tagged , , , ,

Solution: Create unhackable systems

Needless to say, this tweet prompted a number of subtle (and not so subtle) responses; it is just vague enough to not be 100% sure he is actually joking (because the software verification problem is trivial, right?).

Did North Korea hack Sony? I doubt it; perhaps it was from an unexpected agent.

N.B. high-profile cosmologists appear to be quite happy to make bold statements to the media on issues well outside of their expertise…

Tagged , , , , ,

A rational animal

Man is a rational animal — so at least I have been told. Throughout a long life, I have looked diligently for evidence in favour of this statement, but so far I have not had the good fortune to come across it, though I have searched in many countries spread over three continents.

Unpopular Essays (1950)
Bertrand Russell (1872-1970)


Tagged , ,

Simon Jenkins on computer science

In a polemic in The Guardian today, Simon Jenkins argues for a(nother) shake up of the UK’s education system, with less focus on STEM and computer science in particular.

This kind of misinformed ranting on the utilitarian view of STEM and why the UK should focus on being a service industry appears to be his CiF modus operandi — see a similar post from February on mathematics education. In particular, he displays a profound misunderstanding of the difference between digital skills/competencies and the rigorous academic discipline of computer science, as well as a lack of awareness of the profound changes to computing education in England from September for all pupils from age five onwards. He also doesn’t appear to be aware of the increasing demands from pretty much every industrial sector for high-value digital skills (both user and creator skills); see the recently published interim report from the UK Digital Skills Taskforce: Digital Skills for Tomorrow’s World. As for the perceived high unemployment rates for computer science graduates? Well, this isn’t the full picture and is also discussed in detail in the Taskforce report.

While it is tempting to deconstruct and refute his article line by line, I will just link to an excellent response from Chris Mairs, Chief Scientists at Metaswitch Networks and Chair of the UK Forum for Computing Education.

Tagged , , , ,

Department of Dirty

Ever thought the Internet was just too big? Want to help clean up online filth? Welcome to the Department of Dirty. Watch the Department stop one man try to get one over us with his ‘spotted dick’ recipe. The Department of Dirty is working with Internet and mobile companies to stop the dirty Internet. We need you to help us take a stand against blogs, charities and education websites, many of which are thankfully blocked by filters.

(more seriously, please support the work of the Open Rights Group, which exists to preserve and promote your rights in the digital age)

Tagged , , , , ,

Send us your reckons

After moaning about the use of uninformed vox pops on this morning’s BBC Breakfast on banning the use of calculators in maths tests in England, I was reminded of this excellent That Mitchell and Webb Look sketch:

Tagged ,

Simon Jenkins on mathematics education

There has been much discussion online of yesterday’s CiF article by Simon Jenkins (For Britain’s pupils, maths is even more pointless than Latin). Click-bait aside, he has been here before; ignoring the derivation of the now-pervasive “x is the new Latin” meme, as well as overlooking the majority of the straw men and other logic fallacies, the main thrust of the article presents a false dichotomy. It appears to reiterate an antiquated Two Cultures-type of divide between mathematics and “creativity and social and emotional capacities” (which also frequently crops up in discussions on programming and computer science education). Furthermore, it implies the drive to reform mathematics education in the UK is ultimately misguided, with few jobs requiring advanced mathematical skills (STEM agenda? No thank you!), and we would be better served by focusing on numeracy as well as encouraging “key industries”:

If British schools are to be slaves to Gove’s economic dogma, they should be turning out accountants, lawyers, administrators and salespeople. That is where the money is. Britain needs literate and presentable young people, sensitive to culture and the world around them, skilled in health, entertainment, finance, the law and citizenship. The truth is that Gove, like most of Cameron’s ministers, is an old socialist planner at heart.

Now, this is not to say that there are no issues with mathematics education in the UK; ACME has been arguing for a mathematics curriculum fit for the 21st century, supported by Ofsted and reports highlighting the importance of mathematics in the other sciences. Conrad Wolfram has long maintained we have the wrong focus in how we teach mathematics — in a similar way for computer science, contexts and problems must come first. I have long maintained it is socially acceptable to be bad at mathematics — it is rare for people to publicly admit they are unable to read or write, but happily proclaim a lifelong inability to perform basic calculations.

Jenkins has thus thrown together a ragbag of prejudices (a love of the arts, a dislike of international education markers, a sympathy for progressive education) with personal anecdote and concocted an argument completely detached from reality. As epitomised by this quote:

I learned maths. I found it tough and enjoyable. Algebra, trigonometry, differential calculus, logarithms and primes held no mystery, but they were even more pointless than Latin and Greek. Only a handful of my contemporaries went on to use maths afterwards.

…which reminds me of this xkcd comic:

Tagged , , ,

The authority of reason

To argue with a man who has renounced the use and authority of reason, and whose philosophy consists in holding humanity in contempt, is like administering medicine to the dead, or endeavoring to convert an atheist by scripture.

The American Crisis, No. V (1778)
Thomas Paine (1737-1809)

Tagged , , ,

Interview for ITV Wales on effects of computer games

(N.B. due to the privacy settings for this Vimeo clip, you will have to view the video on their website)

Yesterday I was interviewed on Newsweek Wales, ITV Wales’ weekly news summary programme, on the perceived dangers of children playing computer games. This was in response to an ITV Wales News story from a few days before, in which a headteacher from a primary school near Caerphilly had felt he had identified a possible link between violent video games and aggressive behaviour; this story was further contextualised by a nine year old boy from Neath who had written to Prime Minister about his concerns over the availability of age-appropriate computer games.

This rather anecdotal declaration of a causal link between playing computer games (an activity enjoyed by the majority of the population) and increased aggression and violence is frustrating; furthermore, this type of story appears to pop every so often, but is not backed by the evidence base: see here and here, with summaries here and here. As I mentioned in the interview, the demographics of people who play computer games can be surprising, especially average age (over 30) and the gender split (55% male/45% female). While I take the point from the Neath pupil about the availability (and attraction) of age-appropriate computer games, it is interesting to list the top five best-selling computer games of all time (across all platforms):

Ranking Title Release Year Systems Copies Sold
1. Wii Sports 2006 Wii 82 million
2. Super Mario Bros. 1985 NES 40 million
3. Minecraft 2009 Various 36 million
4. Mario Kart Wii 2008 Wii 35 million
5. Tetris 2008 GameBoy/GameBoy Color 35 million

In summary: let’s stick to the evidence and not confuse societal or educational issues as technology problems. Minecraft is a great example of how powerful computer games can be: not only is it incredibly popular, it is also a great resource for education, developing digital literacies, communication skills and basic programming (aside: Ordnance Survey recently released a 22 billion block Minecraft map of the UK as an open data resource).

Tagged , , , ,

The perils of chess

Chess is a mere amusement of a very inferior character, which robs the mind of valuable time that might be devoted to nobler acquirements, while it affords no benefit whatever to the body. Chess has acquired a high reputation as being a means to discipline the mind, but persons engaged in sedentary occupations should never practice this cheerless game; they require out-door exercises — not this sort of mental gladiatorship.

Scientific American, July 1859

Tagged , , , ,

Legal disjunction

A fine example of the problems of grammatically vague statements:

In reproducing the obscure wording of treaty obligations in the Oil in Navigable Waters Act 1955, s 1 of the Act said that if oil were unlawfully discharged from a British ship ‘the owner or master’ of the ship would be guilty of an offence. In Federal Steam Navigation Co v Department of Trade and Industry [1974] 1 WLR 505, both the owner and the master of a ship were convicted. In dismissing their appeals, the House of Lords split three to two.

This interpretation of inclusive disjunction is an example of the golden rule, a form of statutory construction traditionally applied by courts in England and Wales; as per Grey v Pearson (1857) 6 HLC 61: “In construing statutes, and all written instruments, the grammatical and ordinary sense of the words is to be adhered to, unless that would lead to some absurdity or inconsistency with the rest of the instrument, in which case the grammatical and ordinary sense of the words may be modified, so as to avoid that absurdity or inconsistency, but not farther.”.

(thanks to James Davenport)

Tagged , ,

Victory for libel reform: Defamation Act 2013

From today, England and Wales will have a new libel law, with the Defamation Act 2013 coming into effect, after receiving Royal Assent last April*. This is the culmination of five years of work for the Libel Reform Campaign and many others.

As highlighted in posts passim, the previous libel law had been criticised as being antiquated, costly and unfair, resulting in a chilling effect on freedom of expression and the stifling of legitimate debate (particularly for journalists and academics). Some of the new measures include:

  • protection for scientists and academics publishing in peer reviewed journals.
  • protection for those who are publishing material which they reasonably believe is in the public interest.
  • a requirement for companies and individuals to show serious harm to establish a claim.
  • a single publication rule to prevent repeated claims against a person about the same material.
  • a tighter test before claims involving those with little connection to England and Wales can be brought before our courts, addressing libel tourism.
  • a new process enabling website operators to help people complaining about online statements to resolve this direct with the poster of the material.

The Defamation Act will hopefully bring in a new era of libel law that protects freedom of expression and encourages open and honest public debate, while protecting those who feel their reputations have been unjustly attacked; nevertheless, echoing English PEN: we’ve now got a defamation bill but it’s how we act that matters. Read the press release from the Libel Reform Campaign (as well as their initial summary assessment of the Bill), a statement from Lord McNally and this useful guide for journalists.

* to get an idea of how a Bill changes as it passes through both Houses of Parliament, take a look at the tracked changes

Tagged , ,

Logical thinking and bias blind spots

Here’s a simple arithmetic question:

A bat and ball cost a dollar and ten cents. The bat costs a dollar more than the ball. How much does the ball cost?

(the vast majority of people respond quickly and confidently, insisting the ball costs ten cents; this answer is both obvious and wrong — the correct answer is five cents for the ball and a dollar and five cents for the bat)

Here’s another one:

In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half of the lake?

(your first response is probably to take a shortcut and to divide the final answer by half, leading you to 24 days. But that’s wrong — the correct solution is 47 days)

While we like to think (hope) that human beings are rational agents, studies such as the bat and ball question from Daniel Kahneman (a Nobel Laureate and professor of psychology at Princeton) can indicate the opposite: when people face an uncertain situation, they don’t carefully evaluate the information or look up relevant statistics. Instead, their decisions depend on a long list of mental shortcuts, which often lead them to make foolish decisions. These shortcuts aren’t a faster way of doing the calculation; they’re a way of skipping it altogether. Asked about the bat and the ball, we bypass our arithmetic and default to the answer that requires the least mental effort. We assume that intelligence is a buffer against bias, but a 2012 study suggests that, in many instances, smarter people are more vulnerable to these thinking errors.

Find out more about bias blind spots, anchoring bias, framing effects and “cognitive sophistication” in this interesting New Yorker article: “Why Smart People Are Stupid“.

Tagged , , , , ,

“Are you familiar with public-key cryptography…?”

Whitfield Diffie took the stand in Texas on Friday in the courtroom face-off between Newegg and “patent-licensing giant” (a.k.a. patent troll) TQP Development, who has sued hundreds of companies saying it has patented the common Web encryption scheme of combining SSL with RC4.

Enjoy this exchange:

Lawyer: “We’ve heard a good bit in this courtroom about public-key encryption, are you familiar with that?
Diffie: “Yes, I am.
Lawyer: “And how is it that you’re familiar with public-key encryption?
Diffie: “I invented it.

See the full Ars Technica article.

UPDATE: Newegg lost!?

Tagged , , , , ,

Garbage in, garbage out

On two occasions I have been asked “Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?“. In one case a member of the Upper, and in the other a member of the Lower, House put this question.

I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.

Passages from the Life of a Philosopher (1864)
Charles Babbage (1791-1871)

Tagged , ,

Response from Jenny Willott MP on the Draft Defamation Bill

Last month I emailed my local MP, Jenny Willott (Lib Dem, Cardiff Central), regarding the Draft Defamation Bill and the ‘Leveson clause’ using The Libel Reform Campaign‘s Don’t kill the Bill. I received the following response this morning:

Ref: LS/TC/270313/Defamation Bill 2013
8th April 2013

Dear Dr. Crick,

Thank you for your email about the Defamation Bill.

The Defamation Bill is a hugely important piece of legislation that I strongly support. Over recent years, Britain has become a laughing stock as libel tourism has been on the rise and cases have been brought against various high profile scientists and journalists. This has to change, and as my party leader, Deputy Prime Minister Nick Clegg, made clear in a speech on civil liberties last year, libel reform is a very real priority for this government.

That’s why I have been disappointed that this Bill has been held up in the House of Lords by the Conservatives after Labour Peers tabled amendments to introduce the Leveson reforms. However, now that the three main party leaders have agreed the way forward to implement the Leveson reforms, the path is clear for the Defamation Bill to proceed.

As part of the deal between the parties, the Prime Minister announced to the House of Commons that the Government’s legislative programme would now be unblocked, including the Defamation Bill. The changes proposed by Leveson and the reforms in the Defamation Bill are badly needed, and I am pleased that we have now been able to agree a way that we can deliver both.

The right to freedom of speech is a cornerstone of our constitution, and the need to reform our 19th century libel laws is great. The proposals in the Defamation Bill will ensure that the threat of libel proceedings is not used to frustrate robust scientific and academic debate, or to impede responsible investigative journalism and valuable work undertaken by non-governmental organisations. Liberal Democrats have been at the forefront of the campaign for reform, and I am looking forward to the Defamation Bill completing its stages in the Commons in the next few weeks and becoming law.

Thank you again for writing, and please do not hesitate to contact me again if you have further concerns about this or any other issue.

Yours sincerely,

Jenny Willott
MP for Cardiff Central

Tagged , , ,

Get every new post delivered to your Inbox.

Join 377 other followers