Last week Helen and I (and Boris’s biographer Sonia Purnell, who has a new ebook out about , and Martin Hoscik from Mayorwatch and about 1300 others) all trooped down to Marsham St. to see Boris, Jenny, Ken and Brian compered by the affable Clive Anderson, who I suspect would have got elected if he’d been a candidate (nice line in forensic questioning too from Clive, his lawyer training beating the likes of the overrated Jeremy Paxman).

Anyway, Boris has clearly gone big on ‘trust’, which us old crusties recognise as a standard Lynton Crosby tactic of attacking where your candidate is weakest to bring the opponent down to your level.  Let’s see how trust goes in Boris’s world.

Since 2007 and the passing of the Statistics Act Government bodies have been legally constrained in the way they report official statistics by a code of conduct maintained by the independent UK Statistics Authority, headed until earlier this year by Sir Michael Scholar, President of St. John’s College, Oxford and former private secretary to Mrs Thatcher.  Their job, as they put it, is to oversee the ONS and, for our interest:

The Statistics Authority is required to monitor, and may report on, the production and publication of all official statistics (wherever produced), regardless of whether or not they are labelled as ‘National Statistics’

The Statistics Authority must maintain a Code of Practice, a benchmark which allows independent assessment of the quality and integrity of official statistics (produced by ONS, government departments and the devolved administrations), before deciding to give them formal accreditation as ‘National Statistics’.

Notice ‘devolved administrations’ there, and in fact the UKSA came into being during the 2008 Mayoral elections, so has only really been a factor for Boris.

In November 2006, the Government introduced the Statistics and Registration Service Bill which embedded the earlier non-statutory reforms by creating a new Statistics Authority, independent of government reporting directly to Parliament and the devolved legislatures, with overall responsibility to promote and safeguard the production and publication of UK official statistics, wherever produced, and with oversight of the Office for National Statistics. The Statistics and Registration Bill received Royal Assent on 26 July 2007 and all of its provisions came into effect on 1 April 2008 when the new Statistics Authority itself was launched

However, there’s a loophole, in that only statistics classified by the Cabinet Office as ‘Official Statistics’ come under the scrutiny of the UKSA, and things like TfL statistics are not currently classified as such for some reason.  What this essentially means is that Boris is, to quite a large degree, allowed to use statistics from the public bodies under his control (apart from TfL these would include the former London Development Agency as well as the GLA itself) in a manner that doesn’t conform to the ethical and transparency standards that such paragons of virtue as Michael Gove and George Osborne have to follow.

Things came to a head in February 2011 when Boris made one of his usual ‘Crime’s coming down on the transport network’ claims:

  • New figures show travelling across the capital is getting safer
  • Robbery on public transport has fallen by a massive 46.5 per cent in the last three years.
  • Criminal damage to buses and Tube trains has also fallen hugely – by 59.3 per cent.

Now, this kind of cherry picking to prove how great you are has been standard operating practice for politicians since I could walk and it’s telling that I (and presumably a lot of people) tune it out as obvious propaganda.  However, things are changing, the Internet has come along with its near-limitless capacity to store, process, analyse and distribute data.  It’s also generated organisations like FullFact which immediately picked up on an important point:

On the Guardian’s London blog, Dave Hill flagged up some apparent inconsistencies in the numbers cited by the Mayor.

But when tracing similar steps trying to get to the bottom of the statistics, Full Fact encountered a problem.

Nowhere in the Mayor’s press release from last week does it actually link through to the figures cited.

There was no way to check Boris wasn’t making it up.  We had to trust, but not verify.

This prompted a somewhat sniffy letter from Sir Michael on the 16th March pointing out that:

The Code of Practice is very clear about this. It says that statistical reports should be published ‘separately from any other statement or comment about the figures’; that ‘no statement or comment – based on prior knowledge – is issued to the Press or published ahead of the publication of the statistics’; and that ‘no indication of the substance of a statistical report is made public or given to the media’ prior to publication.

In other words, in an era of loss of trust in government, datastores and armies of nerds (guilty, here) with a thirst for public data, don’t publish your PR before your data, so we can check you’re not bullshitting us before the next day’s papers become chip wrappers.  The trust element is stressed by Sir Michael:

Although under the Statistics Act Ministers are now legally obliged to comply with these principles, as matters stand this is not currently the case for you. However, in the interests of restoring public trust in government statistics, may I invite you to undertake in future to comply with the Code, as a matter of principle?

Just to be on the safe side, the letter was copied to Theresa May, Francis Maude (who presumably employs someone who can read it out and explain the difficult words), the National Statistician and Full Fact.  I read into this that it’s a warning shot from a man who clearly thinks the devolved administration in London should come under the same rules as the national government in Westminster, and it’s hard to argue against that.

Boris, however, wrote back on 8th April:

Statistical data for London is regional level data and therefore should not form part of national statistics. I see no good reason to change the current arrangements.  However, it would have been helpful had you approached me first about your concerns about statistical standards for London before publicly releasing your letter.

That’s actually quite breathtakingly rude.  Summarised, first ‘no’ and then ‘how dare you make this public’.  The reaction of a Boris spokesman (Guto Harri?) to later accusations (to try and smear it as a set up job designed to embarrass Boris) is likewise telling of an insecure, arrogant attitude.  The UKSA, being a public body, is not only subject to FoI but has a specific remit to promote openness, and as such has a page where you can see precisely who they’ve been writing to, when and read the letter.  This, I put forward in a non-controversial manner, is a good thing.  I’m not sure if anyone other than Boris has ever complained about this unwarranted intrusion into their public life, including such minor, laid-back characters as Keith Vaz, Francis Maude and Iain Duncan Smith.

As for the substance of Boris’s response, that statistical data for London is ‘regional level’ data –  does this apply to the Met, which is a joint responsibility with the Home Office, nor the British Transport Police, which is run by the Department for Transport and polices the London Underground?  It’s high time we examined the legislation:

6 Official statistics

(1)In this Part “official statistics” means—

(a)statistics produced by—

  • (i)the Board,
  • (ii)a government department,
  • (iii)the Scottish Administration,
  • (iv)a Welsh ministerial authority,
  • (v)a Northern Ireland department, or
  • (vi)any other person acting on behalf of the Crown, and

(b)such other statistics as may be specified by order by—

  • (i)a Minister of the Crown,
  • (ii)the Scottish Ministers,
  • (iii)the Welsh Ministers, or
  • (iv)a Northern Ireland department.
There’s the loophole – Boris is quite content to be a devolved administration when it suits him, but according to the legislation statistics produced by London aren’t considered to fall under the scope of the Act, unlike Wales, Scotland and Northern Ireland.
However, section 6(1)(b) gives the Minister (Francis Maude, in this case, presumably) the power to specify that particular sets of statistics should fall into the Official Statistics category and it’s this suggestion that appears to have had Boris go off on one.
This wasn’t the end of the story by a long shot – the more famous second letter, also published on the UKSA website, was to Keith Vaz rather than Boris, and followed Boris’s appearance on 6th September 2011 before Vaz’s Home Affairs Committee about the riots and in particular his pet scheme at Feltham (Project Daedalus) where…

…it was suggested that re-offending rates in the Heron Wing at Feltham Young Offenders Institution have been cut from 80 per cent to 19 per cent. This was repeated in Press Notices published by the Mayor’s Office and in the national press on 19 September 2011.

Indeed, the national press did cover it on 19th September 2011, in the form of a promising young journalist called Boris Johnson, in the Telegraph:

We are working with the Justice Department to expand the work of the Heron unit in Feltham, where re-offending rates have been brought down from about 80 per cent to about 20 per cent. Bang them up, in other words, but turn them round, too.

Sir Michael was unimpressed:

I attach a note by UK Statistics Authority officials which sets out the facts on this matter, so far as they can discover them. This note shows that the Mayor’s evidence to your Committee is not supported by the Ministry of Justice’s published statistics (the Ministry of Justice is responsible for the relevant National Statistics); nor is it supported by any statistical analysis published by the Mayor’s Office

Note here that reoffending rates definitely *are* covered by the 2007 Act since they’re published by a government department, the Ministry of Justice.  Boris can’t therefore hide behind the ‘regional’ get out.  This raises another loophole – if Ken Clarke had done what Boris did he’d be breaking the law.

Sir Michael pulls no punches:

I should point out that the Statistics Authority does not have any powers to require publication of statistical analysis of this kind; that the work of the Mayor’s office is not covered by the Code of Practice for Official Statistics; and that the Mayor declined to agree with my request several months ago that he should voluntarily undertake to conform with the requirements of the Code.

In other words ‘if Boris doesn’t volunteer we’ll make him’.  Sir Michael’s concern is evident in the attached evidence that lays out precisely how dodgy Boris was being here:

We understand that the figures for reoffending at the Heron Unit come from internal, unpublished, management information. We also understand that the management information contains the following caveat:

“This is based on anecdotal information and does not represent a re-conviction rate
and should not be used publicly”

The figures do not use the recognised Ministry of Justice (MoJ) method for calculating reoffending rates: i.e. that reoffending should be calculated one year after release from custody. It is therefore likely to be an underestimate of reoffending, and is not comparable with the national figure.

MoJ has confirmed that the figure of 78 per cent is not immediately recognised.

‘Not immediately recognised’ – essentially Boris was lying here, is the accusation – deliberately conflating two dissimilar figures to support the position that he was being pro-active and successful in tackling youth crime.  The background, of course, was the August 2011 riots where you might have expected him to be a bit on the defensive – the biggest boos he received at Marsham St. last week were over his refusal to come back from the USA.

Having received a second telling-off from Sir Michael Scholar you might have expected a pause for thought, but Boris of course famously switched into Bullingdon mode when challenged on this by Joanne McCartney:

“There’s this guy Scholar writing me letters who sounds … like some sort of Labour stooge.”

“I am not impressed by the conduct of that particular body and its chief, if I may say so disrespectfully. Nor am I impressed frankly by your whole line of questioning, which I think is petty, political and mind-bogglingly trivial by comparison of what to do, which is to change the lives of young people.”

Got that? Using statistics with either no basis in fact or an explicit caveat against publication is fine, asking questions about it in a public democratic forum is ‘petty, political and mind-bogglingly trivial’.  There’s a pattern here, and it’s not of a man you should trust, not least because Sir Michael was an equal-opportunity politician basher, as Jacqui Smith found.

So what’s Boris’s excuse?:

If I erred, it was not in saying these figures are temporary, they are provisional, they do not represent a final analysis

In the course of a long paragraph of explanation, I used the statistics I had in my head about the success of the Heron Unit, because I want to promote that unit

In other words he tried to get out of an awkward question by using context-free figures he’d memorised, without context and without mentioning the caveat he admitted to knowing.

P.S. Sir Michael Scholar had powerful friends.  Ben Goldacre, no stranger to quacks waving dodgy stats around, is a fan:

This is silly, but also tells a story about the contempt with which politicians hold evidence and statistics. They want to be able to use them as narrative devices, without any responsibility or need for accuracy. They are indignant when this entitlement is questioned. This shows how they really feel about evidence.

In my view this problem is entrenched, and persists, because politicians operate in a permissive environment: popular public scrutiny from the press is hindered, because journalists and editors have the same issues with stats that politicians do. All pretty unfortunate.

Quite.  Finally, Sir Michael’s own college, St. John’s, Oxford, decided to do things properly and passed a ‘Boris is a Berk’ motion banning him from the Junior Common Room.

The motion resolved upon a variety of actions against the mayor. After stating that “Sir Michael is awesome” and “Boris is a fool”, it pledged to “Ban Boris from the JCR”, “Mandate the JCR President to write a letter to Boris informing him of our displeasure and declaring our support for Sir Michael, so long as this does not conflict with our charitable status”, “Print a photo of Boris for fixing to the dart board in the games room” and “Mandate the Bike Rep to never use a ‘Boris bike’”.

That’s him told.

As for Project Daedalus, it’s turned into Project Icarus, in other words it has crashed and burned.  The final figures aren’t even out yet (the updated provisional ones are already showing the Boris was lying about the whole ’19%’ thing), but it’s already been closed as the money’s run out.  So much for Boris’s grand plan as stated to the Home Affairs Select Committee:

I think what we are doing with the Heron wing at Feltham is so important where by taking people, young people, who are willing to turn their lives around, who seem most able to be redeemed if you want, they are educated, they are given inspiration. We cut reoffending rates in that wing from 80% to 19%. That is a model that I think should be replicated around the country.

Why did it close, though?  The BBC have been leaked a report into the scheme that they claim was later amended to airbrush out the failures:

The report authors were told payment-by-results models could exclude charitable organisations from getting involved in future projects, a potentially damaging finding omitted from the final report.

Difficulties arose because the project had to be extended when not enough suitable young offenders could be found from the original six “Diamond” boroughs – Newham, Lambeth, Southwark, Croydon, Lewisham and Hackney.

Omitted was a section of the original draft which said: “It was thought by both parents and young people themselves that Resettlement Brokers may be stretched, and that consequently they did not spend as much time with the young person as was sometimes desired.”

Few inmates who were interviewed felt they had been prepared for the jobs market, also not reported in the published version.

So, a scheme that cost millions, was deliberately targetted at the young offenders who were willing to participate and weren’t disruptive or anti-authority (a rig in itself, I reckon) failed due to the ideological insistence on shoving it all off onto a charity which had to prove results before being paid.  As a model of Conservative public service provision it’s pretty much a warning to us all.

The response to the BBC from City Hall is ironic given the previous misuse of out-of-context statistics by the boss:

“A spokesman said: “It is standard practice for all reports to naturally go through various drafts before being published and the Mayor’s Office, the Ministry of Justice, Youth Justice Board, third sector and boroughs have amended inaccuracies and fed back suggestions to ensure findings are accurately reflected and evidence based

Perhaps they should get Sir Michael to write the BBC one of his letters?

 

One Response to Boris’s Dodgy Statistics – Trust, Don’t Verify

  1. solar penguin says:

    Only £1.37? You’re lucky, I normally have to pay 2 quid. I guess there are some perks to being mayor…

    (Note: the figure of 2 quid in the above paragraph is totally made up. I’m just following the mayor’s approach to statistics as described in the article.)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>