Draft code of practice for statistics
Trustworthiness, Quality and Value (TQV)
1. Our draft refreshed Code is structured around what we have called three pillars of Trustworthiness, Quality and Value. Part 1 of the Code explains what Trustworthiness, Quality and Value mean and why they are important. To what extent do you think that Trustworthiness, Quality and Value capture what is needed to enhance public confidence in statistics? What else might be needed?
These values will help instill more confidence in the production and publication of statistics/numerical information. That said, statistics also need to be relevant in the modern world, support government and wider policy making, and increase public understanding on a broad range of issues. There is a sense that these pillars set out what users and the general public believe/ expect ONS should be doing regardless of the existence of the Code.
Maintaining/protecting confidentiality will be critical in instilling public confidence in statistics. It may be helpful to set out more explicitly how compliance with the Code will be assured, how often audits against it will be carried out, and how instances of non-compliance will be independently arbitrated and upheld in the public interest.
The European Directive 2015 on Re-Use of Public Sector Information offers a number of channels for dispute resolution administered by the Office for Public Sector Information (OPSI). This offers a transparent and open framework for handling policy challenges, mediation, complaints and investigations. It would be good to see a similar framework incorporated within the governance principles (T2) and as an explicit element underpinning the Code of Practice implementation.
2. Have we explained Trustworthiness, Quality and Value clearly in the draft refreshed Code? If not, what is unclear?
Yes. However, a worked example would illustrate how the pillars will be implemented in practice.
Scope of the Code
3. The draft refreshed Code contains a description of the pillars and how they fit together in Part 1. Our intention is that this will encourage those producing a wider range of numerical information to think about what the Code aims to achieve in principle. How well does this meet our intention? What else could we do?
In principle this meets your intention. That said the ultimate test will be the extent to which others, such as private sector organisations, adopt the Code for the numerical information they produce.
4. To what extent do you think that organisations outside the official statistics system would be able to apply the three pillars of Trustworthiness, Quality and Value voluntarily? What do you think might prevent this?
In principle, these pillars are transferable to organisations outside the official statistics system. Uptake may be prevented by existing approaches in said organisations and the extent to which the two approaches align. The intended purpose of the numerical information being produced may also have a bearing on whether this approach is used, e.g. whether the information has a wider utility or value, and the perceived robustness and quality assurance of the data. Cost, particularly for the voluntary sector, may represent a barrier to uptake. We agree that the pillars should be seen as a guide and not as a prescription to organisations outside the official statistics system. In this regard, the pillars provide a helpful framework for statistics production.
Private sector and other non-governmental/unofficial organisations outside the official statistics system might find the principle of integrity (T1) where their behaviours and actions are demonstrably in the public interest difficult to prove and uphold in the face of challenge.
There may be times when the use of recognised standards does not apply/ would not be relevant, for example, for a new and more innovative data set that needed a more flexible approach to demonstrate its value. In such cases, it may not be possible or difficult to validate through comparison with other relevant data sources and statistics as required under Principles Q1: suitable data and Q4: coherence).
5. Part 2 of the draft refreshed Code gives the detail of what we mean by Trustworthiness, Quality and Value. How well do you think the principles and practices in Part 2 reflect Trustworthiness, Quality and Value? What additional practices might reflect these three pillars?
The code is silent on how ONS will engage with users/beneficiaries in the course of its work and how wider stakeholder expertise will be harnessed by ONS in developing its work programme. Greater input from a wider audience on what information is valuable and relevant in a changing world would support ONS in the successful implementation and delivery of the code.
Changes to the principles and practices
6. We have updated the practices of the refreshed draft Code to reflect how statistics and data are changing, while allowing for further developments. Do you have any comments on the detailed content of the practices of the refreshed Code? Are there any other practices we might cover? Is there anything you think we might change?
We do not support the proposed new text in paragraph P4.6 of the revised Code, which seeks to remove the specific reference to geographic referencing and coding in the original text. The updated wording reads, ‘statistics should be consistent over time and between geographical areas while remaining relevant to society. Users should be provided with reasons for any deviation from accepted good practice and explanations of any implications for the use of the statistics’.
We firmly believe that the specific reference to geographic referencing and coding in the code should not be removed, as is being proposed above.
We therefore recommend the retention of the existing paragraph P4.6 of the current code (Version 1.0), which reads ‘promote comparability within the UK and internationally by, for example, adopting common standards, concepts, sampling frames, questions, definitions, statistical units and classifications (including common geographic referencing and coding standards). Make the reasons for any deviations from standard models publicly available’. In our view, the original wording is more likely to result in better quality and more coherent statistics.
There would also be value in reviewing the geographic scale at which relevant datasets/statistics are provided; small scale data and geographies can derive insight, as well as the more traditional larger data geographies used by the ONS. This approach would add value to users of ONS statistics, would ensure their relevance in a local context and would support local/place-based policy making, which is the direction of travel for government policy.
7. The draft refreshed Code includes practices that set out the responsibilities of each person in the organisation: T1.3iv in relation to pre-release access and T2.1 in ensuring the appropriate handling and use of statistics. How well do these practices cover what you think are the responsibilities of the people in producer organisations involved in the use and handling of statistics? What is unclear?
These appear to be fit for purpose.
8. The draft refreshed Code includes more specific practices in relation to the roles and responsibilities of the Head of Profession for statistics. Are the practices relating to Heads of Profession (under Principle T2: Good governance) sufficient to support them in undertaking their role overseeing and advising on the production of statistics and wider numerical information? Are there other aspects of their roles and responsibilities that the Code might include?
The roles and responsibilities of the Chief Statistician/Head of Profession as set out appear to cover the necessary ground within the context of the official statistics system. The scope of the roles could be enhanced by inclusion of a responsibility to develop members of the statistics profession and to advocate more widely around the benefits and value of statistics in policy making, in tune with the three pillars of the code.
The code could be more explicit and detailed about the roles and responsibilities of the Chief Statistician/Head of Profession and Director General for Regulation for overseeing and managing the production of statistics outside the official statistics system. See earlier feedback (under section 1) about the need for a transparent and open framework for handling policy challenges, mediation, complaints and investigations around production of statistics. For example, where there is a complaint that a private sector body claiming to produce an official statistical release is not acting in accordance with the Code.
9. The draft refreshed Code has an expanded range of principles and practices related to Quality, based on the Authority's Quality Assurance of Administrative Data. To what extent does the refreshed Code cover aspects of Quality: robust data, methods and statistics? What other aspects of Quality might they include?
While the code is seeking to ensure statistics are produced through the use of quality data, the missing piece of the jigsaw appears to be that quality, or at least the perception of quality, lies in the interpretation of the data and the insights it reveals. The metric for best professional judgement appears under the trustworthy pillar; but this does not speak to the quality of interpretation. Perhaps it is the interpretation of the data that needs to be assured through the quality pillar, which would in turn instill confidence.
10. The draft refreshed Code includes a specific principle on coherence (Q4), that ‘Statistics should be consistent and comparable, while remaining relevant to society’. It also has a practice (Q3.3) about the use of similar data sources for quality assurance, and drawing insight from related statistical outputs (V2.4). To what extent do you think that the refreshed Code sufficiently covers the statistical practices related to coherence? Are there other aspects of coherence we might include?
It covers the above reasonably well. However, new or changing geographies can occur with changes in government policy, such as with the creation of Local Enterprise Partnerships (LEPs) and their impact on economic geography. The UK’s exit from the European Union may also have an influence on geographical areas, as the UK may no longer need to work with the Nomenclature of Territorial Units for Statistics (NUTS) classification.
11. We have added a new principle: ‘Statistics need to continue to evolve to remain relevant in a changing world’ (V4: Innovative), emphasising innovation. To what extent does this new principle have the right focus to encourage and stimulate creativity and continuous improvement across all aspects of statistics production? What do you think we should add or change?
New entities have been created by government in recent years, such as Local Enterprise Partnerships, which changed the economic development landscape. Statistics at the level of these geographies would support and enhance local policy making and help deliver local economic growth. The element that brings most datasets/ statistics to life is location/geography, as this provides the context for the information. It is the linking of location to other disparate and multiple datasets that drives the insight to strategy and policy development and service delivery in both the public and private sectors. Producing statistics at the most appropriate geographic scale will be increasingly important once the UK exits the European Union.
The principle of continuing to evolve to remain relevant in a changing world is fundamental to the value of the information. However, encouraging innovation in data sources and analysis methods will often challenge the Code Practice principles of Integrity (T1), Confidentiality (T4), and all Quality principles (Q1-Q4). While there is implicit recognition of this within the Code, and how it might be governed and managed within the official statistics system, the Code could go further in promoting innovation by commenting on how synergies between innovations inside and outside the official statistics system might be enabled e.g. by promoting collaborations across public, private and academic sectors in the production and publication of experimental statistics.
Innovation in recent years has been driven by linked data and the semantic web, big data analytics and visualisation. Big data describes the collection of complex and large data sets, such that it is difficult to capture, process, store, search and analyse using conventional database systems. Its uses are shaping the world around us, offering more qualitative insights into our everyday lives. Big data visualisation enables more rapid identification of patterns and trends. Reports generated from big data visualisation tools make it possible to encapsulate complex information on operational, geographic and market conditions in a brief series or even single graphic. This means the information is more vibrant and memorable for users and decision makers.
Adopting this approach would meet the value V4 of the code, which calls for continuous improvements to be made to the presentation and dissemination of statistics, and for new and innovative ways to engage users to be exploited.
12. The detailed practice relating to experimental statistics sits within Principle V4: Innovative in the draft refreshed Code. To what extent do you think the practice on experimental statistics (V4.5) is sufficiently clear to encourage the open and transparent development of new statistics that involves users?
The flexibility to produce and publish experimental statistics is a positive step and should help to encourage development of new datasets and new users of statistics. ONS should be mindful that production of experimental statistics may not meet the proposed test of quality that statistics be consistent over time and comparable, which at the outset they may not be by definition. However, this can be overcome provided the statistics and numerical information represent the best available estimate of what they aim to measure at a particular point in time, and are not materially misleading.
It would be helpful if ONS or the Code provided an illustrative example of what constitutes experimental statistics. Whatever datasets are produced, these need to meet the needs of users and deliver added value over and above information that has already been produced and/ or published. New/ experimental statistics, as with existing datasets, need to be able to pass the ‘So what?’ test.
13. The draft refreshed Code includes a principle that ‘Statistics should help answer society’s important questions’ (V2: Insightful). The principle focuses on ensuring the public value of statistics through clear presentation, demonstrating relevance and supporting use, as well as through working collaboratively. How well do the practices under V2: Insightful set out what is expected of producers?
The draft refreshed code is clear on this point and the proposed practices appear to be fit for purpose.
Communication and audience
14. In previous discussions in the Code Stocktake and subsequent focus groups we found strong interest in the idea of additional guidance to help understand how to apply the Code, with examples of best practice. Which, if any, of the three pillars of Trustworthiness, Quality and Value or their related principles do you think need additional guidance and why? Select all that apply.
Value: Insightful; Value: Innovative
It is not always clear what the words insightful and innovative mean in an objective context. It would therefore be helpful to users and others to have a clear and shared understanding of these terms. The guidance could helpfully provide examples of what does and does not constitute insightful and innovative.
15. To what extent should the guidance be tailored for different audiences? What groups of people in particular might require guidance?
[No answer]
16. Annex A of the consultation document includes a data diagnostic tool that is intended for people who want to use statistics, to consider whether to use particular data sources. To what extent will the data diagnostic tool aid users in assessing the suitability of data for their uses? What more might they need?
[No answer]
17. How clear and understandable is the language used in the Code? How could it be made clearer? Are there any areas of the draft refreshed Code you believe are at risk of becoming out-dated?
All codes of practice have a certain shelf-life; to remain a living and vibrant document, the code would need to be reviewed, and where necessary, updated on a regular/ periodic basis. Best practice would suggest refreshing the code every three years as a minimum, with a wholesale review every five years.
18. Overall, how satisfied are you that the draft refreshed Code meets your needs? Please explain your reasons in the comment box below.
Satisfied.