Measuring the performance of chemical management systems will be one of the high-level discussions at the forthcoming Asian Helsinki Chemicals Forum in Shanghai on 24-25 October. Chemical Watch's science editor Andrew Turley takes a look at the background to this important issue.
Recent years have seen governments put increasing emphasis on the post-implementation analysis of new regulations. In the world of chemicals management, knowing what the impact of a piece of legislation is, and how far it has met its predicted outcomes, can be crucial to planning and meeting policy objectives.
It’s now not uncommon for regulations to contain specific requirements to monitor their own effectiveness.
Nevertheless, Eeva Leinala, principal administrator, hazard assessment and risk reduction programmes at the OECD, says that measuring performance can be one of the most challenging areas in the chemicals policy field.
"For any regulation or programme you put in place, there’s more attention now on the fact that you should be demonstrating the effectiveness. And that is the case, not just with environmental or chemical regulations, but with all different types of regulations. Are your regulations and the actions under them actually achieving what you set out to do?"
Back in February, the OECD published a review of performance measurement of chemicals management systems. The report was based on analysis conducted by Canada and verified by the chemicals committee and the working party on chemicals, pesticides and biotechnology.
It concluded that:
- there is no common and accepted terminology regarding performance measurement;
- there is no common approach in terms of the level where performance is being measured, which might be at the level of outputs, outcomes or impacts; and
- no single performance measurement model would fit all of the chemical management programmes.
"Based on this, there is an opportunity to establish general terminology and indicators," the report says. "This could include the analysis of which indicators can best be measured with available information, and which provide the most meaningful insight into the performance of a chemicals management system and lead to the development of best practices."
Policy makers will look to experts in economics and other fields for the most up-to-date academic knowledge regarding key elements, such as indicators. However, they will also look to the regulatory context for chemicals, which closely tied to politic trends. There are also practical challenges. It is relevant to ask what can be achieved with the data available and what data is missing.
Political factors
Politics can play a major part in driving performance measurement up governmental agendas. The EU, the US and Canada, for example, are all going through, or facing, significant shifts in their political landscapes that may lead to a greater need for data on the effectiveness of regulation.
In the EU, Brexit has, and continues to be both a disruptive and confusing factor. The dust is far from settling on how damaging the UK’s departure will be if and when it leaves the EU. Its effects could play out for a long while yet.
Across the Atlantic, President Trump is loudly opposed to regulation, which he has rolled back at every opportunity. He faces elections in November next year.
Before that, however, Canadians go to the polls for a general election, the outcome of which is set to be critical for chemicals regulation. The government has promised to reform Cepa but any Bill aimed at this would be unlikely to reach parliament before 2020. Therefore, first, there will be an election, which must come on or before 21 October. Cepa reform is dependent on a Liberal Party victory, which looks far from certain following the serious defeat in Ontario’s provincial election last year.
International agreements
Intergovernmental organisations, such as the UN and the OECD also have an interest in measuring the performance of chemicals management systems.
In March the UN published the summary of its Global Chemicals Outlook II report. This said the global goal to minimise adverse impacts of chemicals and waste would not be achieved by 2020. The goal was set out in 2006 under the UN’s global non-binding chemicals programme, the Strategic Approach to International Chemicals Management (Saicm).
Despite international agreement, reached at high-level UN conferences, and significant action already taken, "scientists continue to express concerns regarding the lack of progress" made, the report said.
Chapter two is on reporting schemes and indicators under international agreements and frameworks, including those of:
- the Montreal Protocol
- ILO Convention C170 concerning safety in the use of chemicals at work;
- ILO Convention C174 on prevention of major industrial accidents;
- the WHO international health Regulations;
- the Basel Convention on the control of transboundary movements of hazardous wastes and their disposal;
- the Montreal Protocol on substances that deplete the ozone layer;
- the Stockholm Convention on persistent organic pollutants;
- the Inter-Organization Programme for the Sound Management of Chemicals (IOMC); and
- the Strategic Approach to International Chemicals Management (Saicm), which is the policy framework for the UN’s 2020 goal.
The UN found that, although Saicm has the most comprehensive framework for monitoring progress, its reporting rates have not been satisfactory.
In September, international experts will meet at a workshop with the aim of creating a quantifiable process for measuring the progress an individual country is making towards the objectives of a post-2020 global chemicals and waste framework.
The workshop, hosted by the UK government and run by United Nations Environment Programme's World Conservation Monitoring Centre (WCMC) in Cambridge on 3-5 September, will aim to draft a set of indicators to "enable assessment of global progress".
Data challenges
Ms Leinala says that organisations wishing to measure the performance of their chemicals management systems face various challenges with respect to data. First and foremost, data gaps are everywhere. There are ways to adapt the data you have to the questions you want to answer, but the best solution is always to consider how you will measure performance when you design the system. "I think we're still a bit in that awareness raising phase," she says.
Furthermore, many such systems involve immediate pollution prevention measures. Clearly, this is good for risk management, but it can make it difficult to demonstrate effectiveness via a measurable indicator, as potential releases are actively controlled to avoid the problem before it manifests itself.
Human biomonitoring
One of the indicators currently generating a lot of interest is human biomonitoring (HBM) data.
Human biomonitoring (HBM) is an exposure assessment method for determining the extent to which target substances, found in the environment, are entering humans and the trends associated with this process.
Researchers may measure levels of natural or synthetic substances in a range of biological materials, including: hair, nails, bone, fat and various fluids, such as blood, urine and breast milk. The materials are obtained from a representative sample of the population, along with detailed social, economic and behavioural information. The aim is to connect specific exposures to distinct aspects of human life.
One of the advantages of HBM, compared with other exposure assessment methods, is that it accounts for multiple exposure routes without the need for precise knowledge about them. Additionally, large scale HBM programmes can provide real world data for assessing the effectiveness of regulatory measures designed to reduce unwanted exposures, for example, restrictions and bans.
One of the advantages of HBM, compared with other exposure assessment methods, is that it accounts for multiple exposure routes without the need for precise knowledge about them. Additionally, large scale HBM programmes can provide real world data for assessing the effectiveness of regulatory measures designed to reduce unwanted exposures, for example, restrictions and bans.
Various national governments, including Germany, the US and Canada, have long running HBM programmes. Additionally, in 2016, the EU launched a pan-European project in this area, HBM4EU, with €74m in funding and support from 28 participating countries, comprising 24 EU member states plus Norway, Switzerland, Iceland and Israel.
HBM is useful for those chemicals with which you already you have a problem, says Ms Leinala. In such cases, it can "close the circle" in terms of risk assessment by providing concentrations inside the body.
However, the method is much less useful when you want a prospective approach because it is less clear which chemicals are problematic. This is partly because of the high cost.
How to measure the performance of chemical management systems will be the focus of a panel discussion at the Asian Helsinki Chemical Forum in Shanghai on 24 October. Ms Leinala will chair the discussion with panelists Steven De Regter from BASF, Meg Postle from RPA and a representative of the European Commission.
The panel will aim to answer various questions on this topic, such as:
- what are meaningful indicators to measure success?
- do different stakeholders measure success differently?
- what needs to be measured over time?
- what is the cost of action or inaction? and
- how should we assess the financial value of regulatory systems?
Differing regimes
EU
The EU recently passed two major milestones for measurement concerning the performance of chemicals management systems.
Last year, the European Commission published its report on the second REACH Review.
Under Article 117 of REACH, member states, Echa and the Commission are required to report regularly on the operation of the Regulation. For member states and Echa, this means submitting a report to the Commission every five years.
Echa must also submit a report on the status of implementation and use of non-animal test methods and testing strategies every three years. The member state and Echa reports inform the thinking of the Commission, which is itself required to publish a general report on the REACH review every five years. Its evaluation is carried out as part of the programme for Regulatory Fitness and Performance (REFIT) in accordance with the better regulation guidelines. Chapters V and VI of these guidelines deal with monitoring and evaluation, including fitness checks.
Furthermore, in July, the Commission published the equivalent report for all the other regulations and directives dealing with hazardous chemicals. The report on the fitness check of chemicals legislation excluding REACH covered more than 40 separate pieces of legislation including:
- CLP;
- the biocidal products Regulation;
- legislation for the protection of workers, such as the chemical agents Directive and the carcinogens and mutagens Directive;
- the toy safety Directive;
- the cosmetics Regulation; and
- the detergents Regulation.
Canada
There are similar ongoing discussions in Canada. In 2018, the government’s Auditor General said that the two departments primarily responsible for managing hazardous chemicals were falling short in terms of ensuring the effectiveness of risk management actions. In a report on toxic substances, the Auditor General said that Environment and Climate Change Canada and Health Canada needed to establish a "long-term, systematic approach" to evaluate how effective their actions are in controlling toxic substances. The approach should include setting measurable objectives, monitoring the achievement of these objectives and setting timelines for finalisation.
The government is nearing completion of its mission to address the 4,300 priority chemicals, under the Chemicals Management Plan. It should achieve this next year, with the current programme of work formally ending in March 2021. The government has started to plan for what comes next, and in doing so it will surely reflect on the effectiveness of the CMP.
It will do this against the backdrop of legislative change.
In 2017, a Canadian House of Commons committee report called for widespread changes to the Canadian Environmental Protection Act (Cepa). It included 87 recommendations, many addressing the CMP and government treatment of substances of concern. The following year, the government promised to overhaul Cepa, saying it agreed with the intent of most of the recommendations but wanted to assess them further in 2018 and 2019.
US
There are key differences in the US. The EU’s flagship REACH legislation has been in force since 2007, and while it has experienced some change in that time this has been relatively minor. In contrast, TSCA is much older, but was completely overhauled in 2016.
Consequently, while the EU and Canada have been reflecting on effectiveness, the US has been focused on implementation.
However, the revised TSCA is now entering a critical stage, with publication of draft risk evaluations for the first set of ten substances. The EPA has published four so far this year – those of PV29, 1,4-dioxane, HBCD and 1-bromopropane – and each has led to disagreement between stakeholders.
The OECD's Eeva Leinala will chair a panel on measuring performance on 24 October at the Asian HCF including Steven De Regter, senior manager for global policies, requirements, goals and EHS reporting at BASF, and Meg Postle, director at Risk & Policy Analysts (RPA).