Category Archives: Policy

UKRI go its A.I. policy half right

UKRI AI policy: Authors on the left. Assessors on the right (image generated by DALL.E)

When UKRI released its policy on using generative artificial intelligence (A.I.) in funding applications this September, I found myself nodding until I wasn’t. Like many in the research community, I’ve been watching the integration of A.I. tools into academic work with excitement and trepidation. In contrast, UKRI’s approach is a puzzling mix of Byzantine architecture and modern chic.

The modern chic, the half they got right, is on using A.I. in research proposal development. By adopting what amounts to a “don’t ask, don’t tell” policy, they have side-stepped endless debates that swirl about university circles. Do you want to use an A.I. to help structure your proposal? Go ahead. Do you prefer to use it for brainstorming or polishing your prose? That’s fine, too. Maybe you like to write your proposal on blank sheets of paper using an HB pencil. You’re a responsible adult—we’ll trust you, and please don’t tell us about it.

The approach is sensible. It recognises A.I. as just one of the many tools in the researcher’s arsenal. It is no different in principle from grammar-checkers or reference managers. UKRI has avoided creating artificial distinctions between AI-assisted work and “human work” by not requiring disclosure. Such a distinction also becomes increasingly meaningless as A.I. tools integrate into our daily workflows, often completely unknown to us.

Now let’s turn to the Byzantine—the half UKRI got wrong—the part dealing with assessors of grant proposals. And here, UKRI seems to have lost its nerve. The complete prohibition on using A.I. by assessors feels like a policy from a different era—some time “Before ChatGPT” (B.C.) was released in November 2022. The B.C. policy fails to recognise the enormous potential of A.I. to support and improve human assessors’ judgment.

You’re a senior researcher who’s agreed to review for UKRI. You have just submitted a proposal using an A.I. to clean, polish and improve the work. As an assessor, you are now juggling multiple complex proposals, each crossing traditional disciplinary boundaries (which is increasingly regarded as a positive). You’re probably doing this alongside your day job because that’s how senior researchers work. Wouldn’t it be helpful to have an A.I. assistant to organise key points, flag potential concerns, help clarify technical concepts outside your immediate expertise, act as a sounding board, or provide an intelligent search of the text?

The current policy says no. Assessors must perform every aspect of the review manually, potentially reducing the time they can spend on a deep evaluation of the proposal. The restriction becomes particularly problematic when considering international reviewers, especially those from the Global South. Many brilliant researchers who could offer valuable perspectives might struggle with English as a second language and miss some nuance without support. A.I. could help bridge this gap, but the policy forbids it.

The dual-use policy leads to an ironic situation. Applicants can use A.I. to write their proposals, but assessors can’t use it to support the evaluation of those proposals. It is like allowing Formula 1 teams to use bleeding-edge technology to design their racing cars while insisting that race officials use an hourglass and the naked eye to monitor the race.

Strategically, the situation worries me. Research funding is a global enterprise; other funding bodies are unlikely to maintain such a conservative stance for long. As other funders integrate A.I. into their assessment processes, they will develop best-practice approaches and more efficient workflows. UKRI will fall behind. This could affect the quality of assessments and UKRI’s ability to attract busy reviewers. Why would a busy senior researcher review for UKRI when other funders value their reviewers’ time and encourage efficiency and quality?

There is a path forward. UKRI could maintain its thoughtful approach to applicants while developing more nuanced guidelines for assessors. One approach would be a policy that clearly outlines appropriate A.I. use cases at different stages of assessment, from initial review to technical clarification to quality control. By adding transparency requirements, proper training, and regular policy reviews, UKRI could lead the way with approaches that both protect research integrity and embrace innovation.

If UKRI is nervous, they could start with a pilot program. Evaluate the impact of AI-assisted assessment. Compare it to a traditional approach. This would provide evidence-based insights for policy development while demonstrating leadership in research governance and funding.

The current policy feels half-baked. UKRI has shown they can craft sophisticated policy around A.I. use. The approach to applicants proves this. They need to extend that same thoughtfulness to the assessment process. The goal is not to use A.I. to replace human judgment but to enhance it. It would allow assessors to focus their expertise where it matters most.

This is about more than efficiency and keeping up with technology. It’s about creating the best possible system for identifying and supporting excellent research. If A.I. is a tool to support this process, we should celebrate. When we help assessors do their job more effectively, we help the entire research community.

The research landscape is changing rapidly. UKRI has taken an important first step in allowing A.I. to support the writing of funding grant applications. Now it’s time for the next one—using A.I. to support funding grant evaluation.

Playing Fair: “Horizontality” and the Future of Aid

The arrival of US Aid, “from the American people”.

In his book, Playing Fair, the self-confessed Whig, Ken Binore argued for the redistribution of the “social cake”.

For progress to be made, it is necessary for the affluent to understand that their freedom to enjoy what their “property rights” supposedly secure is actually contingent on the willingness of the less affluent to recognize such “rights”. It is not ordained that things must be the way they are. The common understandings that govern current behavior are constructs and what has been constructed can be reconstructed. If the affluent are willing to surrender some of their relative advantages in return for a more secure environment in which to enjoy those which remain, or in order to generate a larger social cake for division, then everybody can gain. (p.7)

In other words, if we do not share the cake, “they” might burn down the bakery.

I am more idealistic. I have a sense that we should share the social-cake because it is the right thing to do, or maybe it is less the case that redistribution is right than it is wrong to leave people in states of significant disadvantage, particularly when one can do something about it. I am also sufficiently pragmatic not to care what motivates people to extend a hand to others.

Do it because it is right. Do it because it serves your own interests. Do it as a romantic, random act of kindness. I don’t care. The capacity of a dollar to make a difference is not altered. DO IT!

Let me extend this discussion to support offered by more affluent countries to less affluent countries. A couple of days ago I attended a virtual dialogue at Wilton Park as part of their “Future of Aid” series. “Aid” in this context is the (usually financial) assistance provided by one country to another.

Definition; Aid: Late Middle English from Old French aide (noun), aidier (verb), based on Latin adjuvare, from ad- ‘towards’ + juvare ‘to help’.

At least in conceptual origin, country-level aid is about one country doing something towards helping another country. And I would argue that what is really meant (or should be meant) by one country helping another country is that they are helping to improve the lives of the people who live in that country and, in particular, the less affluent and less powerful people.

An important idea emerged in the discussions about aid and that was “horizontality”. Horizontality is the idea that the donor and the recipient countries are equal partners. It is an attempt to move aid beyond neocolonial domination. I applaud this idea, at least I applaud the idea that we should not use aid as a vehicle for exchanging one kind of colonialism for another.

What I hope we are saying when we talk about horizontality is that aid is not about the exercise of power, it is about the redistribution of power. To achieve horizontality, aid can be neither handout, loan nor gift. Aid must be part of a just, redistributive process to improve lives and reduce suffering that recognises we all share one planet, and appreciates that donor and recipient governments are imperfect, though necessary, vehicles for realising these goals.

Horizontality does not mean that aid should be without conditions or accountability. In fact, it means the very opposite. Aid should have strong accountability mechanisms because the purpose of aid is to help people, and governments (and other involved commercial or civil society organisations) are simply vehicles for achieving that goal. The aid is from my people to yours.

If I give money to a homeless person, I am not asking for them to account for how they spend it. I am giving it to them because they need it. Maybe it goes on food or shelter, or maybe some momentary pleasure or relief from misery. If I give money, however, to a charity, I absolutely want them to account for how they spend it, because they are the means to the end and not the end in itself.

COVID-19 has brought the “future of aid” question into stark relief. We need better, more respectful mechanisms for delivering even more aid from more affluent countries to less affluent countries. The aid needs to come with strong accountability mechanisms to ensure that benefits are distributed according to an inverse power-law: the least powerful and the least affluent first. Aid, of all things, should not trickle down. When it does, governments on both sides of the aid-exchange should be held to account, by your people and mine.

What is the optimal number of broken jaws?

I was chatting with a friend recently about the COVID-19 response in different countries. Reflecting on her own country, she said, “It is so hard to know what is right!”; that is, it is so hard to know what the right response to COVID-19 should be.

The variation, for instance, in countries’ lockdown responses is substantial, but which country is doing the right thing? In some countries, there has been no lockdown. The government asked the people to be sensible. In other countries, the government legally confined people to their homes — only one person was allowed out at very specific (restricted) times to buy essentials. Given these two policy extremes (be sensible and house arrest), which one is the right one, and how do you know?

An economist, I have forgotten who once asked tongue-in-cheek, what is the optimal number of dead babies? The very purpose of such a crass question is to make you stop and think. What tradeoffs are you prepared to make to save the lives of babies? Sure, you could be lazy, condemn the questioner as immoral (for even asking you to think), and declare zero dead babies to be the right number. As a simple policy proposition, if zero dead babies is the right number, then all the resources of society should be aimed at preventing neonatal deaths. ALL RESOURCES! Until the policy goal has been achieved, there is more work to be done to reduce the number. One dead baby is too many!!! Farmers may farm, but only to produce the food that supports the workforce that is striving to reduce baby deaths to zero. Teachers may teach, but only to educate the people to fill the jobs to support the policy goal to reduce baby deaths to zero. There is very limited use for art, music, cinema, sport, fashion, restaurants, etc. They will all have to go! If five-year-old deaths increase, that is something to live with, just as long as we can save another baby.

At this point, you’re probably thinking, well that’s stupid. That’s not what I meant when I said the optimal number of dead babies is zero. What I meant was something more along the lines of, “In an ideal world there would be zero dead babies”. Equally, if you were asked about poverty or crime, or amazing works of art, you presumably would have stated the ideals in terms of zero poverty, zero crime, and lots more wonderful art. And this is quite a different proposition. An ideal world is not ideal in virtue of its achievement of a single goal. It is ideal in having achieved all sorts of different outcomes. And that is why the real and the ideal do not intersect. In the real world, we do not achieve the ideal anything. We seek to achieve many ideals, and realistically, we hope to make progress against them, knowing that there is always more to be done. In striving to improve the societal position against a basket of goals, we allocate limited resources and make trade-offs.

This is one part of the COVID-19 problem, and, as my friend observed, why it is so hard to know what is right. What is the right number of COVID-19 deaths? There are lots of important, rational debates to be had around this topic because it is about the tradeoffs we are prepared to make against a basket of societal goals against the myopic achievement of one. Muscular public health responses — effective house arrest — are very good at reducing the number of new COVID-19 cases. They are also very effective at increasing domestic violence, increasing depression, lowering child immunisation rates, degrading child education, increasing poverty and increasing unemployment. If the societal goal should be zero COVID-19 deaths, what is the optimal number of broken jaws, suicide attempts, measles encephalitis cases, illiterate and enumerate children, beggars, and soup kitchens?

All these issues, under normal circumstances, are things of concern to Public Health and maybe, one day, they will be again.

Another part of the COVID-19 problem is that, whether a government “did the right thing” will be determined in hindsight, and by making (inadequate) historical comparisons between the outcomes across countries’. In democracies, at least in the short-term, “did the government do the right thing?” will often be decided at the ballot box. This will surely get the answer wrong. In less-than-democracies, astute rulers will write the history books themselves ensuring that, without regard to the outcome, the government did the right thing.

One of the main reasons that “it is so hard to know what is right!” is that we rarely have a societal view about the long term goals we wish to achieve and the tradeoffs we are prepared to make. Furthermore, we are reluctant to accept the fact that one can do the right thing and still fail. We assume that the right course of action will, by definition, result in success. We are prospective Kantians and retrospective Utilitarians.