No natural predators

Integrity and ethics in the AI windfall

“Spoiled dogs, sleeping belly-up. No natural predators.”

So says Oliver Quick in Saltburn, describing the wealthy family he has infiltrated. I re-watched it recently and this line made me think of Effective Altruism.

I don’t mean that people in EA are spoiled. I do think, though, that EA’s community norms can lull us into a false sense of comfort, and that this makes it easy for us to lose our way, both through bad decisions and by being manipulated.

I believe incredibly strongly that EA has no particular resistance to the dangers associated with money, status and power. In fact, I think it is unusually vulnerable to them.

These risks will be much more acute during the AI windfall. I think we should take steps now to mitigate them.

What happened last time there was a lot of money?

Not going to lie, it wasn’t perfect.

Despite several sincere attempts to get out ahead of it, the EA community took enormous damage, a lot of it self-inflicted.

Among other things, we granted a lot of money that was ultimately clawed back from the recipients; bought a historic abbey, barely used it and later sold it at an enormous loss; and also put someone on a pedestal who turned out to be one of the largest fraudsters of all time (you have probably heard about this one).

There was also other bad behaviour happening at the same time, including serious sexual misconduct issues, which was not obviously linked to the money, but came partly from the same gaps in governance.

The effects of this are still being felt today.

Doesn’t Effective Altruism have antibodies against this stuff?

I don’t think so. This view is partly formed from our track record (above) and partly from reasoning (below).

We tend to assume people in EA are impact-oriented, high integrity and both truthful and truth-seeking. Many people in EA have made real sacrifices in service of impact, like giving away a large share of their income.

Assuming this about people we don’t know makes us vulnerable to manipulation. An Oliver Quick or Elizabeth Holmes would find EAs to be easy prey. This time, the amount at stake is two entire orders of magnitude greater than before, so we should expect more and more serious attempts at infiltration.

Even if you aren’t concerned at all about malignant external actors, there are many other ways that smart EAs have failed at governance historically. These are easier to mitigate, because we can choose to change our own behaviours more easily than we can gatekeep the community.

The community has also historically given very smart young people a lot of trust and responsibility. This has produced a lot of notable successes, but it should make us more concerned about things like governance, risk management, personal conduct around power, and leadership generally, which inexperienced people are typically worse at.

Finally, because EA is relatively small and has some quirky values, we encounter more conflicts of interest than many movements. If you prefer to appoint people you trust personally and who share EA values to positions of responsibility, they will be conflicted more often than in bigger communities.

Last time, we did not do well on conflicts of interest.

Status-seeking is a bigger risk than greed

As George Rosenfeld wrote in 2022,

“Before the increase in funding, it didn’t pay to be EA. In fact, it was rather costly: financially costly because it usually involved a commitment to give away a lot of one’s resources, and socially costly because most people have an intuitive aversion to EA principles.”

Although I don’t think we are immune to the dangers of money, I agree that the community is probably less motivated by personal financial gain than most.

Instead, EAs usually translate money into its potential impact - and, while EAs are not often money-hungry, they are extremely impact-hungry. Increased impact also improves your status in the community, which makes it even more attractive to control more cash. It’s obvious, for example, that the head of a $1bn foundation is higher status than the head of a $100m foundation.

Money also increases your power. Once you have money to distribute, whether that is as a donor, a grantmaker or someone who can hire folks, people will treat you differently. It can be very easy to enjoy your success, without realising that some part of your nicer life comes from people no longer challenging you the way they once might have.

I have felt these things acutely as Ultra Philanthropy has moved more money.

I’m not trying to imply that I have been engaged in sketchy behaviour up to now (for the record, I would deny this!). I am simply saying that it would be extreme hubris for anyone to think that they will be totally unaffected by the chance to gain money, status or power.

ElliotTep has a good list of vanilla, everyday opportunities to be a bit low integrity. I would add to these things I have experienced as an advisor, like:

• Recommending your own work or that of your friends to donors, without truly steelmanning whether it is the best donation option;

• Not disagreeing with something false, because the person saying it is a donor;

• Not writing or saying things that might annoy people with more power and status than you, especially if you might later benefit from their good opinion.

None of these is without nuance. I am simply trying to demonstrate that we should all be on our guard.

I’m not the first person to comment on this, but previous contributions mostly concern external perception, or fundraising behaviour around Anthropic staff. I am concerned with the personal conduct of those already in EA, starting with myself. I am also focusing on governance, rather than just etiquette.

Whose responsibility is this?

Well everyone’s, obviously. However, more for some of us than others.

At EA Global: NYC, I looked around the room and realised with a jolt that my cohort are now the elders in EA. This is quite funny but also scary, because of course we just feel like we always have.

However, I’ve been in the community since about 2011 (GWWC pledge 342. Flex.) If you are my age or older (38), or have been in the community for more than a decade, you are now one of the most important people in EA, even if you don’t feel like one.

Most likely, nobody more senior is coming in above us, because we haven’t historically hired a lot of senior people from outside, and they don’t necessarily want to work here.

Given that we were all here last time it went wrong, and that a significant proportion of the community wasn’t, it’s our job to try to make it go better this time.

It’s our job even if it feels deeply uncomfortable and unreasonably arrogant to think of ourselves as leaders, and even if we don’t want to do it.

I particularly recommend taking this seriously if you expect to accrue significant power or status in the upcoming windfall. You will need to work incredibly hard to continue receiving feedback and to stay on track, and that work needs to start now.

So what am I doing about it?

Ultra Philanthropy runs a fund and advises donors on where to give, which is exactly the kind of conflict I have been describing, and means our work will probably be directly affected by the AI windfall. I don’t want to convince myself that I am somehow too saintly for this to be dangerous.

I am therefore taking some proactive steps, which I list here because others might want to do the same.

First, I have started the process to register Ultra Philanthropy as a 501(c)(3) (it is currently fiscally sponsored). This will mean that I answer directly to a Board, have my pay set by that Board, and do public filings specific to Ultra Philanthropy each year. None of these is foolproof, but they make it harder to do sketchy things without scrutiny.

Of course, a founder-selected Board is only a partial defence. I have a cordial relationship with everyone I am approaching, and some are close friends.

Accordingly, I am deliberately selecting Board members with a specific brief to be ethical watchdogs within the organisation. I have approached people I believe to be unusually high integrity, and unusually willing to challenge or confront me if necessary.

Second, I am asking some people to become personal ethics advisors to me, completely separate from the Board. This is because I think the bar for someone to approach a Board member with feedback is quite high. I think that external ethics advisors might be able to receive more feedback, including “epistemically rigorous gossip”, than charity directors. They might also be perceived as more impartial.

I am delighted that Sjir Hoeijmakers, CEO of Giving What We Can, has already agreed to play this role. Sjir is one of the highest-integrity people I know, already regularly gives me candid feedback and is well placed to play this role within the effective giving space. If you have concerns about Ultra or me, you can reach out to him.

Someone at Coefficient Giving once told me that your five best friends are the five people who will tell you straight if they think you are doing something wrong. I am trying to institutionalise that advice.

Third, I am going to ask the Board to write and publish a conflict of interest policy, a whistleblowing policy and a compensation policy as a first order of business. Again, these are not foolproof defences, but they are important nudges in the right direction.

This will without doubt impose costs on Ultra, and would cost the EA community if implemented at scale.

It will be harder to get things done at maximum speed and more time will be spent securing sign-offs.

However, we tried doing grantmaking at breakneck speed once before and it ultimately went badly. For example, it was harder for charities to protect money that they had been granted from clawbacks because payments were sometimes made via non-charitable subsidiaries in the interests of speed.

Either way, we can mitigate the friction of more rigorous processes by hiring more staff. If we spend some of the extraordinary sums coming our way on compliance, finance, operations and legal staff, not to mention Managing Directors and Chiefs of Staff, we can have better governance and compliance without slowing to a crawl.

What am I calling on others to do?

I understand that people are very short on time, particularly right now. However, I hope that this piece can kick off a public discussion of these issues.

I am particularly keen to see responses and public commitments from other senior folks in the community. You might not think of yourself as an elder, but you might be one anyway, and making public commitments before the money lands gives everyone something to hold you to.

If you read this and thought “what does he mean, “last time”?”, start here and here.

If you run an EA organisation, please publish a conflict of interest, compensation and whistleblowing policy as soon as possible. This goes for the small orgs as much as the big ones - you never know when one of these policies might be needed.

If you are about to make a life-changing amount of money, I gave some advice here. I would particularly recommend asking some close friends to keep you on track, and committing to some giving targets.

Jack Lewars is the founder of Ultra Philanthropy, an independent advisory that helps major donors give for maximum impact, and is the fund manager of its Mid-Stage Global Health Fund. He advises donors giving up to nine figures a year, and is Chair of Trustees at High Impact Athletes. Talk to him about your giving.

I used Claude to help structure my thoughts and to suggest improvements and flag gaps, as well as for proofreading; all views, primary drafting and final edits are mine.