Follow-up on deceased organ donation

Earlier this month, I wrote a post trying to estimate how many people would have to be added to the deceased organ donor registry to result in one additional actual organ donor. I concluded that "the impact of adding a million people to the organ donor registry is likely less than 10 additional deceased organ donors per year, and probably quite a bit smaller than that." Looking at the entire lifetime, rather than one year, I calculated that adding a million people to the donor registry would lead to less than 1,400 extra organ donors.

I pointed to one previous estimate of that value that predicted a figure about 3 times higher, but I hadn't been able to find any other estimates.

During the past month, I looked into extending my estimates to try to calculate the cost-effectiveness of organ donation outreach efforts, and a couple of the papers I looked at calculated estimates for the value I was trying to estimate earlier this month.

Howard and Byrne 2007 try to calculate the value of a marginal registrant, assuming that registrants have differential donation value based on their mortality risk. (The idea driving this is that registered organ donors appear to die substantially less often in ways that allow their organs to be donated than non-registered folks do, suggesting that those people who are easiest to register may not be those who matter the most as donors.) They do a fairly sophisticated calculation that incorporates this issue, registrant age, discounting for future benefits, and family consent policies. They find:

When families are given the right to refuse organ recovery from a registered donor, society should be willing to spend up to $840 for an 18- to 34-year-old registrant [based on a value of $1,087,000 per an organ donor, assuming QALYs are worth $100k]. Because some families will always refuse consent and others will always grant it, the value of registering a donor under a regime of familial consent is always lower than the value under a regime of first-person consent.

Doing the quick division, this means that they estimate the lifetime increase in donations (having already discounted future benefits) for a million marginal 18-34 year old registrants as 840/1.087 = 773 additional donors. That's about half my lifetime estimate from the previous post (1,400), which can mostly be explained by the fact that it's discounted to present value (discounting 30 years at 3% removes 60%). Actually, the authors use an extremely high assessment of the increase in family consent rates associated with registration (45%) based on the Simonoff 2001 paper I discussed in my previous post. They report an alternative scenario as well:

Estimates of the value of registrants under a familial consent regime are probably biased upward. Positive attitudes toward donation are shared by registrants and their families. Thus, the figures from Siminoff and Lawrence almost certainly overstate the impact of registration on consent rates. If registration increases familial consent rates by only 20 percentage points, instead of 45 percentage points, then the value of a registrant is $373. [My estimate in the previous post was based on an increase of 31 percentage points, in between the 20 and 45 estimates.]

This $373 estimate translates to 343 additional discounted lifetime donors per million additional marginal registrants.

Beasley et al. 1999 tried to estimate what the likely returns to an organ donor registry would be, prior to widespread adoption, and turns out to be weirdly prescient, despite a rather crude approach. Reporting that 70% of people claimed to want to donate their organs in a survey, whereas families only consented to donation in 50% of cases, the authors predicted that a registry could lead to to one additional donor for every 5 potential organ donors (i.e. people who die in appropriate ways) registered (70%-50% = 20%). Multiplying out the annual number of potential organ donors, they estimate that:

The number of registrants needed to attain one extra donor above what would have occurred without a registry is given by: 16,700/.2 = 83,300. If we take into account that portion of the donor pool composed of cases that are not identified by hospital staff or not approached for donation by hospital staff (some 27% of the total pool) the number of registrants needed for each additional donor at the margin would be 83,300/.73 or 114,200.

One additional donor per ~100k additional registrants (per year) is just about 10 donors per million additional registrants, which is about what my algebra in the last post suggested as an upper limit for the yield on registrations.

Cost-effectiveness

In the conclusion of my previous post, I tentatively suggested that the organ donation community wasn't spending its money and energy as well as it could be:

My vague impression is that the organ donation community spends a huge amount of its time, money, and effort trying to get more people to sign up as organ donors. My analysis here has been tentative and exploratory, but I think the lack of stronger empirical evidence for an actual impact of those registries is startling, and that it would be worth a fuller exploration by people with more subject-matter expertise than I have. Given that there are a lot of other strategies for improving organ donation, continuing to focus on donor registration outreach seems like it may be leaving a lot on the table.

The key assumption there was that the organ donation community was spending most of its money on trying to get people to register as organ donors. But a quick search reveals shockingly little support for that hypothesis. The total spending by Donate Life America, the big national organ donation promotion organization, was only $1.3 million in 2011 (PDF). Donate Life California spent ~$500k in 2011 (PDF). By contrast, the New York Organ Donor Network spent ~$30 million (PDF), but they're the organ procurement organization, which means that they do all the work around actually harvesting, transporting, etc. organs, in addition to trying to get people to register as donors.

At a budget $1.3 million/year, if Donate Life America is getting one additional donor a year, they're basically breaking even in terms of social impact.

One other interesting datapoint that I heard from a friend this morning is that some folks in the organ donor community--well, one acquaintance of his--estimates that the cost of registering a donor is typically $1.75. At that price, organ donor registries seem like quite a good buy. An organ donor is worth ~$1 million, and we estimated the upper limit on the yield per million additional registered donors as 10 donors/year (though probably substantially fewer). Still, at that cost, we're talking about a potentially really high return on investment, and all the action in trying to assess that ROI will be in the region of impact that's too small for the statistical techniques I used in the last post to yield anything of interest.

For instance, say we're considering whether to significantly increase funding or all the "Donate Life" organizations, which (by hypothesis) spend ~$2 per marginal registrant (and don't face radically increasing costs as they try to get more registrants). That $2 is (pretty easily) worth it as long as one in every ten million marginal registrants end up donating each year. Since the statistical techniques I described in the last post could really only detect a bump of four donors per million additional registrants (or 40 per 10 million), we're massively under-powered to detect the kind of effect size that would matter for practical considerations.

The fact that it only costs <$2 per registrant and folks aren't up in arms over the lack of funding for organizations working to improve registration actually suggests to me that the marginal impact per registrant is probably at the lower end of this spectrum (i.e. closer to 1 additional organ donor per 10 million registrants per year, for the general reasons laid out in this GiveWell blog post), though I don't have the data to really answer that question.

Facebook and organ donation: does getting people to sign up as organ donors actually make a difference?

A year ago today, Facebook launched a PR blitz announcing that they were making it possible to publicize your organ donation status on Facebook. Donate Life America, the national coalition of state organ donation organizations, made it pretty clear that Facebook was launching a concerted PR effort:

In an exclusive interview with ABC News’ Robin Roberts on “Good Morning America” this morning, Facebook founder Mark Zuckerberg announced the new initiative and became one of the first Facebook users to sign up to donate on the social network. Tonight Facebook’s COO and one of America’s most powerful businesswomen Sheryl Sandberg will sit down exclusively with Diane Sawyer. ABC News affiliates, ABC News Radio, Yahoo! and ABCNews.com will also feature coverage of the exciting new initiative. ESPN will be running two powerful organ donation stories throughout the day on SportsCenter that encourage organ and tissue donation.

Aaron Swartz, Kieran Healy and I had a short discussion about the prospects for an effect on Twitter. I was optimistic, but Healy said, "public support for donation hasn't been the bottleneck for a very long time."

The best data I've found on the impact of Facebook's efforts comes from a CBC news article from June 25th:

The Facebook drive had an immediate effect. In the first few days, more than 100,000 people changed their status to indicate they were willing to be donors.

To be official, a willing organ donor needs to sign up with their own government's donor program, so Facebook also provided direct links to local donor registries. Donate Life America (DLA), a non-profit dedicated to raising donor awareness, subsequently reported an average 1,000-per-cent increase in online donor registrations across the U.S. over the six days following the addition of the donor feature.

To put that in concrete terms, during that time period 33,406 people legally registered to be donors; other years the number of signups was much smaller – in 2011 for the same time period, it was 3,288. Since then, the signup numbers have stabilized at around 1,150 per day, which was still more than double the historical daily average of 548.

So the Facebook effort led some people who probably weren't on the list before to sign up as organ donors - if the pace kept up, which seems unlikely, Facebook would have led to around 200,000 additional people joining the list this year. But does getting people who aren't yet donors to sign up make much of a difference?

This may seem like a crazy question, so some context is useful: if you die in a way such that your organs could be donated, even if you're not a registered donor, the "organ procurement organization" (OPO) will still ask your family if they'd like your organs to be donated. The OPOs are good (and getting better) at asking, and most people do support organ donation, so they actually get permission from most families that are asked when their loved one wasn't a registered donor. Conversely, in most cases, even if you're registered as a donor, if your family says they don't want your organs donated, the OPO will follow their wishes (this happens pretty rarely, luckily). So, structurally, it's actually not all that obvious that getting more people to sign up as organ donors will make much of a difference, especially if the people you get to sign up are the ones whose families already support donation (which seems pretty likely).

The ideal way to address this problem would be to run a large randomized control trial: spend a bunch of money on randomly giving specific people messages about registering as donors, hopefully getting a big difference in registration rates between treatment and control groups, and then follow them for years to see if the treatment group actually ends up donating at higher rates. This would be expensive, time consuming, and difficult, and to my knowledge it's never been tried.

There are two easier strategies we can pursue to try to identify the impact of marginal donor registry sign-ups, though they won't have the same rigor as the RCT:

1. compare the actual donation rates of people who are eligible to donate upon death who are and aren't registered donors
2. comparing states based on the proportion of registered people and the proportion of eligible donors who actually donate. The data and code for my analysis are on github here and figshare here. Feel free to email me if you need any help interpreting; I didn't do much to document well.

Neither of these strategies is as good as an RCT, and in particular both will be unable to distinguish whether registration efforts are just leading more people who would donate anyway to sign up on the list, but they still provide useful information.