Rethink Charity

SHIC Workshop Experiment and Revised Impact Strategy 2018

2017-10-31 General Operations

By Tee Barnett and Baxter Bullock

This post details our shift in priorities for the Students for High-Impact Charity (SHIC) program over time, and briefly outlines the revised methods of delivering this approach. We conclude the article by announcing a new SHIC workshop experiment slated to launch in early 2018.

Summary & Sections

I – Background

II – Initial Strategy

III – SHIC Workshop Experiment

IV – Our Revised Approach

I – Background

Shortly after the release of our interim report in early 2017, SHIC became a project under the Rethink Charity umbrella. The organizational reshuffling and insight from 2016 prompted us to reconsider our program objectives and outreach methods for making the largest impact with students.  

 

The program’s overall objectives are to inform students about the greatest problems facing humanity, equip them with the cognitive tools for thinking about how to address these problems, and then suggest possible solutions and avenues for taking effective action.

II – Initial Strategy

Our initial outreach strategy prioritized making SHIC accessible, hoping to reach the largest number of students possible in order to produce broad-scale shifts in perspective and behavior (e.g. survey results across all participants), and perhaps even a significant inflection point for a small fraction. Since mid-2016, we’ve relied solely on volunteer student leaders and teachers to run the program with student participants. The interim report expands on how we found this implementation model rather unstable. We found difficulties in data collection most problematic.

 

Gathering feedback on the curriculum, testing program efficacy, and achieving scale, were largely rolled into a single process. By running the introductory program with as many students as possible, we expected to collect actionable data while simultaneously scaling our presence around the world.

 

SHIC confronted several practical questions that affected how we interpreted our success metrics. As an example, we’d initially intended to inspire dozens of SHIC groups around the world to fundraise for effective charity as a broad indicator of program effectiveness. We reasoned that fundraising participation and performance could serve as a proxy for student engagement. Fundraising also served to hedge uncertainty regarding the overall value of SHIC by prompting donations to high-impact charities, possibly enough to offset the cost of the project. After observing very inconsistent and underwhelming student fundraising numbers, it became clear that using dollars donated as a central goal of program success warranted revisiting.

 

More importantly, we questioned the underlying assumptions of our success metrics. As a program dedicated to helping students achieve the largest prosocial impact, it wasn’t clear that high fundraising numbers or broad shifts in survey results would necessarily be indicative of lasting impact. We increasingly felt that inspiring meaningful value shifts and shaping long-term plans for a smaller proportion of participants would be more impactful.

 

For instance, we had serious reservations that program objectives appeared to privilege present-day action over longer-term skill building. Our previous model did not consider the opportunity-cost of fundraising comparatively nominal amounts for charity, versus skill building in order to have a larger impact in the future. In fact, optimizing SHIC too much in either direction (towards long-term career building or towards short-term action) could be detrimental. Put too much present-day focus in the program, and students likely achieve relatively little good and neglect skill building for the future. Put too much emphasis in skill building for the future, and risk squelching a natural passion for helping others.

 

III – SHIC Workshop Experiment

After reevaluating the program as a whole, we’ve decided on an implementation model that better reflects our revised objectives. By early 2018, SHIC will conduct the most ambitious experiment on the program to date, training in-house instructors to bring SHIC workshops to schools across Vancouver.

 

We gauge interest in the program by opening with mass Giving Game events conducted by SHIC. Provided that a large enough group of students are interested in moving on to the introductory program in a given school, SHIC instructors will return to conduct the full workshop in two additional 1.5-hour installments.

 

An additional component to this experiment will involve collecting longitudinal data on the medium- to long-term effects of our program on student giving behavior. Thanks to our collaboration with Charitable Impact Foundation (CHIMP), a Vancouver-based donation platform that uses a donor-advised fund to facilitate gifts to other registered charities.

 

We will be able to track the giving behavior of workshop participants by individually assigning online Chimp accounts. On a monthly basis, each participant account will be credited with a set amount of money that can be disbursed to any charity in Canada, and also effective charities across the border thanks to our partnership with the Priority Foundation. We hypothesize the SHIC program will at least moderately influence the giving preferences of students.

 

In addition to the longitudinal giving data, surveys administered throughout the subsequent year, qualitative interviews, and classroom-level scouting will help SHIC identify a select cohort of high-potential students eligible for additional programming and mentorship opportunities.

IV – Our Revised Approach

Compared to the inconsistency we experienced with a volunteer implementation model, an instructor approach allows for more rapid feedback and adjustment, reliable data collection, tighter quality control over messaging, and may motivate students to become generally more engaged. By late 2018, we expect to have several vantage points from which to assess the impact of SHIC.

 

More importantly, the instructor model is our best attempt at taking a more targeted approach at influencing students. SHIC will remain committed to keeping our message accessible – students around the world can still download our entire curriculum for free and run their own student clubs, for example – but the majority of our resources and effort will pursue meaningful impact on an individual level. We’ve updated our program to achieve this in the following ways:

 

• Using data to identify high-potential students –  SHIC will collect a variety of metrics to find students most inclined to engage with effective charity in the long-term. Primarily through periodic surveying, qualitative interviews and classroom-level scouting, we will identify a select cohort of students eligible for additional programming, mentorship, and career opportunities.

• Action oriented toward future impact – Our program incorporates more informed insight on balancing long-term skill-building and present-day action. As an example, rather than exclusively recommending raising money for effective charities, we may instead attempt to connect students with high-impact internship opportunities. This new approach empowers students in the present day, facilitates skill-building, and helps build experience and career capital that could reap long-term benefits. The career path carved out by Owen Shen provides a real-world example of this approach. Owen became involved with the rationalism and effective altruism community at the age of 16, eventually volunteering with the SHIC curriculum and outreach teams for a number of months. Owen created his own rationality-focused blog, and subsequently earned a contractor position with the Center for Applied Rationality (CFAR). Our overarching objective is to orient students toward a future with the highest potential for prosocial impact in a way similar to Owen.

• Optimizing curriculum for critical thinking – SHIC goes far beyond philanthropic education. Our application of logic, ethics, epistemology, and metacognition to complex social problems provide transferable skills students can utilize in other educational domains. Students and teachers will explore the scientific method, thematic learning, lateral thinking, and the formulation of an alternative outlook on conventional problem solving. See ‘Level 6 – Cognitive Quirks’ for the sort of thematic principles we will incorporate more of moving forward.

 

More information on the SHIC workshop experiment will be made public in the coming months. You can get notified of the latest developments with this specific project by signing up here.

Credits

Post written by Tee Barnett and Baxter Bullock, with invaluable edits and input from Peter Hurford, David Moss and Catherine Low. A special thanks to 80,000 Hours for publishing their process over the years and communicating the importance of long-term plan changes, and to CHIMP for their inspiration and technical support. We’re happy to discuss this post further in the comments section. You can email Tee at [email protected], and Baxter at [email protected].

EA Survey 2017 Series Part 7: Have EA Priorities Changed Over Time?

2017-10-07 General

By Peter Hurford and Tee Barnett

This is the seventh article in the EA Survey 2017 Series. You can find supporting documents at the bottom of this post, including our prior EA surveys, and an up-to-date list of articles in the series. 

Summary

• We use past survey data to shed light on community shifts in cause area preferences over time.
• Our evidence suggests that EAs are becoming more favorable toward AI and less favorable toward politics.
• EAs in both the 2015 and 2017 surveys shifted away from viewing poverty as a “top” or “near top” cause.
• Newcomers in the 2015 survey were less accepting of global poverty than veterans. However, the reverse was true in the 2017 survey, with newcomers being more accepting of global poverty than veterans.
• There is no indication that EAs are getting less interested in animal welfare with time.

Cause Preference Shifts

Our previous posts in this series were largely descriptive, often reporting on 2015 and 2016 to provide an approximate snapshot of the current EA community. As the series progresses into late 2017, we’ll look to extract further insight from the data, which will include various longitudinal analyses, commentary on the Pledge, and potentially other angles upon request.

 

We turn first to a commonly held narrative within the community – that new EAs are typically attracted to poverty relief as a top cause initially, but subsequently branch out after exploring other EA cause areas. An extension of this line of thinking credits increased familiarity with EA for making AI more palatable as a cause area. In other words, the top of the EA outreach funnel is most relatable to newcomers (poverty), while cause areas toward the bottom of the funnel (AI) seem more appealing with time and further exposure. (For example, see Michael Plant’s post “The marketing gap and a plea for moral inclusivity”.) While we previously reported higher support for global poverty as a top cause, we find reason to support some version of a narrative suggesting that EAs are shifting away from global poverty.

 

There are two ways we’ve looked at changes in preference toward causes over time. First, we took the information on what year EAs joined the community, and compared the cause preferences of earlier EAs to newcomers. Our second method involved taking the population of EAs who took the EA Survey in both 2015 and 2017 and seeing how the same people changed their opinions of their  top cause over this two year gap. The first method has a larger sample size, while the second version captures intrapersonal attitude shifts over time. Both tell a similar tale.

 

Using the longitudinal method, there were 184 people who took both the 2015 and 2017 EA Surveys that we could match (using a hashed email address to preserve anonymity). To get a quick overview of cause preference change over time, we looked at the number of people who shifted toward a cause (they previously had not considered the cause to be a “top priority” or “near the top priority” in 2015, but now do as of 2017) and subtracted the number of people who shifted away from a cause (they previously had considered the cause to be “top” or “near top” and now don’t). This gave us a number we called a “net shift” from a cause.

 

Cause area preferences fluctuated slightly between the 2015 and 2017 EA surveys (Table 1). Poverty remains the clear community favorite, although the net shift in preference broken down by cause area reveals that interest has been waning in poverty since the 2015 EA survey, with a net shift of -8. Interestingly, politics has hemorrhaged the most interest (-13) in the wake of Brexit, Trump’s victory, and other significant political developments in traditional EA hubs. The biggest winner in net gains is AI (+29) and non-AI far future (+14), which suggests at least some directional movement toward long-term concerns over time.

We were compelled to take a closer look at the dropping interest in poverty, particularly due to its continued popularity in the aggregate and traditional status as an EA mainstay. Between the 2015 and the 2017 surveys, 14.13% of EAs in the longitudinal sample changed their mind about how much importance should be placed on the cause (Table 2), with 9.24% of these EAs no longer considering poverty as a “top” or “near top” cause, and 4.89% of EAs upgrading their estimation of poverty’s importance.

However, there has been more movement within the distinction between “top” and “near top”, with 19.02% of EAs in the longitudinal sample relegating poverty from being the top cause two years later and only 5.98% of EAs upgrading their estimation of poverty as the most important cause area (Table 3).

To look at this from another perspective, we took the 2017 EA Survey population and distinguished between whether an EA was more of a “veteran” who learned about EA in 2013 or earlier or was more of a relative newcomer who learned about EA in 2014 or later[1]. The hypothesis is that veteran EAs would have had more time to shift their beliefs in causes and may be predictive of how newcomers will eventually shift.

 

Taking initial preferences into consideration, EAs who joined in 2013 or earlier were far less likely to rank poverty as the “top” or “near top” priority than EAs who joined in 2014 or later (Table 4), though a majority of these veteran EAs still ranked poverty as the “top” or “near top” cause.

One potential explanation for this shift might not be a genuine change in opinion over time, but instead that veteran EAs were always less likely to be into poverty, whereas newer EAs are a lot more likely to be into poverty. To check our base assumption about whether there has been a significant influx of poverty-focused EAs in recent years, we looked back at the 2015 EA Survey and compared it to the 2017 EA Survey (Table 5).

As of the 2015 Survey, newcomers were actually relatively less accepting of global poverty than the veterans, but this effect reverses as of the 2017 EA Survey. This could point to a difference in attitudes for newcomers in 2015 and 2017 skewing the data, rather than newcomers from 2015 changing their minds over time.

 

The data is not entirely clear on whether initially interested EAs change their views away from poverty with time. The perceived separation between veteran EAs being less poverty-focused may be down to initial dispositions, rather than later conversions. The 2017 EA survey data does suggest that most newcomers enter the movement interested in poverty, which may have implications for movement building organizations to bear in mind.

Attitudes Toward AI

Turning to AI, not only has resistance to devoting resources to AI safety reduced substantially since the 2015 EA Survey, but we showed that this set of concerns is now actively competing with other cause areas for top priority billing.

There were more people changing their minds on AI than global poverty (Table 6), with 19.57% of EAs in our longitudinal sample choosing to upgrade the importance of AI in their view to a “top” or “near top” cause and only 3.8% of EAs choosing to downgrade it out of “top” and “near top”. When looking at just top cause area preference, the trends were roughly similar, with 13.04% of EAs in the longitudinal sample promoting AI to the top cause and 7.07% demoting AI from top cause to something else.

Among those veteran EAs who joined in 2013 or earlier, the support for AI as a “top” or “near top” priority was closer to 50-50, whereas for EAs who joined in 2014 or later, there is less support for AI as a “top” or “near top” cause (Table 7). The net shift of aggregate interest toward AI (Table 1), a broad trend favoring AI (Table 6), combined with our knowledge that newer EAs favor AI relatively less (Table 7), would seem to suggest that more exposure to EA increases the likelihood of becoming more inclined to support AI safety over time.

Attitudes Toward Animal Welfare

We were also curious to check the same for animal rights, to see how EA interest in helping animals as a cause has changed over the years.

Here we see that among the 2017 EA Survey respondents, unlike with AI, there is no statistically significant difference between the rate at which newcomers and veterans support animal rights (Table 9). Furthermore, there has been a net shift toward animal welfare among those who took both the 2015 and 2017 EA Surveys (Table 8). Thus, suggestions that EAs are getting less interested in animal welfare over time does not seem to be confirmed by EA Survey data.

Among the 2017 EA Survey respondents, newcomers to EA are relatively more likely to support politics than veterans, though the majority of both newcomers and veterans do not support politics as a “top” or “near top” cause (Table 11). Similarly, among those who took both the 2015 and 2017 EA Surveys, people are shifting away from thinking of politics as a “top” or “near top” cause (Table 10). This may mean that while politics is less popular as an EA cause overall, EAs tend to shift away from it over time. Likewise, it is interesting that it seems like contentious developments of late may have not had any sort of energizing effect on getting EAs interested in politics, as far as we can tell in this survey data.

Endnotes

[1]: This effect is statistically significant at p < 0.00001 for both. We chose 2013 because we felt it properly conveyed “veteran” status before a lot of popular growth in EA in 2014, but this effect remains the same in direction and statistical significance, with similar strength, regardless of your choice for cut-off year (tested with 2011, 2012, 2013, 2014, and 2015 as cut-off years).

Credits

Post written by Peter Hurford and Tee Barnett

 

The annual EA Survey is a volunteer-led project of Rethink Charity that has become a benchmark for better understanding the EA community. A special thanks to Ellen McGeoch, Peter Hurford, and Tom Ash for leading and coordinating the 2017 EA Survey. Additional acknowledgements include: Michael Sadowsky and Gina Stuessy for their contribution to the construction and distribution of the survey, Peter Hurford and Michael Sadowsky for conducting the data analysis, and our volunteers who assisted with beta testing and reporting: Heather Adams, Mario Beraha, Jackie Burhans, and Nick Yeretsian.

 

Thanks once again to Ellen McGeoch for her presentation of the 2017 EA Survey results at EA Global San Francisco.

 

We would also like to express our appreciation to the Centre for Effective Altruism, Scott Alexander via SlateStarCodex, 80,000 Hours, EA London, and Animal Charity Evaluators for their assistance in distributing the survey. Thanks also to everyone who took and shared the survey.

Supporting Documents

EA Survey 2017 Series Articles

I – Distribution and Analysis Methodology

II – Community Demographics & Beliefs

III – Cause Area Preferences

IV – Donation Data

V – Demographics II

VI – Qualitative Comments Summary

VII – Have EA Priorities Changed Over Time?

Please note: this section will be continually updated as new posts are published. All 2017 EA Survey posts will be compiled into a single report at the end of this publishing cycle.

Prior EA Surveys conducted by Rethink Charity (formerly .impact)

The 2015 Survey of Effective Altruists: Results and Analysis

The 2014 Survey of Effective Altruists: Results and Analysis

Monthly Donor Briefing: 09/2017

2017-10-06 General Operations

Thanks for taking the time. These monthly donor briefings are designed to be quick reads that keep you in the know about Rethink Charity. More detailed updates will come in bimonthly/quarterly intervals.

 

Listen to three-minute audio version of this briefing here.

 

Two-sentence RC breakdown – RC is now maturing into a period of evaluation and proactivity since the consolidation of projects announced in May. Key highlights here include plans for a large-scale SHIC workshop experiment, the ongoing LEAN assessment, our release of the 2017 EA Survey, and major developments on regranting avenues in the UK and Canada.

 

---

Your full RC Briefing

Read time: 3 minutes

Project Updates

 

Rethink Charity

We’re taking a more focused approach to our projects since the rebranding in May. This includes throttling down on .impact (independent EA projects) and online peer-to-peer fundraisers, and reassessing the value our major programs (e.g. LEAN assessment). We’re currently prioritizing SHIC and international regranting efforts as the highest-impact use of our time.

 

–  We initiated the EA Survey 2017 Series reporting on 2015 and 2016 survey data with what we perceive as strong interest from the community.

– RC jointly established EAUK with EA London, an official effective altruism charity with regranting capability.

 

SHIC

– SHIC is planning a longitudinal workshop experiment set to launch in early 2018. This will put to the test our data-driven approach to identifying high-potential students most inclined to engage with effective charity in the long-term. More on this next month.

 

LEAN

– LEAN is currently undergoing its most comprehensive impact assessment to date. The future of LEAN within RC is contingent upon the outcome of the assessment.

 

Donor-Advised Funds (DAF)

– Plans are in motion to inherit Charity Science’s DAF that moved ~$600,000 to GiveWell top charities in 2017. The DAF will then be cause-neutral, expanding our services beyond GiveWell-recommended charities.

– Through a partnership with Chimp, we will be able to offer tax benefits to Canadians for both Canadian and non-Canadian charities. This was previously quite restrictive.

 

What’s Next

 

The shift in our priorities reflects what we think will achieve the highest multiplier effects over time, not simply high initial rates of return. This is why we’re favoring regranting efforts over online fundraisers, and allocating more bandwidth to SHIC while LEAN is under assessment.

 

We’re thrilled with your support – all of RC is pushing hard to keep it.