No matter who you are, most of the smartest people work for someone else.  — Bill Joy

Large family offices over $5 billion AUM with five or more years of experience running in-house direct investing programs now have the data to assess their efforts.  From what we hear, for many families, it’s just not worth the effort.

Attribution studies conducted recently by a number of family office investment staffs conclude that they would have been better off buying an index.

Our clients and contacts cite three reasons for their disappointment, public market performance, high costs, and competition.

A few caveats. We usually provide data to back up our thesis, but in this case, due to the sensitivity of our sources (i.e., the career limiting nature of any disclosures), our comments will have to be anecdotal.

Also, the SFOs we spoke with on background regarding direct investing made their money from tech, energy, and manufacturing, not finance.

It’s tough to beat public markets

US equities set a particularly high hurdle for performance this past decade and over longer periods remain a hard mark to beat.

Here’s a snapshot of the S&P over the last twelve years along with some international peers.

(S&P 500 12-yr performance 2022)

Lamentably the stock market took a nasty fall this year after capping an extraordinary run in 2021, but for long term investors it’s hard not to embrace public equities.

Since 1945 the S&P increased on average about 11% annually, posting an inflation-adjusted return of 7.14% per year.  More than enough to provide comfortable payouts for generations of happy heirs.

High costs

A Fidelity survey last year notes that “single family offices are highly focused on keeping costs in check.”

Our last investment office cost study looked at funds with AUM between one billion and thirty-five billion dollars, roughly the same size as the family offices we spoke with for this article.

We found that office costs ranged from 106 to 165 BPS.

Note: We lean heavily on university endowments for our benchmarks because that’s where the data is. Family offices, foundations, and Wall Street firms employ excellent investment professionals, but it’s difficult to extract meaningful data.

An excellent publicly available work on endowment costs was published by Commonfund in 2015 and republished with additional material in 2017.

Their overall best estimate was that total costs for a “well-diversified” endowment office runs between 100 and 175 BPS.

Another study by Citibank’s Private Bank concludes that “family office expenses often amount to approximately 1% to 2% of the family’s total active assets, including investment portfolios, trust assets, and liquid assets.”

But, in addition to the costs above, we then have to add in the price of a “bare-bones” in-house direct investment team which starts at $2.1 million per year according to Nextvest.

Stuart Lucas, family office head and adjunct professor at University of Chicago advises discretion.  “I preach caution when it comes to direct investing unless there is a special angle or large enough scale to recruit world-class talent.”

Competition

Do private investors really think they can consistently beat the best and brightest at KKR, Blackstone, Carlyle, Apollo, and the madding crowd of highly paid, ferociously ambitious top-guns at several thousand competitors?

Ernst & Young counts 4,500 PE firms in the U.S. out prowling for deals while McKinsey estimates the amount of AUM stalking North American targets at over half the $6.3 trillion in global private equity.

In the venture space alone, approximately 1,965 VC firms deploy $548 billion chasing VC gold.  Marc Andreessen, co-founder of a16z, estimates that for even the most successful top-tier VC firms, only about 15 of every 200 VC backed companies reach $100 million in revenue, let alone unicorn status.

As one family office head said to me as we researched this article, “the administrative and workout hassles are endless and the worst thing that can happen is to have a big winner in the first year or two because then you think it’s easy.”

The bar to beat

Top endowments and foundations have invested heavily in private markets for years using mostly outside managers and they are generally considered to be the best in the business at what they do, diversified investing across assets, geographies, and time horizons.

Our chart below shows performance and private market allocations for the eight Ivy endowments plus MIT and Bowdoin.

Endowment Performance & Private equity/Venture Capital Allocations as of June 30, 2022 Source: Charles Skorina & Company https://www.charlesskorina.com
Institution	CEO/CIO	FY2022 %	FY2021 %	10-yr as of FY2020	PE/VC %	HFs %
S&P 500	–	–10.62%	40.79%	14.0%	–	–
Bowdoin*	Bryant, Niles	-7.1	57.4	11.6	44	25
Brown	Dietze Jane	-4.6	51.5	10.2	39	25
Dartmouth	Ruth, Alice A.	-3.1	46.5	10.4	38	23
Yale**	Mendelsohn, Matthew S. T.	0.8	40.2	10.9	38	21
Penn	Ammon, Peter H.	0.0	41.1	9.3	36	26
Harvard	Narvekar, Narv	-1.8	33.6	8.2	34	33
MIT*	Alexander, Seth	-5.3	55.5	11.4	34	17
Cornell*	Miranda, Ken	-1.3	41.9	7.5	33	15
Princeton	Golden, Andrew K.	-1.5	46.9	10.6	30	24
Columbia*	Lew, Kim	-7.6	32.3	9.1	23	28

S&P 500 performance: FY2022 –10.62%,  FY2021 40.79%,  10-yr avg. 2020 14.0%

*Allocations computed from financial statements (approx.)

**Yale asset allocation from 2020 endowment report

For any family office thinking of investing direct, here’s a question.  Do you have the time, attention, and money to build and manage a top-flight internal direct investing team?  A team that can meet or beat the ten to twelve percent compounded return set by the top E&Fs using outside managers?

Direct investing may reduce manager fees but it certainly adds to in-house complexity.

Keep in mind that in any given year, depending on the size of the fund and staff, an E&F office might research and interview anywhere from 100 to 1000 outside managers in private equity, venture capital, natural resources, infrastructure, real estate, and one-off investments.

The Big Eight Canadian pensions, for example, “are able to hire large teams of highly specialized professionals that allow them to invest directly in a wide range of assets and niche markets.” But only because of their size, Wall Street pay packages, and governance structure.

A brief digression

There’s an obvious anomaly in the date for FY2021 and FY2022.

Bowdoin, Brown, and MIT for example announced fivefold gains over their ten-year average in FY2021 and plunging returns in FY2022.

Part of this fluctuation has to do with the timing of reported returns as we discussed last year and part in the difficulty of valuing private assets.

As Matthew Klein noted in the Financial Times a few years back,

If it’s challenging to figure out what a thing is “worth” when some of the smartest people on the planet, armed with the fastest computers and the biggest datasets, are constantly discussing and betting on its value, it’s downright impossible for investment managers focused on illiquid assets to assess the value of anything they own until they exit their positions by selling to someone else.

It’s in the Way That You Use It

Entrepreneurs and business titans mostly made their money by going “all in” on against-the-odds, innovative, highly concentrated investments. And while successful entrepreneurs build competitive advantage into their businesses, that advantage doesn’t naturally transfer outside of their industry.

Professional investors on the other hand practice diversification, risk-management, and patience. And to paraphrase the legendary Marvin Davis (perhaps not the best choice of investors to make my point), they never fall in love with an asset.

There will always be a few entrepreneurs and families that feel comfortable going head-to-head against the best in private markets, however, they usually have a specific skill set or philanthropic focus that drives their efforts. Or they come from Wall Street and understand the challenges.

But for those founder entrepreneurs who love the excitement, like the family office head I spoke with last year, direct investing is often an expensive and frustrating game.

— Charles Skorina

Comments and Rebuttals

We sent this piece to a few savvy and experienced family office heads for comments. Their answers.

Family Office Head

We differentiate direct investing from investing in PE and VC funds.

The cost of our family office operation is offset by not having to pay 2 and 20 to outside managers on direct deals.

Also, it seems the Thrill is gone just as much for the endowments as it is for families. Your headline ought to read “The thrill is gone” for anyone not invested in an index.

Very Large Family office CIO (the AUM, not the CIO)

I would expect a fair amount of engagement on this issue as it is an emotional topic. Your tone and conclusions are reality. Still, this is one of those topics that becomes like a religious/political discussion.