Quantifying the Value of Operational Due Diligence (ODD)

By DiligenceVault: Operational due diligence helps identify and mitigate material business and operational risks associated with asset managers and funds that they manage. ODD generally include review of firm and fund structure, operating controls, regulatory compliance, technology and infrastructure, affiliate and 3rd party risks as well as staff expertise.

What is the Importance of ODD in Fund Investing?

ODD is a critical component of overall due diligence when allocating to external asset managers. A robust ODD framework for fund investing provides an assessment of over 10 non-investment areas via multiple channels:

Sending out ODD questionnaire or reviewing manager’s standard DDQ
Background checks, and media monitoring
Service provider review
Onsite visits of the manager
Document reviews, including the audited financial statements
Regulatory reviews and reference checks
Synthesizing all of these information points into an internal dashboard

The complexities of ODD and required experience, skills, data sources and technologies are key drivers of how an operational due diligence teams structured and empowered at an allocator.

Who is Responsible for Assessing ODD Risks?

How an ODD team is structured varies across the allocator firms, but can be bucketed into four groups: (a) a team which is a generalist across investment and operational due diligence on one bookend (b) the other bookend being an independent internal team with veto rights that conducts ODD, (c) co-sourced solution with an external ODD specialist supporting the internal team structured in the first two ways, or (d) complete outsourcing of ODD to external specialists.

There are pros and cons to each of the four approaches above, and the ODD team structure employed by the asset allocator is driven by a combination of the fiduciary mandate, available expertise, and the complexity of the portfolio.

Lessons Learned from Fund Failures

As we consider the evolving role and value of ODD, let us evaluate two sources. First, a 2003 Capco Study which lists the top causes of operational risk at hedge funds, and the top reason is a combination of operational issues.

Next, we look at 2002 – 2023 to assess the frequency and nature of some of the key risk failures in fund investing:

The Capco Study combined with the review of the 20 fund failures listed in Table 1 highlight three things:

1. The importance of conducting operational due diligence prior to making the allocation to an asset manager. A vast majority of funds listed in Table 1 represent fund managers who were once successful until they failed because of the risk and operational failures. The top three risks when reviewing the table are fraud, risk concentration, and poor operational controls. While the analysis of these events may point us to a solution to focus on these risks during due diligence, allocators cannot base their framework on just known failures to avoid survivorship bias.

In fact, the potential loss analysis should include asset managers who never raised sufficient assets because of operational weaknesses, which can include many other factors.

2. The importance of a scalable due diligence program. There is never just one type of risk, and often the fund failures are attributed to multiple major and minor risk factors coming together. This requires a program which can identify and track risks across each manager, as well as monitor concentration of risks across the portfolio

3. The importance of ongoing monitoring. The risks contributing to fund failures may develop overtime, or new risk factors may be introduced overtime.

For example, data security and AI risks are new risks that ODD professionals have to assess today, which was not a risk area two decades ago.
In some cases, the risks were amplified by market movements, which is often a risk which is masked in favorable market conditions.
Another area where the fund managers may suffer from investment losses would be from operational risks at the underlying portfolio company, which was an example at FTX.

How Do You Quantify the Value of ODD?

ROI Approach: As an illustration, USD 1 billion equally weighted investment portfolio grows by 170mm in five years with a modest return of 17%. However this reduces to USD 90 million if the portfolio has one investment which has a significant ODD related loss. The delta of USD 80 million is significantly greater than the five year cost of an ODD program managed by an internal team or an external ODD service. In addition, having an ODD integration provides higher conviction in investing with winners, and also delivers greater peace of mind to all stakeholders and fiduciaries. While USD 80 million in this example is based on several assumptions, allocators must do a sensitivity analysis to evaluate range of outcomes.

Opportunity Loss Approach: The second approach is to quantify the total portfolio return required to breakeven from the operational risk loss. In the example where the allocators needs to make up the loss in one investment, they need the rest of the portfolio investments to deliver nearly 25% 5 year return which is pretty difficult to achieve even for the most astute allocator.

Benchmark Approach: As seen in Table 1, over 20 years, we have seen 20 operational risk events. However, over the same period S&P 500 had negative total return 4 times.

Given the historical loss statistics and the positive impact of ODD, allocators are increasingly investing in operational due diligence programs.

The Expanding Role of ODD – Never be the Same!

ODD initially gained traction with hedge fund allocators given the complexity of the structures and operations combined with increased occurrences of losses following failures in non-investment areas. Fast forward to 2023, ODD has become mainstream in the traditional as well as the private markets investing.

Another shift is in the frequency. ODD was largely focused on pre-investment diligence, and now has expanded to incorporate ongoing monitoring.

Lastly, the scope of ODD continues to expand as market dynamics and regulations are shifting with newer areas of focus in the past decade include:

AI and issues around data privacy, responsible use and ethics
Cyber risk given the increased frequency of breaches, including wire fraud
ESG and greenwashing
New asset classes – Crypto and DeFi
Regulatory compliance
Service provider diligence, exemplified by SVB and the banking crisis
Valuations given the shifting market cycle

ODD Priorities – 2023 and Beyond

Reflecting on the survey conducted by Goldman Sachs with 89 ODD professionals from allocators and consultants representing USD 1.2 trillion invested in hedge funds, we summarize some of the observations below:

The top three areas of allocators’ ODD focus over the past year has been information security/cyber security, the cash wire processes, and valuation policies. Top of mind operational risk events in evaluating managers response ability include:

A regulatory issue was clearly the primary area of concern, with 53% of allocators ranking it first.
An issue/outage with a financing counterparty came in second with 19%.
A departure/change of a key member of the non-investment staff was third with 15%.

Digital Adoption: 46% of allocators have implemented a system for the collection of manager data, however, only 36% are subsequently performing data analysis on the information they have collected.

Given our expertise in diligence data collection and analysis at DiligenceVault, we are not surprised by the responses around limited analysis. From our experience this is resulting from fours factors: (a) allocators in the early inning of digital data collection, (b) poor response rates from the managers, (c) data collection is focused on qualitative information and documents and on less objective analysis friendly data, and (d) limited interoperability of data from their data collection systems into allocators data warehouses.

Conclusion

The components of ODD are well understood, the ODD processes are well established, operational risk losses are well publicized, which are all necessary ingredients to create a high functional and high ROI ODD function for allocators.

At the same time as the complexity and scope of ODD continues to expand, it is vital to have a trusted and experienced team with the knowledge of known and developing risk areas, robust risk-based processes, and technology to create a defensible ODD program.

Data focus in ODD, while in its infancy will also play a significant role in building a centralized, analysis rich operational due diligence programs of the future which will be the X factor in providing allocators the ability to invest with confidence.

About DiligenceVault

DiligenceVault believes in making due diligence possible for all by creating a new data-driven standard for due diligence in the investment management industry. Today, over 50,000 users leverage the platform in digitalizing and streamlining their investment, ESG, operational and related due diligence framework, moving away from previously manual and document intensive, error-prone, and expensive diligence processes.

DiligenceVault is thrilled to be rated as #1 platform in the industry by both investors and asset managers and is proud to win awards including FinTech 100, WealthTech 100, Private Equity Wire and more! Founded in 2014, DiligenceVault is backed by Goldman Sachs, and delivers a global support promise with teams in Boston, New York, London, India, Singapore, and Australia.