The risk-profiling tools used by advisers have previously been described as a “ticking time-bomb”.
These tools have not been scrutinised for years, and now there are fresh concerns over the wildly different asset allocations different tools produce.
Determining what risk a client can take with their money – and then suggesting an appropriate investment plan to meet these needs – is a central part of the advice process.
So there is bound to be keen interest in a new report this week from former regulator Rory Percival, particularly as it is the first review of the sector since then regulator the FSA’s damning report in 2011, which found that nine out of 11 risk-profiling tools “had weaknesses which could, in certain circumstances, lead to flawed outputs”.
This has led to questions about whether these profiling tools were really fit for purpose.
The new report attempts to answer this question, by taking an in-depth look at six of the biggest risk-profiling tools used by advisers.
The six under scrutiny are A2Risk, EValue, Dynamic Planner, FinaMetrica, Morningstar and Oxford Risk. These companies also provide risk-profiling services through other third-party brands, for example Iress and Defaqto. Although Percival found that risk-profiling tools had improved significantly since the earlier FSA review, and are now far more integrated within the advice process, not one of these tools gets a completely clean bill of health.
He notes all these tools “to a greater or lesser extent involve limitations”.
Asset allocation angst
Of particular concern has been the way some of these tools recommend asset allocations for portfolios. Percival finds there is often little consistency between these tools when it comes to matching a given risk profile to an investment portfolio.
Percival tells Money Marketing: “We are talking about significant and material differences in asset allocations, which can lead to very different investment portfolios.”
In some cases, the exposure to equities and property may be up to 30 per cent higher, when comparing these risk-profile tools side by side.
But this is not the only problem flagged by the report. Percival also highlights problems with the way risk descriptions are quantified by some providers, and the way scoring algorithms are used which can, in some cases, lead to incorrect results.
Percival says: “No risk profiling tool is perfect. No tool is guaranteed to provide the right answer in every scenario. They are tools to help [advisers] do their job, rather than machines creating a flawless result.”
The report has been broadly welcomed by the industry. Ben Goss, the chief executive at Distribution Technology – which sells the Dynamic Planner risk profiling tool – says: “We strongly support Rory’s view that the process of risk profiling should be part of a wider risk-based plan developed between adviser and client, one that takes into consideration their wealth, cashflow as well as current and future lifestyle needs.”
We are talking about significant differences in asset allocations, which can lead to very different portfolios
The report not only highlights the limitations of each particular system, but gives specific advice on how advisers can address these shortcomings in their advisory process.
Percival says he hopes this will help advisers use this technology more effectively, and ultimately deliver better outcomes for clients.
The report is also published shortly before the introduction of the EU’s Mifid II rules, which will require advisers to independently assess the risk-profiling tools they use, and identify potential limitations within the advice process.
Rating the tools
Percival has previously said he did not want to “score” each risk-profile, or present the results in a league table. Despite this, the report uses a red, amber, and green system to review each risk-profiling tool against 11 key criteria based on the parameters set out in the previous FSA review. For advisers, this shows certain risk profiling tools have far more concerns flagged than others.
Two of the risk-profiling tools come out ahead of the others: Dynamic Planner and FinaMetrica get just two amber flags, all other aspects of the risk-profiling process are rated green.
EValue has three amber flags. Morningstar gets one amber flag, and one red. Oxford Risk gets two amber flags and one red; while A2Risk gets five amber scores and one red.
With both Oxford Risk and A2Risk the red flag is for the risk descriptions used by these tools, which Percival claims are not always clear.
For example, when A2Risk outlines its “cautious” category, Percival says it isn’t clear whether this covers no risk clients, very low risk clients or both.
Meanwhile, Morningstar gets a red flag because its risk-profiling tool does not generate a “not suitable for investing” category at the outset.
Percival points out this is not highlighted in the information given to advisers, but it is important that advisers using this system identify such clients at the outset before proceeding with this tool.
This was an issue that tripped up most risk-profilers. With the exception of Dynamic Planner, they all received an amber warning on this particular test.
Many of the other amber flags concern the way questions might be open to interpretation (particularly in relation to ‘middle’ answers), or whether a question, incorrectly answered, has the scope to tip the client into the next risk category. There were also concerns raised about whether these tools could potentially conflate a client’s risk profile with their capacity for loss and knowledge and experience of investing.
Percival points out that under regulatory guidance these separate aspects should not be conflated into a single score.
Profilers fight back
But not all profilers agree with Percival’s assessment. A2Risk research co-ordinator Alistair Haig says: “We don’t agree with all the findings of this report, and have given this feedback.
“When you look at the sections of the report that explain how advisers should deal with these limitations of each risk-profiling tool, ours is the shortest in the report. There seems in some respects a lot less to criticise.”
Haig points out that the risk profiling and psychometric testing offered by A2Risk was designed to complement stochastic packages, generating cashflow models and asset allocation.
He says: “We don’t offer an all-in solution, as this is not what our advisers want. But this report seems to favour those that offer more of an all-in package.”
Haig says the company updates and improves its service on a regular basis and will take these comments on board.
Thameside Financial Planning director Tom Kean
I have always been sceptical about the efficacy of risk profiling tools as a means to an end, which was further tested when the FSA muddied the waters by allowing two separate “scales” of risk. In other words, it can either be out of seven or out of 10. From the start this anomaly has caused great confusion.
We have always regarded these risk profiling questionnaires as something we have to do. We have built them into our processes but they are just part of wider discussions with a client. We always say to our clients it is nice to have a computer view of their risk profile, but we can always disagree with it. We use cashflow modelling to ensure we analyse risk and capacity for loss to the highest level, which may diminish the importance of the risk profiling tool.
As for possible asset allocation models, I tend to steer clear of them. We would rather use our own judgement.
A Morningstar spokeswoman said the firm would be amending its guidance to advisers following the report, but disagreed with the conclusions that some of their questions could be misinterpreted by clients.
FinMetrica director Paul Resnik says he supports this “sensible guidance” on how advisers can mitigate the limitations of these tools.
He adds: “The disparity in growth asset exposures that Rory has unearthed represents the systemic risk clients and advisers face if they use systems that are poorly calibrated. This exposes advisers and their clients to unnecessary compliance and investment risk.”
He adds that he is confident FinaMetrica’s mapping of risk scores to equity allocations was robust, as it was based on data taken from more than one million tests, going back over a 20-year period.
Resnik says: “Although the regulator never named the two risk profiling tools that ‘passed’ the 2011 FSA test, we were always convinced that FinaMetrica was one.
“We’re even more certain of that now given the application of a similar test methodology. The visual ‘red, amber, green’ ratings in this guide illustrate the high-level conclusions. But the devil is in the detail of the analysis. We would encourage everyone to read the words very carefully to get a clearer picture of the different hues of the amber ratings in particular.”
Weightings weighed up
Some of the most significant findings of this report concern asset allocation. Percival says he has serious concerns about the way some of these risk-profiling tools produce very different results when it comes to asset allocation.
Not all the tools offer this additional process. Neither Oxford Risk nor A2Risk, for example, include asset allocation as part of their risk descriptions. The report looked at different hypothetical situations to see what asset allocations would be recommended for a cautious, medium or more adventurous investor.
As the tables opposite show, the results can differ widely. For example, an adviser using the EValue tool could end up putting a cautious investor into a portfolio where 50 per cent of the portfolio is invested in equities and property. With FinaMetrica just 22 per cent of the portfolio would be in such assets.
A similar discrepancy can be seen when it comes to more adventurous investors: by selecting the second- highest risk answer on these risk questionnaires, clients could have a portfolio that is 95 per cent invested in equities and property with Dynamic Planner, but just 68 per cent exposure to these assets if their adviser used Morningstar’s risk-profiling tools. Percival stresses that this report does not pass judgement on which asset allocation is correct, but points out that with such large discrepancies, it is unlikely both would be suitable for the same client.
He says advisers need to look carefully at how these risk categories are mapped onto asset allocations. “Most of the risk-profiling tools produce results based on psychometric testing which plot clients – on a comparative basis – on a normal distribution curve created from extensive testing on a large population of clients. They then divide up this distribution curve into a number of risk categories ranging from five to 10. This is about clients, not investments. So, for example a client in the middle of this distribution curve is medium risk only insofar as he or she is on the middle of the risk spectrum. This is different from saying they are medium risk in terms of investments.”
Percival adds it is important not to solely rely on the risk profiler’s risk category when selecting asset allocation – but ensure this is part of the advisory process, taking into account factors like capacity for loss, investment knowledge and experience, investment term, and individual objectives.
EValue chief executive Paul McNamara says the report is right to highlight these discrepancies and says this is an issue that needs to be “urgently addressed” by advisers. He says consumers are being short-changed by the wildly different methodologies used. He says: “The report correctly identifies that the mapping of investments and asset allocations to the output of risk profile questionnaires is sometimes problematic, and could nullify the good work done by an adviser using a risk questionnaire.” McNamara adds he was “disappointed” the report did not probe further into this issue.
“These recommendations need to go further. What is required in an in-depth analysis of the risk characteristics of each different asset class and the diversification benefits available within a multi-asset portfolio. Advisers need to review this area since inadequate risk mapping of investments can completely undermine the results obtained from a robustly developed risk questionnaire.”
Threesixty managing director Russell Facer
The new guide reiterates the importance of the FSA’s finalised guidance, Assessing Suitability, in 2011. In essence, the output from any risk-profiling questionnaire should be taken as the starting point for assessing the level of risk a client is willing to take, rather than an irrefutable truth.
To demonstrate the suitability of the risk level assigned to a client, we recommend files evidence discussions regarding responses which appear to be out of line with the output, potentially conflicting answers and the risk descriptors.
While some risk-profiling tools do pose questions designed to test capacity for loss, we recommend clear factfinding to assess this area, the most important aspects being: assets, liabilities, income, expenditure and expected future expenditure.
The importance of demonstrating the client will be able to maintain their standard of living through
retirement cannot be understated, particularly when a defined benefit transfer is recommended.
Where possible, we advocate the use of cashflow modelling and appropriate stress testing. Importantly, discussions held with clients regarding what they consider to be essential spending should be documented.
Mifid II highlights the importance of advisers understanding the limitations of risk-profiling tools and actively mitigating them through the suitability assessment process.
Now would be an appropriate time to revisit your firm’s risk profiling tool, to ensure detailed information regarding the assumptions and scoring mechanism have been obtained and analysed.
Many in the industry point out that risk profiling tools can help ensure consistency. Psigma chief executive Jonathan Howard-Smith says they are useful when building model portfolios. “There are larger investment firms I know where the individual investment managers have got such a leeway that two investment managers working for the same firm could build entirely different portfolios for the client with the same circumstances and that is completely wrong.”
Goss adds: “It’s the nature of asset allocation theory that an almost unlimited combination of asset classes can be used, in principle to deliver a given risk level. In practice though many of these permutations are impractical or unpalatable to advisers and clients. We have worked with advice firms to find the right balance of robust portfolios which deliver good returns for the level of risk the client is willing and able to take.”
Many in the industry are already all too aware of the limitations of these risk-profiling tools. Professional Partnerships IFA business consultant Gill Cardy says: “The vast variation in outcomes when you move from the risk profile to the suggested asset allocation has always worried me. And some of the suggested portfolios have not always been very diversified.”
She adds there is a tendency for some of these tools to come up with too high-risk assessments for clients.
“You had to put in a lot of really low risk answers to get back to anything like the four to six area where you would expect most results to sit.”
Aspect 8 financial planner Claire Walsh uses the risk-profiling questionnaires designed by Oxford Risk, which do not map a recommended asset allocation for the client, but leaves this up to the adviser. Walsh says: “These questions are the starting point for a broader discussion about risk, the client’s financial situation and capacity for loss. Many of my clients come out quite high on the risk profile score, however they don’t need to achieve high growth, and when discussing it we often agree on a lower risk strategy when it comes to allocating their investments.”