View more on these topics

Risky business: Will profilers stand up to fresh scrutiny?

The risk-profiling tools used by advisers have previously been described as a “ticking time-bomb”.

These tools have not been scrutinised for years, and now there are fresh concerns over the wildly different asset allocations different tools produce.

Determining what risk a client can take with their money – and then suggesting an appropriate investment plan to meet these needs – is a central part of the advice process.

So there is bound to be keen interest in a new report this week from former regulator Rory Percival, particularly as it is the first review of the sector since then regulator the FSA’s damning report in 2011, which found that nine out of 11 risk-profiling tools “had weaknesses which could, in certain circumstances, lead to flawed outputs”.

This has led to questions about whether these profiling tools were really fit for purpose.

The new report attempts to answer this question, by taking an in-depth look at six of the biggest risk-profiling tools used by advisers.

The six under scrutiny are A2Risk, EValue, Dynamic Planner, FinaMetrica, Morningstar and Oxford Risk. These companies also provide risk-profiling services through other third-party brands, for example Iress and Defaqto. Although Percival found that risk-profiling tools had improved significantly since the earlier FSA review, and are now far more integrated within the advice process, not one of these tools gets a completely clean bill of health.

He notes all these tools “to a greater or lesser extent involve limitations”.

How do advisers sort the good risk-profiling tools from the bad?

Asset allocation angst

Of particular concern has been the way some of these tools recommend asset allocations for portfolios. Percival finds there is often little consistency between these tools when it comes to matching a given risk profile to an investment portfolio.

Percival tells Money Marketing: “We are talking about significant and material differences in asset allocations, which can lead to very different investment portfolios.”

In some cases, the exposure to equities and property may be up to 30 per cent higher, when comparing these risk-profile tools side by side.

But this is not the only problem flagged by the report. Percival also highlights problems with the way risk descriptions are quantified by some providers, and the way scoring algorithms are used which can, in some cases, lead to incorrect results.

Percival says: “No risk profiling tool is perfect. No tool is guaranteed to provide the right answer in every scenario. They are tools to help [advisers] do their job, rather than machines creating a flawless result.”

The report has been broadly welcomed by the industry. Ben Goss, the chief executive at Distribution Technology – which sells the Dynamic Planner risk profiling tool – says: “We strongly support Rory’s view that the process of risk profiling should be part of a wider risk-based plan developed between adviser and client, one that takes into consideration their wealth, cashflow as well as current and future lifestyle needs.”

We are talking about significant differences in asset allocations, which can lead to very different portfolios 

The report not only highlights the limitations of each particular system, but gives specific advice on how advisers can address these shortcomings in their advisory process.

Percival says he hopes this will help advisers use this technology more effectively, and ultimately deliver better outcomes for clients.

The report is also published shortly before the introduction of the EU’s Mifid II rules, which will require advisers to independently assess the risk-profiling tools they use, and identify potential limitations within the advice process.

Rating the tools

Percival has previously said he did not want to “score” each risk-profile, or present the results in a league table. Despite this, the report uses a red, amber, and green system to review each risk-profiling tool against 11 key criteria based on the parameters set out in the previous FSA review. For advisers, this shows certain risk profiling tools have far more concerns flagged than others.

Two of the risk-profiling tools come out ahead of the others: Dynamic Planner and FinaMetrica get just two amber flags, all other aspects of the risk-profiling process are rated green.

EValue has three amber flags. Morningstar gets one amber flag, and one red. Oxford Risk gets two amber flags and one red; while A2Risk gets five amber scores and one red.

With both Oxford Risk and A2Risk the red flag is for the risk descriptions used by these tools, which Percival claims are not always clear.

For example, when A2Risk outlines its “cautious” category, Percival says it isn’t clear whether this covers no risk clients, very low risk clients or both.

Meanwhile, Morningstar gets a red flag because its risk-profiling tool does not generate a “not suitable for investing” category at the outset.

Percival points out this is not highlighted in the information given to advisers, but it is  important that advisers using this system identify such clients at the outset before proceeding with this tool.

This was an issue that tripped up most risk-profilers. With the exception of Dynamic Planner, they all received an amber warning on this particular test.

Many of the other amber flags concern the way questions might be open to interpretation (particularly in relation to ‘middle’ answers), or whether a question, incorrectly answered, has the scope to tip the client into the next risk category. There were also concerns raised about whether these tools could potentially conflate a client’s risk profile with their capacity for loss and knowledge and experience of investing.

Percival points out that under regulatory guidance these separate aspects should not be conflated into a single score.

Profilers fight back

But not all profilers agree with Percival’s assessment. A2Risk research co-ordinator Alistair Haig says: “We don’t agree with all the findings of this report, and have given this feedback.

“When you look at the sections of the report that explain how advisers should deal with these limitations of each risk-profiling tool, ours is the shortest in the report. There seems in some respects a lot less to criticise.”

Haig points out that the risk profiling and psychometric testing offered by A2Risk was designed to complement stochastic packages, generating cashflow models and asset allocation.

He says: “We don’t offer an all-in solution, as this is not what our advisers want. But this report seems to favour those that offer more of an all-in package.”

Haig says the company updates and improves its service on a regular basis and will take these comments on board.

Adviser view

Thameside Financial Planning director Tom Kean

I have always been sceptical about the efficacy of risk profiling tools as a means to an end, which was further tested when the FSA muddied the waters by allowing two separate “scales” of risk. In other words, it can either be out of seven or out of 10. From the start this anomaly has caused great confusion.

We have always regarded these risk profiling questionnaires as something we have to do. We have built them into our processes but they are just part of wider discussions with a client. We always say to our clients it is nice to have a computer view of their risk profile, but we can always disagree with it. We use cashflow modelling to ensure we analyse risk and capacity for loss to the highest level, which may diminish the importance of the risk profiling tool.

As for possible asset allocation models, I tend to steer clear of them. We would rather use our own judgement.

A Morningstar spokeswoman said the firm would be amending its guidance to advisers following the report, but disagreed with the conclusions that some of their questions could be misinterpreted by clients.

FinMetrica director Paul Resnik says he supports this “sensible guidance” on how advisers can mitigate the limitations of these tools.

He adds: “The disparity in growth asset exposures that Rory has unearthed represents the systemic risk clients and advisers face if they use systems that are poorly calibrated. This exposes advisers and their clients to unnecessary compliance and investment risk.”

He adds that he is confident FinaMetrica’s mapping of risk scores to equity allocations was robust, as it was based on data taken from more than one million tests, going back over a 20-year period.

Resnik says: “Although the regulator never named the two risk profiling tools that ‘passed’ the 2011 FSA test, we were always convinced that FinaMetrica was one.

“We’re even more certain of that now given the application of a similar test methodology. The visual ‘red, amber, green’ ratings in this guide illustrate the high-level conclusions. But the devil is in the detail of the analysis. We would encourage everyone to read the words very carefully to get a clearer picture of the different hues of the amber ratings in particular.”

Weightings weighed up 

Some of the most significant findings of this report concern asset allocation. Percival says he has serious concerns about the way some of these risk-profiling tools produce very different results when it comes to asset allocation.

Not all the tools offer this additional process. Neither Oxford Risk nor A2Risk, for example, include asset allocation as part of their risk descriptions. The report looked at different hypothetical situations to see what asset allocations would be recommended for a cautious, medium or more adventurous investor.

As the tables opposite show, the results can differ widely. For example, an adviser using the EValue tool could end up putting a cautious investor into a portfolio where 50 per cent of the portfolio is invested in equities and property. With FinaMetrica just 22 per cent of the portfolio would be in such assets.

Regulatory action over asset allocation a worry for IFAs, says Pru

A similar discrepancy can be seen when it comes to more adventurous investors: by selecting the second- highest risk answer on these risk questionnaires, clients could have a portfolio that is 95 per cent invested in equities and property with Dynamic Planner, but just 68 per cent exposure to these assets if their adviser used Morningstar’s risk-profiling tools. Percival stresses that this report does not pass judgement on which asset allocation is correct, but points out that with such large discrepancies, it is unlikely both would be suitable for the same client.

He says advisers need to look carefully at how these risk categories are mapped onto asset allocations. “Most of the risk-profiling tools produce results based on psychometric testing which plot clients – on a comparative basis – on a normal distribution curve created from extensive testing on a large population of clients. They then divide up this distribution curve into a number of risk categories ranging from five to 10. This is about clients, not investments. So, for example a client in the middle of this distribution curve is medium risk only insofar as he or she is on the middle of the risk spectrum. This is different from saying they are medium risk in terms of investments.”

Percival adds it is important not to solely rely on the risk profiler’s risk category when selecting asset allocation – but ensure this is part of the advisory process, taking into account factors like capacity for loss, investment knowledge and experience, investment term, and individual objectives.

‘Urgent’ action

EValue chief executive Paul McNamara says the report is right to highlight these discrepancies and says this is an issue that needs to be “urgently addressed” by advisers. He says consumers are being short-changed by the wildly different methodologies used. He says: “The report correctly identifies that the mapping of investments and asset allocations to the output of risk profile questionnaires is sometimes problematic, and could nullify the good work done by an adviser using a risk questionnaire.” McNamara adds he was “disappointed” the report did not probe further into this issue.

“These recommendations need to go further. What is required in an in-depth analysis of the risk characteristics of each different asset class and the diversification benefits available within a multi-asset portfolio. Advisers need to review this area since inadequate risk mapping of investments can completely undermine the results obtained from a robustly developed risk questionnaire.”

Expert view

Threesixty managing director Russell Facer

The new guide reiterates the importance of the FSA’s finalised guidance, Assessing Suitability, in 2011. In essence, the output from any risk-profiling questionnaire should be taken as the starting point for assessing the level of risk a client is willing to take, rather than an irrefutable truth.

To demonstrate the suitability of the risk level assigned to a client, we recommend files evidence discussions regarding responses which appear to be out of line with the output, potentially conflicting answers and the risk descriptors.

While some risk-profiling tools do pose questions designed to test capacity for loss, we recommend clear factfinding to assess this area, the most important aspects being: assets, liabilities, income, expenditure and expected future expenditure.

The importance of demonstrating the client will be able to maintain their standard of living through
retirement cannot be understated, particularly when a defined benefit transfer is recommended.

Where possible, we advocate the use of cashflow modelling and appropriate stress testing. Importantly, discussions held with clients regarding what they consider to be essential spending should be documented.

Mifid II highlights the importance of advisers understanding the limitations of risk-profiling tools and actively mitigating them through the suitability assessment process.

Now would be an appropriate time to revisit your firm’s risk profiling tool, to ensure detailed information regarding the assumptions and scoring mechanism have been obtained and analysed.

Many in the industry point out that risk profiling tools can help ensure consistency. Psigma chief executive Jonathan Howard-Smith says they are useful when building model portfolios. “There are larger investment firms I know where the individual investment managers have got such a leeway that two investment managers working for the same firm could build entirely different portfolios for the client with the same circumstances and that is completely wrong.”

Goss adds: “It’s the nature of asset allocation theory that an almost unlimited combination of asset classes can be used, in principle to deliver a given risk level. In practice though many of these permutations are impractical or unpalatable to advisers and clients. We have worked with advice firms to find the right balance of robust portfolios which deliver good returns for the level of risk the client is willing and able to take.”

Many in the industry are already all too aware of the limitations of these risk-profiling tools. Professional Partnerships IFA business consultant Gill Cardy says: “The vast variation in outcomes when you move from the risk profile to the suggested asset allocation has always worried me. And some of the suggested portfolios have not always been very diversified.”

She adds there is a tendency for some of these tools to come up with too high-risk assessments for clients.

“You had to put in a lot of really low risk answers to get back to anything like the four to six area where you would expect most results to sit.”

Aspect 8 financial planner Claire Walsh uses the risk-profiling questionnaires designed by Oxford Risk, which do not map a recommended asset allocation for the client, but leaves this up to the adviser. Walsh says: “These questions are the starting point for a broader discussion about risk, the client’s financial situation and capacity for loss. Many of my clients come out quite high on the risk profile score, however they don’t need to achieve high growth, and when discussing it we often agree on a lower risk strategy when it comes to allocating their investments.”

Recommended

Bitcoin-700x450.jpg

FCA warns on cryptocurrency fundraising as Bitcoin soars

The FCA has warned that investors in a cryptocurrency fundraising mechanism should be prepared to lose their entire stake as Bitcoin trades at record highs, almost quintupling year to date. Initial Coin Offerings (ICO) raise money for a specific project or company through the issue of a physical coin in exchange for a cryptocurrency, such […]

Lessons to be learnt

DFM masterclass: markets, strategy and asset allocation

Watch Nick Georgiadis, Head of DFM, discuss asset allocation and investment strategy on Asset TV’s DFM masterclass. In this masterclass on discretionary fund management, a panel of industry experts discuss their views on the current market, inflation figures, investment strategies, asset allocation and what they will be focusing on when managing portfolios over the next […]

What triggers the MPAA?

Jim Grant – Senior Product Insight & Technical Support Analyst There’s sometimes confusion around what triggers the money purchase annual allowance. Find out what does and what doesn’t trigger the MPAA. The money purchase annual allowance (MPAA) is a reduced annual allowance that can apply to contributions to defined contribution (DC) schemes. The following table […]

Newsletter

News and expert analysis straight to your inbox

Sign up

Comments

There are 14 comments at the moment, we would love to hear your opinion too.

  1. These tools are and always have been a compliance placebo.
    Their use is really just a rough guide used (as it ought to be) as a detailed further discussulion with the client.
    As to these being used as an asset allocation tool points yet again to the conclusion that advisers followig these ought not to be giving investment advice at all as it evidently shows that they are not capable of constructing an appropriate portfolio.

  2. An interesting article, raising many issues but with few solutions! Let’s start with the actual questionnaires (RQ). All they tell you is where the client sits relative to the populace as a whole. If you have enough clients, you can plot the graph yourself! There is no apparent reason why different RQs will give the same absolute graphs but the distributions can be divided up in say 10 sections. So, you can say something like “You attitude to risk with your money is just below the average for the populace as a whole” (call this risk 4).
    The next step is to discuss with the client the graphs calculated from a sequential loss calculator. This gives the likely returns for a given growth rate and standard deviation. More importantly it gives some idea of what risk (standard deviation) and returns are needed to meet the client’s expectations and aims and can illustrate the potential for loss.
    The next bit is the tricky part! Take (say) three asset classes, A B and C. There are already 6 ways of combing these, even without any weightings. With relative weightings there are in principle an infinite number of combinations. Using historical data (yes I know!) you can combine these three assets classes in many many ways, to obtain average returns and standard deviations. There is no reason why A/3 + B/3 + C/3 should give any different results to A/4 + B/2 + C/4. With say ten asset classes to choose from, it would be no surprise to find many mixes that produce the same results. As long as you can show this, it matters not that different systems may give different asset allocations and yet predict the same results.
    The final step is to plot the returns and standard deviations and then match them to the client’s requirements OR you could divide the results into 10 different areas and match those to the RQ results (a lot less satisfactory method but one used quite often).
    Any predictions on asset allocations based on a stochastic model of future investment parameters may give wildly different results for even small changes to the input parameters.
    Finally, for larger IFAs with many clients, you can back test your investment decisions – surely the strongest proof of the robustness of your planning procedures.

  3. “Percival stresses that this report does not pass judgement on which asset allocation is correct, but points out that with such large discrepancies, it is unlikely both would be suitable for the same client.”

    This is typical of both consultants and regulators (I guess a consultant who was a regulator gives a double dose!). Happy to tell you something is wrong or probably wrong but will never tell you when it’s right.

    Then again, this report is a commercial exercise designed to sell reports and drum up consultancy. There is an open conflict of interest here in this respect – there’s no money to be made from a report that says everything is OK.

    And would MM be willing to give as much air time to all the other compliance consultancies if they did the same? Or is Mr Percival a special case because of his background?

  4. I am very surprised that anyone familiar with all the assumptions and the variety of parameter controls which go into these ‘models’ should be surprised at the wide ‘discrepancy’ in output. these are all opinions and, as others have pointed out, theoretically there is almost and infinite number of ‘correct’ answers, i.e. combinations which provide the same level of given risk. To expect otherwise is like expecting all economists to produce the same forecasts! The bigger problem is those firms who then slavishly implement the precise asset allocation dictated by one such tool without further thought.

  5. Any modelling tool is dangerous, as they have to make assumptions.

    Illustrations are no better, as no one knows the future.

    These documents are only guide lines and frankly, just like the illustrations which the regulator insisted had to be given in the past, had to assume a growth rate of, they will be used against advisers in the future.

    The whole system is now unworkable, advice is by nature subjective, investment carries risk, clients by nature change their minds according to outcomes.

    The whole system has become to complicated, costly and regulation is at the very hart of the problem.

  6. This would be funny if it weren’t so serious.
    Another report that talks a lot but tells us nothing we didn’t already know.
    There are legions of advisory firms out there running model portfolio’s based on the output from these things, and I’ll wager the vast majority are essentially set up slavishly to be as near as possible to the output (where applicable) of that risk profiler’s asset allocation result for that clients so called risk profile.
    As I’ve pointed out in a previous post, some of these asset allocation models are wildly different from one another (and I take Ken Hayden’s point on this in terms of combinations and results), because they are founded on either a long term strategic asset allocation basis, or on one which employs some form of tactical asset allocation overlay adjusted periodically. A case in point for the former, is that you still see low risk as being predominantly Government Bonds, Corporate Bonds and High Yield Bonds, with some cash, but right now, with bond yields of all variants at an all time low and asset prices across the board being at ridiculous levels, how in God’s name could one currently describe Bonds as a low risk investment suited to a cautious client? It’s all relative to market position, which its self is more unpredictable than ever due to geo-political factors now being a major component.
    We have had FCA comments that claim not to like advisors applying asset allocation models to a client that differ from that produced from the question set output of the particular risk profiling system they are using, and that if you do, you are in deep water. Next it’s that you shouldn’t accept this and should apply adjustments according to your discussions, too (which is what sensible advisers were doing but were in some cases being forced subsequently not to do by the Quant brigade).
    I’ve known clients come out as definitely within the spectrum of an above average (whatever that is) attitude to investment risk scale, yet comment that they could only afford to suffer little or no losses, even if this were, in actuality, not the case (what they actually meant, was that they wouldn’t LIKE to suffer losses as it would upset them). So; are they above average risk takers (because they wouldn’t like low returns, either), are they cautious, or are the to be moved to somewhere in between, in which case you satisfy neither camp.
    It is ALL about the discussion and knowing the client; any risk profiling questionnaire system is at best be a guide and nothing more. Let’s stop chasing rainbows, shall we?

    • @Whittington Dick

      “It is ALL about the discussion and knowing the client; any risk profiling questionnaire system is at best be a guide and nothing more. Let’s stop chasing rainbows, shall we?”

      Whilst you are doubtless correct, it’s a politically unacceptable position and so the politically answerable regulator must ignore or fudge. They do both and throw in a dash of generic berating for show. Consultants love this as they feed off giving nearly answers to impossible-to-answer questions. Some consultants assist this process through studies which confirm the impossible to answer questions and where possible pose new ones.

      The reason? In practice what you describe is what good advisers do and have done for many years. Good stuff. However, RDR has made advice expensive (no one saw that coming…) and the political fix is automation, i.e. what is generally referred to as robo-advice. The inconvenient truth here is that this relies solely on tools to assess client risk and feed them into models. That creates a conundrum for the regulator because surely what’s good for the robo-goose must surely be good for the adviser gander. Except it’s clearly not and expectations of advisers are palpably higher which is a dual standard based on the same rules.

      Perhaps face to face advisers could just use a ‘robo-tool’ when it comes to assessing risk and choosing an investment and not discuss any of that part with the client. If it’s the same tool used by a robo-adviser that the regulator is comfortable with then presumably that won’t be an issue…

  7. PS – and some of these profilers – of which Dynamic Planner is one – have questions about time scale, but the answer to that question does not have any impact on the asset allocation output… ’nuff said.

    • In Dynamic Planner, we don’t suggest what the recommended risk profile should be. We show the results of the attitude questionnaire, but how to adjust that based on the clients knowledge, experience and capacity is best left to the adviser who is having the conversation with the client.

      Joshua Knight, Product Manager, Dynamic Planner

  8. Excellent comments section on this article with all the relevant issues being highlighted!

    Whilst Mr Percival’s report will no doubt shed some light, it was never going to address the most important part – the asset allocation. And even within the same asset allocation, wildly different strategies could populate it. But let’s be clear, short of the FCA mandating a risk profiling process and asset allocation, there is no answer to this conundrum. These are matters of opinion, not fact. I just hope that the FCA acknowledge this unequivocally.
    Then there’s the question of FOS – how do they assess what asset allocation is suitable?

    • The FOS assess it based on what they consider to be fair and reasonable in the circumstances. Legally, this allows them to do pretty much what they like.

      It can only be challenged successfully in court if you can demonstrate that the decision “is so unreasonable that no reasonable authority could ever have come to it”. Read that carefully, whimper quietly. You could show that the decision was unreasonable, stupid, or even against the FCA’s rules and still lose. Good luck.

  9. I have to agree with the comments about asset allocation and risk profiles and this is something that came up at the Paraplanner Pow Wow yesterday.

    One interesting concept that was shared yesterday was to establish what level of growth the client needs to achieve their objective and work back from that?

    One only has to look at the risk rating that DT has awarded a UK Residential Property fund to understand that the methodology is flawed.

Leave a comment