View more on these topics

PFS chief defends new exam format

Reading some of the recent press comments and blogs, there is an adviser body that thinks the new diploma in regulated financial planning has been dumbed down because it is tested online through multiplechoice questions. In fairness, some of the PFS members attending the regional presentations on qualifications have also raised similar concerns.

So is this a valid criticism or just a perception?

Having challenged quite a few people, both within the CII and externally, I am persuaded that the new diploma is not an easier qualification. The new diploma was developed by advisers and examiners and the testing methodology was carefully considered before the change was introduced.

Objective testing is a tried and tested approach within higher education. Increasing numbers of universities and professional bodies, including those in the medical profession, have adopted this method of assessment even at levels above QCF level four.

Because the questions typically take candidates less time to complete, with answers not having to be written out, it is possible to have a greater number of questions, thereby testing a greater proportion of the syllabus. The breadth of this coverage lessens candidates’ ability to “question/subject spot” since more of the syllabus will be covered.

Additionally, the testing methodology will extend beyond mere knowledge recall, involving comprehension, application and analysis-based questions that focus on the specified learning outcomes for the units in question.

Objective testing also removes any possibility of variations in marking due to subjective factors such as personal bias. All questions are considered to ensure no bias exists in the questions themselves. A range of techniques is also employed to ensure that the objective testing is robust and to minimise the chances of candidates making a lucky guess. This includes, for instance, checks to ensure the removal of obviously wrong answers, ensuring no inadvertent narrowing of the answer pool.

There are also significant benefits associated with testing this way as it enables online examinations and electronic marking, meaning candidates can get immediate provisional results notification, with full confirmation sent within seven days. And in the event that a candidate fails they can enter to re-sit in as little as seven days. This addresses a concern raised by many existing diploma entrants that three exams sessions a year, as is the case with the existing diploma, isn’t flexible enough when trying to balance work/life commitments.

Finally, Gary Bottriell, a highly respected IFA and Fellow of the PFS, has tested the methodology by sitting a 100-question mock exam as part of the rigorous pre-launch assessment. His comment says it all: “Anyone who doesn’t prepare for this exam, thinking it will be easy, is in for a shock.”

Fay Goddard is chief executive of the PFS

Newsletter

News and expert analysis straight to your inbox

Sign up

Comments

There are 23 comments at the moment, we would love to hear your opinion too.

  1. A well argued response – Ms Goddard has clearly done a number of hand written essay papers in her time.

    I took the UK CFA’s Investment Management Certificate, which was Level 4 at the time I believe. It was 3 hours MCQ exam and was definitley not easy.

    The other advantage of MCQs over hand written papers is that people with poor handwriting will not be penalised.

  2. In my view MCQ exams work best where students lose a mark when they get the question wrong. My husband used this type of exam when he was a tutor to discourage medical students from guessing an answer.

    It is good to hear though that the CII will be dealing with many of the problems with the previous exams that advisers were obliged to sit. And about time too!

  3. MR. JOGGA SINGH TEIDY 11th June 2010 at 3:09 pm

    handwritten question/answers are labour intensive, since the examiner has to trawl through verbage to find the candiates answer and perhaps to undersyand it as well…

    mulitichoice pose a clear choice….where there is right and a wrong answer,overcomming the above difficulty..

    I think same number of grey cells have to fire…be the response written or a tick.

    The multi choice favours allows those not too good at their word power to suceed

    Important thing is..has the applicant learnet and mastered the subject matter…and of course security …it is the candidates work…

  4. problem with MCQ is that a monkey typing at random gets 25%. 30 years since I did maths O level and can’t remeber what the expected distribution of scores would be but woud guess that if 1,000 monkeys took such an exam there’s probably a fair chance 1 of them would pass.

  5. Peter Davies @ Create Wealth Management 11th June 2010 at 3:29 pm

    Its absolutely laughable and a total slap on the face to all the advisers who already have gained Diploma and Chartered status.

    I would call on Fay Goddard to resign.

  6. I can see advantages of both frankly, particularly as I’m someone who still has exams to take.

    An advantage of the written is, as I undersatnd it, one of positive marking, so I suppose you can more or less empty your head on to the paper, where as with multi choice theres no where to hide, especially as some of the questions I’ve seen on past written papers can be ambiguous.

    By contract, an advantage of the multi choice is that I imagine you’ll get your result quickly and hopefully will not have to wait 3 months to re-sit if you fail.

    Furthermore, as I’m sure there are a number of IFAs still to sit exams, are there going to enough people qualified to mark them?

  7. Yippee – much easier to pass multiple choice! It’s just FPC with a bit more to remember for that exam, then just forget about it and get on with the job.

  8. p.s. I see the CII benefit from saving lots of time marking papers. I wonder if the cost of the exams will come down?

  9. Of course this is dumbing down, this has been done for cynical commercial reasons.

    If you are asked a question and the answer is one of four in front of you, then you have a 1 in 4 chance of guessing it correctly without any knowledge. With a little knowledge you can narrow this to a 1 in 3 or 1 in 2 chance of a correct guess. That’s completely different to being asked a question with a blank space underneath it.

    Tim – You are not marked on the standard of your handwriting, (mine’s very poor but it hasn’t affected me passing the written JOs or AFs). And I think you’ll find that IMC is level 3. This is strange considering that it is an appropriate exam for discretionary permissions yet AF4 (which is level 6) doesn’t.

  10. I have a degree in Statistics and have sat quite a few CII MCQ exams, many of which included questions where none of the answers was right and some where the intent of the question itself was far from clear.

    A cynic might suggest that the CII has produced a sausage machine which will churn out as many Level 4 qualifications as possible in the shortest period of time. Keep taking then every week and you’re bound to pass eventually and in the meantime will have done wonders for the CII’s cashflow!

    Can I ass-u-me that the exam fees will fall sharply as a result of this change? J08 is currently £272 for the exam and the course material, which is, frankly, a rip-off.

  11. As I see it there are pros & cons to both ways – I’ve a degree/masters/various AF papers, studied IMC/etc etc and have seen both formats

    Here’s a thought: Why not use MCQ, but have 6-8 possibles and a 1/2pt deduction for any incorrect answers – then it would weed out E-Zee and the likes who don’t do any work … …

    … and yes, the cost SHOULD be forced to come down …

  12. I’m told by a very good source R0 exams have come about simply because the Banks complained that their guys found the exams too hard & it was costing them a fortune for them to fail……

    After taking J02 in April – fingernails bitten for results next Friday!!..- I do have sympathy though. 130 marks in a 2 hour exam on Trusts and it felt more like a speed writing test.. Where’s the common sense in that??!!!

    FPC2 was multiple choice and it was by far the hardest of the 3 FPC exams.

    I just like the fact that I’ll know the result straight away like ER1 etc

  13. My colleague had a stroke, a while ago, and hasn’t fully regained the use of his writing hand, although he can work fine in all other respects. This type of exam would certainly solve problem for him.

  14. I’ve been doing CII exams for the last 30 years but now I wonder why I ever bothered – this is a real slap in the face for those of us that have put in time and effort consistently throughout our careers. This has been done simply to get the many, who couldn’t be previously be bothered, qualified in time and just panders to the lazy.
    The only advantage I see if that there will now be a big knowledge gap between the new R papers and the AF papers, so achieving chartered status will be more difficult.
    As for it not being dumbed down, how can 10 points for the new pensions paper possibly compare to JO4 and JO5 which would give 40 points.
    I’m not convinced by the arguments at all I’m afraid, although I can see that instant results and more frequent sittings are beneficial.

  15. How can a multiple choice exam be as difficult to pass as a written exam? You’re giving me the answer; it’s on the piece of paper in front of me!

    The CII just aren’t prepared to pay for the additional marking costs that would be incurred if the papers stayed in their current format.

    I agree that the cost of exams should come down but doubt they will. Bunch of thieving toe-rags.

  16. The fact is that if you sat CF2, CF6, CF7 and CF8 ebfore 2007 you are awarded “Diploma” level points. All of these were multiple choice exams!

    Therefore, many advisers that are near to or already have QCA Level 4 have got through mainly via multiple choice exams. What the CII are doing is correcting the doublestandard.

    No one appeared to make a fuss about this at the time, so why now are people making a fuss over the same system?

    Until we’ve seen the actual test don’t assume it’s a walk in the park. There are many ways to make tests like this more difficult e.g. negative marking, which will heavily penalise those that try to guess their way through the exam.

  17. I think all those who criticise the fact the CII have finally seen sense and believe this is dumbing down should sit the enw exam before they criticise it……
    Like the lady earlier who has just sat J02, I sat it before and by the end of 2 hours, could hardly move my arm as the only use my arms get is typing or kayaking, nit writing things by hand. Much of the syllabus is not relevant to the work I do and at least with mutli choice I can see that my handwriting was not the problem and consider whether the bits I failed are material to the owkr I do (or not). If not, what’s the problem?

  18. Richard Brooks 14th June 2010 at 7:31 pm

    The pass mark is much higher for MCQ exams, it can be set as high as the CII wishes in order to ensure the required standard is met.

    My belief is that the article’s author hasn’t sat any CII examinations so it would be better, in my opinion, if such articles were left to those who know what they’re talking about.

  19. Fay and PFS want us to be be compared with solicitors and accountants. Do they sit multiple choice exams to gain their professional qualifications – I hope not.

  20. Trust the halfwits in this profession to have a go at Fay’s argument…

    Anyone who sat FPC2 will remember how hard the second section was. You simply can’t comment without having seen the papers.

    As for the CII reducing their costs – that would be a first. One of the biggest rip off organisations ever created now sees a great opportunity to make more money. I bet there will be more people taking it regularly and they stand to make more money than running 3 sessions per year.

    My guess is that the exam won’t be as easy as some dumbos think. More will have to be learned which increases the probability of failure. More money for the “non profit making” CII

  21. I have many FS qualifications as well as a Degree in Psychology which was assessed both MCQ and written.

    MCQ’s entail recognition memory whereas written requires recall memery (the latter generally being accepted as harder); however there are issues regarding quality and bias.

    Applied in practice, the adviser would generally rely on recognition using the tools of her/his trade and not exam style writing (off the cuff) for providing advice. MCQ’s would therefore seem appropriate.

    However, the most important thing is not how the exam is passed, but how competence is maintained in the long run.

  22. Having sat FPC and G60 and G10 I am in a good position to comment on both. MCQ’s are easier no doubt. The argument that you have to remember more for MCQ’s doesnt add up. To pass G60 required full knowledge of the subject as you didnt know that you would be questioned on. With MCQ’s the question is in front of you and by careful answer them by eliminating incorrect options. It is amazing that the PFS (and yes I am a member) has suddenly embraced MCQ’s when in the past it was such a bastion of written exams for ‘higher’ level subjects.

  23. Not this old chestnut again.

    For heavens sake lets focus on what is important. Using “handwriting” as a method of conveying intelligence should be dead and buried. All exams should be taken by computer. Think of the poor folk who have to sit and mark a handwritten exam paper written by someone with penmanship worse that a Dr. Who on earth spends hours writing page after page of prose, technical or otherwise.

    Lets get modern about this please.

Leave a comment

Close

Why register with Money Marketing ?

Providing trusted insight for professional advisers.  Since 1985 Money Marketing has helped promote and analyse the financial adviser community in the UK and continues to be the trusted industry brand for independent insight and advice.

News & analysis delivered directly to your inbox
Register today to receive our range of news alerts including daily and weekly briefings

Money Marketing Events
Be the first to hear about our industry leading conferences, awards, roundtables and more.

Research and insight
Take part in and see the results of Money Marketing's flagship investigations into industry trends.

Have your say
Only registered users can post comments. As the voice of the adviser community, our content generates robust debate. Sign up today and make your voice heard.

Register now

Having problems?

Contact us on +44 (0)20 7292 3712

Lines are open Monday to Friday 9:00am -5.00pm

Email: customerservices@moneymarketing.com