Thereareanumber of reasonableprocedurestoguardagainstmodel risk.Thefollowing
list is moreofa check"1 thana process for rigorousanalytictesting of theproposed model.
Understand that model riskisinevitable,hecognizantof the limitations ofagiven model,and besure modelsareappliedin the appropriatesituation.
Analyze if themodel’s performance is sensitive tokey assumptions.
Verifythat the modelworks correctlyforsimple problemswithknownsolutions.
Choose thesimplestmodel thatprovidesusefulresults. Onlyaddcomplexity ifthere Lsa strongneed.
Place moreconfidenceinamodel drat has been backtested andstress tested.
When applicable, simulatemodel risk toprovide usefulinformationonitsperformance.
Thecaveatis,ofcourse,that notall riskiscaptured andhence, the model riskwill exceed theestimate.
Payattention tosmall problems,whichcanhe indicativeoflarger problemswithdie model.
Conductsimplediagnostic testing of residualsto reveal informationaboutthe model’s predictive abilityandidentifyoutliersor other abnormalitiesin thedata.Usenon- parametric testsifappropriate.
Reevaluateand update models regularlytostaycurrentwithrecem trendsandchanging market conditions.
The Role ofSeniorManagers
AIM39.5:Summarizethe role ofsenior managersin managing model risk.
“Whilesenior managersneed notheexpertmodelersperse, it isimperative they understand thefundamentals of model riskandestablishanorganizationalframework thatimplements sound riskmanagementprocedures.Managing model riskitsimilar tomanagingother
casks; managersshould question modelfeatures,overseeinternalprotocol,andlisten to subordinateconcerns.In addition, managersmust overseeindividual position-takingand instill riskmanagement as a meansto providecorporatestabilityrather dian as anobstacle
©2013Kaplan,Inc.
Page 90
Topic39 CrossReferencetoGARP AssignedReading—Dowd,Chapter16
toindividual{excessive) risk-talcing, especially iftied tocompensation.Managersmust track model performance,implementation,andmisuse.
AIM39.6:Describe proceduresforvettingand reviewingamodel.
AIM39.7: Explain the function ofan independent risk oversight(IRO) unit.
Models should bevettedforcompleteness beforeimplementation.Adetailedprocedure would involvereviewofdocumentation,includingtheoriginalcode,verification of model appropriatenessbased on portfoliocomposition, and establishmentandtesting against benchmark and no-arbitragerelationships. Vettingismosteffective againsta backdropof organizationalfreedom toindependentlymonitorand evaluateriskmanagement functions.
Thisisparticularly trueif the evaluators will bepartof the risk managementdivision.
Regularauditsof the models and proceduresand efficient record keepingarealso important featuresof thevettingprocess.
Theindependentriskoversight(IRO) unit isideallyamiddle-officeunit responsible for overall riskmanagementandmeasurement.Thelinkheadshouldreportdireedyto theCEOandmayevensitontheboardof directorstocommunicateperformanceand strategy. Procedureandposidonlimitswillvarybydieoverall risk-takingbehavior of the firm.Sound practice willinsulate the unitfrom the pressures of the front-office and traders.
Additionally,compensatenwill notbebased on theperformanceofotherunits.
CrossReferencetoCARPAssignedReading-Dowd,Chapter16
KEY CONCEPTS
AIM 39.1
Model riskis the risk associated withusingfinancial modelstosimulatecomplex relationships.
All modelsareinherentlyflawed,sostriveforsimplicityand judgemodels on their predictive ability rather thanon (often incorrect) assumptions.
Model riskmayarisefrom incorrectmodel applications, implementationrisks,calibration
errors, programmingerrors, anddata problems.
AIM 39.2
Measuringmodel risk assumesanunderlying distributionforasingle unknown parameter
andgenerates aconfidence intervalforVaR.
AIM39.3
Simulationmethodsareusedfor estimating model risk formultiple parameters.
AIM39.4
Managerscan protectagainstmodel riskbyemployingrealitychecksof the model, including testingagainst knownsolutions, backtesting andstresstesting, updatingthe model,andanalyzingsensitivity to parameterassumptions.
AIM39.5
Senior managers need not beexperts inmodeling, but shouldbeknowledgeable enough to
questionmodel inputs, uses,and limitations.
AIM39.6
Vettinginvolvesdetailed review ofdocumentation, includingoriginalcode, verificationof model appropriatenessbasedonportfolio composition, andestablishing and testing against benchmarkandno-arbitragerelationships.
AIM39.7
Theindependent riskoversight(IRO) unit is responsible for overall riskmanagementand
measurement;itshouldoperate independendyof other businessunits.
©2013Kaplan,Inc.
Page 92
Topic39 CrossReferencetoGARPAssignedReading—Dowd,Chapter16
CONCEPT CHECKERS
Whichof thefollowingisnota sourceof model risk?
A. Programmingerrors.
B. Failuretorecalibratemodels.
C. Minimal roundingerrorsfromalgorithms.
D. Implementationof models for situations for which die modelswerenot originallydesigned.
Financial models:
A. arenotlikely tobecomeoutdated.
B. areinherentlyaccurate instruments.
C. should be used in theIRO unitexclusively.
D. containheterogeneouslevelsof model risk.
1.
2.
Riskmanagers should manage model riskusingwhichof thefollowingapproaches?
Managers should:
I. alwaysadd complexity to Lhe model.
II. identifyoutliers within die model'sdata 3.
A. Ionly.
B. IIonly.
C. BothIandII.
D. NeitherI norII.
Whichof thefollowingstatementsconcerningquantifyingmodel riskisfalse?
A. Quantifying modelriskoften usesVaRestimatestomeasurethe loss level.
B. Parameter estimationmayuseparametricor non-parametrictests.
C. Estimatesof model riskarelikelyto underestimate thetruerisk level.
D. Quantifyingmodel riskissimplifiedduetodie unknown distributionof the
parameter.
Managerscan bestguardagainst model riskbyperformingwhichofthefollowing oversightactivities?
A. Backtestingandstress testingestimates.
B. Testingmodelsagainst unknown problems.
C. Minimizing small problemsas inherentinparameter estimation.
D. Developingcomplex modelsand testingwithbodi parametric andnon- parametrictests.
Foradditional Book3, Topic39practice questionsseer 4.
5.
Self-Test Questions:#5 (page273)
PastFRM Exam Questions:#10-15(page280)
Topic
CrossReferencetoGARP Assigned Reading— Dowd,ChapterId
CONCEPT CHECKER ANSWERS
1„ C Minimal roundingerrors isanattractivefeatureofanalgorithm.
2. D Model risk willvaryfrommodeltomodelbasedoncomplexity data qualityand implementation,
3. B Managing model risk involvesbeingcognizant ofthe limitationsofagiven model,
reevaluatingandupdatingmodelsregularly andidentifyingoutliersorother abnormalitiesin the model’s data. The manager should choose thesimplestmodel thatprovides usefulresults andonlyadd complexityif there isa strongneed.
4. D Modelriskmustassumeadistributionof theunderlyingparameterThisincreasesthe model riskcomparedtoknowledgeof theunderlyingparameterdistribution.
5. A Backtestingandstresstestingarcstandardprocedurefor tesdng models.Modelsshould he testedagainstknownproblems,notunknownproblems.Smallproblemscanbeindicativeof larger modelingissuesand shouldnotbeignored, Simple modelsarepreferredunless thereis strongreason toadd complexity,
Page94 ©2013Kaplan,Inc.
Thefollowingis a(hiewof dieOperationalantiTnleÿtaledRiskManagement,ptincipies designedLoaddress theAIMsiaiementssetforthby GART®,ThisLopit isalso coveredin: