Have you ever wondered how banks make their lending decisions? Working with the Scorecard module in STATISTICA has given me quite a bit of insight into the decision-making process regarding lending in some banks. One new feature coming out in the next version of STATISTICA is Calibration Tests. This tool makes it possible for banks to test whether or not the forecast probability of default (PD) has been the PD that has actually occurred. If it has, then all is well and good. However, if the forecast PD and the realized PD are different to a large degree, a banker somewhere may need to reevaluate their lending model.
Banks typically divide customers up into segments of alike customers, realizing that each separate segment will have a certain number of customers who meet their credit obligations and a certain number who will not. Based upon the model the bank has agreed upon, each segment has a forecast PD. After the bank has used this model for a specified length of time, the bankers want to check and make sure the forecast PD and the realized PD match. The banks will use two types of data, or files, to test their model. They will have the master file, which has the forecast PD for each segment of their customer profile. Then they have the credit activity file, which has information about what segments customers in a specific time frame belonged to and whether those customers met their obligations or defaulted. These are the files that STATISTICA uses to test the forecast model.
Once you tell STATISTICA about the two files, it’s really very easy to run the test and interpret the results. There is even a built-in "traffic light approach", which you can read about in a great reference on guidelines in credit risk management (Oesterreichishe Nationalbank, 2004). In the dialog shown below, you will see all the default settings that are used in Calibration Tests in STATISTICA.
Once you click OK, you get the results. A portion of those results are below.
Yes, it is as intuitive as it seems. The "traffic light approach" tells you that those segments with a yellow light probabily have an underestimated PD and those with the red light definitely have an underestimated PD. The officers at this bank would need to spend some time updating their model for sure.
It can, of course, be more complicated. However, even with something like Calibration Tests, STATISTICA offers the tools and resources that will provide the best solutions to customers including those in the banking industry.
Oesterreichishe Nationalbank. (2004). Guidelines on credit risk management: Rating models and validation. Vienna, Austria: Oesterreichishe Nationalbank.