The rationale for using linear regression and the r-squared (R^2) value as a measure of fit is that the predictions of a model may be SYSTEMATICALLY off target in a way well accounted for by a linear model. Implicit in the use of linear regression between the predicted or estimated development effort (independent variable) and actual effort (dependent variable), is the definition of a new composite model: the model formed by machine induction plus the linear model formed through regression). Using such a composite model, we would generate a prediction using the composite model and then adjust it using a previously derived best fitting line. This assessment measure may better match our intuitions and keeps assessment focused on MRE values exclusively. Rather than giving the R^2 values, we could evaluate an inductive model's predictions by simply looking at the MRE's of predictions stemming from the composite model. This is what we do in the table showing results with Kemerer's data (corresponds to Table 1 of article): Cart' Est + regress eq. Cart' (102.5 + 0.075x, Act Est Table 2 of article) (truncated) effort error(%) <-- relative to Act 287 1893 244.5 14.8 82 162 114.65 38.97 1107 11400 957.5 13.53 86 243 120.73 38.93 336 6600 597.5 77.67 84 129 112.19 33.56 23 129 112.19 383.58 130 243 120.73 7.34 116 1272 197.9 70.6 72 129 112.19 55.82 258 243 120.73 53.33 230 243 120.73 47.67 157 243 120.73 23.1 246 243 120.73 51.1 69 129 112.17 60.47 MRE 64.71 (compared to 364% MRE) Using the best fitting line (Table 2) constructed from all of Kemerer's data may lead to an overly optimistic MRE that we would realize in practice when using the compound model. Rather, we can do a take-one-out (N-fold cross validation) strategy -- use 14 (of the 15) predicted effort values for Kemerer's projects. In the table below, we summarize the results in the table above, but additionally show the linear equation of best fit generated over each set of 14 (of the 15) and used to adject the effort prediction for the 15th model. A different equation for estimation is generated for each project (using the remaining 14 projects). Compound Cart' Model Compound Model Ests. using N-fold CV Act Est Est ------------------------------------------------ (truncated) linear eq. R Est. effort error (%) 287 1893 244.5 99.70 + .0750x .9132 241.72 15.78 82 162 114.65 105.41 + .0748x .9113 117.53 42.46 1107 11400 957.5 125.99 + .0351x .6234 525.92 52.50 86 243 120.73 105.46 + .0748x .9116 123.64 42.28 336 6600 597.5 106.65 + .0874x .9621 683.30 103.18 84 129 112.19 105.07 + .0748x .9112 114.73 36.58 23 129 112.19 110.51 + .0746x .9136 120.08 417.59 130 243 120.73 101.71 + .0752x .9119 119.99 7.91 116 1272 197.9 108.70 + .0749x .9156 204.01 75.87 72 129 112.19 106.14 + .0747x .9114 115.79 60.82 258 243 120.73 82.52 + .0775x .9325 101.36 60.82 230 243 120.73 92.90 + .0762x .9200 111.42 51.70 157 243 120.73 98.16 + .0756x .9122 116.54 25.77 246 243 120.73 91.48 + .0764x .9222 110.04 55.43 69 129 112.17 73.07 + .0790x .9085 83.27 19.13 MRE 71.19 In sum, the high R^2 (0.83) for the CART' estimates of Table 1 (of article) translates into a relatively low MRE (64.71% or 71.19% depending on methodology) compared to Cart' MRE of 364%, after using the line of best fit to adjust Cart' estimates -- apparently, Cart's errors are systematic and can be adjusted to good effect. *The Act and Cart' Estimated values are truncated for presentation purposes only -- see Table 1 of article for values used here.