Predicting response from load : an application of multivariate analysis techniques to computer performance data

Abstract

This thesis takes the form of a case-study. It is concerned with the application of principal component analysis and regression analysis to computer performance data. The problem considered was that of finding a way to quantitatively assess the effect of changes in a computer's scheduling parameters on its response. Two variables (# responses for PQI/CMQ swapin, denoted by y(l), and ticks for PQI/CMQ swapin response, denoted by y(2)) were taken as measures of response and a procedure devised to assess, for a given load, the change in these variables brought about by a change in the parameters. In order to make such an assessment, a single measure of load is required. There are 26 different measures of the load on different parts of the system and these measurements are recorded automatically every minute that the system is operating. The data available for this study was all these measurements for the months April, May and June 1980. A subset of 1000 points was taken from the April and May data and a principal component analysis done on this subset. Some properties of the first principal component were studied and this component was accepted as an index of load. The index was then used to divide points from the April and May data set into groups of similar load and regression equations relating some of the load variables to each of y(l) and y(2) were then estimated for each group. A satisfactory equation for y (1) could be found for each group but not for y(2). After a process of trial and error it was found that an equation for the transformed variable Z = log(y(2)) could be found for most but not all of the groups. Then it was found that the groups could be further combined without significant loss of information into three groups and that equations could be fitted for y (1) to all three groups but for Z only to two of the groups. The ability of these equations to predict response from load was then tested by drawing a sample of points from the June data and comparing the observed values of y (1) and Z with the values predicted by the equations. Thus the procedure to assess the effect of a parameter change was to select a sample of points from immediately after such a change and use the equations (which have been estimated using data from before the change) to predict the response variables. These predictions represent estimates of the expected response for these points under the old parameter settings and the observed values represent the actual response under the new settings. Thus a comparison between the observed and the actual response gives a measure of the change in response brought about by the parameter change. Finally this procedure was applied to some data taken from September 1980 (after a change in the parameter values) and was found to perform satisfactorily.