|
AboutSVMPackage
Why we use it?In order to improve the accuracy of falling detection, we try to use the SVM algorithm to achieve the goal. Information About the SVM Package这是SVM工具包下载链接: http://hcsfsp.googlecode.com/files/libsvm-2.9.zip 这是SVM工具包的pdf详细说明文档: http://hcsfsp.googlecode.com/files/guide.pdf 下面是开发者个人主页的超链接: http://www.csie.ntu.edu.tw/~cjlin/ Let’s introduce SVM to you briefly. What is SVM? SVM, Support Vector Machine, in short it is a bit like neural network, but today most often it is brought to make categories. In other words, if I have a bunch of things which have been divided into many categories, but the principle of the classification is unknown. Then when we receive something new, SVM can predict where the new information should go. It sounds wonderful, but the SVM based on statistical learning theory can resolve this issue beautifully within a reasonable time. An example of SVM: In order to get examples to illustrate the problem, assuming that I marked a bunch of space classification using color-point. The color is its type and the location is its information, then SVM can find the formula to cut apart these segments of points , thereby separating the different areas one by one; when we get a new point, as long as we know which region it belongs to,we are able to determine which color (category) the point is. The distribution of original data The result by using SVM When you have a overview of the SVM, Let’s talk about how the program we wrote. We use “SVM package” which can help us realize the SVM algorithm. The brief flow of how the algorithm works are as follows: • Transform data to the format of an SVM package • Conduct simple scaling on the data •Consider the RBF kernel K(x,y) = e(〖-γ||x-y||〗2 ) • Use cross-validation to find the best parameter C and γ • Use the best parameter C and γ to train the whole training set • Test The Programs: SVMTRAIN: It is used to train data, and the reason why running SVM has been dubbed "open-Train" also came from this program. Train will only accept the particular form of input, and then it results in a "Model" file. This model can be understood the SVM's internal information, because we need model to predict, and we cannot directly use original data. Assuming that train itself is a very time-consuming step while after training information can be maintained in some form of internal data, then next time when we want to predict, the internal data can be directly loaded for predicting. SVMPREDICT: In accordance with training model, coupled with a given input (new value),we can get the output prediction type. SVMSCALE: Rescale data. Because the scope of the original data may be too large or too small, svmscale can scale data to the appropriate range of data. File Format: label index1:value1 index2:value2 ... label index1:value1 index2:value2 ... A data line, such as: +1 1:0.708 2:1 3:1 4:-0.320 5:-0.105 6:-1 7:4.3 8:0.23 Label Or we should say class which are the classification types, and they are usually integers. Index It is a sequence of the index, usually we put consecutive integers. Value It is used to train the data, usually a bunch of real numbers. The structure of each line is the same, the meaning is: I have a row of data, namely, value1, value2, ... ..., valueN, (and they were orderly specified by the indexN), the result of these is label. The input training data is a vector, which is a row of x1, x2, x3, ... ... these values are valueN, and in x n, n is specified by indexN. These things are also known as the "attribute". Most of the time we have a lot of information on a given "characteristic (feature)" or "attribute", so the input would be a group. Take the drawing points partition for an example, each point has its own X and Y coordinates, then each point has two kinds of attribute. If I have two points: (0,3) and (5,8) at the label (class) 1 and 2, it will be written as 1 1:0 2:3 2 1:5 2:8 Similarly, the three-dimensional coordinate space is equivalent to three sets of attribute. The greatest advantage of this file format is that you can use the sparse matrix, or that some data of the attribute cannot exist.Svmtrain The general syntax of Svmtrain is: Svmtrainoptionstrrain_set_filemodel_file train_set_file is the previous format, if you do not give a certain name, the model_file will be called trrain_set_file. model.Options can be not given at first. The implementation of the following programs will have heart_scale.Model file:(the screen output is not very important,and no error is ok.) Microsoft Windows XP [版本 5.1.2600] (C) 版权所有 1985-2001 Microsoft Corp. C:\libsvm-2.9\windows>svm-train heart_scale optimization finished, #iter = 162 nu = 0.431029 obj = -100.877288, rho = 0.424462 nSV = 132, nBSV = 107 Total nSV = 132 The general syntax of Svmpredict is: Svmpredict test_file model_file output_file test_file is the information that we want to predict . Its format is the same with the svmtrain input, that is, train_set_file, predict will take the value of prediction and the value of test_file for comparison.The labels in the test_file is the real classification results, if we use them by comparing them with the outcome of our predictions,then we can know whether the predictions are correct or not. Because of its same format,we can just take the original training set as test_file then use svmpredict to predict, according to the correct rate, we are ready to transfer parameters. model_file is the out file of svmtrain, output_file keeps the output files. The output format is very simple, each line there is a label, corresponding to each line in test_file. The implementation of the following programs will have heart_scale.Out Microsoft Windows XP [版本 5.1.2600] (C) 版权所有 1985-2001 Microsoft Corp. C:\libsvm-2.9\windows>svm-predict heart_scale heart_scale.model heart_scale.out Accuracy = 86.6667% (234/270) (classification) Therefore,by using the original input to predict, the first line of the output is the accuracy of the prediction.If your input has no label, then it will be a real prediction. Scaling The appropriate scale can contribute to make a good choice of parameters and the speed of svm. Svmscale will scale every attribute. Range with l, u specified, usually [0,1] or [-1,1], the output is saved in the stdout, we should pay attention to the testing data and training data, because they have to scale at the same time. In Svm-scale the hardest thing is that there is no way to specify the testing data / training data (different file) and then do the scale at the same time. Cross Validation In general, the approach of using SVM(in determining the parameters) is often like this: 1. First, the pile has been divided into many different types of information. 2. Split them into several groups of training set. 3. We use a certain set of parameters to train and then predict other groups to see correct rate. 4. If the accuracy is not enough, then change parameters to repeat the process of train / predict. If we are lucky to find a set of nice parameters, we will take this group of parameters to build model and use them to make the final prediction of the unknown data. This whole process is called cross validation. In the process of finding parameters, we can use svmtrain built-in cross validation for help. -v n, n is the number of groups to split into , such as n = 3,the dataset will be split into three groups, and then first we cast the one and two to train model and predict three to get the correct rate; second we cast the two and three to train model and predict one to get the correct rate; at last we cast the one and three to train model and predict two to get the correct rate, and so on. If there is no cross-validation, it is likely to find good parameter only at some particular inputs. Some important parameters: In general, the important parameter is gamma (abbreviated as g) and cost (referred to as c), while the parameters of the cross validation (-v) is set as 5 usually. The default value of cost is 1,the default value of gamma is 1 / k, k is equal to the amount of input data. How to determine parameters: It is totally working by trying, trying to find a better parameter values. In the process of trying, we increase or decrease the parameter value by exponential growth to approach a better result, which is 2 ^ n (2 n-th power). Because there are two sets of parameters, it is equivalent to try n n = n ^ 2 times, this process is not a continuous growth. We can imagine like this: in an XY plane within a specified range, we need find a group of lattice points, through the conversion, X and Y of each lattice point are look upon as the cost and the gamma to do cross validation. |