Support Vector Machine Based on Type-2 Fuzzy Training Samples

  • cc icon
  • ABSTRACT

    In order to deal with the classification problems of type-2 fuzzy training samples on generalized credibility space. Firstly the type-2 fuzzy training samples are reduced to ordinary fuzzy samples by the mean reduction method. Secondly the definition of strong fuzzy linear separable data for type-2 fuzzy samples on generalized credibility space is introduced. Further, by utilizing fuzzy chance-constrained programming and classic support vector machine, a support vector machine based on type-2 fuzzy training samples and established on generalized credibility space is given. An example shows the efficiency of the support vector machine.


  • KEYWORD

    Type-2 Fuzzy Training Samples , Mean Reduction Method , Fuzzy Chance-Constrained Programming , Support Vector Machine

  • 1. INTRODUCTION

    The fuzzy set was proposed by Zadeh (1965) in 1965, it is an interesting extension of classical set. In classical sets the characteristic function can take only one of two values from the {0, 1} , in fuzzy sets the membership function can take any value from the interval [0, 1] . The membership function plays an important role in fuzzy set theory and its applications. But in some actual problems, it is often very difficult to determine the value of membership function (or membership degree for short). To solve this problem, the concept of type-2 fuzzy set was introduced by Zadeh (1975) in 1975. A type-2 fuzzy set is characterized by a special membership function, the value of which for each element of this set is a fuzzy set in [0, 1] . Compared with ordinary fuzzy set, type-2 fuzzy set could describe the objective phenomenon more accurately because of secondary membership function. It has been applied more efficiently to pattern recognition and other machine learning fields successfully (Mitchell, 2005).

    Support vector machine built on statistical learning theory is a kind of effective machine learning method, has advantaged superiority in dealing with small samples classification problems, and has become the standard tool in machine learning field now (Cristianini and Shawe-Taylor, 2000). The traditional support vector machine is based on the real valued random samples and established on the probability space, and it is difficult to handle the small samples classification problems with non-real random samples on non-probability space. Naturally, it is a very interesting and valuable research direc-tion to construct the support vector machine based on non-real valued random samples and established on nonprobability space or probability space. On this basis, Lin and Wang (2002) proposed the support vector machine based on a non-real valued random samples-fuzzy samples, constructed the fuzzy support vector machine, which determines membership degree by different weight for each sample. Then Ha et al. (2009, 2011) improved the fuzzy support vector machine in determining membership degree, and they constituted a new fuzzy support vector machine and an intuitionistic fuzzy support vector machine. Considering that above support vector machines only make the samples fuzzified and the training samples are not fuzzy input directly, Ji et al. (2010) constructed a kind of support vector machine for classification based on fuzzy training data, and discussed its application in the diagnosis of coronary. Nevertheless the support vector machine is limited to the ordinary fuzzy number samples-the triangular fuzzy samples, and it is difficult to deal with the classification problems of type- 2 fuzzy number training samples which is a generalization of the ordinary fuzzy number samples. In order to handle the problem, this paper firstly discusses the support vector machine based on type-2 fuzzy training samples and established on generalized credibility space.

    2. PRELIMINARIES

       2.1 The Type-2 Fuzzy Set

    Definition 2.1(Zadeh, 1965): A fuzzy set A in X is characterized by a mapping μA : X → [0, 1], χμA(χ), μA is called the membership function of A and μA(χ) is called the membership degree of χ in A. The symbol

    image

    represents A. A collection of all fuzzy sets of X can be denoted by

    image

    Definition 2.2: Suppose a, b, cR. a is a fuzzy set in R, and its membership function is as follows

    image

    Then a is called triangular fuzzy number, and is represented by a = (a, b, c) (refer to Ha et al., 2010)

    Definition 2.3(Zadeh, 1975): A type-2 fuzzy set in X , denoted

    image

    is characterized by a secondary membership function

    image

    where χX, uJ x ⊆ [0,1], and

    image
    image

    can also be expressed as

    image

    Definition 2.4 (Qin et al., 2011): A type-2 fuzzy set

    image

    is called triangular if its secondary membership function

    image

    is

    image

    for x ∈ [r1, r2], and

    image

    for x ∈ [r2 , r3], where θ1, θ2 ∈ [0, 1] are two parameters characterizing the degree of uncertainty that

    image

    takes the value x. For simplicity, we denote the type-2 triangular fuzzy number

    image

    with the above condition by (r1, r2, r3 ; θ1, θr).

       2.2 The Mean Reduction Method

    Definition 2.5(Qin et al., 2011): Let X be a nonempty set, P( X ) be the class of all subsets of X , Pos be a possibility measure in P(X). Suppose a is a fuzzy number, then the possibility measure of fuzzy event ab is defined by Pos({ab} ) = sup {μa(χ), χR, χb} . Similarly, Pos({ a < b} ) = sup{μa(χ), χR, χ < b}, Pos({a = b}) = μa(b).

    Definition 2.6(Qin et al., 2011): Let A be a fuzzy set in X, and

    image

    be a type-2 fuzzy set in X. A is the reduction of

    image

    via the mean reduction method, if it satisfies

    image

    We denote

    image

    Example 2.1(Qin et al., 2011): Let

    image

    be a type-2 triangular fuzzy number. According to the definition 2.6, we have the reduction a of

    image

    and the membership function of a is as follows.

    image

       2.3 The Generalized Credibility Measure

    Definition 2.7(Qin et al., 2011): Suppose a is a fuzzy number. The generalized credibility measure

    image

    of the event {ab} is defined by

    image
    image

    Theorem 2.1(Qin et al., 2011): Let ai be the mean reduction of type-2 fuzzy triangular number

    image

    i = 1, 2, … , n. Suppose a1, a 2, … ,an are mutually independent, and ki ≥ 0, i = 1, 2, … , n. Given the generalized credibility level

    image

    is equivalent to

    image

    Proof: It can be easily got by type-2 fuzzy set theory in the reference.

    3. SUPPORT VECTOR MACHINE BASED ON TYPE-2 FUZZY TRAINING SAMPLES AND APPLICATION EXAMPLE

       3.1 Support Vector Machine Based on Type-2 Fuzzy Training Samples

    Consider the fuzzy type-2 training samples set

    image

    where

    image

    and

    image

    i = 1, 2, …, m, j = 1,2 … , n, when yi = 1, then

    image

    is called a positive class; when yi = -1, then

    image

    is called a negative class. The classification based on the type-2 fuzzy training set

    image

    is to find a decision function

    image

    such that the positive class and the negative class can be separated with the low classification error and good generalization performance.

    Definition 3.1(Ji et al., 2010): For the type-2 fuzzy training samples set

    image

    if for a given level

    image

    there exist wRn, bR, such that

    image

    Then the type-2 fuzzy training samples set

    image

    is strong type-2 fuzzy linear separable.

    The support vector machine for strong linear separable type-2 fuzzy sample set is to solve the fuzzy chance constrained programming:

    image

    To solve the above programming,

    image

    is equivalent to

    image

    i = 1, 2, …, m, j = 1, 2, … , n according to Theorem 2.1. Then the fuzzy chance constrained programming is converted into the following classic convex quadratic programming:

    image

    We can obtain its dual problem:

    image

    w* , b* are the solution of above dual programming. For a given confidence level

    image

    if

    image

    then

    image

    belongs to the positive class; if

    image

    then

    image

    belongs to the negative class.

       3.2 An Example

    We shall apply the support vector machine for twoclass classification with type-2 fuzzy training samples to the diagnosis of Coronary. The data is sourced from the reference (Ji et al., 2010), the half of which are healthy (yi = 1), the others are Coronary patients (yi = -1). Let θl,1i = θ r,1i = 0.1, θl,2i = θr,2i = 0.2, we can obtain type-2 triangular fuzzy numbers

    image

    and

    image

    The programming was solved by SVM toolbox of matlab, when parameter C =1, λ = 0.8. The accurate rate of diagnosis is 80% . The result of example illustrates that the proposed support vector machine is effective.

    4. CONCLUSIONS

    This paper constructs firstly the support vector machine in which the training samples is type-2 fuzzy number input, and the proposed support vector machine is an interesting extension of the traditional support vector machine and the fuzzy support vector machine. We shall focus on the proposed support vector machine’s applications in some practical problems.

  • 1. Cristianini N., Shawe-Taylor J. (2000) An Introduction to Support Vector Machines and Other Kernelbased Learning Methods google
  • 2. Ha M. H., Peng G. B., Zhao Q. H., Ma L. J. (2009) A New fuzzy support vector machine [Computer Engineering and Applications] Vol.45 P.151-153 google
  • 3. Ha M. H., Wang C., Zhang Z. M., Tian D. Z. (2010) Uncertainty Statistical Learning Theory google
  • 4. Ha M. H., Huang S., Wang C., Wang X. L. (2011) Intuitionistic fuzzy support vector machine [Journal of Hebei University (Natural Science Edition)] Vol.3 P.225-229 google
  • 5. Ji A. B., Pang J. H., Qiu H. J. (2010) Support vector machine for classification based on fuzzy training data [Expert Systems with Applications] Vol.37 P.3495-3498 google doi
  • 6. Liu B. D. (2002) Theory and Practice of Uncertain Programming google
  • 7. Lin C. F., Wang S. D. (2002) Fuzzy support vector machines [IEEE Transaction on Neural Networks] Vol.13 P.464-471 google doi
  • 8. Mitchell H. B. (2005) Pattern recognition using type-II fuzzy sets [Information Sciences] Vol.2 P.409-418 google
  • 9. Qin R., Liu Y. K., Liu Z. Q. (2011) Critical values reduction methods for type-2 fuzzy variables and their applications [Journal of Computational and Applied Mathematics Mathematics] Vol.1 P.1454-1481 google
  • 10. Qin R., Liu Y. K., Liu Z. Q. (2011) Modeling fuzzy data envelopment analysis by parametric programming method [Expert Systems with Applications] Vol.38 P.8648-8663 google doi
  • 11. Vapnik V. N. (1995) The Nature of Statistical Learning Theory google
  • 12. Zadeh L. A. (1965) Fuzzy sets [Information and Control] Vol.8 P.338-353 google doi
  • 13. Zadeh L. A. (1975) The concept of a linguistic variable and its application to approximate reasoning-I [Information Sciences] Vol.8 P.199-249 google doi