Support Vector Machine (SVM) employs Structural Risk Minimization (SRM) principle to generalize better than conventional machine learning methods employing the traditional Empirical Risk Minimization (ERM) principle. However, training SVM requires large memory and long cpu time when training set is large. One way to circumvent this computational burden is to select some of training patterns in advance which contain most information given to learning. One of the merits of SVM theory distinguishable from other learning algorithms is that it is clear that which patterns are of importance to training. Those are called support vectors (SVs), distributed near the decision boundary, and fully and succinctly define the classifiation task at hand. Furthermore, on the same training set, the SVMs trained with different kernel functions, i.e., RBF, polynomial, and sigmoid, have selected almost identical subset as support vectors. Therefore, it is worth fiding such would-be support vectors prior to SVM training. In the thesis, we propose neighborhood property based pattern selection algorithm (NPPS) which selects the patterns near the decision boundary based on the neighborhood properties. We utilizes k nearest neighbors to look around the pattern's periphery. The first neighborhood property is that a pattern located near the decision boundary tends to have more heterogeneous neighbors in their class-membership.'' The second neighborhood property dictates that an overlap or a noisy pattern tends to belong to a different class from its neighbors." The first one is used for identifying those patterns located near the decision boundary. The second one is used for removing the patterns located on the wrong side of the decision boundary. These properties are first implemented as a naive form with time complexity O(M2) where M is the number of given training patterns. Then, to accelerate the pattern selection procedure we utilize another property. The third neighborhood property is that the neighbors of a pattern located near the decision boundary tend to be located near the decision boundary as well.'' The third one skips calculation of unnecessary distances between patterns. This advanced form of algorithm, fast NPPS, has a practical time complexity of O(vM) where v is the number of patterns located in the overlap region. The number of patterns located in the overlap region v is closely related to determine a parameter of NPPS, the number of neighbors k, accordingly we provide a heuristic method to set the value of k. Then, we prove invariance of the neighborhood relation under the input to feature space mapping, which assures that the patterns selected by NPPS in input space are likely to be located near decision boundary in feature space in where SVs are defined. NPPS is demonstrated on synthetic as well as real-world problems. The results show that NPPS reduces SVM training time up to almost two orders of magnitude with virtually no loss of accuracy. In addition, to exemplify a practical usage of the algorithm, we applies NPPS to marketing domain problem - response modeling. This experiment is rather a domain-oriented application than a simple algorithm-oriented application. We diagnose the domain specific difficulties which can arise in practice when applying SVM to response modeling, then propose how to alleviate and solve those difficulties: informative sampling, different costs for different classes, and use of distance to decision boundary. Again, SVM with NPPS achieved the accuracies and uplifts comparable to those of SVM with 80–100\% random samples, while the computational cost was comparable to those of SVM with 10–20\% random samples.