Off-line handwriting recognition is currently an unsolved problem, as different people have different handwriting styles. In this paper, we describe a new pyramidal neural network system for high accuracy recognition of separated handwritten characters. The proposed architecture is initiated by two types of alternative layers: pyramidal layers perform feature extraction and subsampling layers reduce the dimensionality. Shared weights are applied among sliding windows that collect input on the pyramidal layers. Consequently, full-connected linear layers connect subsampling layers and classification output, constituting the third type of layers of the architecture. This study also shows that some preprocessing procedures may significantly improve the recognition rate, including locating center of mass and predicting a possible subset. Experimental results and comparisons are given based on National Institute of Standards and Technology (NIST) special database 19. The approach achieves better accuracy and faster training and recognition process than traditional convolution neural network.