diva.sketch.recognition

Class BasicStrokeRecognizer

java.lang.Object

diva.sketch.recognition.BasicStrokeRecognizer

All Implemented Interfaces:

StrokeRecognizer

Direct Known Subclasses:

PanZoomInterpreter.LocalRecognizer

public class BasicStrokeRecognizer
extends Object
implements StrokeRecognizer

BasicStrokeRecognizer performs recognition on completed strokes by filtering them, extracting features from them (e.g. aspect ratio), and then passing the extracted features into a classifier that performs classification on the feature set. It also provides methods to train the classifier if the classifier is trainable.

A BasicStrokeRecognizer uses a classifier to perform classification. Different kinds of classifiers use different classification algorithm (e.g. Gaussian, Laplace, parametric, nonparametric, etc.). Suppose a classifier has been trained on square, circle, and triangle. When a user sketches a stroke, the classifier will try to determine how closely the strokes resembles each of the trained strokes by generating a confidence value for each stroke. On a scale of 0 through 100, if a stroke receives 95 for being a square, 40 for circle, and 5 for triangle, then it is verly likely that the strokes is a square.

Version:

$Revision: 1.10 $

Author:

Michael Shilman (michaels@eecs.berkeley.edu), Heloise Hse (hwawen@eecs.berkeley.edu)

Constructor Summary

Constructors

Constructor and Description
BasicStrokeRecognizer() Construct a basic recognizer that performs recognition using a WeightedEuclideanClassifier, the default features, and an ApproximateStrokeFilter.
BasicStrokeRecognizer(Classifier classifier) Construct a basic recognizer that performs recognition using the given classifier, the default features, and an ApproximateStrokeFilter.
BasicStrokeRecognizer(Classifier classifier, FeatureExtractor[] extractors, StrokeFilter filter) Construct a classifying recognizer that classifies with the given classifier, set of feature extractors, and filter.
BasicStrokeRecognizer(Reader trainingFile) Construct a basic recognizer that trains on the gestures in the training file.
BasicStrokeRecognizer(Reader[] trainingFiles) Construct a basic recognizer that trains on the gestures in the training files.

Method Summary

Modifier and Type	Method and Description
static FeatureExtractor[]	defaultFeatureExtractors() Initialize an array of feature extractors which are used to produce a feature set from a gesture.
FeatureSet	extractFeatures(TimedStroke s) First filter the stroke, then extract the features from the filtered copy stroke by running each of the feature extractors on the copy, in order, and setting the corresponding i'th feature in the returned feature set to the extracted value from the i'th feature extractor.
Classifier	getClassifier() Return the classifier that is used to perform classification.
StrokeFilter	getStrokeFilter() Return the stroke filter that is used to preprocess the strokes.
RecognitionSet	strokeCompleted(TimedStroke s) Perform recognition on the given completed stroke.
RecognitionSet	strokeModified(TimedStroke s) Returns NO_RECOGNITION; BasicStrokeRecognizer only operates on completed strokes.
RecognitionSet	strokeStarted(TimedStroke s) Returns NO_RECOGNITION; BasicStrokeRecognizer only operates on completed strokes.
void	train(SSTrainingModel model) This function takes in a SSTrainingModel, builds a TrainingSet object from the examples in the model, and trains the classifier with the training set data.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BasicStrokeRecognizer

public BasicStrokeRecognizer()

Construct a basic recognizer that performs recognition using a WeightedEuclideanClassifier, the default features, and an ApproximateStrokeFilter.

See Also:

defaultFeatureExtractors()

BasicStrokeRecognizer

public BasicStrokeRecognizer(Classifier classifier)

Construct a basic recognizer that performs recognition using the given classifier, the default features, and an ApproximateStrokeFilter.

See Also:

defaultFeatureExtractors()

BasicStrokeRecognizer

public BasicStrokeRecognizer(Classifier classifier,
                             FeatureExtractor[] extractors,
                             StrokeFilter filter)

Construct a classifying recognizer that classifies with the given classifier, set of feature extractors, and filter. A null filter indicates no filtering should be performed.

BasicStrokeRecognizer

public BasicStrokeRecognizer(Reader trainingFile)
                      throws Exception

Construct a basic recognizer that trains on the gestures in the training file.

Throws:

Exception

BasicStrokeRecognizer

public BasicStrokeRecognizer(Reader[] trainingFiles)
                      throws Exception

Construct a basic recognizer that trains on the gestures in the training files. A SSTrainingParser is used to combine the gestures in the training files and form a training model data structure. The recognizer then trains its classifier on the training model.

Throws:

Exception

Method Detail

defaultFeatureExtractors

public static final FeatureExtractor[] defaultFeatureExtractors()

Initialize an array of feature extractors which are used to produce a feature set from a gesture.

Be default the following set of feature extractors are used:

1. CosInitAngleFE
2. SineInitAngleFE
3. BBoxDiagonalAngleFE
4. DistanceStartEndPtsFE
5. CosFirstLastPtsFE
6. SineFirstLastPtsFE
7. SumOfAnglesFE
8. SumOfAbsoluteAnglesFE
9. SumOfSquaredAnglesFE
10. AreaRatioFE
11. AspectRatioFE
12. CornerFE

extractFeatures

public FeatureSet extractFeatures(TimedStroke s)
                           throws ClassifierException

First filter the stroke, then extract the features from the filtered copy stroke by running each of the feature extractors on the copy, in order, and setting the corresponding i'th feature in the returned feature set to the extracted value from the i'th feature extractor.

If the feature extractor array is empty return null.

Throws:

ClassifierException

getStrokeFilter

public StrokeFilter getStrokeFilter()

Return the stroke filter that is used to preprocess the strokes.

getClassifier

public Classifier getClassifier()

Return the classifier that is used to perform classification.

strokeCompleted

public RecognitionSet strokeCompleted(TimedStroke s)

Perform recognition on the given completed stroke. This occurs when the mouse up event has been detected. The recognition consists of the following steps:

1. (optionally) filter the stroke.
2. extract features from the stroke, producing a feature set.
3. classify the feature set using the default or user-specified classifier.
4. create a Recognition object for each classification type. This object stores the type and confidence of the stroke. For example, the stroke may receive 95% confidence for being a square type.
5. Return the Recognition objects.

Specified by:

strokeCompleted in interface StrokeRecognizer

strokeModified

public RecognitionSet strokeModified(TimedStroke s)

Returns NO_RECOGNITION; BasicStrokeRecognizer only operates on completed strokes.

Specified by:

strokeModified in interface StrokeRecognizer

strokeStarted

public RecognitionSet strokeStarted(TimedStroke s)

Returns NO_RECOGNITION; BasicStrokeRecognizer only operates on completed strokes.

Specified by:

strokeStarted in interface StrokeRecognizer

train

public void train(SSTrainingModel model)
           throws ClassifierException

This function takes in a SSTrainingModel, builds a TrainingSet object from the examples in the model, and trains the classifier with the training set data. This only makes sense if the classifier used by this recognizer is trainable. Otherwise, a ClassifierException will be thrown. If the trainable classifier is incremental, then its train function will be called on the training set. Otherwise, we will first clear its weight sets, and then rebuild on the training data.

The following algorithm is applied:

1. Preprocess strokes (e.g. filter strokes).
2. Extract features from each strokes.
3. Build a TrainingSet object from the types and the feature sets.
4. Train classifier with the training set.

Throws:

ClassifierException