CSPR – Continuous Gesture Recognition

CSPR is a plugin for SSI which is able to recognize gestures from a continuous data stream without needing to know when a gesture start or ends. The system is real-time capable, scale and translation invariant and — when used in combination with the Microsoft Kinect — also largely independent of environmental noise such as changing lighting, irregular backgrounds or additional persons in the scene. At its core is an incremental algorithm for managing gesture candidates extracted from continuous input streams. It allows the processing of gesture candidates by common gesture recognizers as if temporal data segmentation is given.

Downloads

[downloads query=”limit=5&category=2&orderby=date&order=DESC” format=”4″ before=”” after=”” wrap=””]

Name

Version

Date

Download URL

Pipeline Example

<pipeline ssi-v="1">	
	<register>
		<load name="ssiskeleton.dll" />
		<load name="ssimicrosoftkinect.dll" />
		<load name="ssimodel.dll" />
		<load name="ssisignal.dll" />
		<load name="ssiioput.dll" />
		<load name="ssigraphic.dll" />
		<load name="ssievent.dll"/>
		<load name="ssicspr.dll"/>
	</register>
 
	<!-- set framework options -->		
	<framework console="true" cpos="200,400,600,400"/>	
 
	<!-- set painter options -->	
	<painter arrange="true" apos="1,1,200,0,400,400"/>	
 
	<!-- sensor -->
	<sensor create="ssi_sensor_MicrosoftKinect">
		<provider channel="skeleton" pin="skeleton_ms" />
		<provider channel="rgb" pin="scene" />
	</sensor>		
 
	<!-- select finger -->
	<transformer create="ssi_filter_SkeletonConverter">
		<input pin="skeleton_ms" frame="1" />
		<output pin="skeleton" />
	</transformer>	
	<transformer create="ssi_filter_SkeletonSelector" joints="11" values="0,1,3">
		<input pin="skeleton" frame="1" />
		<output pin="hand" />
	</transformer>
 
	<!-- visualization -->
	<consumer create="ssi_consumer_VideoPainter" name="scene" mirror="false" flip="false">
		<input pin="scene" frame="1"/>
	</consumer>	
 
	<!-- gesture classifier -->
	<consumer create="ssi_consumer_Classifier" console="True" trainer="cspr" sname="classifier" ename="gesture" socket="true" host="127.0.0.1" port="343">
		<input pin="hand" frame="0.07s" delta="0"/>
	</consumer>
 
	<!-- event board -->
	<listener create="ssi_listener_EventMonitor" mpos="600,0,400,400">
		<input listen="@classifier" span="20000"/>
	</listener>	
</pipeline>

Options

Name	Type	Description
indx	SSI_INT	dimension of x coordinate (def 0)
indy	SSI_INT	dimension of y coordinate (def 1)
indconf	SSI_INT	dimension of y coordinate (def 2). Use -1 to disable confidence checking
recognizer	SSI_CHAR	name of recognition algorithm cspr should use (CGR, $1, DM, $1_ci1) (def CGR)
noiseFilter	SSI_FLOAT	power of the noise filter (def 0.007)
matchingMode	SSI_INT	mode of computing gesture matches (0 = MIN_PROB, 1 = MIN_PROB_AND_MIN_DIFF, 2 = MIN_PROB_OR_MIN_DIFF, 3 = MIN_DIFF, 4 = LAST_MAN_STANDING) (def 1)
minProbForMatch	SSI_FLOAT	minimum probability a blob has to reach to become a match (def 0.007)
minProbDiffForMatch	SSI_FLOAT	minimum probability difference between the best two hypotheses of a blob, for the first hypothesis to become a match (def 0.2)
avgProbPerClass	SSI_BOOL	use the average prob of all examples of a class instead of the best prob (def false)
relativeCostScoring	SSI_BOOL	RCS ignores the constant maximum cost and dynamically sets the current worst match’s cost as max cost (def true)
maxBlobs	SSI_SIZE	maximum number of parallel blobs/threads (def 43). For beste recontion should be equal to maxBlobAge
maxBlobAge	SSI_SIZE	maximum age a blob can reach (def 43). maxBlobAge * frame size = max gesture length in seconds
duplicateTimeout	SSI_FLOAT	duration in seconds after a match in which the system treats all other matches as duplicates and deletes them (def 1). Use 0 to disable feature.
duplicateMaxAgeDiff	SSI_SIZE	after a match, all blobs within this age range get deleted (def 7). Use 0 to disable feature.
sampleTmplSpace	SSI_BOOL	sample template space by creating additional templates from interpolation
numPointsInGesture	SSI_SIZE	How many points do we use to represent a gesture (def 32)
pointsInterval	SSI_SIZE	The distance between two adjacent points in a gesture path (def 20)
f_PathDistance	SSI_FLOAT	computes euclidean disatance between two paths ($1). This option sets the weight of this feature used when comparing two paths (def 1). Use 0 to disable feature.
f_pd_rotInvariance	SSI_BOOL	whether the algorithm should try to be rotational invariant (def false)
f_pd_max	SSI_FLOAT	maximum distance between two paths (def 350)
f_DirectionVectors	SSI_FLOAT	parses the path for direction vectors (N,W,S,E ..). This option sets the weight of this feature used when comparing two paths (def 3). Use 0 to disable feature.
f_dv_Compress	SSI_BOOL	compress direction vectors (def true)
f_dv_FuzzyFilter	SSI_FLOAT	direction vector filter to reduce small bumps. This option sets the filtering threshold in degrees (def 15). Use 0 to disable filter.
f_dv_FilterStartJerks	SSI_BOOL	ignore direction jerks at path start when parsing direction vectors (def false)
f_dv_Shifting	SSI_BOOL	shift dirrection vectors to find optimal comparisson (DTW style) (def false)
f_dv_max	SSI_FLOAT	maximum distance value this feature can return (def 5)
f_NumDirectionVectors	SSI_FLOAT	computes the number of direction vectors of a path. This option sets the weight of this feature used when comparing two paths (def 1). Use 0 to disable feature.
f_ConnectedShape	SSI_FLOAT	computes whether a path is connected or not. This option sets the weight of this feature used when comparing two paths (def 4). Use 0 to disable feature.
f_cs_MinDistMult	SSI_INT	multiplier applied to minimal distance calculation. The smaller this value the closer the start and end of a path have to be for the path to be connected (def 3)
f_StopPoints	SSI_FLOAT	computes the number of stop points in a path. This option sets the weight of this feature when comparing two paths (def 1). Use 0 to disable feature.
f_sp_AvgVelMult	SSI_FLOAT	multiplier used to determine stop points based on average velocity (def 0.25)
f_sp_max	SSI_FLOAT	maximum distance value this feature can return (def 5)
f_PathLength	SSI_FLOAT	computes the length of a path. This option sets the weight of this feature used when comparing two paths (def 1). Use 0 to disable feature.
f_PenaltySize	SSI_FLOAT	weight of the feature score penalty size which penalizes the scores based on the difference between the paths’ sizes (def 0.9)
f_PenaltyDirection	SSI_FLOAT	weight of the feature score penalty direction which penalizes the scores based on the difference between the paths’ draw direction (def 0.0)