Download

SSI is written in C/C++ and primarily developed with Visual Studio (CMake files are available but less tested). To receive your copy of SSI including full source code and documentation please use the public repository (see below). You can visit our documentation site to learn how to get started with SSI. Also have a look at our various plugins. If you use SSI in your research project please cite this paper.

The latest version of SSI for developers is available on github. The repository requires ca. 3.45 Gb of disk space.

Audio Processing:

• EmoVoice: Emotional speech recognition
• LibSoxFilter: Wrapper for SoXLib, the Swiss Army knife of sound processing
• OpenSmile: Pitch and spectral audio features based on the OpenSmile library
• OpenSmileWrapper: A wrapper allowing it to run OpenSmile scripts
• Praat: Wrapper for Praat, a computer software for the analysis of speech in phonetics

Body Processing:

• Fubi: Full Body Interaction Framework for recognizing full body gestures and postures
• Skeleton: Conversion between different skeleton types

Eye Gaze Processing:

• Eyesig: Algorithms for the analysis of gazing patterns

Generic Signal Processing:

• DSPFilters: Wrapper for DSPFiters, a collection of classes for digital signal processing
• Signal: Basic signal processing algorithms

Machine Learning:

• Libsvm: A Wrapper for the famous LibSVM library
• Fusion: A collection of segmentation-based fusion algorithms
• Model: Basic pattern recognition algorithms
• Torch: Wrapper for Torch, a machine-learning library
• VectorFusion: Event-based fusion algorithms
• WEKA: Wrapper for Weka, a collection of machine learning algorithms for data mining tasks

Physiologal Data Processing:

• Biosig: Algorithms for the analysis of physiological signals

Sensor Devices:

• Audio: Audio sensor
• Camera: Camera sensor
• E-Health: e-Health Sensor Shield for Arduino and Raspberry Pi
• IOM: IOM biofeedback system (Wild Divine)
• MicrosoftKinect: Kinect for Windows sensor (1+2)
• Mouse: Mouse sensor
• TheEyeTribe: Eye Tribe Tracker
• WiiMote: Wii Remote, the primary controller for Nintendo’s Wii console
• GazeTracker: Gaze tracking and recording for devices from Pupil Labs

Video Processing:

• ARTKplus: Wrapper for ARToolKit+, a software library to calculate camera position and orientation relative to physical markers
• Avi: Avi writer/reader
• FFMpeg: Wrapper for FFMpeg, a software to record, convert and stream audio and video
• Image: Wrapper for OpenCV, a library for computer vision
• OpenFace: Wrapper for OpenFace, a a facial behavior analysis toolkit

After downloading and setting up SSI, we recommend you to have a look at the /pipes folder. It includes example pipelnes that demonstrate the possibilities of SSI. To open and edit the pipelines we recommend a text editor such as Notepad++.

SSI is open source and freely distributed under the terms of the GNU General Public License v3 and Lesser GNU General Public License v3 (see invidual license files).

If you use SSI in your research projects please reference the following paper:

Johannes Wagner, Florian Lingenfelser, Tobias Baur, Ionut Damian, Felix Kistler, and Elisabeth André. 2013. The social signal interpretation (SSI) framework: multimodal signal processing and recognition in real-time. In Proceedings of the 21st ACM international conference on Multimedia (MM ’13). ACM, New York, NY, USA, 831-834. DOI=10.1145/2502081.2502223 http://doi.acm.org/10.1145/2502081.2502223 [pdf]

@inproceedings{Wagner13,
 author = {Wagner, Johannes and Lingenfelser, Florian and Baur, Tobias and Damian, Ionut and Kistler, Felix and Andr{\'e}, Elisabeth},
 title = {The social signal interpretation (SSI) framework: multimodal signal processing and recognition in real-time},
 booktitle = {Proceedings of the 21st ACM international conference on Multimedia},
 series = {MM '13},
 year = {2013},
 isbn = {978-1-4503-2404-5},
 location = {Barcelona, Spain},
 pages = {831--834},
 numpages = {4},
 url = {http://doi.acm.org/10.1145/2502081.2502223},
 doi = {10.1145/2502081.2502223},
 acmid = {2502223},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {multimodal fusion, open source framework, real-time pattern recognition, social signal processing},
}

Older paper:

J. Wagner, F. Lingenfelser and E. André, “The Social Signal Interpretation Framework (SSI) for Real Time Signal Processing and Recognition”, in Proceedings of Interspeech, 2011. [pdf]

@inproceedings {wagner11,
 author = {Wagner, Johannes and Lingenfelser, Florian and Andr{\'e}, Elisabeth},
 location={Florence, Italy},
 booktitle={Proc. of Interspeech 2011},
 title={The Social Signal Interpretation Framework (SSI) for Real Time Signal Processing and Recognition},
 year={2011}
}