Statistically Integrated Semiotics that Enables Mutual

Inference
Between Linguistic and Behavioral Symbols for Humanoid
Robots

Wataru Takano and Yoshihiko Nakamura

ICRA 2009, Kobe Japan

Leon Palafox
June 4th 2006
 Outline

• Introduction
• Past Work
• Approximation
• Motion Language Model
• Natural Language Model
• Integration of both Models
• Conclusions
 Introduction

• The language is fundamental for human

communication
• It consists in:
– Icon: Reference to an object through physical
similarity
– Index: Based in physical and temporal relation of
Icons.
– Symbol: More abstract reference based in other
symbols.
 Past Work
Previous Algorithms

• Recurrent Neural Network

• Module Selection and Identification for control.
• Hidden Markov Model

Problems

• All of these have a common issue

• Do not deal with particles or nouns
• They only are able to interpret verbs
 Approximation

• A new Framework
– Motion Language model
– Natural Language Model
• It is a two ways algorithm where a robot can interpret the symbolic input.
• It can also create new outputs.
 Motion Language Model

• The motion language model represent the stochastic association

of Morpheme words with proto symbols via latent states.
• The motion language is defined by two kinds of parameters
– Probability that a morpheme word is generated by a latent variable
– Probability that a latent variable is generated by a proto symbol
 Natural Language Model

• Natural language model

represents the dynamics of
word classes by Hidden
Markov Models.
• The node ci corresponds to the
word class.
• Transition from the node cj to
the node ci is implemented
with probability P(ci|cj ).
• The morpheme word ωk is
generated by the node ci with
conditional probability P(ωk|ci).
 Integration of both models

• Represents relationship among

proto symbols and morpheme
words via latent variables as a
graph structure.
• Where nodes indicate the proto
symbols, the morpheme words and
the latent variables respectively.
• The natural language model
represents the dynamics of
language which means the order of
words in sentences.
• The motion language model and
the natural language model are
equivalent to semantics and
syntax.
 Conclusion

• The method seems robust for interpreting movement

signals and interpreting them as sentences.
• Depending on the language it may be a different
approach.
• The implementation in ISpace of this exact algorithm
seems difficult since it relies on heavy sampling of each
movement.