Software Learns Language Using Visual Perception

Share Article

Experience-Based Language Acquisition (EBLA) is an open source software system that enables a computer to learn simple language from scratch based on visual perception. It is the first "grounded" language system capable of learning both nouns and verbs. Moreover, once EBLA has established a vocabulary, it can perform basic scene analysis to generate descriptions of novel videos.

New Roads, LA -- February 14, 2003 -- Announcing <b><i>Experience-Based Language Acquisition (EBLA)</i></b>, an open source software system designed to investigate how computers might be enabled to understand natural language in a more humanlike way. Based, in part, on cognitive development in infants, EBLA is a computational framework for visual perception and grounded language acquisition. EBLA can “watch” a series of short videos and acquire a simple language of nouns and verbs corresponding to the objects and object-object relations in those videos. Upon acquiring this <I>protolanguage</I>, EBLA can perform basic scene analysis to generate descriptions of novel videos.

The general architecture of EBLA is comprised of three stages: vision processing, entity extraction, and lexical resolution. In the vision processing stage, EBLA processes the individual frames in short videos, using a variation of the mean shift analysis image segmentation algorithm to identify and store information about significant objects. In the entity extraction stage, EBLA abstracts information about the significant objects in each video and the relationships among those objects into internal representations called entities. Finally, in the lexical acquisition stage, EBLA extracts the individual lexemes (words) from simple descriptions of each video and attempts to generate entity-lexeme mappings using an inference technique called cross-situational learning. EBLA is not primed with a base lexicon, so it faces the task of bootstrapping its lexicon from scratch.

While there have been several systems capable of learning object or event labels for videos, EBLA is the first known system to acquire <i>both</i> nouns <i>and</i> verbs using a grounded computer vision system.

EBLA was developed in conjunction with the <a href="Robotics" onclick="linkClick(this.href)" rel="nofollow">">Robotics Research Laboratory at LSU</a>. It is written entirely in <a href="Java">Java</a>; and uses a <a href="PostgreSQL">PostgreSQL</a>; database as a backend. The complete source code is available from the <a href="SourceForge" onclick="linkClick(this.href)" rel="nofollow">">SourceForge EBLA project site</a>.

For more information on EBLA, please visit <a href="GreatMindsWorking.com"></a>;, a web site dedicated to research on computer models of human language acquisition.

Share article on social media or email:

View article via:

Pdf Print

Contact Author

Brian E. Pangburn, Ph.D.
Visit website