Proposer:Nils Goroll, Phone +44 131 650-2698 (C17 80 SB), +44 131 552-0604 (Home), Email: nilsg@dai.ed.ac.uk
Suggested Supervisors:
Phone (work): (0131) 650 3959 or (0131) 650 3960
Phone (home): (0131) 669 1573
Fax: (0131) 650 3962
E-mail: jim@ling.ed.ac.uk or J.Hurford@ed.ac.uk
Phone: +44-131-650-2711
E-mail: chrism@dai.ed.ac.uk
Description:
[Hurford 1999] gives an overview about current research in Edinburgh and elsewhere into simulations of language evolution within some population of "agents". The experiments described therein show how artificial languages with somehow complex syntactic structures can evolve amongst individuals of a simulated population, given some semantic meaning to communicate. All these experiments use symbolic representations.
The basic task of this project is to reimplement related work [Batali 1996], in which a connectionist approach to the subject is described: Members of a population of Elman networks (feed-forward with recurrent connections from the hidden layer to the input layer) are trained to convey to each other a set of binary encoded meanings through a limited sequence of characters. Of the meaning vector, six bits are used to encode arbitrary predicates and four bits to encode referents as arguments to these predicates. Of the 2^10 (=1024) possible combinations of the meaning vector only 100 are used for valid meanings. The sequences to be communicated between the networks are up to 30 characters long and each character can be one of four possible.
The layout of all networks of the population is identical and consists of one input-, one hidden- and one output-layer. The input layer is made up of the current character to be communicated to/from another network and the context input which is the output of the hidden layer after the previous activation was propagated through the network = recurrent connection. The output vector represents the meaning associated by the network given the current input and context. When acting as a 'hearer', each network is trained using back-propagation in order to associate the meaning vector of a 'speaker' with the character string being received from it. As the networks have only unidirectional connections, they can not produce mappings from meaning to characters directly. Thus to make a network produce an utterance we choose, from all possible characters for each position in the string, the one that would produce the most correct output if it was heard.
The results of the reimplementation are to be compared with those of the original work. Then the basic implementation is to be extended in order to investigate the effects of methods departing from this simple approach. Examples could be: other learning algorithms (in order to produce direct mappings from meaning to utterances as well as vice versa); different and possibly more complex representations; or changes towards a minimal model as proposed in [Hurford 1999] Chapter 4.
Resources Required:Some programming environment to implement the simulation (could be Matlab (if performant enough), C(++) or even Allegro Common LISP).
Degree of Difficulty:
The basic reimplementation task should be moderately difficult - it
does only involve inplementation of some well known neural network
techniques and a simulation environment. But as the project
is quite open-ended, it could be extended to more diffucult tasks as
appropriate.
Background Needed:
CC1, TNLP1
Some insight into fundamental linguistic questions, neural networks
and general programming skills needed.
Classification:
(1) Natural Language Processing
(2) Neural Networks/Connectionist Computing
(3) Evolutionary Computing
Applicable Degrees:MSc
in AI
References: