Connectionist-Based Information Systems: A Proposed Research Theme

Principal Investigators:

Assoc. Prof. Nikola K. Kasabov1

Dr Martin K. Purvis

Professor Philip J. Sallis

Department of Information Science

University of Otago

February 1996

General Characteristics of the Theme

• Emerging technology with rapidly growing practical applications

• Nationally and internationally recognised leadership of the University of Otago

• Already established organisation for research and working teams

• Growing number of postgraduate students working on the theme

• Growing number of research projects in this area

• Growing number of publications by members of the team

1 Introduction

Connectionist-based information systems (CBIS) are emerging computer systems (both in software and in hardware systems) whose central processing component is derived from parallel and distributed computing architectures. At the core of these systems are connectionist-based computing modules, such as neural networks, fuzzy inference mechanisms, or genetic algorithms, which may be used individually or in concert. Their power derives from their methods of coordinating distributed, local computations in such a way that the overall system exhibits "high- level" inferencing capabilities, such as learning, generalisation, adaptation, and intelligent knowledge acquisition, and these capabilities are collectively termed computational intelligence. This coordination may be achieved by a range of methods, from "hard-wired" artificial neuron connections to the more flexible means of intelligent autonomous agents. CBIS have been demonstrated to be particularly useful in the context of large quantities of information collected from multiple sources, since such databases are often characterised by such attributes as inconsistencies, incompleteness, and noise that are best handled by connectionist analysis. With the growth of global internetworking and communication bandwidth leading to an explosive increase of available electronic information, CBIS will play a leading role in many areas of industry and social life. A recent industry report [Schwartz Associates, USA 1993] identifies CBIS as one of the fastest growing technologies, with an expected revenue by 1999 to be ten times that of 1994.

The CBIS researchers at the University of Otago have been recognised nationally as a centre of excellence in New Zealand. This is reflected in the number of research projects in this area conducted here and the number of postgraduate students supervised. The Department of Information Science at Otago is at present the host of the ANNES (Artificial Neural Networks and Expert Systems) Special Interest Group of the New Zealand Computer Society, with more than 200 ANNES SIG members. Work on practical applications of CBIS has already been supported through two grants from the Public Good Science Fund of FoRST, grants from private industry, from the University of Otago and other sources.

The CBIS theme at the University of Otago has also been internationally recognised. The bi-
annual international conference of the ANNES SIG, ANNES'93, was successfully organised and run by individuals from the University of Otago, mainly from the Department of Information Science. The next conference, ANNES'95, is to be held in November 1995. The ANNES conferences attract researchers and practitioners from universities and industry across a range of research areas. Another international bi-annual conference, SE:E&P'96 (Software Engineering: Education and Practice), which focuses on software implementation issues, is also being hosted for the second time by the Department of Information Science.

A team of researchers from New Zealand and overseas has been organised by the Department of Information Science CBIS investigators to work on problems in a coordinated fashion (Figure 1).

The central column shown in Figure 1 indicates core CBIS activities at separate architectural layers. Note that hardware/software codesign is an important element, since hardware implementations incorporating parallelism can significantly enhance the performance of these systems. Shaded areas indicate significant applications that will enhance the overall performance of widespread applicability of CBIS. The combined and coordinated efforts in the various research areas will accelerate progress in this field.

2 Participants

The Connectionist-Based Information Systems research team has been organised by the Department of Information Science at the University of Otago in association with other departments within the university, with other universities, with other research organisations in New Zealand, and with overseas groups.

The principal researchers of the CBIS theme are: Dr Nik Kasabov, Dr Martin Purvis and Professor Philip Sallis from the Department of Information Science. The team includes participants from other universities in New Zealand as follows: University of Auckland (Dr George Coghill, Department of Electrical and Electronic Engineering); Massey University (Professor R. Hodgson, Department of Production Technology). The team also includes participants from New Zealand CRIs and industrial concerns: IRL Auckland (Dr David Tuck, Dr Natalie Spooner); IRL Lower Hutt (Dr Max Kennedy), and Waste Solutions Ltd, Dunedin (Dr Tico Cohen), which is emerging as a nationwide leader in the use of CBIS in process control.

There is a strong liaison with other departments from this university: Department of Computer Science (Dr W.Yeap, Dr A.Robins); Department of Zoology (Dr M. Paulin); Advanced Business Programme (Dr A. Everett); Department of Surgery (C. Solomon, S. Craig); Department of Maori Studies (M. Laws); Centre of Bioethics (Professor G. Gilett); Department of Linguistics (Dr J.Taylor); Department of Psychology (Associate Professor N. McNaughton, Dr D. O'Hare, Dr R. O'Shea); Department of English (Dr J.Hale).

International collaboration in CBIS has already been established. Joint research has started on the following topics: spoken language recognition, with Advanced Telecommunication Research Laboratory in Japan (Dr. Yamazaki); adaptive neuro-fuzzy systems, with University of Keiserslautern, Germany (Dr Uwe Zimmer); soft computing, with Kyushu Institute of Technology, Japan (Professor T. Yamakawa); adaptive decision making systems, with University of Trento, Italy (Professor M. Fedrizzi).

3 CBIS-related Topics of Research and Applications

(a) Work on the following research topics has been conducted by the team so far. All these topics relate either directly or indirectly to the CBIS theme:

•	Neural networks: experimenting and building connectionist models and their implementation [N.Kasabov, G.Coghill, M.Purvis, A.Robins]

•	Fuzzy systems: developing reasoning models based on fuzzy concepts, such as "big", "likely", etc. and human-like rules expressed in a linguistic form [N.Kasabov].

•	Expert systems: development of models and tools for building systems for problem solving based on if-then rules [A.Yeap, N.Kasabov, A.Everett, P.Sallis, M.Purvis].

•	Knowledge acquisition: extracting explicit if-then rules from raw data [N.Kasabov].

•	Autonomous agents: Computational models which consist of small intelligent, interacting elements [M.Purvis, S.Cranefield]

•	Software systems design: designing and implementing software systems [M.Purvis, P.Sallis, S.MacDonell, N.Kasabov]

•	Modelling the software development process: [P.Sallis, S.MacDonell]

•	Petri nets and distributed dynamic modelling: computational models for simulation of dynamic processes based on data flow through a net of simple process nodes [M.Purvis, G.Benwell]

•	Multimedia systems: systems which include text, graphics, sound and video [P.Sallis]

(b) Research results from the CBIS theme have been applied to various applications. Some of them are listed below:

•	Speech recognition: development of computer systems which recognise spoken language as input data [N.Kasabov]

•	Process control in bio-technology: CBIS for controlling anaerobic processes in a sewage plant [T.Cohen, N.Kasabov]

•	Natural language processing: computer systems for text understanding in English [P.Sallis]

•	Software forensics: identification of authorship in computer fraud detection [P.Sallis, S.MacDonell]

•	Medical data analysis and decision making: CBIS for decision making based on varicose veins data [C.Solomon, S.Craig, N.Kasabov]

•	Zoology: CBIS for dolphin echo-location [M.Paulin]

•	Environment, Agriculture, Spatial Information Processing: CBIS for predicting fruit development [G.Benwell, N.Kasabov, M.Purvis]

•	New Zealand culture and heritage: CBIS for building English-to-Maori talking dictionary [N.Kasabov, M.Laws]

4 Selected Titles of Research Projects

The following selected research projects give a good example of the range of the problems from the CBIS theme which have been investigated by the CBIS principle investigators in the last few years:

1995/96, FoRST, Spatial Analysis Systems, $609,000, 3 years

1995, Otago Research Grant, Methods and Tools for Building Adaptable Speech Interfaces to Standard and Fuzzy Databases, $25,193

1995, Fisher&Paykel, Fuzzy Control (through and with Waste Solutions. Ltd), $20,000,

1996, Agent-Based Technologies for Distributed Multi-platform Software Integration, Otago Research Grant, $47,819

1994/95, FRST, Development of a Spatial Analysis Tool Box, $57,000

1994, TELECOM New Zealand Ltd, Automatic Speech Recognition, $24,500

1994, Departmental Research Grant, Hybrid Connectionist AI Systems - FuzzyCOPE, $10,000

1994, Otago Postgraduate Award, $12,000 plus tuition fees for 3 years

5 Selected Publications

The following is a selected list of some of the major publications published by the principal researchers in the area of CBIS in the last several years.

(a) Books

Kasabov N.K. Neural Networks, Fuzzy Systems and Knowledge Engineering. Cambridge, MA, MIT Press (being printed) 550p

Sallis P.J., Tate G. and MacDonell S.G. Software Engineering: practice, management and improvement. Addison-Wesley, 1995, 215p

(b) Book Chapters

Kasabov N. and Clarke G. A template-based implementation of connectionist knowledge based systems for classification and learning. In Advances in Neural Networks, vol.4. O.Omidvar ed. USA, Ablex Publishing Company (1995) 137-156

Kasabov N. Building comprehensive AI and the task of speech recognition. In Applications of Neural Networks to Telecommunications, 2, ed. J.Alspector, R.Goodman, T.Brown, Laurence Erlbaum (1995) 178-187

Kasabov, N. and Nikovski, D. Prognostic expert systems on a hybrid connectionist environment. In Artificial Intelligence V Methodology, Systems, Applications. B. du Boulay and V.Sgurev eds., Elsevier Science Publ, North Holland (1992) 141-148

Kasabov N. Hybrid connectionist rule based systems. In Artificial Intelligence IV Methodology, Systems, Applications. P.Jorrand and V.Sgurev Eds., North-Holland (1990) 227- 235

(c) Publications in Refereed Journals

Kasabov, N., Lavington S., Li S. and Wang C. A model for exploiting parallel associative matching in AI production systems. Knowledge-Based Systems 8(1):1-7 (1995)

Kasabov, N. Adaptable connectionist production systems. Neurocomputing (1995) (being printed)

Kasabov, N. Hybrid connectionist fuzzy systems for speech recognition. Lecture Notes in Computer Science/Artificial Intelligence (1995) (being printed)

Kasabov, N. Learning fuzzy rules and approximate reasoning in fuzzy neural networks and hybrid systems. Fuzzy Sets and Systems (1995) (being printed)

Kasabov, N. Hybrid Connectionist Fuzzy Production Systems - Towards Building Comprehensive AI, Intelligent Automation and Soft Computing, (1995) (being printed)

Kasabov, N. Connectionist fuzzy production systems. in: Fuzzy Logic in Artificial Intelligence, A. Ralescu ed, Lecture Notes in Artificial Intelligence 847:114-128 (1994)

Kasabov, N. Hybrid connectionist production systems. Journal of Systems Engineering 3 (1), 15-
21 (1993)

Kasabov, N. and Shishkov, S. A connectionist production system with partial match and its use for approximate reasoning. Connection Science 5(3&4):275-305 (1993)

Kasabov, N. Incorporating neural networks into production systems and a practical approach towards the realization of fuzzy expert systems. Computer Science and Informatics 21(2):26-34 (1991)

M. K. Purvis, M. A. Purvis, and G. L. Benwell, "Modelling and Simulation of the New Zealand Resource Management Act", to appear in the Journal of Law and Information Science, 1995.

S. Cranefield, P. Gorman, and M. Purvis, "Communicating Agents: An Emerging Approach for Distributed Heterogeneous Systems", New Zealand Journal of Computing, 6:1B, August 1995, pp. 337-343.

M. K. Purvis, G. L. Benwell, and M. A. Purvis, "Dynamic Modelling of the Resource Management Act", New Zealand Journal of Computing, 5:1, 1994, pp. 45-56.

M. K. Purvis, G. L. Benwell, and M. A. Purvis, "Dynamic Modelling of the Resource Consent Process in the Resource Management Act", New Zealand Surveyor (Journal of the New Zealand Institute of Surveyors), No. 285, March 1995, pp. 13-20.

Sallis, P.J., Anderson, R. and Yeap, W.K. Enhancing a hypertext application using Natural Language Processing techniques. Journal of Information Science, 17(1), 49-56 (1991)

Benwell,G.L., Firns, P.G. and Sallis, P.J. Deriving semantic data models from structured descriptions of reality. Journal of Information Technology, 6(1), 15-25 (1991)

Sallis, P.J. Contemporary Computing Methods for solving the authorship characterisation problem in computational linguistics. New Zealand Journal of Computing, 5(1), 85-91 (1994)

(d) Publications in Proceedings of International Conferences

N.Kasabov (ed) Proceedings of the First New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems, IEEE Computer Society Press, Los Alamitos, California, 1993.

N.Kasabov and G.Coghill (eds) Proceedings of the Second New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems, IEEE Computer Society Press, Los Alamitos, California, 1995.

M. K. Purvis (ed.), Software Education Conference (SRIG-ET'94), IEEE Computer Society Press, Los Alamitos, CA, 1995.

M. K. Purvis and X. Li, "Connectionist Learning Using an Optical Thin-Film Model", to appear in Proceedings of the Second New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems, IEEE Computer Society Press, Los Alamitos, California, 1995.

M. K. Purvis, and S. J. S. Cranefield, "Causal Agent Modelling: a Unifying Paradigm for Systems and Organisations", Proceedings of Pan-Pacific Conference XII, Dunedin and Queenstown, New Zealand, 1995, pp. 394-396.

M. K. Purvis and M. A. Purvis, "Modelling Environmental Legislative Processes with Petri Nets", Modelling and Simulation (Proceedings of the International Association of Science and Technology for Development International Conference), IASTED-ACTA Press, Anaheim, CA, 1995, pp. 238-246.

M. K. Purvis and L. Xiaodong, "Connectionist Computations Based on an Optical Thin- Film Model", in Proceedings of the First New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems, IEEE Computer Society Press, Los Alamitos, California, 1993, pp. 130-133.

M. K. Purvis and G. L. Benwell, "A Causal Agent Approach for Modelling Dynamic Systems", in Proceedings of the 13th New Zealand Computer Society Conference, New Zealand Computer Society, Auckland, August 1993, pp. 598-604.

D. W. Franke and M. K. Purvis, "Hardware/Software CoDesign: A Perspective", in Proceedings of the 13th International Conference on Software Engineering, Austin, TX, U.S.A., May 13-16, 1991, pp. 344-352.

Sallis, P.J. Neural Networks and Ambiguity in Natural Language. Proceedings of the First New Zealand International Conference on Artificial Neural Networks and Expert Systems. Dunedin 1993, 102-104

Sallis, P.J. Spatially related knowledge: the multimedia experience. Proceedings of the Third Annual Spatial Information Processing Colloquium. Dunedin, May 1991, 107-118.

Sallis, P.J. and Yeap, W.K. Alpha and Omega: an AI perspective. Proceedings of the XIth Annual South-East Asian Computer Confederation Conference. Kuala Lumpur, Aug 1992, Vol 2, 381-388

Sallis, P.J. Information retrieval using a natural language interface to a GIS database. Proceedings of the Second Annual Spatial Information Processing Colloquium. Dunedin, Nov 1990, 100-111.

Barrow, F. and Sallis, P.J. NLP and GIS Techniques for a Railway Safety Audit Reporting System. Proceedings of the Seventh Annual Colloquium of the Spatial Information Research Centre, Eds. P.G. Firns and N.C. Sutherland, Palmerston North, 1995, 229-236.