Hybrid Fuzzy Connectionist Rule-based Systems

and the Role of Fuzzy Rules Extraction

Dr Nikola K. Kasabov1

Computer and Information Science

University of Otago

November 1994

Abstract

The paper presents the major principles of building complex hybrid systems for knowledge engineering where at the centre of the design process is the task of learning (extracting) fuzzy rules from data. An experimental environment FuzzyCOPE, which facilitates this process, is described. It consists of a fuzzy rules extraction module, a neural networks module, module fuzzy inference methods and a production rules module. Such an environment makes possible the use of the three paradigms, i.e. fuzzy rules, neural networks and symbolic production rules, in one system. Automatic rules extraction from data and choosing the most appropriate reasoning mechanism is also provided. Using FuzzyCOPE for building hybrid systems for decision making and speech recognition is discussed and illustrated.

1 Introduction

Knowledge engineering is an area where many methods have been developed and used so far for solving difficult AI problems, like decision making, speech recognition and language understanding. The problem solving process can be viewed as mapping the domain space into the solution space of the problem with the use of the existing problem knowledge (rules, historic data, rules and data, text, etc.).

Among the knowledge engineering methods used so far, are the symbolic AI methods, the fuzzy logic methods, the methods of neural networks. Figure 1 shows different pathways for mapping the domain space into the solution space for a particular problem. This mapping can be 'one- shot', for example training a neural network with existing data and finding the solution, but it can be very complicated as well. The solution can be represented as many paths processed in parallel or in a given sequence, as loops for experimenting with different methods and finding the best one, as a very low blended solution, like setting a neural network by a set of rough fuzzy rules, training it, and extracting refined parameters of the fuzzy rules, etc.

Figure 1: Different pathways for mapping the domain space into the solution space

The three main groups of methods, namely the AI symbolic, the fuzzy-, and the connectionist, use different representations ranging from symbolic to numeric. The three groups of methods overlap on this range.

We can view the above three main groups of methods as complementary, i.e. each of them facilitates dealing with different kinds of knowledge representation and thus facilitates solving different sub-tasks of a global task. The area of integrating ordinary artificial intelligence, neural networks and fuzzy systems is called in [5] "comprehensive artificial intelligence" and is one of the long term objectives of many AI laboratories, including LIFE in Japan. Comprehensive artificial intelligence deals with hierarchical, multiparadigm systems.

Going back to the beginning of the hybrid AI systems as an area of knowledge engineering [1], a general two-level hierarchical model for knowledge processing was suggested in [2]. The low level performs pattern matching, 'unconscious' tasks, fast pre-processing, etc., but the final decision is communicated at a higher level, where strategic knowledge, conceptual knowledge, 'conscious' reasoning is applied. Implementing the two-level model requires appropriate methods. For example, connectionist methods for the low level processing and rule-based methods for the higher level processing. Fuzzy logic methods are suitable for both low level and higher level processing when the process can be linguistically described. Fuzzy knowledge can be used as a mediator between low level data representation and higher level rule representation and also as an unifying media.

The idea of using neural networks for low level pattern matching and symbolic rule-based systems - for a higher level processing - led to the development of hybrid connectionist rule- based systems and software environments [1,2,3]. One of them, COPE (COnnectionist Production system Environment) [3] consists of a set of neural network simulators and a production language CLIPS for higher level processing and control. CLIPS (the abbreviation stands for C Language Interpreter of Production Systems) is being developed by NASA [6]. Functions which perform initialisation, training and recall of different types of neural networks are compiled together with the CLIPS functions to obtain the COPE functionality. This makes all these functions callable from a rule-based program written in COPE.

Now, new software environments are required to facilitate all the possible pathways for problem solving. Such environments should facilitate the representation and processing of symbolic, fuzzy and connectionist knowledge and also integrating them into one system.

2 FuzzyCOPE - a Fuzzy COnnectionist Production system Environment

FuzzyCOPE is a software environment which has five main modules (Figure 2):

• FuzzyCOPE Production System Module - it is an extension of FuzzyCLIPS with functions to call fuzzy inference methods, to call rules extraction methods, to call neural networks and data processing methods, all of them being available for experimenting within the corresponding modules of the environment. A user program can be written in FuzzyCOPE production language and compiled to executable code.

• Fuzzy Inference Module - it offers different fuzzy inference methods for experimentation while tuning the fuzzy reasoning over a set fuzzy rules. The fuzzy control inference methods [9] as well as decompositional methods for reasoning with fuzzy inputs and fuzzy outputs [10, 17] are implemented. This module processes a set of fuzzy rules either designed by the user or the expert, or automatically extracted from raw or pre-processed data by the rules extraction module.

• Rules Extraction Module - it includes several connectionist methods for rules extraction from raw or fuzzified data. One of them is for extracting a set of rough rules when the membership functions for the fuzzy predicates are chosen of a standard triangular type. The method is based on a standard neural network architecture. Another method refines the membership functions and the rough fuzzy rules by using a fuzzy neural network. All the used neural network structures are initialised, trained and tested either in the neural network module, or directly in the production system at the higher level, where the neural network functions are called as actions in the production rules.

• Neural Networks Module - it contains several types of neural networks which can be initialised, trained with raw or fuzzified data, and tested. The fuzzification of the raw data is done in the data processing module. Four types of neural networks are currently being implemented - feedforward neural network with backpropagation algorithm [8], Kohonen self organising topological map [7], recurrent network and fuzzy neural network [11,18].

• Data processing module - it performs operations on raw data, like fuzzification, normalisation, clusterisation. The output of this module can be used as input to the other modules.

Figure 2: A block diagram of FuzzyCOPE

FuzzyCOPE has been created to facilitate:

• building complex multimodular and hierarchical hybrid systems;

• experimenting with different pathways for solving a given problem before choosing and compiling the best one(s);

• mixing explicit expert knowledge with raw data and with extracted knowledge from data when creating a system;

• tuning and revising the existing or the extracted knowledge;

• improving the understanding of the problem to solve through the process of finding its solution by extracting, refining and interpreting fuzzy rules.

All the modules in the FuzzyCOPE environment, and the rationale for their use, are presented below. We start with a discussion on the use of neural networks as a low level processing technique and then discuss the higher level processing performed in the symbolic production systems.

3 Neural Networks - are they suitable for low level processing only?

Neural networks (NN) are a very powerful paradigm for solving problems when problem knowledge is in the form of past, historical data.

It is now well known that neural networks are universal approximators, i.e. for any continuous function in a compact domain, given as set of points, there will be a NN which approximates this function to any desired degree of accuracy [13]. The model of approximation is represented by the neural network structure and the initial conditions for training. The process of choosing the appropriate neural network structure is a process of choosing a model for approximation. In order to choose an appropriate neural network architecture, especially if the volume of data is small, one has to carefully analyse the data set. Efficient methods for choosing the optimal architecture have been proposed, for example - fuzzy clustering of data, pruning of the neural network while training. In order to avoid exhaustive search for the best architecture in the neural architectures space, which is extremely time consuming, data analysis has to be performed in advance. The data processing module is to provide methods for such analysis.

Neural networks can learn spatial or temporal patterns. They are excellent 'blind' pattern matchers, i.e. they find similarity in patterns without 'knowing' what the patterns really mean. In addition, neural networks can learn to associate membership functions between fuzzy predicates in a fuzzy rule [3,12]. Fuzzy neural networks can learn fuzzy rules and the correct membership functions for the fuzzy predicates from a set of rough rules and standard membership functions [11,18].

Problem solving with the use of a NN can be represented as a structural and a functional mapping of the problem solving process into a neural network, i.e. the domain space is mapped into the input space of the NN, the problem knowledge is mapped into the connection weights, and the problem solution space is mapped into the output neuronal space.

The neural network module of FuzzyCOPE has several types of neural networks available for simulations and experimenting, namely: feedforward neural networks with a backpropagation training algorithm [8]; Kohonen self organising topological network [7]; recurrent network; fuzzy neural network [11,18]. Which one to use for as particular problem, depends on the problem.

Recurrent networks are suitable for time series learning and approximation, like stock exchange index prediction, speech recognition, language understanding. They can learn time dependencies.

The fuzzy neural networks are usually 'pre-wired' according to a set of rough fuzzy rules and rough form of the membership functions for the fuzzy predicates in them [11,18]. After training with real, crisp data, the network refines the rules and the membership functions. One of these types, called the filtering neuron, is presented in [18]. A fuzzy neural network has the following advantages when compared to a standard feedforward network [11]:

• it learns much faster (a matter of 3-5 orders of magnitude);

• it generalises better, avoiding such phenomena like 'overfitting' and 'local minimum';

• the fuzzy rules are 'clearly' presented in the network and they explain the behaviour of the fuzzy neural network and the problem solving method derived.

Fuzzy neural networks have an important role for learning fuzzy rules in FuzzyCOPE. An example of using different types of neural networks for the task of stock exchange index prediction is given in section 7.

4 Fuzzy rule-based systems

Fuzzy rule-based systems (FS) offer definite advantages when used for knowledge engineering:

• They are based on well defined fuzzy logic theory, which uses the theory of fuzzy sets originated by Lotfi Zadeh, 1965, and further developed by many scientists.

• FS facilitate representing common sense knowledge, vague knowledge, ambiguous knowledge.

• FS facilitate representing uncertain knowledge. The way they process uncertainties is inherent in their theory.

• FS use linguistic terms as fuzzy predicates, but they have to have a numerical representation defined by membership functions. This makes the reasoning process tunable.

• Fuzzy logic theory offers a variety of fuzzy inference methods.

• The fuzzy controllers are universal function approximators, i.e. for any continuous function there will be a fuzzy system which can approximate this function to any desired degree of accuracy.

Fuzzy (rule-based) systems are characterised by three major elements:

• fuzzy variables and their linguistic fuzzy predicates;

• set of fuzzy rules;

• fuzzy inference method.

The software environment FuzzyCOPE provides help when dealing with the following difficult problems in fuzzy engineering:

• Articulating the membership functions for the fuzzy predicates is difficult. The problem is, if we do not know them, should we articulate them in a standard way adopting a standard, for example, triangular, type? A fuzzy neural network can be used to refine the membership functions of the fuzzy predicates.

• The number of the fuzzy rules increases dramatically with the increase of the fuzzy variables and their fuzzy values. For example, all the possible rules for 5 input variables, each having 3 fuzzy values, is 35 = 243. How can we reduce the number of all possible fuzzy rules and still cover the whole problem space? A solution to this problem can be found by using gradual rules or by using fuzzy rule interpolation, but these techniques are still not ready to be implemented in a software environment [9,12]. This problem can be solved in FuzzyCOPE by partitioning the problem space and covering the different parts by different subsystems, e.g. a fuzzy system, a neural network(s), a production rule system.

• Difficulties in choosing appropriate fuzzy inference method for a set of fuzzy rules. The majority of the fuzzy engineering tools offer a limited number of such inference methods, mainly aimed at developing fuzzy control systems. Tuning the fuzzy inference to a particular application is achieved in the FINEST system (developed at LIFE [15]) by realising the fuzzy inference in a connectionist structure and representing the fuzzy inference operators by analytical formulas each depending on one or two parameters of the network. The operators used for defining the method of fuzzy inference are: aggregation operator (the aggregation between the truth values of several condition elements in one rule), implication (between the whole antecedent part of a rule and its consequent part) and combination - the operation between the inferred membership functions by all the satisfied fuzzy rules at a given execution cycle. Supervised learning is applied on the neural network when the input and output values are fuzzy, represented by their membership function. After training, the values of the parameters which define the operators can be analysed and the type of the operator linguistically articulated, e.g. "the aggregation operator is closer to the MIN operator rather than to the algebraic product". In this case the operators are approximately defined, but the inference is tuned on a neural network by training with examples to represent adequately the application. It is known that a neural network can be trained with a set of fuzzy rules by presenting fuzzy input values and producing fuzzy output values [12, 13] .This method can also be realised in FuzzyCLIPS by using a fuzzy neural network.

The fuzzy inference module in FuzzyCOPE offers both decompositional inference methods [10, 17] and fuzzy control inference methods (fuzzification - evaluation - defuzzification) [9,12].

For the decompositional method of inference fifteen implications (of the type <condition element - rule consequent> [10], three aggregation operators (compositions) and three combination operators (else-links) to choose one of them for a particular inference method. Figure 3 below shows a decompositional method for a simple fuzzy rule IF X1 is A1 and X2 is A2 THEN Y is B.

Figure 3: Decompositional fuzzy inference over a simple fuzzy rule

Among the fifteen implicational relations implemented in FuzzyCOPE [10] are Ra, Rm, Rc, Rb, Rs, Rg, Rgs, Rgg, Rsg, Rss. The Max-min, Max- and Max- compositional operators are implemented. The three 'else-links' (also called combination operators) facilitated in the fuzzy inference module of FuzzyCOPE are [10, 17]:

• AND-link, the final result is obtained after a MIN operation over the corresponding values of the fuzzy membership functions inferred by all the rules;

• OR-link, similar to the AND-link, but MAX operation is applied;

• Truth qualification-link: a coefficient Tbi' is calculated for the inferred fuzzy set Bi' by every rule Ri. The result obtained by a rule Rj with the maximum coefficient, is taken as a final result:

Tbj= MAX { Tbi }, i=1,2,..., N, Tbi= B'(v) / B(v), for all v V (the universum for the fuzzy variable Y). The selection of the 'else-link' depends on the context in which the rules are written [17].

A fuzzy inference method is considered here as a triple Fi=(I, C, L), where I is an implicational operator, C is one of the possible compositional operators, L is one of the possible 'else-links'. For example, the following function will perform a decompositional inference over fuzzy input "inp" over a set of rules which are in the file "loan.rul", when implication 3, composition 1 and else-link 3 will be used:

(dcmfuzzy "loan.rul" inp 3 1 3)

The following function will perform a fuzzification-evaluation-defuzzification inference over crisp input values "inp" and rule base "pendulum.rul":

(fidfuzzy "pendulum.rul" inp)

Having available so many decompositional inference methods the question of which one to choose for a particular application is quite reasonable. It depends on the inference laws we want our system to obey. Different implication methods satisfy different laws of logical inference. Mitzumoto [10] for example, showed which of the implications given above satisfy the laws of syllogism and contrapositive and which of them satisfy the modus ponens and modus tolens laws and their modifications.

As the fuzzy inference functions are available either for direct use from the fuzzy inference module, or they can be called from a FuzzyCOPE higher level production rules, the selection of the most suitable fuzzy inference method can be automatically carried out, as shown in the following exemplar program:

Example:

Rule 1:

IF (a set of fuzzy rules is available - "fuzz1.rul")

AND (there is a fuzzy inference method fi which has not been tested)

THEN (run the fuzzy inference method fi over the set of fuzzy rules for a test data "test.dat")

(save the method f_best which produces best results)

Rule2:

IF (a set of fuzzy rules is available - "fuzz1.rul")

(new input data is available - "new.dat")

THEN (run the fuzzy inference method f_best on "fuzz1.rul" for "new.dat")

(assert the results in the working memory)

5 A (fuzzy) rule is worth more than a thousand data examples

One (fuzzy) rule may represent a 'thousand' data points. Knowing the rules is really knowing how to solve the problem and not only getting a good solution from a 'black box'.

Learning explicit symbolic knowledge from data has been well explored in the AI community by using AI machine learning methods. But a dramatic development in learning methods is on- going due to the learning characteristics of neural networks. NN can be used for both learning rough fuzzy rules when membership functions for the fuzzy predicates are defined as standard ones, and for tuning the fuzzy rules and the membership functions. For example, in [12] a differential competitive clustering method is developed for learning simple fuzzy rules of the form:

Rulei: IF X1 is A1i AND X2 is A2i AND ...AND Xn is Ani THEN Y is Bi

A method for learning generalised fuzzy rules is published in [14]. This method uses a feedforward NN, trained with fuzzified data (the membership functions of the fuzzy predicates are supposed to be either known or articulated as standard ones). The learned generalised fuzzy rules are of the form of [16]:

Rulei: IF [X1 is A1i (DI1i) AND .... Xn is Ani (DIni)] [NTi, SFi, RFi] THEN (Y1 is B1i AND ... Ym is Bmi) [CFi]

where: (Xj is Aji) are fuzzy or crisp propositions; DIji are the degrees of importance of the condition elements Cji; NTi is noise tolerance of the antecedent part of the rule Rulei, meaning a level of data support below which the rule cannot be satisfied; SFi is a sensitivity factor defining how much fast the truth value for the whole antecedent part of the rules Rulei would grow with the growth of matching level of the present data; CFi is a certainty factor for the consequent part of the rule. The parameter "sensitivity factor" SFi is a generalised parameter which includes "synergism" and "cancellation parameters" [15] as partial cases.

The above method has been implemented in the rules extraction module of FuzzyCOPE. NTi, SFi and RFi are not learned but assumed to have their default values instead. As a partial case, in a special learning mode, simple fuzzy rules can be learned by this method.

Tuning fuzzy rules and membership functions by using fuzzy neural networks is becoming very much used approach in fuzzy engineering [11, 18]. This group of methods use 'pre-wired' NN and rough form of fuzzy rules and standard membership functions for the fuzzy predicates. After training the network, rules in the form of:

IF X1i is A1i and...Xni is Ani THEN y is Bi (strength factor)

can be extracted with the membership functions of the fuzzy linguistic values Aji and Bi. In this case the trained NN and the extracted fuzzy rules with membership functions produce equivalent results.

In section 7 the prediction of the SE40 index performed by using extracted rough rules and tuned rules is shown.

6 Symbolic production systems for higher level control and decision making

Symbolic AI systems deal with symbols. There are well developed and broadly used logic systems to manipulate symbols correctly, e.g. propositional logic, predicate logic.

Production rule-based systems (PS) belong to the set of symbolic AI systems [6]. They consist of three major parts:

• production memory, where the rules reside;

• working memory, where the facts reside;

• inference machine which performs a control cycle (match - select - act).

When executed, a production rule may assert or retract facts to the working memory, call functions, perform I/O operations, etc. The exact match implemented in the standard versions of production languages, for example CLIPS [6], has been extended to partial matching and dealing with fuzzy variables and fuzzy inference in FuzzyCLIPS (NRC Canada, released 1994). FuzzyCLIPS was used as a production language on top of which FuzzyCOPE was developed.

PS are characterised by the following features:

• PS are universal computational mechanisms, i.e. equivalent to the Turing machine. PS can realise finite automata.

• PS prove with chain reasoning over production rules.

• Different reasoning strategies are facilitated when selecting which of all the satisfied rules, after the current match phase, should be executed, e.g. LEX, MEA, Random [6].

• PS deal with symbolic variables.

• PS have good explanation facilities.

The above characteristics of the PS make them suitable for high level control and decision making in a hybrid system. Two examples of using production rules in a hybrid system are given below:

Example 1. A PS can help automating a hybrid system design:

Rule 1:

IF (past data is available)

AND (a higher level processing fuzzy rules "fuzz1.rul" are available)

THEN (train a neural network with data "data.dat" for a low level processing - "net1.net")

(train a neural network "net2.net" with data "data2.dat" for rules extraction)

(extract fuzzy rules "fuzz2.rul" for higher level processing from the network "net2.net")

(initialise a fuzzy network "fnet.net" with the set of fuzzy rules "fuzz2.rul" and train it with data "data2.dat")

(extract tuned fuzzy rules "fuzz3.rul" from the network "fnet.net")

(add the tuned fuzzy rules "fuzz3.rul" to the existing fuzzy rules in "fuzz1.rul")

Rule 2:

IF (the design process is completed)

THEN (compile all the modules of the hybrid system in the given order into one executable module)

Example2. A PS for controlling the execution process of a hybrid system

Rule 1:

IF (a neural network "net1.net" for a low level processing has been trained) AND

(a fuzzy rule system "fuzz1.rul" for a higher level processing is available)

THEN (read new input data "inp")

(recall the network net1 with the input data "inp"; bind the resulting vector to res1)

(run a fuzzy inference method over fuzzy rules "fuzz1.rul" for input vector res1; bind the resulting vector in res2)

(assert res2 in the working memory)

7 Building decision support systems in FuzzyCOPE

Building decision support systems, whatever type of knowledge is available, requires: good explanation facilities and preferably presentation of the decision rules used; dealing with vague as well as with crisp knowledge; dealing with contradictory knowledge; dealing with large data bases with a lot of redundant information, or dealing with not enough data. FuzzyCOPE ideally suits these requirements.

Example 1: Stock exchange index prediction.

Figure 4 presents experimental results relating to the use of different types of neural networks and inferences over fuzzy rules in FuzzyCOPE for the stock exchange market SE40 prediction. The best prediction has been achieved using a fuzzy neural network. Obviously, the best SE40 prediction should be achieved by using a hybrid system which combines the predicted value from the fuzzy neural network and expert rules on the current political and economic situation in the country, in the region and in the world.

Figure 4: Stock exchange SE40 index prediction using FuzzyCOPE: (a) the training data; (b) the test data: expected values - dark solid line, and the predicted values by using: a feedforward neural network - dotted line; recurrent neural network - dashed line; a fuzzy correlation minimum method over extracted rough fuzzy rules - light solid line; fuzzy neural network (filtering neuron) - dashed/dotted line.

Example 2. Loan approval system [17]

The following is the rule base "loan.rul" in the FuzzyCOPE format:

MEMBERSHIPS INPUT

CScore:{150,155,160,165,170,175,180,185,190,195,200}

> High: { 0, 0, 0, 0, 0, 0, 0.2, 0.7, 1, 1, 1}

> Low: { 1, 1, 0.8, 0.5, 0.2, 0, 0, 0, 0 ,0 ,0}

INPUT

CRatio:{0.1,0.3,0.4,0.41,0.42,0.43,0.44,0.45,0.5,0.7,1.0}

> Good_CR:{ 1 ,1 , 0.7, 0.3, 0 ,0, 0, 0, 0, 0, 0}

> Bad_CR: { 0, 0, 0, 0, 0, 0, 0, 0.3, 0.7, 1, 1}

INPUT CCredit: {0,1,2,3,4,5,6,7,8,9,10}

> Good_CC: { 1, 1, 1, 0.7, 0.3, 0, 0, 0, 0, 0, 0}

> Bad_CC: { 0, 0, 0, 0, 0, 0, 0.3, 0.7, 1, 1, 1}

OUTPUT Decision: {0,1,2,3,4,5,6,7,8,9,10}

> Approve: { 0, 0, 0, 0, 0, 0, 0.3, 0.7, 1, 1, 1}

> Reject: { 1, 1, 1, 0.7, 0.3, 0, 0, 0, 0, 0, 0}

RULES

if <CScore is High> and <CRatio is Good_CR> and <CCredit is Good_CC>

then <Decision is Approve>

else

if <CScore is Low> and <CRatio is Bad_CR> or <CCredit is Bad_CC>

then <Decision is Reject>

The following FuzzyCOPE production rule calls a function for fuzzy inference and then calls a Kohonen self organising topological map for quantising the resulting membership functions into two dimensional decision space:

(defrule loan

(input_data $?x)

=>

(bind $?y (dcmfuzzy "loan.rul" $?x 1 2 1))

(assert (result $?y))

(bind $?z (recallkoh "koh.net" $?y)))

8 Building speech and language recognition systems in FuzzyCOPE

Speech and language knowledge is ambiguous and difficult to articulate and to process. It requires good pattern matching as well as rules which reflect the ambiguity in the language. Higher level speech and language rule-based systems must have facilities to represent and process ambiguous, contradictory, incomplete, and uncertain knowledge. Articulating rules for language processing, which are fuzzy by nature, is a difficult task for language experts. Such rules, used by humans, reflect their knowledge on the language which is not a simple 'flat' set of rules, but possibly a hierarchical structure of many layers of such rules. The more we learn and practice a language, the more layers we build in our brain, the more complex speech constructions we are able to recognise and comprehend.

A module for extracting fuzzy rules from speech and language data is an important part of a software environment for developing speech and language recognition systems. This module makes the environment adaptable, though it is not clear that complex speech and language recognition fuzzy rules can be captured in an automatic way to the same degree to which humans have learned them.

FuzzyCOPE is being currently used for different speech recognition tasks like:

• building speech interfaces to fuzzy and standard databases; the main idea here is to use neural networks for low level recognition and fuzzy rule-based systems for higher level knowledge processing;

• phonemes recognition;

• spoken digits recognition for telecommunication;

• simple language translation tasks.

FuzzyCOPE is currently being upgraded with signal processing modules for the purpose of using it for real-time speech recognition tasks.

9 Conclusions

The hybrid fuzzy connectionist production system environment presented here makes possible:

• building complex multimodular and hierarchical hybrid systems;

• experimenting with different pathways for solving a given problem before choosing and compiling the best one;

• extracting explicit knowledge from raw data;

• mixing explicit expert knowledge with raw data and with extracted knowledge from data when creating a system;

• tuning and revising existing or extracted knowledge.

• improving the understanding of problems by extracting, refining and interpreting fuzzy rules.

The software environment FuzzyCOPE is currently under development. Future research is anticipated in the following directions:

• further development of FuzzyCOPE by integrating more data analysis techniques and connectionist learning methods;

• further use of FuzzyCOPE for decision making, speech and language recognition, building speech interfaces to standard and fuzzy databases, real time mobile robot navigation, control.

Acknowledgements

The programming work for creating the software environment FuzzyCOPE was done by a student team, Jay Garden, Tim Albertson and Paul Jones, of the Department of Information Science at the University of Otago. The experiments on the stock exchange index SE40 data were done by Max Bailey - also a student at Otago.

References

1. Hendler, J. (1989) On the Need for Hybrid Systems, Connection Science, special issue on Hybrid Connectionist/ Symbolic Systems, 1(3).

2. Kasabov, N.K.(1990) Hybrid Connectionist Rule-Based Systems, Jorrand, Ph. and V. Sgurev (ed), Artificial Intelligence - Methodology, Systems, Applications, Elsevier Science Publ. B.V., pp.227-235.

3. Kasabov, N. (1993) Hybrid Connectionist Production Systems, Journal of Systems Engineering, vol.3, No.1, pp.15-
21, Springer Verlag London.

4. Zadeh, L. (1965) Fuzzy sets, Information and Control, 8, 338-353.

5. Terano, T. (1993) Long-term View on Fuzzy Technology and LIFE Projects, LIFE Technical News, Vol.4, No.1, November, pp. 1-10.

6. Giarratano, J., G.Riley (1989) Expert Systems. Principles and programming. PWS Publ., Boston.

7. Kohonen, T. (1989) Self Organization and Associative Memory, Springer Verlag, Berlin.

8. Rumelhart, D.E., McClelland, J.L. and PDP Research Group (1986) Parallel Distributed Processing, Vol.1, Foundations, MIT Press, MA.

9. Terano, T., Asai, K., Sugeno, M. (1992) Fuzzy Systems Theory and Its Applications. Academic Press, Inc.

10. Mizumoto, M. and H.Zimmermann (1982) Comparison of Fuzzy Reasoning Methods, Fuzzy Sets and Systems, vol.18, pp.253- 283.

11. Furuhashi, T., Hasegawa, T., Horikawa S., Uchikawa, Y. (1993) An Adaptive Fuzzy Controler Using Fuzzy Neural Networks, in: Proceedings of Fifth IFSA World Congress, pp.769-772.

12. Kosko, B. (1992) Neural Networks and Fuzzy Systems. Prentice Hall, Englewood Cliffs.

13. Cybenko, G. (1989) Approximation by Superpositions of Sigmoidal Function. Mathematics of Control, Signals, and Systems, 2, 303-314.

14. Kasabov, N. (1993) Learning Fuzzy Production Rules for Approximate Reasoning with Connectionist Production Systems, in: S.Gielen and B. Kappen (Eds) Proceedings of ICANN'93, Amsterdam, 13-16 September), pp.337-345, Springer Verlag

15. Tano, S., Oyama,T., Arnould, T., Bastian, A. (1994) Definition and Tuning of Unit-based Fuzzy Systems in FINEST, IEEE, 0-7803-1896-X/94, pp.436-440.

16. Kasabov, N., Shishkov, S. (1993) A Connectionist Production System and the Use of its Partial Match for Approximate Reasoning, Connection Science, vol. 5, Nos.3&4, pp. 275-305, Carfax Publishing Company

17. Lim, M. & Takefuji, Y. (1990) Implementing fuzzy rule-based systems on silicon chips. IEEE Expert, February, pp.31-45.

18. Kasabov, N. (1994) A Filtering neuron and its Application for Building Connectionist Production Systems, Proc. of ICONIP'94, Seoul,17-21 October.