STATISTICA Automated Neural Networks

Moving from STATISTICA Version 7 Neural Networks (SNN) to STATISTICA Version 8 Automated Neural Networks

Automated Neural Networks Software

Contents

• A Brief Overview
  
• Technical Description
  
• Example Applications
  
• Optional Source Code Generator Add-on Available
  

The popularity of neural network methodology is rapidly growing in a wide variety of areas from basic research to data mining applications, business forecasting and risk management, engineering, and others (see Example Applications). STATISTICA Automated Neural Networks (SANN) is one of the most advanced and best performing neural networks applications on the market. It offers numerous unique advantages and will appeal not only to neural network experts (by offering them a wide selection of network types and training algorithms), but also to new users in the field of neural computing (via the unique Automated Network Search, a tool that guides the user through the necessary procedures for creating neural networks).

STATISTICA Automated Neural Networks - a technical description

• STATISTICA Automated Neural Networks - Tackling the Real Issues in Neural Computing
• Input Data
• Data Scaling and Nominal Value Preparation
• Selecting a Neural Network Model
• The Automated Network Search (Automated evaluation and selection of multiple network architectures)
• Training a Neural Network
• Probing and Testing a Neural Network
• Embedded Solutions
• Training Algorithm Summary
• System Requirements
• Size Limitations
• Electronic Manual
• Examples of Real-life Applications
• Optional Source Code Generator Add-on

• Integrated pre- and post-processing, including data selection, nominal-value encoding, scaling, normalization, and missing value substitution, with interpretation for classification, regression, and time series problems.
• Exceptional ease of use coupled with unsurpassed analytic power; for example, a unique wizard-style Automated Network Search (ANS) guides the user step-by-step through the procedure of creating a variety of different networks and choosing the network with the best performance (a task that would otherwise require a lengthy "trial-and-error" process and a solid background in the underlying theory).
• State-of-the-art, highly optimized training algorithms (including Conjugate Gradient Descent and BFGS); full control over all aspects that influence the network performance such as activation and error functions or network complexity.
• Support for combinations of networks and network architectures of practically unlimited sizes organized in network sets for forming ensembles.
• Comprehensive graphical and statistical feedback that facilitates interactive exploratory analyses.
• Full integration with the STATISTICA system; all results, graphs, reports, etc. can be further modified with STATISTICA's powerful graphics and analytic tools (e.g., to perform further analyses of prediction residuals, generate annotated summary reports, and so on).
• Full integration with STATISTICA's powerful tools for automation; record complete macros for all analyses; program custom neural network analyses and applications in the STATISTICA Visual Basic environment, or call STATISTICA Automated Neural Networks from any application that supports the Component Object Model (COM; e.g., Automatedally perform neural network analyses in Microsoft Excel spreadsheets, or incorporate neural network procedures in your own custom applications developed in C, C++, C#, Java, etc.).

STATISTICA Automated Neural Networks - Tackling the Real Issues in Neural Computing

Using neural networks involves more than simply feeding data to a neural network.

STATISTICA Automated Neural Networks (SANN) has the functionality to assist you through the critical design stages, including not only state-of-the-art Neural Network Architectures and Training Algorithms, but also innovative new approaches to network architecture design by using specific and meaningful error functions that allow the iterpretation of the output results. Moreover, software developers and those users who experiment with customized applications will appreciate the fact that once your prototyping experiments are completed using STATISTICA Automated Neural Networks' simple and intuitive user interface, neural networks analyses can be incorporated in custom applications by using either the STATISTICA library of COM functions that fully expose all functionality of the program or by using the C/C++ code generated by the program to aid in the deployment of fully trained networks.

Input Data

STATISTICA Automated Neural Networks is fully integrated with the STATISTICA system, so a large selection of tools for editing (preparing) data for analyses is available (transformations, case selection conditions, data verification tools, etc.). Like all STATISTICA analyses, the program can be "connected" to remote databases via the tools for in-place-database processing, or it can be linked to active data so that models are retrained or applied (e.g., to compute predicted values or classifications) Automatedally every time the data change.

Data Scaling and Nominal Value Preparation

In general, data must be specifically prepared for input into neural networks, and also it is important that the network output can be interpreted correctly. STATISTICA Automated Neural Networks (SANN) includes Automated data scaling for both inputs and outputs; there is also Automated recoding of Nominal valued variables (e.g., Sex = {Male, Female}), including one-of-N encoding. SANN also has facilities to handle missing data. There are special data prepartaion and interpretation facilities for use with Time Series. A large number of relevant tools are also included in STATISTICA.

For classification problems, SANN assigns cases to class memberships and interprets network outputs as true probabilities. In combination with SANN's specialized Softmax activation function and cross-entropy error functions, this supports a principled, probabilistic approach to classification.

The range of neural network models and the number of parameters that must be decided upon (including network size, and training algorithm control parameters) can seem bewildering [the Automated Network Search (ANS) is available to Automatedally search through numerous network architectures of varying complexities; see below]. STATISTICA Automated Neural Networks (SANN) supports the most important classes of neural networks for real world problem solving, including:

• Multilayer Perceptrons
• Radial Basis Function networks
• Self-Organizing Feature Maps
• Linear Networks

The above architecture can be used for regression, classification, regression time series, classification time series, and cluster analysis.

In addition, ANS supports Ensembles networks formed from arbitrary (when meaningful) combinations of the network types listed above. Combining networks to form Ensemble predictions are particularly easy to use in SANN, especially for noisey or small datasets.

SANN contains numerous facilities to aid in selecting an appropriate network architecture. SANN's statistical and graphical feedback includes histograms, matrices and graphs of individual and overall case errors, summaries of classification/misclassification performance, and vital statistics such as regression correlation - all Automatedally calculated.

For data visualization, SANN can also display scatterplots and 3D response surfaces to help the user understand the network's "behavior."

Naturally, you can use information from any of these sources for further analyses with other STATISTICA tools or for inclusion in your reports, further analysis, or customization.

SANN Automatedally retains copies of the best networks found as you experiment on a problem, which can be retrieved at any time. The usefulness and predictive validity of the network can Automatedally be assessed by including test and validation samples and by evaluating the size and efficiency of the network as well as the cost of misclassification.

For enhanced performance, STATISTICA Automated Neural Networks supports a number of network customization options. You can specify a linear output layer for networks used in (but not restricted to) regression problems or softmax activation functions for probability-estimation in classification problems. Cross-entropy error functions, based on information-theory models, are also included, and there is a range of specialized activiation functions, including Exponential, Tangent Hyperbolic, Logistic Sigmoid, and Sine functions for both hidden and output neurons.

The Automated Network Search (Automated evaluation and selection of multiple network architectures)

Included with STATISTICA Automated Neural Networks (SANN) is a tool that can Automatedally evaluate a large number of different neural network architectures of varying complexities, and select the best set of specific architectures for the problem at hand. It is known as the Automated Network Search (ANS)

A significant amount of time during the design of a neural network is spent on the selection of appropriate variables, and then optimizing the network architecture by heuristic search. SANN takes the pain out of the process by Automatedally conducting a heuristic search for you. This search includes network types, network sizes and architectures, activation functions, and even error functions when appropriate.

The Automated Network Search is an extremely effective tool that uses sophisticated techniques to search Automatedally for optimal network architectures. Why labor over a terminal for hours, when you can let STATISTICA Automated Neural Networks do the work for you?

As you experiment with architectures and network types, you rely critically on the quality and speed of the network training algorithms. STATISTICA Automated Neural Networks (SANN) supports the best known state-of-the-art training algorithms.

SANN naturally includes fast, second-order training algorithms: Conjugate Gradient Descent and BSFGS. There is also a memory-less version of BFGS to which SANN Automatedally switches whenever the amount of memory on your computer is at critical levels. These algorithms typically converge far more quickly than first order algorithms such as Gradient Descent.

STATISTICA Automated Neural Networks also includes a range of training algorithms for Cluster analysis, which is based on the well known Kohonen algorithm for self organizing feature maps.

Once you have trained a network, you'll want to test its performance and explore its characteristics. STATISTICA Automated Neural Networks (SANN) uses a range of statistics and graphical facilities.

You may select multiple models (and ensembles), in which case, wherever possible, SANN will display any results generated in a comparative fashion (e.g. by plotting the response curves for several models on a single graph, or presenting the predictions of several models in a single spreadsheet). This feature is particularly useful for comparing various models trained on the same data set.

All statistics are generated independently for the training, test, and validation samples or combinations of your choice.

Overall statistics calculated include mean network error, the so-called confusion matrix for classification problems (which summarizes correct and incorrect classification across all classes), and the correlation for regression problems - all Automatedally calculated. Kohonen networks include a Topological Map window, which enables you to visually inspect unit activations during data analysis.

Embedded Solutions (custom applications that use the STATISTICA Automated Neural Networks engines)

STATISTICA Automated Neural Networks' simple and efficient user-interface enables you to rapidly prototype neural network solutions to your problems.

In some applications, you may want to embed these solutions in your own systems and, for example, build them into larger computing environments (such as pre-designed procedures built into enterprise-wide computing systems).

Trained neural networks can be applied to new data (for prediction) in several ways: you can save the trained neural networks, and later retrieve them to be applied to new data (for prediction, predicted classification, or forecasting). You can also use the optional code generator to save fully trained neural network models in C programming language that is ready to compile and can be called from external applications and environments, such as visual basic, for deployment and predicting new data. Finally, all functionality of STATISTICA, including STATISTICA Automated Neural Networks, can be accessed as COM (Component Object Model) functions from other applications (e.g., from Java, Microsoft Excel, C#, VB.NET, etc.). For example, you could embed Automated analyses via STATISTICA Automated Neural Networks into your Microsoft Excel spreadsheets.

Training Algorithm Summary

• Gradient Descent
• Conjugate Gradient Descent
• BFGS
• Kohonen training
• k-Means Center Assignment for Radial Basis networks

STATISTICA System Requirements
Minimum:

The networks can be of practically unlimited size (that is, they can be much larger than what would ever be practical or reasonable). For all practical purposes, the program is effectively limited only by the hardware of the computer.

Electronic Manual

STATISTICA Neural Networks includes a well-illustrated manual, with a comprehensive, conceptual introduction to Neural Networks (and tutorials), and extensive context sensitive Help accessible from every dialog.

Examples of Real-life Applications

Neural networks can be used in virtually any situation where the objective is to determine an unknown variable or attribute from known observations or registered measurements (i.e., various forms of regression, classification, and time series), where there is a sufficient amount of historical data, and where there actually exists a tractable underlying relationship or a set of relationships (networks are relatively noise tolerant). In addition, neural networks can be used for exploratory analysis by looking for data clustering (Kohonen networks).

A comprehensive discussion of theoretical considerations related to the issue of when neural network applications are most likely to be successful can be found in the chapter on neural networks in the StatSoft Electronic Statistics Textbook (available on the StatSoft web site). The following list includes a selection of representative examples that by no means exhaust all areas where neural networks can be used.

• Optical Character Recognition, including Signature Recognition (e.g., a company has developed a device which identifies signatures, using not just appearance but also pen-velocity while signing, which makes it more difficult to perpetrate fraud).
• Image Processing (e.g., a system was developed which scanned images of London subway stations, and could tell if the station was Full, Empty, Half-Full etc. and was invariant across light conditions and presence/absence of trains).
• Financial Time Series Prediction (e.g., one trading company has claimed to have significantly improved trading performance using Multilayer Perceptrons to predict stock prices).
• Credit Worthiness (credit scoring; a classic problem - decide whether someone is a good credit risk, based on questionnaire information).
• Bulk mail targeting (i.e., identify customers who are more likely to respond positively to a mail-out, based on database information).
• Detection and evaluation of medical phenomena (e.g., detection of epileptic attacks, estimation of prostate tumor size).
• Condition monitoring of machinery (e.g., detecting when something has gone wrong with a machine based on vibration or acoustic signatures, so that preventative maintenance can be scheduled).
• Speech synthesis from text (e.g., the famous early experiment was Nettalk, which learned to produce phonemes from written text).
• Chaotic Time Series Prediction (a number of researchers have demonstrated good prediction capability on chaotic time series data).
• Process control (e.g., monitoring industrial process machinery and continuously adjusting control parameters).
• Engine management systems (estimating fuel consumption from sensor measurements and adjusting - a form of process control).
• Language analysis (e.g., using unsupervised techniques to identify key phrases, words, etc. in native South American languages).
• Real-time triggering systems of High Energy Physics detectors. Neural networks are noise tolerant and allow for robust pattern recognition of particle physics data with large statistical noise.

Optional Source Code Generator Add-on

2300 East 14th Street, Tulsa, OK 74104
Phone: (918) 749-1119; Fax: (918) 749-2217
e-mail: info@statsoft.com

©Copyright StatSoft, Inc., 1984-2008.
StatSoft, StatSoft logo, and STATISTICA, are trademarks of StatSoft, Inc.