Engelbrecht Andries P. Computational Intelligence: An Introduction
Подождите немного. Документ загружается.
List of Figures
Figure Page
1.1 Computational Intelligence Paradigms .................. 5
1.2 ABiologicalNeuron ............................ 6
1.3 AnArtificialNeuron ............................ 6
1.4 AnArtificialNeuralNetwork ....................... 7
2.1 AnArtificialNeuron ............................ 17
2.2 ActivationFunctions............................ 19
2.3 Artificial Neuron Boundary ........................ 21
2.4 LinearSeparableBooleanPerceptrons .................. 22
2.5 XOR Decision Boundaries . . ....................... 23
2.6 GradientDescentIllustrated........................ 24
3.1 FeedforwardNeuralNetwork ....................... 28
3.2 FunctionalLinkNeuralNetwork ..................... 29
3.3 ElmanSimpleRecurrentNeuralNetwork ................ 32
3.4 JordanSimpleRecurrentNeuralNetwork ................ 33
3.5 ASingleTime-DelayNeuron ....................... 34
3.6 CascadeNeuralNetwork.......................... 36
3.7 Product Unit Neural Network Search Space for f(z)=z
3
....... 44
3.8 Feedforward Neural Network Classification Boundary Illustration . . . 51
3.9 HiddenUnitFunctioningforFunctionApproximation ......... 52
3.10EnsembleNeuralNetwork......................... 53
4.1 UnsupervisedNeuralNetwork....................... 56
4.2 LearningVectorQuantizertoIllustrateClustering ........... 60
4.3 Self-organizing Map ............................ 63
4.4 VisualizationofSOMClustersforIrisClassification .......... 70
5.1 RadialBasisFunctionNeuralNetwork.................. 74
6.1 ReinforcementLearningProblem ..................... 84
7.1 IllustrationofOverfitting ......................... 96
7.2 EffectofOutliers.............................. 101
7.3 SumSquaredErrorObjectiveFunction ................. 101
7.4 HuberObjectiveFunction......................... 102
7.5 EffectofLearningRate .......................... 108
7.6 AdaptiveSigmoid.............................. 115
7.7 PassivevsActiveLearning......................... 119
8.1 HammingDistanceforBinaryandGrayCoding ............ 131
9.1 CrossoverOperatorsforBinaryRepresentations............. 147
xix
xx LIST OF FIGURES
9.2 IllustrationofMulti-parentCenterofMassCrossoverOperators.... 151
9.3 Diagonal Crossover ............................. 153
9.4 MutationOperatorsforBinaryRepresentations............. 155
9.5 AnIslandGAModel............................ 163
10.1Tree-RepresentationofXOR........................ 178
10.2Tree-RepresentationforMathematicalExpressions ........... 179
10.3GeneticProgrammingCrossover ..................... 181
10.4 Genetic Programming Mutation Operators . . ............. 183
11.1 Finite-State Machine [278] ......................... 208
12.1IllustrationofMutationDistributionsforES .............. 219
12.2DirectedMutationOperatorforES.................... 225
12.3BiasedMutationforEvolutionStrategies ................ 230
13.1DifferentialEvolutionMutationandCrossoverIllustrated ....... 243
13.2AngleModulationIllustrated ....................... 254
14.1 Illustration of Population and Belief Spaces of Cultural Algorithms . . 263
14.2IllustrationofBeliefCells ......................... 272
16.1GeometricalIllustrationofVelocityandPositionUpdates ....... 294
16.2 Multi-particle gbest PSOIllustration ................... 295
16.3 Illustration of lbest PSO .......................... 296
16.4ExampleSocialNetworkStructures.................... 302
16.5EffectsofVelocityClamping........................ 305
16.6 Stochastic Particle Trajectory for w =0.9andc
1
= c
2
=2.0...... 315
17.1BinaryBridgeExperiment......................... 361
17.2ShortestPathSelectionbyForagerAnts................. 362
17.3GraphforShortestPathProblems .................... 365
17.42-optand3-optLocalSearchHeuristic.................. 408
18.1Antigen-Antibody-Complex ........................ 417
18.2WhiteCellTypes.............................. 418
18.3LifeCycleofALymphocyte........................ 418
18.4B-CellDevelopsintoPlasmaCell,ProducingAntibodies........ 419
18.5MacrophageandNKTC .......................... 420
18.6Co-StimulationofT-CellbyanAPC................... 423
19.1 r-ContinuousMatchingRule ....................... 428
19.2AdaptedNegativeSelection ........................ 430
20.1IllustrationofMembershipFunctionforTwo-ValuedSets ....... 455
20.2 Illustration of tall MembershipFunction ................. 455
20.3ExampleMembershipFunctionsforFuzzySets ............. 458
20.4IllustrationofFuzzySetContainment .................. 458
20.5IllustrationofFuzzyOperators ...................... 460
LIST OF FIGURES xxi
20.6MembershipFunctionsforAssignments1and2............. 463
21.1FuzzyRule-BasedReasoningSystem................... 468
21.2InterpretingaFuzzyRule ......................... 469
21.3DefuzzificationMethodsforCentroidCalculation............ 473
21.4MembershipFunctionsforAssignments2and3............. 474
22.1AFuzzyController............................. 476
23.1RoughSetIllustration ........................... 481
A.1 TypesofOptimaforUnconstrainedProblems.............. 553
A.2 Problem with Multiple Global Optima, with f(x)=sin
6
(5πx)..... 554
A.3 ConstrainedProblemIllustration..................... 562
A.4 IllustrationoftheEffectofPenaltyFunctions.............. 564
A.5 IllustrationofDominance ......................... 572
A.6 ExamplePareto-OptimalFronts...................... 574
A.7 Dynamic Parabola Objective Function . . . ............... 577
A.8 DynamicObjectiveFunction ....................... 579
List of Tables
Table Page
3.1 SUsandPUsNeededforSimpleFunctions................ 43
8.1 BinaryandGrayCoding.......................... 132
10.1XORTruthTable.............................. 178
10.2 Genetic Programming Applications .................... 185
11.1ResponseofFinite-StateMachine..................... 207
11.2 Real-World Applications of Evolutionary Programming . . ....... 210
12.1ApplicationsofEvolutionStrategies ................... 235
13.1ApplicationsofDifferentialEvolution................... 259
15.1 Algorithms Used to Achieve Adaptation in CCE . . . ......... 280
15.2ApplicationsofCompetitiveCoevolution................. 281
16.1ApplicationsofParticleSwarmOptimization .............. 358
17.1GeneralACOAlgorithmParameters ................... 383
17.2 Ant Algorithm Applications ........................ 405
xxiii
List of Algorithms
Algorithm Page
3.1 StochasticGradientDescentLearningAlgorithm ........... 41
3.2 Conjugate Gradient Neural Network Training Algorithm . ...... 45
3.3 Fletcher-ReevesConjugateGradientAlgorithm ............ 46
3.4 Scaled Conjugate Gradient Algorithm .................. 47
3.5 LeapFrog Algorithm ............................ 50
4.1 Hebbian Learning Algorithm ....................... 57
4.2 Learning Vector Quantizer-I Training Algorithm ............ 61
4.3 BatchSelf-OrganizingMap........................ 65
4.4 FastBatchSelf-OrganizingMap..................... 66
4.5 Growing Self-Organizing Map Algorithm ................ 67
4.6 ShortcutWinnerSearch ......................... 68
5.1 TraininganRBFNNwithFixedCenters ................ 77
5.2 GradientDescentTrainingofRBFNN ................. 78
5.3 Two-PhaseRBFNNTraining....................... 79
6.1 RPROP Neural Network Training Algorithm .............. 88
6.2 Connectionist Q-Learning Update Algorithm .............. 90
6.3 Q(λ)ConnectionistUpdateAlgorithm ................. 90
7.1 Lambda-GammaTrainingRule ..................... 117
7.2 Generic Active Learning Algorithm ................... 123
8.1 Generic Evolutionary Algorithm ..................... 128
8.2 RouletteWheelSelection......................... 136
8.3 StochasticUniversalSampling...................... 137
9.1 Generic Algorithm for Bitstring Crossover ............... 145
9.2 One-PointCrossoverMaskCalculation ................. 145
9.3 Two-PointCrossoverMaskCalculation................. 146
9.4 UniformCrossoverMaskCalculation .................. 146
9.5 GeneScanningCrossoverOperator ................... 153
9.6 Uniform/RandomMutation ....................... 155
9.7 InorderMutation ............................. 155
9.8 MessyGeneticAlgorithm......................... 160
9.9 InteractiveEvolutionAlgorithm..................... 161
9.10 Algorithm to Maintain an Archive of Nondominated Solutions .... 171
9.11 Nondominated Sorting Genetic Algorithm . . ............. 173
11.1 BasicEvolutionaryProgrammingAlgorithm .............. 188
11.2 Solis and Wets Random Search Algorithm for Function Minimization 204
xxv
xxvi LIST OF ALGORITHMS
11.3 Extinction Evolutionary Programming for Function Minimization . . 205
12.1 EvolutionStrategyAlgorithm ...................... 215
12.2 PolarEvolutionStrategy......................... 227
12.3 ParetoArchivedEvolutionStrategy................... 232
12.4 ArchiveUpdateAlgorithmusedbyPAES ............... 232
13.1 DifferentialEvolutionBinomialCrossover ............... 240
13.2 DifferentialEvolutionExponentialCrossover.............. 240
13.3 GeneralDifferentialEvolutionAlgorithm................ 241
13.4 Hybrid Differential Evolution with Acceleration and Migration .... 246
13.5 StochasticGradientDescentforNeuralNetworkTraining....... 247
13.6 DifferentialEvolutionwithStochasticGradientDescent........ 248
13.7 Rank-BasedCrossoverOperatorforDifferentialEvolution ...... 249
13.8 Rank-BasedMutationOperatorforDifferentialEvolution ...... 249
13.9 AngleModulatedDifferentialEvolution................. 254
13.10BinaryDifferentialEvolutionAlgorithm ................ 255
13.11InitializationofQuantumIndividuals.................. 258
14.1 Cultural Algorithm ............................ 262
14.2 Fuzzy Rule-base for Cultural Algorithm Acceptance Function . . . . 267
15.1 Competitive Coevolutionary Algorithm with Two Populations . . . . 279
15.2 Single Population Competitive Coevolutionary Algorithm . ...... 279
15.3 CoevolutionaryTrainingofGameAgents................ 282
16.1 gbest PSO ................................. 291
16.2 lbest PSO ................................. 293
16.3 Particle Clustering Algorithm ...................... 299
16.4 CalculationofSpatialNeighborhoods.................. 318
16.5 Selection-BasedPSO ........................... 321
16.6 Global Best Spawning Algorithm . . . ................. 323
16.7 Life-CyclePSO .............................. 330
16.8 Cooperative Split PSO Algorithm .................... 331
16.9 HybridofCooperativeSplitPSOandGCPSO............. 333
16.10SelectionofParticlestoReinitialize................... 336
16.11 Self-Organized Criticality PSO ...................... 337
16.12Multi-startParticleSwarmOptimization................ 338
16.13 binary PSO ................................ 341
16.14CoelloCoelloandLechugaMOPSO................... 345
16.15NichePSOAlgorithm ........................... 351
16.16 NichePSO Sub-swarm Creation Algorithm . . . ............ 353
16.17 PSO Coevolutionary Game Training Algorithm ............ 357
17.1 ArtificialAntDecisionProcess...................... 363
LIST OF ALGORITHMS xxvii
17.2 SimpleACOAlgorithm.......................... 367
17.3 Ant System Algorithm .......................... 371
17.4 Ant Colony System Algorithm . . .................... 374
17.5 MMAS Algorithm with Periodic Use of the Global-Best Path .... 377
17.6 ANTSAlgorithm ............................. 382
17.7 Lumer–Faieta Ant Colony Clustering Algorithm ............ 388
17.8 Continuous Ant Colony Optimization Algorithm ............ 397
17.9 MultipleColonyACOLocalSharingMechanism............ 400
19.1 BasicAISAlgorithm ........................... 427
19.2 Training ALCs with Negative Selection ................. 429
19.3 CLONALG Algorithm for Pattern Recognition ............. 432
19.4 A Multi-layered AIS Algorithm . . ................... 435
19.5 ArtificialImmuneNetwork(AINE) ................... 437
19.6 ResourceAllocationintheArtificialImmuneNetwork ........ 438
19.7 Self Stabilizing AIS . . . ......................... 439
19.8 aiNet Learning Algorithm ........................ 443
19.9 InitializationPhaseforanAdaptiveMailbox.............. 446
19.10 Running Phase for an Adaptive Mailbox . . . ............. 447
A.1 GeneralLocalSearchAlgorithm..................... 556
A.2 Simulated Annealing Algorithm ..................... 558