位置:成果数据库 > 期刊 > 期刊详情页
Negative effects of sufficiently small initial weights on back-propagation neural networks
  • 时间:0
  • 分类:TP18[自动化与计算机技术—控制科学与工程;自动化与计算机技术—控制理论与控制工程]
  • 作者机构:[1]School of Mathematical Sciences, Dalian University of Technology, Dalian 116024, China, [2]School of Information Science and Engineering, Dalian Polytechnic University, Dalian 116035, China, [3]Department of Mathematics and Computational Science, Hengyang Normnal University, Hengyang 421002, China
  • 相关基金:Project supported by the National Natural Science Foundation of China (No. 11171367) and the Fundamental Research Funds for the Central Universities, China
中文摘要:

在训练前馈控制神经网络,起始的重量应该在大小是小的以便阻止早熟的浸透,这通常被建议。这份报纸的目的是指出这个故事的另外的方面:在一些情况中,错误功能的坡度不仅为无穷地大的重量而且为零重量是零。在训练过程的开始的慢集中经常是足够地小的起始的重量的结果。因此,我们建议在这些情况中,重量的起始的价值也应该大,也不太小。例如,起始的重量的选择的一个典型范围可能是相似的一些东西(0.4,0.1 )(0.1, 0.4 ) ,而非(0.1, 0.1 ) 是由平常的策略建议了。我们中等尺寸重量应该被使用的理论也被扩大了到一些通常使用的转移功能和错误功能。数字实验被执行支持我们的理论调查结果。

英文摘要:

In the training of feedforward neural networks, it is usually suggested that the initial weights should be small in magnitude in order to prevent premature saturation. The aim of this paper is to point out the other side of the story: In some cases, the gradient of the error functions is zero not only for infinitely large weights but also for zero weights. Slow convergence in the beginning of the training procedure is often the result of sufficiently small initial weights. Therefore, we suggest that, in these cases, the initial values of the weights should be neither too large, nor too small. For instance, a typical range of choices of the initial weights might be something like (-0.4,-0.1) U (0.1,0.4), rather than (-0.1,0.1) as suggested by the usual strategy. Our theory that medium size weights should be used has also been extended to a few commonly used transfer functions and error functions. Numerical experiments are carried out to support our theoretical findings.

同期刊论文项目
同项目期刊论文