Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
1.3k views
in Technique[技术] by (71.8m points)

tensorflow - Are dropout layer weights used at execution time?

I'm trying to gain a good understanding of the mechanics of how neural nets execute. To that end I am working through some Tensor Flow examples and attempting to implement my own execution implementation.

While playing around with the MNSET example I grasped that the Dropout(0.2) (here) layer is is actually part of the struture of the trained net in the end. You can examine its weights using model.get_weights()[2]. The documentation suggests that Dropout is a noop at execution time. If that's true, are the weights ignored? Or are the trained weights used just like the regular Dense layer at execution time?

question from:https://stackoverflow.com/questions/65866542/are-dropout-layer-weights-used-at-execution-time

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

from the documentation located here:

The Dropout layer randomly sets input units to 0 with a frequency of rate at each step during training time, which helps prevent overfitting. Inputs not set to 0 are scaled up by 1/(1 - rate) such that the sum over all inputs is unchanged.

Note that the Dropout layer only applies when training is set to True 
such that no values are dropped during inference. 
When using model.fit, training will be appropriately set to True 
automatically, and in other contexts, you can set the 
kwarg explicitly to True when calling the layer.

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

2.1m questions

2.1m answers

60 comments

57.0k users

...