Swish function

The swish function is a family of <a href="/facts/Function_(mathematics)/CdReEKCh">mathematical function</a> defined as follows:

swish
 
 β
 
 
 ⁡
 (
 x
 )
 =
 x
 sigmoid
 ⁡
 (
 β
 x
 )
 =
 
 
 x
 
 1
 +
 
 e
 
 −
 β
 x
 
 
 
 
 
 .
 
 
 {\displaystyle \operatorname {swish} _{\beta }(x)=x\operatorname {sigmoid} (\beta x)={\frac {x}{1+e^{-\beta x}}}.}
 
<a class="footnote-ref" id="fnref:1" href="#fn:1">1</a>
where 
 
 
 
 β
 
 
 {\displaystyle \beta }
 
 can be constant (usually set to 1) or <a href="/facts/Trainable_parameter/V3mMQb2G">trainable</a>.
The swish family was designed to smoothly <a href="/facts/Interpolation/8PyetX4f">interpolate</a> between a linear function and the ReLU function.
When considering positive values, Swish is a particular case of doubly parameterized sigmoid shrinkage function defined in : Eq 3 . Variants of the swish function include <a href="/facts/Mish_(function)/pwzdU9jF">Mish</a>.

Swish function open-in-new

Swish function