jsat.classifiers.neuralnetwork.activations

Class ReLU

java.lang.Object

jsat.classifiers.neuralnetwork.activations.ReLU

All Implemented Interfaces:

java.io.Serializable, ActivationLayer

public class ReLU
extends java.lang.Object
implements ActivationLayer

This Activation Layer is for Rectified Linear Units. A ReLU activation is simply f(x) = max(0, x), and is thus very fast to compute.
See: Nair, V., & Hinton, G. E. (2010). Rectified Linear Units Improve Restricted Boltzmann Machines. Proceedings of the 27th International Conference on Machine Learning, 807–814.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ReLU()

Method Summary

Modifier and Type	Method and Description
void	activate(Matrix input, Matrix output, boolean rowMajor) Computes the activation function of this layer on the given input.
void	activate(Vec input, Vec output) Computes the activation function of this layer on the given input.
void	backprop(Matrix input, Matrix output, Matrix delta_partial, Matrix errout, boolean rowMajor) This method computes the backpropagated error to a given layer.
void	backprop(Vec input, Vec output, Vec delta_partial, Vec errout) This method computes the backpropagated error to a given layer.
ReLU	clone()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ReLU

public ReLU()

Method Detail

activate

public void activate(Vec input,
                     Vec output)

Description copied from interface: ActivationLayer

Computes the activation function of this layer on the given input.

Specified by:

activate in interface ActivationLayer

Parameters:

input - the raw input to compute the activation for

output - the location to store the activation in

activate

public void activate(Matrix input,
                     Matrix output,
                     boolean rowMajor)

Description copied from interface: ActivationLayer

Computes the activation function of this layer on the given input.

Specified by:

activate in interface ActivationLayer

Parameters:

input - the raw input to compute the activation for

output - the location to store the activation in

rowMajor - true if the information per input is stored in rows, false if the inputs were stored by column. This parameter does not indicate if the matrices themselves are backed by a row or column major implementation

backprop

public void backprop(Vec input,
                     Vec output,
                     Vec delta_partial,
                     Vec errout)

Description copied from interface: ActivationLayer

This method computes the backpropagated error to a given layer. Often denoted as δ^l = w^{l+1 T} δ^l+1 ⊗ ∂ f(x^l), where ∂ is the Hadamard product and ∂ f(x^l) is the derivative of this activation function on the input that was feed into this activation.
delta_partial and errout may point to the same vector object

Specified by:

backprop in interface ActivationLayer

Parameters:

input - the input to this layer that was feed in to be activated

output - the activation that was produced for this layer

delta_partial - the error assigned to this layer from the above layer, sans the hamard product with the derivative of the layer activation. Often denoted as w^{l+1 T} δ^l+1

errout - the delta value or error produced for this layer

backprop

public void backprop(Matrix input,
                     Matrix output,
                     Matrix delta_partial,
                     Matrix errout,
                     boolean rowMajor)

Description copied from interface: ActivationLayer

This method computes the backpropagated error to a given layer. Often denoted as δ^l = w^{l+1 T} δ^l+1 ⊗ ∂ f(x^l), where ∂ is the Hadamard product and ∂ f(x^l) is the derivative of this activation function on the input that was feed into this activation.
delta_partial and errout may point to the same vector object

Specified by:

backprop in interface ActivationLayer

Parameters:

input - the input to this layer that was feed in to be activated

output - the activation that was produced for this layer

delta_partial - the error assigned to this layer from the above layer, sans the hamard product with the derivative of the layer activation. Often denoted as w^{l+1 T} δ^l+1

errout - the delta value or error produced for this layer

rowMajor - true if the information per input is stored in rows, false if the inputs were stored by column. This parameter does not indicate if the matrices themselves are backed by a row or column major implementation

clone

public ReLU clone()

Specified by:

clone in interface ActivationLayer

Overrides:

clone in class java.lang.Object