phitodeep.layers.activation
===========================

.. py:module:: phitodeep.layers.activation


Classes
-------

.. autoapisummary::

   phitodeep.layers.activation.ReLu
   phitodeep.layers.activation.LeakyReLu
   phitodeep.layers.activation.GELU
   phitodeep.layers.activation.Swish
   phitodeep.layers.activation.Sigmoid
   phitodeep.layers.activation.Tanh
   phitodeep.layers.activation.Softmax
   phitodeep.layers.activation.ELU


Module Contents
---------------

.. py:class:: ReLu

   Bases: :py:obj:`phitodeep.layers.base.Layer`


   Base class for all layers in the network.


   .. py:method:: forward(X)


   .. py:method:: backward(dL_dZ)

      Backpropagate through ReLU activation.
      ReLU derivative: 1 if X > 0, else 0



   .. py:method:: copy()


.. py:class:: LeakyReLu(alpha=0.01)

   Bases: :py:obj:`phitodeep.layers.base.Layer`


   Base class for all layers in the network.


   .. py:attribute:: alpha
      :value: 0.01



   .. py:method:: forward(X)


   .. py:method:: backward(dL_dZ)

      Backward pass through the block.

      :param dL_dZ: gradient of loss w.r.t. output of this block

      :returns: gradient of loss w.r.t. input (to pass to previous layer)
      :rtype: dL_dX



   .. py:method:: copy()


.. py:class:: GELU

   Bases: :py:obj:`phitodeep.layers.base.Layer`


   Base class for all layers in the network.


   .. py:attribute:: constant
      :value: 0.044715



   .. py:method:: forward(X)


   .. py:method:: backward(dL_dZ)

      Backward pass through the block.

      :param dL_dZ: gradient of loss w.r.t. output of this block

      :returns: gradient of loss w.r.t. input (to pass to previous layer)
      :rtype: dL_dX



   .. py:method:: copy()


.. py:class:: Swish

   Bases: :py:obj:`phitodeep.layers.base.Layer`


   Base class for all layers in the network.


   .. py:method:: forward(X)


   .. py:method:: backward(dL_dZ)

      Backward pass through the block.

      :param dL_dZ: gradient of loss w.r.t. output of this block

      :returns: gradient of loss w.r.t. input (to pass to previous layer)
      :rtype: dL_dX



   .. py:method:: copy()


.. py:class:: Sigmoid

   Bases: :py:obj:`phitodeep.layers.base.Layer`


   Base class for all layers in the network.


   .. py:method:: forward(X)


   .. py:method:: backward(dL_dZ)

      Backpropagate through Sigmoid activation.
      Sigmoid derivative: sigmoid(Z) * (1 - sigmoid(Z))



   .. py:method:: copy()


.. py:class:: Tanh

   Bases: :py:obj:`phitodeep.layers.base.Layer`


   Base class for all layers in the network.


   .. py:method:: forward(X)


   .. py:method:: backward(dL_dZ)

      Backpropagate through Tanh activation.
      Tanh derivative: 1 - tanh(Z)^2



   .. py:method:: copy()


.. py:class:: Softmax

   Bases: :py:obj:`phitodeep.layers.base.Layer`


   Base class for all layers in the network.


   .. py:method:: forward(X)


   .. py:method:: backward(dL_dZ)

      Backpropagate through Softmax activation.
      When paired with CategoricalCrossEntropy, the combined gradient
      (y_pred - one_hot(y_true)) / N is computed entirely in the loss,
      so this layer is a straight pass-through.



   .. py:method:: copy()


.. py:class:: ELU(alpha=1.0)

   Bases: :py:obj:`phitodeep.layers.base.Layer`


   Base class for all layers in the network.


   .. py:attribute:: alpha_activation
      :value: 1.0



   .. py:method:: forward(X)


   .. py:method:: backward(dL_dZ)

      Backpropagate through ELU activation.
      ELU derivative: 1 if X > 0, else alpha * exp(X)



   .. py:method:: copy()