Layer Components and calling sequence

TensorFlow General Discussion
, ,
1

Hi,

I am new to tensor flow and currently am reading Keras. So I see a code like this:

tf.keras.layers.Dense(
    units,
    activation=None,
    use_bias=True,
    kernel_initializer="glorot_uniform",
    bias_initializer="zeros",
    kernel_regularizer=None,
    bias_regularizer=None,
    activity_regularizer=None,
    kernel_constraint=None,
    bias_constraint=None,
    **kwargs
)

I see that there are couple of initializers and Regularizers and Constraint BOTH for Bias as well as kernel. I have searched on the internet but could not find the calling sequence information. For example is the calling sequence like this:
use_bias → bias_initializer → bias_regularizer → bias_constraint AND
kernel_initializer → kernel_regularizer → kernel_constraint

or is the constraint called before regularizer.
Also is kernel = activation function or kernel is something that runs activation function?

3

Hi,

A couple of days ago I talked about something very similar here: https://tensorflow-prod.ospodiscourse.com/t/what-is-effect-of-use-bias/3336/3?u=lgusm

The Dense layers computes:
output = activation(dot(input, kernel) + bias)

A dense layer is trying to calculate: y= Ax + b

input (the x in the equation) are the data used for training
Kernel (the A in the equation) are the weights your layer is trying to find
Bias (b in the equation) is a vector. The use_bias parameter configures the layer to calculate a bias vector together with with the weights. The initializer, regularizer and constraint will be used with the bias calculation is enabled.

for more information: https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense