How significant it is to add non-linearity in a Deep Learning model?

how significant it is to add non-linearity in a model?
and is it necessary to add it in any sort of layering and any type model.

Highly, non linearity is one of the reason why deep learning models achieves good result.

Can you elaborate this

Non-Linearity is very important. Think about a function that takes an input x and outputs y,
This means f(x) = y, Now if the function f is linear what you can get is to some linear-combination of the input x, examples f(x) = A*x + B . But think about a task in which you want to map a x==image_of_a_monkey to monkey. Since the input-space deals with images and output space deals with names. It becomes imperative to use non-linear functions that can do the task.

4 Likes

can the concept of adding non-linearity always be used in models whether they are purely ML based or purely DL based.
does different types of layering(sequential, convolution, activation, cropping) affect this concept or vice-versa

I think it can depend on the type of data you are looking at. If your data shows high linear correlation, you can stick with good ol linear regression to model or predict. Just most things aren’t correlated linearly so its pretty significant to use nonlinearity.

2 Likes