The network training is fine without adding any contrast normalization layer (all types), but ones add the contrast normalization layers, after several iterations the net gets nan values. I tried different values of the size, scale and pow values, and tried to place the layer before and after pooling layer.

I have a similar issue. Does it still work when you remove the contrast normalization layer?