deep learning - Why does CNN ignore some input images?

Question

I am working on a super-resolution problem for downscaling of the responses fields of a dynamical system y=f(x,t), where x is the material property, t is time.

In the super-resolution task, we useed CNN to learn the relationship y_{LR,t} -> y_{HR,t}. It worked relatively well with some details lossed. So we tried to recover the high frequency details with an additional regression net to learn the relationship f(·): (x_HR, t) -> y_{HR,t}. The regression net also worked well (i.e. given (x_HR, t) as inputs, it predicted the y_HR time series)

In implementation, two nets were respectively used to extract feature map from y_{LR,t} and (x_HR, t). These feature maps were then concatenated together to feed into an upsampler to recover y_{HR,t}. For this hybrid super-resolution net, the inputs are { x_HR, y_{LR,t} }, the ouput is y_{HR,t}, with t=1 or 2.

But the hybrid net always produced the output for y_{HR,1} regardless of the inputs are { x_HR, y_{LR,1} } or { x_HR, y_{LR,2} }. It seems that the net focused on the x_HR input and ignore y_{LR,t}.

The hybrid net successfully recovered y_{HR,1}

For another test example, the hybrid net failed to recover y_{HR,2} even y_{LR,2} was given as the input. It recovered y_{HR,1} instead

Any help would be greatly appreciated!

question from:https://stackoverflow.com/questions/65878616/why-does-cnn-ignore-some-input-images

Answer 1

Waitting for answers