snowflake cloud data platform - SnowPipe Paused ( NULL result in a non-nullable colum )

Question

I have set up snow pipe and it reads from S3 (JSON File) and for some reason it has paused suddenly, when I queried COPY_HISTORY table it has error saying "NULL result in a non-nullable column"

My JSon File :only the first Key-value pair is not nullable, rest can be null.

I have checked the entire JSON file, but couldnt find where it is null. Can some one tell me how to pin point the exact data where It is failng? Thanks

question from:https://stackoverflow.com/questions/65907140/snowpipe-paused-null-result-in-a-non-nullable-colum

Answer 1

I could see two options to troubleshoot this problem One Option is Create External Table for this S3 Json File and build View on top of it to view the Json data , check possible null values coming from Json key-value pair elements

If you think your JSON File is heavy and viewing data out of External Table using View is not giving the expected query performance to troubleshoot this issue then Second Option is to load this file into your internal stage and then build view on top of it to analyze your null key value pair elements.