python - Issue for insert using psycopg

Question

I am reading data from a .mat file using the Pytables module. After reading the data, I want to insert this data into the database using psycopg. Here is a sample code piece:

file = tables.openFile(matFile)
x = 0
#populate the matData list
for var in dest:
   data = file.getNode('/' + var)[:]
   matData.append(data) 
   x = x+1 
#insert into db
for i in range(0,x):
   cur.execute("""INSERT INTO "%s" (%s) VALUES (%s)""" % tableName,dest[i],matData[i]) )

I am getting the following error:

Traceback (most recent call last):
  File "./loadDBFromMAT.py", line 111, in <module>
    readInputFileAndLoad(args.matFileName,args.tableName)
  File "./loadDBFromMAT.py", line 77, in readInputFileAndLoad
    cur.execute("INSERT INTO "%s" (%s) VALUES (%s)" % (tableName,dest[i],matData[i]) )
psycopg2.ProgrammingError: syntax error at or near "["
LINE 1: INSERT INTO "DUMMY1km" (data) VALUES ([[-3000 -3000 -3000 .....

It would be great if anyone can suggest a workaround for this. Thanks!

Answer 1

The INSERT statement has invalid syntax. There something wrong inside the for loop you mention.
_{You should include the for loop in the question.}

INSERT INTO "DUMMY1km" (data) VALUES ([[-3000 -3000 -3000 .....

A valid statement could look like this - assuming your column is of type integer[].
_{... which you should also include in the question.}

INSERT INTO "DUMMY1km"(data) VALUES ('{-3000, -3000}'::int[])

or

INSERT INTO "DUMMY1km"(data) VALUES (ARRAY[-3000, -3000])  -- note the "ARRAY"

or for a 2-dimensional array (looks a bit like that in the error msg.):

INSERT INTO "DUMMY1km"(data) VALUES ('{{-3000, -3000}, {-3000, -3000}}'::int[])

or

INSERT INTO "DUMMY1km"(data) VALUES (ARRAY[[-3000, -3000],[-3000, -3000]])

More on array value input in the manual.

Ergo:

matData[i] needs to contain ARRAY[-3000, -3000] or one of the other listed variants of valid syntax instead of [[-3000 -3000 -3000 ... which isn't valid for an integer array.

Psychopg automatically converts a PostgreSQL array into a Python list. When building the INSERT, you need to convert the list back to an array. I quote from here:

Python lists are converted into PostgreSQL ARRAYs:

>>> cur.mogrify("SELECT %s;", ([10, 20, 30], ))
'SELECT ARRAY[10, 20, 30];'

_{Disclaimer: I am an expert with PostgreSQL, not so much with Python. For somebody who knows Python better than me, it should be easy to format the string accordingly. I found the above quote in a quick research on the web.}