python - Tensorflow Sparse Matrix element wise calculation takes more time than Dense Matrix

Question

I am trying to optimize my model by moving from dense matrix calculations to sparse matrices on Tensorflow. But unexpectedly sparse matrix calculations take significantly more time compared to dense matrix - at least 5 times.

Here is a subsection of my code with some dummy data with 75% zeros:

Creating data

import numpy as np
import tensorflow as tf
import time
n_samples, S, N = 10,100, 100000
trunc = 0.5

dist_sqrd = tf.constant(np.float32(np.random.rand(N,n_samples*S)))
comparison = tf.less_equal( dist_sqrd, tf.square(trunc))
dist_sqrd_trunc = dist_sqrd*tf.cast(comparison, tf.float32)
var_s = tf.constant(np.random.rand( 1, n_samples* S),dtype=np.float32)

pcnt_zero = 1 - tf.reduce_sum(tf.cast(comparison, tf.float32))/ (N*n_samples*S)
print('pcnt_zero :' + str(pcnt_zero))
-------------------------    
pcnt_zero :tf.Tensor(0.75000405, shape=(), dtype=float32)

Dense matrix calculation (run time : 0.15)

start_main = time.time()

trunc_numerat = var_s * dist_sqrd_trunc 

end = time.time()
print('----Time---- : ' + str(end - start_main))

Sparse matrix Calculation (run time : 1.22)

dist_sqrd_sp = tf.sparse.from_dense(dist_sqrd_trunc) 

start_main = time.time()

trunc_numerat = tf.raw_ops.SparseDenseCwiseMul(sp_indices = dist_sqrd_sp.indices, sp_values = dist_sqrd_sp.values, sp_shape =dist_sqrd_sp.shape , dense =var_s , name=None)

end = time.time()
print('----Time---- : ' + str(end - start_main))

Is there a way to optimise this or this is the nature of this?

question from:https://stackoverflow.com/questions/65842910/tensorflow-sparse-matrix-element-wise-calculation-takes-more-time-than-dense-mat

Answer 1

Waitting for answers