python - How can I avoid value errors when using numpy.random.multinomial?

Question

When I use this random generator: numpy.random.multinomial, I keep getting:

ValueError: sum(pvals[:-1]) > 1.0

I am always passing the output of this softmax function:

def softmax(w, t = 1.0):
    e = numpy.exp(numpy.array(w) / t)
    dist = e / np.sum(e)
    return dist

except now that I am getting this error, I also added this for the parameter (pvals):

while numpy.sum(pvals) > 1:
    pvals /= (1+1e-5)

but that didn't solve it. What is the right way to make sure I avoid this error?

EDIT: here is function that includes this code

def get_MDN_prediction(vec):
    coeffs = vec[::3]
    means = vec[1::3]
    stds = np.log(1+np.exp(vec[2::3]))
    stds = np.maximum(stds, min_std)
    coe = softmax(coeffs)
    while np.sum(coe) > 1-1e-9:
        coe /= (1+1e-5)
    coeff = unhot(np.random.multinomial(1, coe))
    return np.random.normal(means[coeff], stds[coeff])

Answer 1

I also encountered this problem during my language modelling work.

The root of this problem rises from numpy's implicit data casting: the output of my sorfmax() is in float32 type, however, numpy.random.multinomial() will cast the pval into float64 type IMPLICITLY. This data type casting would cause pval.sum() exceed 1.0 sometimes due to numerical rounding.

This issue is recognized and posted here