如何在Python中实现Softmax函数

从Udacity的深度学习课程中，y_i的softmax仅仅是指数除以整个Y向量的指数之和:

其中S(y_i)是y_i的软最大函数e是指数函数j是no。输入向量Y中的列。

我试过以下几种方法:

import numpy as np

def softmax(x):
    """Compute softmax values for each sets of scores in x."""
    e_x = np.exp(x - np.max(x))
    return e_x / e_x.sum()

scores = [3.0, 1.0, 0.2]
print(softmax(scores))

[ 0.8360188   0.11314284  0.05083836]

但建议的解决方案是:

def softmax(x):
    """Compute softmax values for each sets of scores in x."""
    return np.exp(x) / np.sum(np.exp(x), axis=0)

它产生与第一个实现相同的输出，尽管第一个实现显式地取每列与Max的差值，然后除以和。

有人能用数学方法解释一下吗?一个是对的，另一个是错的?

实现在代码和时间复杂度方面是否相似?哪个更有效率?

当前回答

import tensorflow as tf
import numpy as np

def softmax(x):
    return (np.exp(x).T / np.exp(x).sum(axis=-1)).T

logits = np.array([[1, 2, 3], [3, 10, 1], [1, 2, 5], [4, 6.5, 1.2], [3, 6, 1]])

sess = tf.Session()
print(softmax(logits))
print(sess.run(tf.nn.softmax(logits)))
sess.close()

2019-07-13 14:38:27

其他回答

更简明的说法是:

def softmax(x):
    return np.exp(x) / np.exp(x).sum(axis=0)

2016-09-06 20:08:40

从数学的角度看，两边是相等的。

这很容易证明。m = max (x)。现在你的函数softmax返回一个向量，它的第i个坐标等于

注意，这适用于任何m，因为对于所有(甚至是复数)数e^m != 0

from computational complexity point of view they are also equivalent and both run in O(n) time, where n is the size of a vector. from numerical stability point of view, the first solution is preferred, because e^x grows very fast and even for pretty small values of x it will overflow. Subtracting the maximum value allows to get rid of this overflow. To practically experience the stuff I was talking about try to feed x = np.array([1000, 5]) into both of your functions. One will return correct probability, the second will overflow with nan your solution works only for vectors (Udacity quiz wants you to calculate it for matrices as well). In order to fix it you need to use sum(axis=0)

2016-11-23 05:51:30

我的建议是:

def softmax(z):
    z_norm=np.exp(z-np.max(z,axis=0,keepdims=True))
    return(np.divide(z_norm,np.sum(z_norm,axis=0,keepdims=True)))

它既适用于随机，也适用于批量。欲了解更多详情，请参阅: https://medium.com/@ravish1729/analysis-of-softmax-function-ad058d6a564d

2018-08-18 09:44:16

这也适用于np. remodeling。

   def softmax( scores):
        """
        Compute softmax scores given the raw output from the model

        :param scores: raw scores from the model (N, num_classes)
        :return:
            prob: softmax probabilities (N, num_classes)
        """
        prob = None

        exponential = np.exp(
            scores - np.max(scores, axis=1).reshape(-1, 1)
        )  # subract the largest number https://jamesmccaffrey.wordpress.com/2016/03/04/the-max-trick-when-computing-softmax/
        prob = exponential / exponential.sum(axis=1).reshape(-1, 1)

        

        return prob

2020-08-26 04:35:59

根据所有的回复和CS231n的注释，请允许我总结如下:

def softmax(x, axis):
    x -= np.max(x, axis=axis, keepdims=True)
    return np.exp(x) / np.exp(x).sum(axis=axis, keepdims=True)

用法:

x = np.array([[1, 0, 2,-1],
              [2, 4, 6, 8], 
              [3, 2, 1, 0]])
softmax(x, axis=1).round(2)

输出:

array([[0.24, 0.09, 0.64, 0.03],
       [0.  , 0.02, 0.12, 0.86],
       [0.64, 0.24, 0.09, 0.03]])

2019-10-19 12:48:52

如何在Python中实现Softmax函数

推荐文章

标签

最新文章