寫(xiě)在前面
- 態(tài)度決定高度！讓優(yōu)秀成為一種習(xí)慣！
- 世界上沒(méi)有什么事兒是加一次班解決不了的，如果有，就加兩次！（- - -茂強(qiáng)）
詞袋模型文本分類
- 數(shù)據(jù)準(zhǔn)備
如圖數(shù)據(jù)
- 數(shù)據(jù)清晰
數(shù)據(jù)讀取與清晰，這里只過(guò)濾出中文，并且兩個(gè)字以上的詞
target = [] #存儲(chǔ)句子的正負(fù)面標(biāo)簽，1代表正面，0代表負(fù)面
texts = [] #存儲(chǔ)句子
with open("c:/traindatav.txt", "r", encoding="utf-8") as f:
for line in f.readlines():
text = line.split(" => ")
if len(text) == 2:
lable = text[0].strip()
sentence = " ".join([w for w in text[1].split(" ") if re.match("[\u4e00-\u9fa5]+", w) and len(w) >= 2])
if lable == "正面":
target.append(1)
else:
target.append(0)
texts.append(sentence)
最后得到texts和target連個(gè)list
-
句子的長(zhǎng)度不能過(guò)長(zhǎng)，因此我們需要確定一個(gè)最大的句子長(zhǎng)度，這樣我們需要看一下句子長(zhǎng)度的分布是如何的
text_lengths = [len(x.split()) for x in texts] text_lengths = [x for x in text_lengths if x < 100] plt.hist(text_lengths, bins=25) plt.title('Histogram of # of Words in Texts') plt.show()
如圖：
從圖上可以看出，長(zhǎng)度取60時(shí)已經(jīng)涵蓋了大部分的句子
因此聲明
sentence_size = 60
min_word_freq = 3
- 清晰地?cái)?shù)據(jù)轉(zhuǎn)化成tensorflow能夠接受的數(shù)據(jù)，這一點(diǎn)，在tensorflow中在 learn.preprocessing包下有一個(gè)內(nèi)置函數(shù)VocabularyProcessor()
vocab_processor = learn.preprocessing.VocabularyProcessor(sentence_size, min_frequency=min_word_freq)
vocab_processor.fit_transform(texts)
vocab_processor.vocabulary_
我們來(lái)看一下，vocab_processor.vocabulary_中到底是什么
其中_ferq這個(gè)就是詞頻的統(tǒng)計(jì)dict
其中_mapping是一個(gè)對(duì)每個(gè)詞編輯一個(gè)索引
還有一個(gè)
就是上一個(gè)的reverse，只不過(guò)用了list表示
其他的就不解釋了
- 把數(shù)據(jù)集分成訓(xùn)練集于測(cè)試集
train_indices = np.random.choice(len(texts), round(len(texts)*0.8), replace=False)
test_indices = np.array(list(set(range(len(texts))) - set(train_indices)))
texts_train = [x for ix, x in enumerate(texts) if ix in train_indices]
texts_test = [x for ix, x in enumerate(texts) if ix in test_indices]
target_train = [x for ix, x in enumerate(target) if ix in train_indices]
target_test = [x for ix, x in enumerate(target) if ix in test_indices]
解釋一個(gè)數(shù)據(jù)內(nèi)容
聲明一個(gè)embedding矩陣
identity_mat = tf.diag(tf.ones(shape=[embedding_size]))然后聲明各logistic回歸的變量和placeholder
A = tf.Variable(tf.random_normal(shape=[embedding_size,1]))
b = tf.Variable(tf.random_normal(shape=[1,1]))
# Initialize placeholders
x_data = tf.placeholder(shape=[sentence_size], dtype=tf.int32)
y_target = tf.placeholder(shape=[1, 1], dtype=tf.float32)-
不得不說(shuō)的tf.nn.embedding_lookup函數(shù)
其實(shí)embedding_lookup的原理很簡(jiǎn)單，相當(dāng)于在np.array中直接采用下標(biāo)數(shù)組獲取數(shù)據(jù)。細(xì)節(jié)是返回的tensor的dtype和傳入的被查詢的tensor的dtype保持一致；和ids的dtype無(wú)關(guān)。
下面看個(gè)例子import tensorflow as tf import numpy as np sess = tf.InteractiveSession() mat1 = tf.reshape(tf.range(1, 10, name="m1"), shape=[3, 3]) [[1, 2, 3], [4, 5, 6], [7, 8, 9]] ids = [[1,2], [0,1]] res = tf.nn.embedding_lookup(mat1, ids) res.eval()
從上邊的例子看出， ids = [[1,2], [0,1]]決定要取矩陣的哪一行數(shù)據(jù)
-
不得不說(shuō)的tf.reduce_sum函數(shù)
還是老規(guī)矩，先看例子# 'x' is [[1, 1, 1] # [1, 1, 1]] tf.reduce_sum(x) ==> 6 tf.reduce_sum(x, 0) ==> [2, 2, 2] tf.reduce_sum(x, 1) ==> [3, 3] tf.reduce_sum(x, 1, keep_dims=True) ==> [[3], [3]] tf.reduce_sum(x, [0, 1]) ==> 6
例子看完我想你就明白了吧
-
下面就是如何把一個(gè)句子轉(zhuǎn)成向量了
x_embed = tf.nn.embedding_lookup(identity_mat, x_data)
x_col_sums = tf.reduce_sum(x_embed, 0)
首先是根據(jù)diag生成的one-hot矩陣，根據(jù)輸入的x_data（也就是每個(gè)句子中每個(gè)詞的索引向量），比如“我們 是 中國(guó)人”在vocab_processor.vocabulary_中的_mapping中的索引分別是[20, 3, 134]
那么embedding_lookup會(huì)從diag矩陣中找到對(duì)應(yīng)的行號(hào)（20,3,134）行的數(shù)據(jù)，也就是每個(gè)詞的詞向量，然后再reduce_sum注意參數(shù)0我們可以從以上例子中看到，其實(shí)就是把每個(gè)詞的向量按index相加，就生成該句子的向量，而對(duì)應(yīng)的20,3,134列的數(shù)字就是1其他都是0
所以x_col_sums就代表一個(gè)句子向量# 't' is a tensor of shape [2] shape(expand_dims(t, 0)) ==> [1, 2] shape(expand_dims(t, 1)) ==> [2, 1] shape(expand_dims(t, -1)) ==> [2, 1] # 't2' is a tensor of shape [2, 3, 5] shape(expand_dims(t2, 0)) ==> [1, 2, 3, 5] shape(expand_dims(t2, 2)) ==> [2, 3, 1, 5] shape(expand_dims(t2, 3)) ==> [2, 3, 5, 1]
如果看不明白，就用一個(gè)例子來(lái)看
labels = [1,2,3]
x = tf.expand_dims(labels, 0)
[[1 2 3]] #結(jié)果增加了一個(gè)維度
x = tf.expand_dims(labels, 1)
[[1]
[2]
[3]]
看了上邊的例子就能夠有個(gè)理解了
變換與計(jì)算
x_col_sums_2D = tf.expand_dims(x_col_sums, 0)
model_output = tf.add(tf.matmul(x_col_sums_2D, A), b)
首先把x_col_sums按照0方式變換增加一維，主要是為了矩陣運(yùn)算，然后計(jì)算y=AX+B然后定義損失函數(shù)
loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(model_output, y_target))
其實(shí)就是邏輯回歸表達(dá)式然后定義激活函數(shù)
prediction = tf.sigmoid(model_output)接下來(lái)就是優(yōu)化算法
my_opt = tf.train.GradientDescentOptimizer(0.001)
train_step = my_opt.minimize(loss)緊接著就是參數(shù)初始化
init = tf.initialize_all_variables()
sess.run(init)接著對(duì)所有的句子進(jìn)行迭代訓(xùn)練
loss_vec = []
train_acc_all = []
train_acc_avg = []
for ix, t in enumerate(vocab_processor.fit_transform(texts_train)):y_data = [[target_train[ix]]]
sess.run(train_step, feed_dict={x_data: t, y_target: y_data})
temp_loss = sess.run(loss, feed_dict={x_data: t, y_target: y_data})
loss_vec.append(temp_loss)
if (ix+1)%10==0:
print('Training Observation #' + str(ix+1) + ': Loss = ' +str(temp_loss))
# Keep trailing average of past 50 observations accuracy
# Get prediction of single observation
[[temp_pred]] = sess.run(prediction, feed_dict={x_data:t, y_target:y_data})
# Get True/False if prediction is accurate
train_acc_temp = target_train[ix]==np.round(temp_pred)
train_acc_all.append(train_acc_temp)
if len(train_acc_all) >= 50:
train_acc_avg.append(np.mean(train_acc_all[-50:]))
loss_vec存放的是每次訓(xùn)練的損失值，train_acc_all存放的是所有的acc值，train_acc_avg存放的是每50次的平均acc值
聲明一點(diǎn)，這里是對(duì)每一個(gè)句子進(jìn)行迭代的，而不是批計(jì)算的最后就是測(cè)試
print('Getting Test Set Accuracy')
test_acc_all = []
for ix, t in enumerate(vocab_processor.fit_transform(texts_test)):
y_data = [[target_test[ix]]]
if (ix+1)%50==0:
print('Test Observation #' + str(ix+1))
# Keep trailing average of past 50 observations accuracy
# Get prediction of single observation
[[temp_pred]] = sess.run(prediction, feed_dict={x_data:t,y_target:y_data})
# Get True/False if prediction is accurate
test_acc_temp = target_test[ix]==np.round(temp_pred)
test_acc_all.append(test_acc_temp)
print('\nOverall Test Accuracy: {}'.format(np.mean(test_acc_all)))
