Изменения

Scikit-learn<ref>[https://scikit-learn.org/stable/ Библиотека scikit-learn]</ref> {{---}} библиотека машинного обучения на языке программирования Python с открытым исходным кодом. Содержит реализации практически всех возможных преобразований, и нередко ее одной хватает для полной реализации модели. В данной библиотеки реализованы методы разбиения датасета на тестовый и обучающий, вычисление основных метрик над наборами данных, проведение [[кросс-валидации|Кросс-валидация]]^{[на 28.01.19 не создан]}. В библиотеке также есть основные алгоритмы машинного обучения: [[Линейная регрессия|линейной регрессии, (]]^{[на 28.01.19 не создан]} и ее её модификаций Лассо, гребневой регрессии, [[Метод опорных векторов (SVM), |опорных векторов]]^{[на 28.01.19 не создан]}, [[Дерево решений и случайный лес|решающих деревьев и лесов ]] и др. Есть и реализации основных методов [[Кластеризация|кластеризации]]. Кроме того, библиотека содержит постоянно используемые исследователями методы работы с параметрами (фичами)признаками: например, понижение размерности [[Метод главных компонент (PCA)|методом главных компонент]]^{[на 28.01.19 не создан]}. Частью пакета является библиотека imblearn<ref>[https://imbalanced-learn.readthedocs.io/en/stable/index.html Библиотека imbalanced-learn]</ref>, позволяющая работать с разбалансированными выборками и генерировать новые значения. 

Основная статья: [[{{Main|Логистическая регрессия]]}}

'''from ''' sklearn.datasets '''import ''' load_digits

'''import ''' numpy '''as ''' np '''import ''' matplotlib.pyplot '''as ''' plt

plt.figure(figsize=(<font color="blue">20</font>,<font color="blue">4</font>)) '''for ''' index, (image, label) '''in ''' enumerate(zip(digits.data[<font color="blue">0</font>:<font color="blue">3</font>], digits.target[<font color="blue">0</font>:<font color="blue">3</font>])): plt.subplot(<font color="blue">1</font>, <font color="blue">3</font>, index + <font color="blue">1</font>) plt.imshow(np.reshape(image, (<font color="blue">8</font>,<font color="blue">8</font>)), cmap=plt.cm.gray) plt.title(<font color="red">'Training: %i\n' </font> % label, fontsize = <font color="blue">20</font>) [[File:Digits-sklearn.png|800px|none|super]]

'''from ''' sklearn.model_selection '''import ''' train_test_split x_train, x_test, y_train, y_test = train_test_split(digits.data, digits.target, test_size=<font color="blue">0.25</font>, random_state=<font color="blue">0</font>)

'''from ''' sklearn.linear_model '''import ''' LogisticRegression

Основная статья: [[{{Main|Нейронные сети, перцептрон]]}}

'''from ''' sklearn '''import ''' datasets

'''from ''' sklearn.model_selection '''import ''' train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = <font color="blue">0.20</font>)

'''from ''' sklearn.preprocessing '''import ''' StandardScaler

'''from ''' sklearn.neural_network '''import ''' MLPClassifier mlp = MLPClassifier(hidden_layer_sizes=(<font color="blue">10</font>, <font color="blue">10</font>, <font color="blue">10</font>), max_iter=<font color="blue">1000</font>)

'''from ''' sklearn.metrics '''import ''' classification_report, confusion_matrix print(confusion_matrix(y_test,predictions)) print(classification_report(y_test,predictions))  > '''[[ 7 0 0]''' '''[ 0 8 1]''' '''[ 0 2 12]]''' '''precision recall f1-score support''' '''0 1.00 1.00 1.00 7''' '''1 0.80 0.89 0.84 9''' '''2 0.92 0.86 0.89 14''' '''micro avg 0.90 0.90 0.90 30''' '''macro avg 0.91 0.92 0.91 30''' '''weighted avg 0.90 0.90 0.90 30''' ====Метрический классификатор и метод ближайших соседей===={{Main|Метрический классификатор и метод ближайших соседей#Пример использования (через scikit-learn)|l1=Метрический классификатор и метод ближайших соседей: пример через scikit-learn}} ====Дерево решений и случайный лес===={{Main|Дерево решений и случайный лес#Примеры использования (в scikit-learn)|l1=Дерево решений и случайный лес: пример через scikit-learn}} ====Обработка естественного языка===={{Main|Обработка естественного языка}} Загрузка датасета: '''from''' sklearn '''import''' fetch_20newsgroups twenty_train = fetch_20newsgroups(subset=<font color="red">'train'</font>, shuffle='''True''', random_state=<font color="blue">42</font>) Вывод первых трех строк первого тренивочного файла и его класса: print(<font color="red">"\n"</font>.join(twenty_train.data[<font color="blue">0</font>].split(<font color="red">"\n"</font>)[:<font color="blue">3</font>])) print(twenty_train.target_names[twenty_train.target[<font color="blue">0</font>]])  > '''From: lerxst@wam.umd.edu (where's my thing)''' '''Subject: WHAT car is this!?''' '''Nntp-Posting-Host: rac3.wam.umd.edu''' '''rec.autos''' Построение и обучение двух моделей. Первая на основе [[Байесовская классификация | Байесовской классификации]]^{[на 28.01.19 не создан]}, а вторая использует метод опорных векторов: '''from''' sklearn.pipeline '''import''' Pipeline '''from''' sklearn.feature_extraction.text '''import''' CountVectorizer, TfidfTransformer '''from''' sklearn.naive_bayes '''import''' MultinomialNB text_clf1 = Pipeline([ (<font color="red">'vect'</font>, CountVectorizer()), (<font color="red">'tfidf'</font>, TfidfTransformer()), (<font color="red">'clf'</font>, MultinomialNB()), ]) '''from''' sklearn.linear_model '''import''' SGDClassifier text_clf2 = Pipeline([ (<font color="red">'vect'</font>, CountVectorizer()), (<font color="red">'tfidf'</font>, TfidfTransformer()), (<font color="red">'clf'</font>, SGDClassifier(loss=<font color="red">'hinge'</font>, penalty=<font color="red">'l2'</font>, alpha=<font color="blue">1e-3</font>, random_state=<font color="blue">42</font>, max_iter=<font color="blue">5</font>, tol='''None''')), ]) text_clf1.fit(twenty_train.data, twenty_train.target) text_clf2.fit(twenty_train.data, twenty_train.target) Оценка алгоритмов: twenty_test = fetch_20newsgroups(subset=<font color="red">'test'</font>, shuffle='''True''', random_state=<font color="blue">42</font>) docs_test = twenty_test.data predicted1 = text_clf1.predict(docs_test) predicted2 = text_clf2.predict(docs_test) print(<font color="red">"Score: %.3f"</font> % np.mean(predicted1 == twenty_test.target)) print(<font color="red">"Score: %.3f"</font> % np.mean(predicted2 == twenty_test.target))  > '''Score for naive Bayes: 0.774''' '''Score for SVM: 0.824''' ====Кросс-валилация и подбор параметров===={{Main|Кросс-валидация}} Возьмем предыдущий пример с обработкой естественного языка и попробуем увеличить точность алгоритма за счет кросс-валидации и подбора параметров: '''from''' sklearn.model_selection '''import''' GridSearchCV parameters = { <font color="red">'vect__ngram_range'</font>: [(<font color="blue">1</font>, <font color="blue">1</font>), (<font color="blue">1</font>, <font color="blue">2</font>)], <font color="red">'tfidf__use_idf'</font>: ('''True''', '''False'''), <font color="red">'clf__alpha'</font>: (<font color="blue">1e-2</font>, <font color="blue">1e-3</font>), } gs_clf = GridSearchCV(text_clf2, parameters, cv=<font color="blue">5</font>, iid='''False''', n_jobs=<font color="blue">-1</font>) gs_clf = gs_clf.fit(twenty_train.data, twenty_train.target) print(<font color="red">"Best score: %.3f"</font> % gs_clf.best_score_) '''for''' param_name '''in''' sorted(parameters.keys()): print(<font color="red">"%s: %r"</font> % (param_name, gs_clf.best_params_[param_name]))  > '''Best score: 0.904''' '''clf__alpha: 0.001''' '''tfidf__use_idf: True''' '''vect__ngram_range: (1, 2)''' ====Метод опорных векторов (SVM)===={{Main|Метод опорных векторов (SVM)|l1=Метод опорных векторов (SVM) ^{[на 28.01.19 не создан]}}} Загрузка датасета: '''from''' sklearn '''import''' datasets iris = datasets.load_iris() Разбиение датасета на тестовый и тренировочный: '''from''' sklearn.model_selection '''import''' train_test_split x_train, x_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=<font color="blue">0.25</font>, random_state=<font color="blue">0</font>) Построение и обучение модели: clf = svm.SVC(kernel=<font color="red">'linear'</font>, C=<font color="blue">1.0</font>) clf.fit(x_train, y_train) predictions = clf.predict(x_test) Оценка алгоритма: '''from''' sklearn.metrics '''import''' classification_report, confusion_matrix

Реализация логистической регрессии для классификации цифр из датасета MNIST<ref >'''[[13 0 0]''' '''[ 0 15 1]''' '''[https://en.wikipedia.org/wiki/MNIST_database Датасет MNIST0 0 9]]</ref>:''' '''precision recall f1-score from __future__ import print_functionsupport''' __docformat__ = '''0 1.00 1.00 1.00 13''' '''1 1.00 0.94 0.97 16''' '''2 0.90 1.00 0.95 9''restructedtext en' import six '''micro avg 0.97 0.97 0.97 38''' '''macro avg 0.97 0.98 0.97 38''' '''weighted avg 0.98 0.moves97 0.cPickle as pickle97 38'''  import gzip====EM-алгоритм==== import os{{Main|EM-алгоритм|l1=EM-алгоритм^{[на 28.01.19 не создан]}}} import sys '''import timeit''' numpy '''as''' np '''import numpy''' matplotlib.pyplot '''as''' plt '''from''' matplotlib.colors '''import theano''' LogNorm '''from''' sklearn '''import theano.tensor as T''' mixture

class LogisticRegression(object): def __init__(self, input, n_in, n_out): self.W n_samples = theano.shared( value<font color=numpy.zeros( (n_in, n_out), dtype=theano.config.floatX ), name='W', borrow=True ) self.b = theano.shared( value=numpy.zeros( (n_out,), dtype=theano.config.floatX ), name='b', borrow=True ) self.p_y_given_x = T.nnet.softmax(T.dot(input, self.W) + self.b) self.y_pred = T.argmax(self.p_y_given_x, axis=1) self.params = [self.W, self.b] self.input = input def negative_log_likelihood(self, y): return -T.mean(T.log(self.p_y_given_x)[T.arange(y.shape[0]), y])"blue">300</font>

def load_data(dataset): data_dir, data_file = os.path.split(dataset) if data_dir =<font color= "green" and not os># generate random sample, two components</font> np.pathrandom.isfileseed(dataset): new_path <font color= os.path.join( os.path.split(__file__)[0], "..blue", "data", dataset ) if os.path.isfile(new_path) or data_file == 'mnist.pkl.gz': dataset = new_path if (not os.path.isfile(dataset)) and data_file == 'mnist.pkl.gz': from six.moves import urllib origin = ( 'http://www.iro.umontreal.ca/~lisa/deep/data>0</mnist/mnist.pkl.gz' ) print('Downloading data from %s' % origin) urllib.request.urlretrieve(origin, dataset) with gzip.open(dataset, 'rb') as f: try: train_set, valid_set, test_set = pickle.load(f, encoding='latin1') except: train_set, valid_set, test_set = pickle.load(f) def shared_dataset(data_xy, borrow=Truefont>): data_x, data_y = data_xy shared_x = theano.shared(numpy.asarray(data_x, dtype=theano.config.floatX), borrow=borrow) shared_y = theano.shared(numpy.asarray(data_y, dtype=theano.config.floatX), borrow=borrow) return shared_x, T.cast(shared_y, 'int32') test_set_x, test_set_y = shared_dataset(test_set) valid_set_x, valid_set_y = shared_dataset(valid_set) train_set_x, train_set_y = shared_dataset(train_set) rval = [(train_set_x, train_set_y), (valid_set_x, valid_set_y), (test_set_x, test_set_y)] return rval

def sgd_optimization_mnist(learning_rate=0.13, n_epochs=1000, dataset<font color='mnist.pkl.gz', batch_size=600): datasets = load_data"green"># generate spherical data centered on (dataset) train_set_x20, train_set_y = datasets[0] valid_set_x, valid_set_y = datasets[1] test_set_x, test_set_y = datasets[2] n_train_batches = train_set_x.get_value(borrow=True20).shape[0] n_valid_batches = valid_set_x.get_value(borrow=True).shape[0] n_test_batches = test_set_x.get_value(borrow=True).shape[0]</font> index shifted_gaussian = Tnp.lscalar() x = Trandom.matrixrandn('x') y = T.ivector('y') classifier = LogisticRegression(input=xn_samples, n_in<font color=28 * 28, n_out=10) cost = classifier.negative_log_likelihood(y) test_model = theano.function( inputs=[index], outputs=classifier.errors(y), givens={ x: test_set_x[index * batch_size: (index + 1"blue">2</font>) * batch_size], y: test_set_y[index * batch_size: (index + 1) * batch_size] } ) validate_model = theanonp.functionarray( inputs=[index], outputs=classifier.errors(y), givens={ x: valid_set_x[index * batch_size: (index + 1) * batch_size], y: valid_set_y[index * batch_size: (index + 1) * batch_size] } ) g_W = T.grad(cost=cost, wrt=classifier.W) g_b = T.grad(cost=cost, wrt=classifier.b) updates = [(classifier.W, classifier.W - learning_rate * g_W), (classifier.b, classifier.b - learning_rate * g_b)] train_model = theano.function( inputs=[index], outputs=cost, updates=updates, givens={ x: train_set_x[index * batch_size: (index + 1) * batch_size], y: train_set_y[index * batch_size: (index + 1) * batch_size] } ) patience = 5000 patience_increase = 2 improvement_threshold <font color= 0.995 validation_frequency = min(n_train_batches, patience // 2) best_validation_loss = numpy.inf test_score = 0. start_time = timeit.default_timer() done_looping = False epoch = 0 while (epoch "blue">20< n_epochs) and (not done_looping): epoch = epoch + 1 for minibatch_index in range(n_train_batches): minibatch_avg_cost = train_model(minibatch_index) iter = (epoch - 1) * n_train_batches + minibatch_index if (iter + 1) % validation_frequency == 0: validation_losses = [validate_model(i) for i in range(n_valid_batches)] this_validation_loss = numpy.mean(validation_losses) print( 'epoch %i, minibatch %i/%ifont>, validation error %f %%' % ( epoch, minibatch_index + 1, n_train_batches, this_validation_loss * 100. ) ) if this_validation_loss < best_validation_loss: if this_validation_loss < best_validation_loss * \ improvement_threshold: patience = max(patience, iter * patience_increase) best_validation_loss = this_validation_loss test_losses = [test_model(i) for i in range(n_test_batches)] test_score font color= numpy.mean(test_losses) print( ( ' epoch %i, minibatch %i/%i, test error of' ' best model %f %%' ) % ( epoch, minibatch_index + 1, n_train_batches, test_score * 100. ) ) with open('best_model.pkl', 'wb') as f: pickle.dump(classifier, f) if patience "blue">20<= iter: done_looping = True break end_time = timeit.default_timer() print( ( 'Optimization complete with best validation score of %f %%,' 'with test performance %f %%' ) % (best_validation_loss * 100., test_score * 100.) ) print('The code run for %d epochs, with %f epochs/sec' % ( epoch, 1. * epoch / (end_time - start_time))) print(('The code for file ' + os.path.split(__file__)[1font>] + ' ran for %.1fs' % ((end_time - start_time))), file=sys.stderr)

def predict():<font color="green"># generate zero centered stretched Gaussian data</font> classifier C = picklenp.loadarray(open('best_model[[<font color="blue">0.pkl')) predict_model </font>, <font color= theano"blue">-0.function( inputs7</font>], [<font color=[classifier"blue">3.input]5</font>, outputs<font color=classifier"blue">.y_pred7</font>]]) dataset stretched_gaussian ='mnistnp.dot(np.pklrandom.gz' datasets = load_datarandn(dataset) test_set_xn_samples, test_set_y = datasets[2] test_set_x = test_set_x.get_value() predicted_values <font color= predict_model(test_set_x[:10]) print("Predicted values for the first 10 examples in test set:blue">2</font>) print(predicted_values, C)

if __name__ <font color="green"># concatenate the two datasets into the final training set</font> X_train = np.vstack([shifted_gaussian, stretched_gaussian]) <font color="green"># fit a Gaussian Mixture Model with two components</font> clf = mixture.GaussianMixture(n_components=<font color="blue">2</font>, covariance_type=<font color="red">'full'</font>) clf.fit(X_train) <font color="green"># display predicted scores by the model as a contour plot</font> x = np.linspace(<font color="blue">-20.</font>, <font color="blue">30.</font>) y = np.linspace(<font color="blue">-20.</font>, <font color="blue">40.</font>) X, Y = np.meshgrid(x, y) XX = np.array([X.ravel(), Y.ravel()]).T Z = -clf.score_samples(XX) Z = Z.reshape(X.shape) CS = plt.contour(X, Y, Z, norm=LogNorm(vmin=<font color="blue">1.0</font>, vmax=<font color="blue">1000.0</font>), levels=np.logspace(<font color="blue">0</font>, <font color="blue">3</font>, <font color="blue">10</font>)) CB = plt.colorbar(CS, shrink=<font color="blue">0.8</font>, extend=<font color= "red">'__main__both'</font>) plt.scatter(X_train[:, <font color="blue">0</font>], X_train[:, <font color="blue">1</font>], <font color="blue">.8</font>) sgd_optimization_mnist plt.title(<font color="red">'Negative log-likelihood predicted by a GMM'</font>) plt.axis(<font color="red">'tight'</font>) plt.show() [[File:Em.png|400px|none|super]] ====Уменьшение размерности===={{Main|Уменьшение размерности#Пример кода scikit-learn|l1=Уменьшение размерности: пример через scikit-learn}} 

Основная статья: [[{{Main|Сверточные нейронные сети]].}}

Реализация сверточной нейронной сети для классификации цифр из датасета MNIST<ref>[https://en.wikipedia.org/wiki/MNIST_database Датасет MNIST]</ref>: '''from ''' __future__ '''import ''' division, print_function, absolute_import '''import ''' tensorflow '''as ''' tf <font color="green"># Import MNIST data</font> '''from ''' tensorflow.examples.tutorials.mnist '''import ''' input_data mnist = input_data.read_data_sets(<font color="red">"/tmp/data/"</font>, one_hot='''True''')

<font color="green"># Training Parameters</font> learning_rate = <font color="blue">0.001</font> num_steps = <font color="blue">200</font> batch_size = <font color="blue">128</font> display_step = <font color="blue">10</font>

<font color="green"># Network Parameters</font> num_input = <font color="blue">784 </font> <font color="green"># MNIST data input (img shape: 28*28)</font> num_classes = <font color="blue">10 </font> <font color="green"># MNIST total classes (0-9 digits)</font> dropout = <font color="blue">0.75 </font> <font color="green"># Dropout, probability to keep units</font>

<font color="green"># tf Graph input</font> X = tf.placeholder(tf.float32, ['''None''', num_input]) Y = tf.placeholder(tf.float32, ['''None''', num_classes]) keep_prob = tf.placeholder(tf.float32) <font color="green"># dropout (keep probability)</font>

<font color="green"># Create some wrappers for simplicity</font> '''def ''' conv2d(x, W, b, strides=<font color="blue">1</font>): <font color="green"># Conv2D wrapper, with bias and relu activation</font> x = tf.nn.conv2d(x, W, strides=[<font color="blue">1</font>, strides, strides, <font color="blue">1</font>], padding=<font color="red">'SAME'</font>)

'''return ''' tf.nn.relu(x)

'''def ''' maxpool2d(x, k=<font color="blue">2</font>): <font color="green"># MaxPool2D wrapper</font> '''return ''' tf.nn.max_pool(x, ksize=[<font color="blue">1</font>, k, k, <font color="blue">1</font>], strides=[<font color="blue">1</font>, k, k, <font color="blue">1</font>], padding=<font color="red">'SAME'</font>)

<font color="green"># Create model</font> '''def ''' conv_net(x, weights, biases, dropout): <font color="green"># MNIST data input is a 1-D vector of 784 features (28*28 pixels)

# Tensor input become 4-D: [Batch Size, Height, Width, Channel]</font> x = tf.reshape(x, shape=[<font color="blue">-1</font>, <font color="blue">28</font>, <font color="blue">28</font>, <font color="blue">1</font>]) <font color="green"># Convolution Layer</font> conv1 = conv2d(x, weights[<font color="red">'wc1'</font>], biases[<font color="red">'bc1'</font>]) <font color="green"># Max Pooling (down-sampling)</font> conv1 = maxpool2d(conv1, k=<font color="blue">2</font>) <font color="green"># Convolution Layer</font> conv2 = conv2d(conv1, weights[<font color="red">'wc2'</font>], biases[<font color="red">'bc2'</font>]) <font color="green"># Max Pooling (down-sampling)</font> conv2 = maxpool2d(conv2, k=<font color="blue">2</font>) <font color="green"># Fully connected layer # Reshape conv2 output to fit fully connected layer input</font> fc1 = tf.reshape(conv2, [<font color="blue">-1</font>, weights[<font color="red">'wd1'</font>].get_shape().as_list()[<font color="blue">0</font>]]) fc1 = tf.add(tf.matmul(fc1, weights[<font color="red">'wd1'</font>]), biases[<font color="red">'bd1'</font>])

<font color="green"># Apply Dropout</font>

<font color="green"># Output, class prediction</font> out = tf.add(tf.matmul(fc1, weights[<font color="red">'out'</font>]), biases[<font color="red">'out'</font>]) '''return ''' out

<font color="green"># Store layers weight & bias</font>

<font color="green"># 5x5 conv, 1 input, 32 outputs</font> <font color="red">'wc1'</font>: tf.Variable(tf.random_normal([<font color="blue">5</font>, <font color="blue">5</font>, <font color="blue">1</font>, <font color="blue">32</font>])), <font color="green"># 5x5 conv, 32 inputs, 64 outputs</font> <font color="red">'wc2'</font>: tf.Variable(tf.random_normal([<font color="blue">5</font>, <font color="blue">5</font>, <font color="blue">32</font>, <font color="blue">64</font>])), <font color="green"># fully connected, 7*7*64 inputs, 1024 outputs</font> <font color="red">'wd1'</font>: tf.Variable(tf.random_normal([<font color="blue">7</font>*<font color="blue">7</font>*<font color="blue">64</font>, <font color="blue">1024</font>])), <font color="green"># 1024 inputs, 10 outputs (class prediction)</font> <font color="red">'out'</font>: tf.Variable(tf.random_normal([<font color="blue">1024</font>, num_classes]))

<font color="red">'bc1'</font>: tf.Variable(tf.random_normal([<font color="blue">32</font>])), <font color="red">'bc2'</font>: tf.Variable(tf.random_normal([<font color="blue">64</font>])), <font color="red">'bd1'</font>: tf.Variable(tf.random_normal([<font color="blue">1024</font>])), <font color="red">'out'</font>: tf.Variable(tf.random_normal([num_classes]))

<font color="green"># Construct model</font>

<font color="green"># Define loss and optimizer</font>

<font color="green"># Evaluate model</font> correct_pred = tf.equal(tf.argmax(prediction, <font color="blue">1</font>), tf.argmax(Y, <font color="blue">1</font>))

<font color="green"># Initialize the variables (i.e. assign their default value)</font>

<font color="green"># Start training</font> '''with ''' tf.Session() '''as ''' sess: <font color="green"># Run the initializer</font>

'''for ''' step '''in ''' '''range'''(<font color="blue">1</font>, num_steps+<font color="blue">1</font>):

<font color="green"># Run optimization op (backprop)</font> sess.run(train_op, feed_dict={X: batch_x, Y: batch_y, keep_prob: <font color="blue">0.8</font>}) '''if ''' step % display_step == <font color="blue">0 </font> '''or ''' step == <font color="blue">1</font>: <font color="green"># Calculate batch loss and accuracy</font>

keep_prob: <font color="blue">1.0</font>}) print(<font color="red">"Step " </font> + str(step) + <font color="red">", Minibatch Loss= " </font> + \ <font color="red">"{:.4f}"</font>.format(loss) + <font color="red">", Training Accuracy= " </font> + \ <font color="red">"{:.3f}"</font>.format(acc)) print(<font color="red">"Optimization Finished!"</font>) <font color="green"># Calculate accuracy for 256 MNIST test images</font> print(<font color="red">"Testing Accuracy:"</font>, \ sess.run(accuracy, feed_dict={X: mnist.test.images[:<font color="blue">256</font>], Y: mnist.test.labels[:<font color="blue">256</font>], keep_prob: <font color="blue">1.0</font>}))  > '''Step 1, Minibatch Loss= 41724.0586, Training Accuracy= 0.156''' '''Step 10, Minibatch Loss= 17748.7500, Training Accuracy= 0.242''' '''Step 20, Minibatch Loss= 8307.6162, Training Accuracy= 0.578''' '''Step 30, Minibatch Loss= 3108.5703, Training Accuracy= 0.766''' '''Step 40, Minibatch Loss= 3273.2749, Training Accuracy= 0.727''' '''Step 50, Minibatch Loss= 2754.2861, Training Accuracy= 0.820''' '''Step 60, Minibatch Loss= 2467.7925, Training Accuracy= 0.844''' '''Step 70, Minibatch Loss= 1423.8140, Training Accuracy= 0.914''' '''Step 80, Minibatch Loss= 1651.4656, Training Accuracy= 0.875''' '''Step 90, Minibatch Loss= 2105.9263, Training Accuracy= 0.867''' '''Step 100, Minibatch Loss= 1153.5090, Training Accuracy= 0.867''' '''Step 110, Minibatch Loss= 1751.1400, Training Accuracy= 0.898''' '''Step 120, Minibatch Loss= 1446.2119, Training Accuracy= 0.922''' '''Step 130, Minibatch Loss= 1403.7135, Training Accuracy= 0.859''' '''Step 140, Minibatch Loss= 1089.7897, Training Accuracy= 0.930''' '''Step 150, Minibatch Loss= 1147.0751, Training Accuracy= 0.898''' '''Step 160, Minibatch Loss= 1963.3733, Training Accuracy= 0.883''' '''Step 170, Minibatch Loss= 1544.2725, Training Accuracy= 0.859''' '''Step 180, Minibatch Loss= 977.9219, Training Accuracy= 0.914''' '''Step 190, Minibatch Loss= 857.7977, Training Accuracy= 0.930''' '''Step 200, Minibatch Loss= 430.4735, Training Accuracy= 0.953''' '''Optimization Finished!''' '''Testing Accuracy: 0.94140625''' 

Основная статья: [[{{Main|Сверточные нейронные сети]].}}

'''from ''' __future__ '''import ''' print_function '''from ''' keras.preprocessing '''import ''' sequence '''from ''' keras.models '''import ''' Sequential '''from ''' keras.layers '''import ''' Dense, Dropout, Activation '''from ''' keras.layers '''import ''' Embedding '''from ''' keras.layers '''import ''' Conv1D, GlobalMaxPooling1D '''from ''' keras.datasets '''import ''' imdb

<font color="green"># set parameters:</font> max_features = <font color="blue">5000</font> maxlen = <font color="blue">400</font> batch_size = <font color="blue">32</font> embedding_dims = <font color="blue">50</font> filters = <font color="blue">250</font> kernel_size = <font color="blue">3</font> hidden_dims = <font color="blue">250</font> epochs = <font color="blue">2</font>

print(len(x_train), <font color="red">'train sequences'</font>) print(len(x_test), <font color="red">'test sequences'</font>)  > '''25000 train sequences''' '''25000 test sequences'''  print(<font color="red">'Pad sequences (samples x time)'</font>)

print(<font color="red">'x_train shape:'</font>, x_train.shape) print(<font color="red">'x_test shape:'</font>, x_test.shape)  > '''Pad sequences (samples x time)''' '''x_train shape: (25000, 400)''' '''x_test shape: (25000, 400)''' 

model.add(Dropout(<font color="blue">0.2</font>))

padding=<font color="red">'valid'</font>, activation=<font color="red">'relu'</font>, strides=<font color="blue">1</font>))

model.add(Dropout(<font color="blue">0.2</font>)) model.add(Activation(<font color="red">'relu'</font>)) model.add(Dense(<font color="blue">1</font>)) model.add(Activation(<font color="red">'sigmoid'</font>))

model.compile(loss=<font color="red">'binary_crossentropy'</font>, optimizer=<font color="red">'adam'</font>, metrics=[<font color="red">'accuracy'</font>])