MNIST handwriting data set

Table of Contents

MNIST handwriting data set

Introduction

Dataset description

Download and import data

data visualization

Data preprocessing

Build and train models

Model evaluation

Summarize

Practical application scenarios

Sample code

MNIST handwritten digits dataset

shortcoming

similar data sets

MNIST handwriting data set

Introduction

MNIST is a very classic handwritten digit data set, organized and annotated by the National Institute of Standards and Technology (NIST) in the 1980s. This dataset contains a series of images of handwritten digits from 0 to 9 and is used for image classification tasks in machine learning. The MNIST dataset is widely used to train and verify the performance of machine learning models.

Dataset description

The MNIST data set contains 60,000 training images and 10,000 test images. Each image is a 28*28 pixel grayscale image (single channel). The gray value of each pixel is between 0 and 255, indicating the brightness of the pixel. In addition to image data, the MNIST data set also provides corresponding label data. Labels are numbers between 0 and 9, representing handwritten numbers on the image.

Download and import data

In Python, there are several machine learning libraries available to download and import the MNIST dataset. The following is sample code using TensorFlow and Keras libraries:

pythonCopy codeimport tensorflow as tf
from tensorflow.keras.datasets import mnist
# Download and import the MNIST data set
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()

data visualization

To better understand the MNIST dataset, we can visualize the images in it. The following code will display the first 25 images of handwritten digits in the training set and display the corresponding labels.

pythonCopy codeimport matplotlib.pyplot as plt
# Visualize the first 25 images
plt.figure(figsize=(10, 10))
for i in range(25):
    plt.subplot(5, 5, i + 1)
    plt.xticks([])
    plt.yticks([])
    plt.grid(False)
    plt.imshow(train_images[i], cmap=plt.cm.binary)
    plt.xlabel(train_labels[i])
plt.show()

data preprocessing

Before training a machine learning model, we usually need to preprocess the data. For the MNIST dataset, common preprocessing steps include:

  • Data normalization: Normalize the gray value of image pixels from 0-255 to 0-1 to speed up the training speed and performance of the model.
  • Data expansion: Expand the 28*28 image into a 784-dimensional vector to adapt to the input requirements of most machine learning algorithms. The following is sample code to preprocess the MNIST dataset:
pythonCopy code# Data normalization
train_images = train_images / 255.0
test_images = test_images/255.0
# Data expansion
train_images = train_images.reshape((-1, 784))
test_images = test_images.reshape((-1, 784))

Build and train model

Using the preprocessed data, we can start building and training a machine learning model. Here we use the Sequential model of the Keras library to build a simple fully connected neural network classifier. The input layer of the model has 784 nodes, the output layer has 10 nodes, and the Softmax activation function is used for multi-classification. Here is sample code to build and train a model:

pythonCopy codefrom tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense
# Build model
model = Sequential()
model.add(Dense(128, activation='relu', input_shape=(784,)))
model.add(Dense(10, activation='softmax'))
# Compile model
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])
#Train model
history = model.fit(train_images, train_labels, epochs=10, validation_split=0.2)

Model Evaluation

After training is complete, we can evaluate the model using the test set. The following code calculates and prints out the model’s classification accuracy on the test set.

pythonCopy code# Evaluate the model on the test set
test_loss, test_acc = model.evaluate(test_images, test_labels)
print('Test accuracy:', test_acc)

Summary

The MNIST handwriting data set is a classic data set used for image classification tasks and is widely used in the field of machine learning. By downloading, preprocessing, model building, and evaluating the MNIST dataset, we can better understand the training and performance evaluation process of machine learning models. I hope that through the introduction of this article, readers can have a more comprehensive understanding of the MNIST data set.

Actual application scenario

The MNIST handwriting dataset has many uses in practical applications. Some common application scenarios include:

  • Digit recognition: Use the MNIST data set to train a machine learning model to recognize handwritten digits.
  • Automated filling: Use the MNIST data set in combination with optical character recognition (OCR) technology to realize automatic filling in forms and other functions.
  • Target detection and tracking: By training the model and using handwritten digits in the MNIST data set as targets, target detection and tracking in images or videos is achieved.

sample code

The following is an example code for training a simple handwritten digit recognition model using the MNIST dataset:

pythonCopy codeimport tensorflow as tf
from tensorflow.keras.datasets import mnist
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense
# Download and import the MNIST data set
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
#Data preprocessing
train_images = train_images / 255.0
test_images = test_images/255.0
# Build model
model = Sequential()
model.add(Dense(128, activation='relu', input_shape=(784,)))
model.add(Dense(10, activation='softmax'))
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])
#Train model
model.fit(train_images, train_labels, epochs=10)
# Evaluate the model on the test set
test_loss, test_acc = model.evaluate(test_images, test_labels)
print('Test accuracy:', test_acc)
# Make predictions for a single image
import numpy as np
image_to_predict = test_images[0]
image_to_predict = np.expand_dims(image_to_predict, axis=0)
prediction = model.predict(image_to_predict)
predicted_label = np.argmax(prediction[0])
print('Predicted label:', predicted_label)

This code first downloads and imports the MNIST dataset from Keras using the mnist.load_data() function. Then the data is preprocessed to normalize the pixel gray value to between 0-1. Next, use the Sequential model to build a simple fully connected neural network model and compile the model. The model is then trained using the training set data and the accuracy of the model is evaluated on the test set. Finally, use the model to predict an image and output the predicted label.

MNIST Handwritten Digits Data Set

The MNIST handwritten digits dataset is a commonly used machine learning dataset, transformed from the large-scale handwritten digits dataset from the National Institute of Standards and Technology (NIST). The data set contains 60,000 training samples and 10,000 test samples. Each sample is a grayscale image representing a handwritten number between 0 and 9. The image size is 28×28 pixels. The MNIST dataset is widely used for handwritten digit recognition tasks and is often used to evaluate the performance of machine learning algorithms or artificial neural network models.

Disadvantages

Although the MNIST dataset is widely used in the machine learning community, it also has some shortcomings:

  1. Simplicity: The MNIST dataset is relatively simple and presents fewer challenges. This allows some advanced machine learning algorithms to achieve almost perfect accuracy on MNIST, but it does not mean that these algorithms can also perform well on more complex tasks.
  2. Obsolescence: As deep learning evolves, more complex data sets and tasks become more common. The MNIST data set appears somewhat outdated in this regard and cannot cover current more complex problems such as image classification, object detection, and image generation.
  3. Uneven data distribution: The number of samples in each category in the MNIST data set is basically equal. This balanced distribution does not conform to the actual scenario. Real data sets often have uneven category distribution.
  4. Lack of diversity: The handwritten digits in the MNIST dataset are all written by Americans and therefore may not apply to handwriting styles in other countries or regions, limiting the diversity and generalization capabilities of the dataset.

Similar Dataset

With the development of machine learning and deep learning, many datasets similar to MNIST have emerged for a wider range of and complex tasks. Some similar datasets include:

  1. Fashion-MNIST dataset: Similar to the MNIST dataset, but used for image classification tasks of clothing and footwear.
  2. CIFAR-10 and CIFAR-100 datasets: Contains 10 and 100 color images of different categories for image classification and object detection tasks.
  3. ImageNet dataset: Contains over one million labeled high-resolution images for tasks such as image classification, object detection, and image generation.
  4. COCO dataset: A dataset for complex vision tasks such as target detection, image segmentation, and human pose estimation. These data sets are more complex, real, and diverse than MNIST, and are more suitable for evaluating and developing more powerful machine learning algorithms and models.

The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge. Python entry skill treeHomepageOverview 383467 people are learning the system