Anomaly Detection in TensorFlow and Keras Utilizing the Autoencoder Methodology | by Rashida Nasrin Sucky | Sep, 2023

A cutting-edge unsupervised technique for noise removing, dimensionality discount, anomaly detection, and extra

All of the tutorials about TensorFlow and neural networks I’ve shared till now have been about supervised studying. This one will likely be in regards to the Autoenocder which is an unsupervised studying approach. If I wish to specific it merely, autoencoders cut back the noises from the information by compressing the enter information, and encoding and reconstructing the information. That means autoencoders can cut back the dimensionality or the noise of the information and deal with the actual point of interest of the enter information.

As you’ll be able to see from the introduction to the autoencoders right here there may be multiple course of required.

1. First, a mannequin to compress the enter information which is the encoder mannequin.
2. Then one other mannequin to reconstruct the compressed information that ought to be as shut because the enter information which is a decoder mannequin.

On this course of, it might take away the noise, cut back the dimensionality, and clear up the enter information.

On this tutorial, I’ll clarify intimately how an autoencoder works with a working instance.

For this instance, I selected to make use of a public dataset (Apache License 2.0) named deep_weeds.

import tensorflow as tf
import tensorflow_datasets as tfds
ds = tfds.load('deep_weeds', break up='practice', shuffle_files=True)

Information Preparation

We have to put together a dataset for this unsupervised anomaly detection instance. Just one class will likely be taken as our important class that will likely be thought-about because the legitimate class. And I’ll put a number of information from one other class as an anomaly. Then we are going to develop the mannequin to see if we are able to discover that few anomaly information.

I selected class 5 because the legitimate class and sophistication 1 because the anomaly. Within the code block beneath, I’m taking all the information of lessons 5 and 1 first and creating lists of the photographs and their corresponding labels.

import numpy as np
images_main = []
images_anomaly = []
labels_main= []
labels_anomaly = []
ds = ds.prefetch(tf.information.AUTOTUNE)
for instance in ds…