Smruti Sahoo

Image Classification Using Apache Spark with Linear SVM

Image Classification

Suppose you have got a problem to distinguish between Male and Female, in a set of images (by set, I mean a set of millions of images). How will you do that? Or how does Facebook recognizes your face when you and your close buddies have taken a selfie?

So one thing that comes into the mind is- ‘there must be some logic, algorithm and maths behind this‘. There must be some program which they are using in that algorithm, some mathematics which is running on those images to classify them.

So, this blog is all about discussing an algorithm and a program implementing such an algorithm which will help us in classifying images(male or female) based on some previously known/defined images.

Overview: SVM (Support Vector Machine)

SVM is a supervised learning algorithm which is used for classification and regression analysis of data-set through pattern matching. General Pattern analysis algorithms study general types of relations in data-sets such as correlations and classifications. The data in the Objects/images need to be converted into n-dimensional vector with each coordinate having some numerical value. In short, object in its numerical representable form.

SVM works smartly without computing all the coordinates of the objects. Instead, it uses a method known as kernel trick, which calculates the dot product between , (e.g. a measure of ‘similarity’ between ) where and are the transformed vectors of two different images. Through this transformed data it finds a decision boundary between the possible outputs.

As earlier mentioned, SVM can be used for classification and linear regression. Here we will discuss a classification example. Particularly, the focus will be on Linear SVM.

Linear SVM

Linear SVM search for a hyperplane that symmetrically separates the data points in the training set between different classes. Here this is known as decision boundary, that separates the space into two halves: one half for class '0', and the other half for class '1'. This explanation applies only to dataset having two data classes.

.2000px-DBSCAN-Gaussian-data.svg

A two-class, linearly separable dataset.

 
Training this dataset yields the below decision boundary. As the set of data points are easily linearly separable, the SVM is able to draw a margin that suitably separates the training data.

Svm_max_sep_hyperplane_with_margin

Data points arrangement post linear SVM

 

The problem statement

We have a set of Grey-scaled images having fixed sizes in pixel. The image set consists of different facial expression of around 1000 different males and females. Now our challenge is to implement liner SVM in a program to decide if a given image is of a male or female, with the help of existing images. These images will be used as training and test data.

Technology Stack:

  • Java 1.8
  • Apache Spark

Apache Spark

It is a framework for performing general data analytics on distributed computing cluster like Hadoop. It provides in-memory computations for increased speed and data process over map-reduce. It runs on top of existing Hadoop cluster and access Hadoop data store (HDFS) can also process structured data in Hive and streaming data from HDFS, Flume, Kafka, Twitter (Ref: http://aptuz.com/blog/is-apache-spark-going-to-replace-hadoop/).

MLlib Machine Learning Library

MLlib is a distributed machine learning framework on top of Spark. It implements many common machine learning and statistical algorithms to simplify large scale machine learning pipelines. MLlib also implements Linear SVM. (Ref: https://en.wikipedia.org/wiki/Apache_Spark)

The process

We will use Java for scaling, converting the colored images into Grey scaled images and converting grey scaled images to vectors. The vectors will be written to a text file.

namart.1RGB Image

144161733689107Grey Scaled Image

 
The vectors will be used by Apache Spark’s MLlib program or Linear SVM as training and test data. I will not cover how to set up Apache Spark with Java. Please refer to-http://www.robertomarchetto.com/spark_java_maven_example for details on this.
 
Program 1: Java Program to write the Vectors of images to a text file

 /** * sample program to read images belonging to a sample Class e.g. Male/Female . Then writing the images to Vector format into a text file. These text files will be used by Apache Spark for Linear SVM analysis */

  import java.awt.AlphaComposite;
  import java.awt.Color;
  import java.awt.Graphics2D;
  import java.awt.image.BufferedImage;
  import java.awt.image.DataBufferByte;
  import java.awt.image.Raster;
  import java.io.BufferedWriter;
  import java.io.File;
  import java.io.FileWriter;
  import java.io.IOException;
  import java.io.PrintWriter;
  import java.nio.file.FileVisitResult;
  import java.nio.file.Files;
  import java.nio.file.Path;
  import java.nio.file.Paths;
  import java.nio.file.SimpleFileVisitor;
  import java.nio.file.attribute.BasicFileAttributes;
  import java.util.ArrayList;

  import javax.imageio.ImageIO;

  /**
 * @author smruti
 *
 */
  public class ImageParser {

  /**
 * @param args
 */

  public static void toGray(BufferedImage image) {
  int width = image.getWidth();
  int height = image.getHeight();
  for(int i=0; i<height; i++){
  for(int j=0; j<width; j++){
  Color c = new Color(image.getRGB(j, i));
  int red = (int)(c.getRed() * 0.21);
  int green = (int)(c.getGreen() * 0.72);
  int blue = (int)(c.getBlue() *0.07);
  int sum = red + green + blue;
  Color newColor = new Color(sum,sum,sum);
  image.setRGB(j,i,newColor.getRGB());
  }
  }
  }

  public static BufferedImage createResizedCopy(BufferedImage originalImage,
  int scaledWidth, int scaledHeight,
  boolean preserveAlpha)
  {
  System.out.println("resizing...");
  int imageType = preserveAlpha ? BufferedImage.TYPE_INT_RGB : BufferedImage.TYPE_INT_ARGB;
  BufferedImage scaledBI = new BufferedImage(scaledWidth, scaledHeight, imageType);
  Graphics2D g = scaledBI.createGraphics();
  if (preserveAlpha) {
  g.setComposite(AlphaComposite.Src);
  }
  g.drawImage(originalImage, 0, 0, scaledWidth, scaledHeight, null);
  g.dispose();
  return scaledBI;
  }

  public static void main(String[] args) throws IOException {

  /* args[0] is the pixel to which the image will be converted */
  String[] piexels=args[0].split("x");

  int scaledWidth= Integer.parseInt(piexels[0]);
  int scaledHeight= Integer.parseInt(piexels[1]);

  ArrayList<String> paths = new ArrayList<String>();

  /* Traverse All the files inside the Folder and sub folder. args[1] is the path of the folder having the images */
  Files.walkFileTree(Paths.get(args[1].toString()), new SimpleFileVisitor<Path>() {
  @Override
  public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
  //Files.delete(file);
  paths.add(file.toFile().getAbsolutePath());
  return FileVisitResult.CONTINUE;
  }

  @Override
  public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
  //Files.delete(dir);
  return FileVisitResult.CONTINUE;
  }

  });

  for(String file : paths){

  String extension = "";

  int extIndex = file.indexOf('.');
  extension = file.substring(extIndex+1);

  System.out.println(file+" extension "+ extension);

  File input = new File(file.toString());
  BufferedImage image =createResizedCopy(ImageIO.read(input),scaledWidth,scaledHeight,Boolean.TRUE);
  toGray(image);
  File output = new File(file.toString());
  ImageIO.write(image, extension, output);
  }

  try
  {
  /* args[2] is the Class of the image, Class = Male/Female. Vector will be written into a text file */
  try(PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter("input-"+args[2]+".csv", true)))) {

  for(String file : paths){

  File file1 = new File(file.toString());
  BufferedImage img= ImageIO.read(file1);
  if (img == null) continue;
  Raster raster=img.getData();
  int w=raster.getWidth(),h=raster.getHeight();
  out.print(file1.getName());
  out.print(","+args[2]+",");
  for (int x=0;x<w;x++)
  {
  for(int y=0;y<h;y++)
  {
  out.print(raster.getSample(x,y,0)+" ");
  }
  out.print(" ");
  }
  out.println("");
  }

  }catch (IOException e) {
  //exception handling skipped for the reader
  }

  }
  catch (Exception e)
  {
  //exception handling skipped for the reader
  }

  }
  }

 

Program reference : http://introcs.cs.princeton.edu/java/31datatype/GrayPicture.java

Images Used in the example can be obtained from http://cswww.essex.ac.uk/mv/allfaces/faces96.zip
 

Input Command 1 (to generate male training set):

smruti@smruti-pc:~$ ImageParser 180×200 /home/smruti/spark-test/svm-test-images-male/ 1
  • ImageParser – Java class name of the file
  • 180 x 200 – The pixel dimension to which the image is to be resized.
  • /home/smruti/spark-test/svm-test-images-male/ – Folder having all the images belonging to Male.
  • 1 – is the class of the persona . Here it represents a Male.

 
Output: The output will be a file input-1.csv in the Java_home folder having the vectors of the images with the below format.

Class name, Image Name, Image vector

 
Sample input-1.csv contents:

1,andy.jpg,119 119 125 127 128 128 129 129 128 128 133 131 132 134 133 129 127 128 128 129 130……..
1,amar.jpg,114 114 114 113 112 110 108 107 107 106 103 102 99 100 96 92 89 90 90 88 86 50 55 67…….
1,bharat.jpg,53 53 52 52 52 52 52 52 52 52 53 52 50 49 49 50 52 53 50 51 53 54 54 53 51 50 53 53……..
1,bryan.jpg,67 67 67 67 67 66 66 65 65 66 73 70 69 70 70 71 69 67 64 65 64 65 65 65 63 62 61 61……..

 
Input Command 2 (to generate female training set)

smruti@smruti-pc:~$ ImageParser 180×200 /home/smruti/spark-test/svm-test-images-female/ 0
  • ImageParser – Java class name of the file
  • 180×200 – The pixel to which the images to be converted
  • /home/smruti/spark-test/svm-test-images-female/ – Folder having all the images belonging to Male
  • 0 – is the class of the persona . Here it represents a Female.

 
Output: The output will be a file input-0.csv in the Java_home folder having the vectors of the images with the below format.

0,virvi.jpg,140 141 142 145 145 144 143 142 145 147 146 143 141 141 140 137 137 138 142 144.
0,kyra.jpg,117 120 121 122 120 116 114 114 112 113 115 118 120 119 119 118 121 120 119 118
0,shipra.jpg,145 143 143 143 143 143 144 142 144 144 145 146 146 147 147 148 148 146 144 143
0,shally.jpg,133 136 137 136 134 135 133 130 133 129 129 129 125 126 128 124 123 124 125 125

 
Now we have two files having Class 0 (Female) and Class 1 (Male). We can further break down individual files for training and test data.
 
Program 2: Java Program that uses Apache Spark for Image classification:

package svmdemo;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaPairRDD;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.api.java.function.Function;
import org.apache.spark.api.java.function.PairFunction;
import org.apache.spark.mllib.classification.NaiveBayes;
import org.apache.spark.mllib.classification.NaiveBayesModel;
import org.apache.spark.mllib.classification.SVMModel;
import org.apache.spark.mllib.classification.SVMWithSGD;
import org.apache.spark.mllib.linalg.Vectors;
import org.apache.spark.mllib.regression.LabeledPoint;

import scala.Tuple2;

public class SVM {

public static void main(String[] args) {

// Create Java spark context
SparkConf conf = new SparkConf().setAppName("SVM vs Navie Bayes");
JavaSparkContext sc = new JavaSparkContext(conf);

//RDD training = MLUtils.loadLabeledData(sc, args[0]);
//RDD test = MLUtils.loadLabeledData(sc, args[1]); // test set

JavaRDD training = sc.textFile(args[0]).cache().map(new Function<String,LabeledPoint> () {

@Override
public LabeledPoint call(String v1) throws Exception {
double label = Double.parseDouble(v1.substring(0, v1.indexOf(",")));
String featureString[] = v1.substring(v1.indexOf(",") + 1).trim().split(" ");
double[] v = new double[featureString.length];
int i = 0;
for (String s : featureString) {
if (s.trim().equals(""))
continue;
v[i++] = Double.parseDouble(s.trim());
}
return new LabeledPoint(label, Vectors.dense(v));
}

});
System.out.println(training.count());
JavaRDD test = sc.textFile(args[1]).cache().map(new Function<String,LabeledPoint> () {

@Override
public LabeledPoint call(String v1) throws Exception {
double label = Double.parseDouble(v1.substring(0, v1.indexOf(",")));
String featureString[] = v1.substring(v1.indexOf(",") + 1).trim().split(" ");
double[] v = new double[featureString.length];
int i = 0;
for (String s : featureString) {
if (s.trim().equals(""))
continue;
v[i++] = Double.parseDouble(s.trim());
}
return new LabeledPoint(label, Vectors.dense(v));
}

});
System.out.println(test.count());
final NaiveBayesModel model = NaiveBayes.train(training.rdd(), 1.0);

JavaPairRDD<Double, Double> predictionAndLabel = test.mapToPair(new PairFunction<LabeledPoint, Double, Double>() {
@Override
public Tuple2<Double, Double> call(LabeledPoint p) {
return new Tuple2<Double, Double>(model.predict(p
.features()), p.label());
}
});
double accuracy = 1.0
* predictionAndLabel.filter(
new Function<Tuple2<Double, Double>, Boolean>() {
@Override
public Boolean call(Tuple2<Double, Double> pl) {
//System.out.println(pl._1() + " -- " + pl._2());
return pl._1().intValue() == pl._2().intValue();
}
}).count() / (double)test.count();
System.out.println("navie bayes accuracy : " + accuracy);

final SVMModel svmModel = SVMWithSGD.train(training.rdd(), Integer.parseInt(args[2]));

JavaPairRDD<Double, Double> predictionAndLabelSVM = test.mapToPair(new PairFunction<LabeledPoint, Double, Double>() {
@Override
public Tuple2<Double, Double> call(LabeledPoint p) {
return new Tuple2<Double, Double>(svmModel.predict(p
.features()), p.label());
}
});
double accuracySVM = 1.0
* predictionAndLabelSVM.filter(
new Function<Tuple2<Double, Double>, Boolean>() {
@Override
public Boolean call(Tuple2<Double, Double> pl) {
//System.out.println(pl._1() + " -- " + pl._2());
return pl._1().intValue() == pl._2().intValue();
}
}).count() / (double)test.count();
System.out.println("svm accuracy : " + accuracySVM);

}
}

 
We will convert this above program to a jar file i.e. svmdemo.jar . Please refer https://docs.oracle.com/javase/tutorial/deployment/jar/build.html
 
Input Command 3 (for image classification of test data)

smruti@smruti-pc:~$ bin/spark-submit –jars /opt/poc/spark-1.3.1-bin-hadoop2.6/mllib/spark-mllib_2.10-1.0.0.jar –class “SVM” –master local[4] /home/smruti/spark-test/svmdemo.jar /home/smruti/spark-test/SVM/training /home/smruti/spark-test/SVM/test 100
  • /home/smruti/spark-test/svmdemo.jar – is the jar file we have created before in Program 2
  • –class “SVM” – SVM is the java class file name to be executed in the jar
  • /home/smruti/spark-test/SVM/training – folder contains the training csv file we have generated in Program 1
  • /home/smruti/spark-test/SVM/test – folder contains the testing image csv files
  • 100 – is number of iterations to perform without reaching convergence.

 
Output:

tom_exp.11.jpg:1  1
jbierl.201.jpg:0  0
mike_exp.11.jpg:1  1
tjdyke.11.jpg:0  0
virvi.171.jpg:0  0
dav_exp.11.jpg:1  1
labenm.11.jpg:0  0
and_exp.11.jpg:1  1
namart.11.jpg:1  0
glen_exp.21.jpg:0  1
pat_exp.11.jpg:1  1
svm accuracy : 0.8181818181818182

 
Decoding The Output:
The output explains <test image name>: <Class (Male/Female) predicted by SVM> — <Expected Class (Male/Female)>

Here we can see out of 11 tests we have nine passed test cases and two failed test cases with an accuracy of 82%.

Passed test cases:

tom_exp.1 jbierl.20 mike_exp.1 tjdyke.1 labenm.1

tom_exp.11.jpg jbierl.201.jpg mike_exp.11.jpg tjdyke.11.jpg virvi.17.1jpg

virvi.17dav_exp.1 and_exp.1 pat_exp.1

labenm.11.jpg and_exp.11.jpg pat_exp.21.jpg dav_exp.11.jpg

 

Failed test cases:

glen_exp.2 namart.1

glen_exp.11.jpg namart.21.jpg

 

Further Experiments

The same program can be used to classify individual user persona as well. In that case, we should have many images per persona as training dataset which is essentially a face recognition system.

Thank you for reading. 🙂