Novelty Detection on an Arduino using One-Class SVM

Deploying machine learning (ML) to the Edge, or even onboard your embedded device takes a good amount of customisation and tweaking. This is especially true if you’re using a small embedded C chip with limited RAM. Most ML algorithms take a long time to develop and deploy, but Support Vector Machines are an exception. The model data for these algorithms requires only a small amount of program memory, and a smaller amount of RAM to calculate at runtime. This means they’re ideally suited to a small microcontroller such as an Arduino. (Note: other microcontrollers are available, just not as good for SEO).

https://github.com/Rotario/noveltyDetection

Novelty Detection or One-Class Classification

I’m going to cover aspecific form of Classification in this article, called One-Class Classification. This is useful when you’ve got many datapoints for one case, such as sensor data from a working pump, and very little or no datapoints for another case, such as sensor data from a broken pump. There’s a library for standard SVM classification on microcontrollers (link). But it doesn’t support one-class, which I think is part of the attraction for SVMs.

In the case described above you’d like to find out whether a pump is working or not, given historical data of it working. What we’re interested here is finding the “novelty” or the new data that doesn’t fit the old. In this case when the pump breaks.

If, however, you have example datapoints for both cases (working and non-working) then you should probably just use a standard SVM.

Training Data

Training data must be in the form of discrete events, this system is not applicable to time series data in some cases (if in doubt, try it).

Your model must be trained offline. To do this you can use LIBSVM, a python library called scikit-learn or other machine learning environment of your choice. I have a script that can generate these but it’s not finished yet. Send me an email if you want it and I’ll hurry up!

There are ample tutorials online for training your model, find links at the bottom of this page.(training this model can be as complicated as you want, there’s a lot to tweak!). What you want to produce is a model file and a scaling file. LIBSVM generates these automatically, so I’d recommend using it for testing. But sci-kit is easier to automate in python.

Loading model file into your Arduino

Now you’ve generated your model file and scaling file, these need to be loaded onto your microcontroller. There’s a number of ways to do this:

1 — SD Card (supported by the library)

2 — SPIFFS (ESP8266 only) (find examples in the ESP branch of the repository)

3 — Hard coded (not supported yet)

If you’ve got a board that has an EEPROM and SD card in one package (like a Teensy) then no 1 is the preferred solution, and the library can be used without modification.

Just call

SVM_readModelFromSD(SVM_MODEL_FILENAME, SVM_SCALE_PARAMETERS_FILENAME)

to load your SVM model into the EEPROM memory. The default filenames are “svm.mod” for the model file and “svm.par” for the parameter file.

Then measure your data, from temperature/pressure sensors with analogRead, or with their digital interface. Create a float array with your measured data (make sure each sensor value is in the same position it was in when you trained your model!):

float * data[] = {analogRead(A0), analogRead(A1), analogRead(A2)};

Then predict whether this data is like the training data or unlike it:

float ret = SVM_predictEEPROM(data, 3);
int recognisedClass;
if (ret > 0) {
    Serial.println("Not Novelty");
    recognisedClass = 1;
  } else {
    Serial.println("Novelty");
    recognisedClass = -1;
  }

Here the predict function returns a float, where greater than 0 means it’s like the training dataset, and less than one means it’s not like the dataset, or a novelty.

Now go and monitor your home (or industrial) automation solutions!

Send your questions (or code requests) to rowan@workyourtech.com

Model Training Guides:

libsvm guide — PDF DOWNLOAD

scikit-learn — LINK

6 Replies to “Novelty Detection on an Arduino using One-Class SVM

  1. I am new to embeded machine learning. I have a question.
    can i have samples data that you load from the sd card ?
    Ihow do i get the model from scikit learn? and what is the scale file that you mentioned in the article ?
    thank you
    M.

    1. Hi Manolis,
      If you look on the github repo, on the esp branch, there’s a new folder called utility.
      This has example data, with a python script that trains your data for you.
      It also inclueds svm.mod and svm.par, which are the model and parameter files generated by the training script.
      Link to the utility folder here:
      https://github.com/Rotario/noveltyDetection/tree/esp/utility
      I can do a 30 minute hangout with you to work through the questions you’ve got! Email me
      Thanks,
      Rowan

  2. hello Rowan
    i was wondering if we could design the code just to to read the data from a csv file and to run the one class svm from the flash.
    suppose we have two sounds and we want to distinquish them,
    but if a sound signal comes and it is not what we want to clasify then the system reject it.

    1. Hi Emmanuel,
      What microcontoller are you using? Do you want to load a CSV file from flash or an SD card?
      Right now you can read the SVM model parameters from an SD card or from SPI Flash on an ESP8266. You can then predict with data you gather while you’re running the program, or on data from any source, as long as you pass each data point as an array into svm.predict.
      If you’ve got a sound file, then I would suggest looking at reducing it down into a few datapoints, maybe like this: https://towardsdatascience.com/extract-features-of-music-75a3f9bc265d?gi=b0cd7a9b9f79

      How much progress have you made already? It’s worth pointing out the steps you need to do to be able to use the SVM model to predict things with:
      1. Gather representative data
      2. Extract features from dataset
      3. Use my utility (on [ESP branch on github](https://github.com/Rotario/noveltyDetection/tree/esp/utility)) to train and generate svm.mod and svm.par files with this dataset
      4. Put both of these files on your SD card, or SPIFFS on ESP8266
      5. Run loadSVMmodelFromSDCard, as shown in the example

      1. Hi Rowan,
        I am using Arduino uno,
        i have a sound sensor and recording 3 sounds and make 3 classes, from almost 15 samples of each one. I use fft to create the features and that is my dataset. i want to load it from flash not from sd.
        I train ofline the model and load it to Uno.

        1. Hi,
          If you want to load the model from flash, it should be fairly easy – I had a look but I can’t do it right now I’m afraid.
          If you take the loadModelfromSD function and remove anything to do with the SD library and instead read from a char pointer, that should work the same as reading from an SD file.
          I can help you with it in a couple of weeks, I’ve got deadlines to meet before then. If you want this job done quicker, you could put a bounty on it!
          Thanks,
          Rowan

Leave a Reply

Your email address will not be published. Required fields are marked *