Watch Recognition from scratch

I started the write up on this over 7 months ago, but didn’t have the courage to finish.

What is this project about and how did it start?

In January 2023, I decided to try to make an ML model as I had never tried machine learning before. I had a new interest in watches, and decided to experiment with Tensorflow.

I wished to make a ML model that could identify a watch model for a picture of a watch face. This ambition led to me creating not 1 but 2 models (so far!), a web app, learn about image formats and OpenCV.

Aspects of the design (and what makes it the ultimate project)

• Web scraping - Python
• Image Validation - Python + Rust
• Image Tinder (web app) - SolidJs
• Custom Tensorflow models
• ResNet

I have no clue if I should write about this project in chronological or logical order, as these versions would not be equivalent. I’m going to probaly summarize the chronology and then divide everything up in a technical aspect.

Chronology

Web scraping

Key takeaways

• Scraped over 100,000 images from several watch websites with custom python scripts.
• Created my own datastructure to organize the scraped image links.
• Built a SolidJS app to sort the images in wanted/unwanted categories. This let made it possible to have 2 folders: pictures of watches faces and pictures of watches that don’t contain a watch dial.
• Made a machine learning model that would classify watch images as having a watch face vs not having one with 99% accuracy.
• Used that ML model to organize the rest of the dataset.
• Download all the images (as the ML model classifies each image, the link to that image moves in the dataset from the “untreated” array to the “wanted” array)
• Devised a custom (NO ResNet involved!) model to classify the images of these watches by brand, to 96% accuracy.

Web Scraping

I had never done web scraping before this project, however I can now say I’m really comfortable with it. I started with Seiko’s USA website, a tribute to my first watch.

This was a mistake, Seiko’s website is a mess. In fact, most of the watch maker websites are in shambles.

Image Validation

Image Tinder

This was a fun part of the project for about 5 minutes, when I discovered Signals.

Why did I have to make my own Tinder?

Scraping the images was easy, but you collect A LOT of garbage (useless images). The images from the Orient Bambino Version 2 below will demonstrate what I mean:

---	---

We would only want the 3 first images here.

The images above have all been losslessly compressed (in the native JPG/PNG forms) and then converted to .webp for the sake of this page, however the file names are unchanged (stripped the suffixes that auto-crop them for the site, such as ?v=1631228511&width=1000&height=1110&crop=center). Inspect element or look at the network request to see their names if you’re curious. Hopefully you see why I was unable to auto-filter erroneous images with 100% accuracy.

In this case, the naming is petty straightforward, but this isn’t always the case. There was no good method to get rid of the rogue images #4 and #5. Let’s use ML and some manual labor then.

The idea was the follwing: Since I don’t download images until the download.py file is run, I could just restructure my scraped url JSON files to have 3 arrays for each watch: wantedUrls, unwantedUrls and untreatedUrls. Here is a visualization:

{
  "watch_name": {
    "collection": "watch_collection",
    "wantedUrls": ["urls go here"],
    "unwantedUrls": ["urls go here"],
    "untreatedUrls": ["more urls go here"]
  }
}

Due to this restrcuture, web scraping had to change: if the image URL was never seen before (meaning it is none of the arrays), it would be added to untreatedUrls. Here is an example of what a Nomos Glashütte Tangente watch being scraped for the first time would looks like :

{
  "125": {
    "collection": "tangente",
    "wantedUrls": [],
    "unwantedUrls": [],
    "untreatedUrls": [
      "https://cdn.nomos-glashuette.com/media/image/7f/41/ce/320xauto-q80-bg238238238/0125_tangente_33_grau_2022_2d_front-masked.jpg",
      "https://cdn.nomos-glashuette.com/media/image/7f/41/ce/1024xauto-q80-bg238238238/0125_tangente_33_grau_2022_2d_front-masked.jpg",
      "https://cdn.nomos-glashuette.com/media/image/2e/22/c1/125-detail-22-3.jpg",
      "https://cdn.nomos-glashuette.com/media/image/18/65/0b/320xauto-q80-bg238238238/0123_0125_0151_Tangente_33_GB_1Bn_PR.jpg",
      "https://cdn.nomos-glashuette.com/media/image/2e/22/c1/1024xauto-q80/125-detail-22-3.jpg",
      "https://cdn.nomos-glashuette.com/media/image/18/65/0b/480xauto-q80-bg238238238/0123_0125_0151_Tangente_33_GB_1Bn_PR.jpg",
      "https://cdn.nomos-glashuette.com/media/image/fb/6b/24/320xauto-q80/125-tangente-33-grau-detail-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/fb/6b/24/480xauto-q80/125-tangente-33-grau-detail-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/fb/6b/24/125-tangente-33-grau-detail-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/fb/6b/24/1024xauto-q80/125-tangente-33-grau-detail-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/2e/22/c1/320xauto-q80/125-detail-22-3.jpg",
      "https://cdn.nomos-glashuette.com/media/image/18/65/0b/1024xauto-q80-bg238238238/0123_0125_0151_Tangente_33_GB_1Bn_PR.jpg",
      "https://cdn.nomos-glashuette.com/media/image/2e/22/c1/480xauto-q80/125-detail-22-3.jpg"
      "https://cdn.nomos-glashuette.com/media/image/7f/41/ce/480xauto-q80-bg238238238/0125_tangente_33_grau_2022_2d_front-masked.jpg",
      "https://cdn.nomos-glashuette.com/media/image/43/7e/95/320xauto-q80/125-detail-22-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/43/7e/95/1024xauto-q80/125-detail-22-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/43/7e/95/125-detail-22-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/43/7e/95/480xauto-q80/125-detail-22-1.jpg"
      "https://cdn.nomos-glashuette.com/media/image/41/9a/16/1024xauto-q80/125-tangente-33-grau-wrist-still.jpg",
      "https://cdn.nomos-glashuette.com/media/image/41/9a/16/320xauto-q80/125-tangente-33-grau-wrist-still.jpg",
      "https://cdn.nomos-glashuette.com/media/image/41/9a/16/125-tangente-33-grau-wrist-still.jpg",
      "https://cdn.nomos-glashuette.com/media/image/41/9a/16/480xauto-q80/125-tangente-33-grau-wrist-still.jpg"
    ]
  }
}

Then, I would load the .json file, and sort the image manually, shifting the arrays around. Downloading the JSON, here is what I would obtain for this particular watch model:

{
  "125": {
    "collection": "tangente",
    "wantedUrls": [
      "https://cdn.nomos-glashuette.com/media/image/43/7e/95/1024xauto-q80/125-detail-22-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/43/7e/95/480xauto-q80/125-detail-22-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/43/7e/95/125-detail-22-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/7f/41/ce/1024xauto-q80-bg238238238/0125_tangente_33_grau_2022_2d_front-masked.jpg",
      "https://cdn.nomos-glashuette.com/media/image/7f/41/ce/480xauto-q80-bg238238238/0125_tangente_33_grau_2022_2d_front-masked.jpg",
      "https://cdn.nomos-glashuette.com/media/image/7f/41/ce/320xauto-q80-bg238238238/0125_tangente_33_grau_2022_2d_front-masked.jpg",
      "https://cdn.nomos-glashuette.com/media/image/43/7e/95/320xauto-q80/125-detail-22-1.jpg"
    ],
    "unwantedUrls": [
      "https://cdn.nomos-glashuette.com/media/image/41/9a/16/480xauto-q80/125-tangente-33-grau-wrist-still.jpg",
      "https://cdn.nomos-glashuette.com/media/image/2e/22/c1/480xauto-q80/125-detail-22-3.jpg",
      "https://cdn.nomos-glashuette.com/media/image/2e/22/c1/1024xauto-q80/125-detail-22-3.jpg",
      "https://cdn.nomos-glashuette.com/media/image/2e/22/c1/125-detail-22-3.jpg",
      "https://cdn.nomos-glashuette.com/media/image/18/65/0b/320xauto-q80-bg238238238/0123_0125_0151_Tangente_33_GB_1Bn_PR.jpg",
      "https://cdn.nomos-glashuette.com/media/image/2e/22/c1/320xauto-q80/125-detail-22-3.jpg",
      "https://cdn.nomos-glashuette.com/media/image/18/65/0b/1024xauto-q80-bg238238238/0123_0125_0151_Tangente_33_GB_1Bn_PR.jpg",
      "https://cdn.nomos-glashuette.com/media/image/fb/6b/24/320xauto-q80/125-tangente-33-grau-detail-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/18/65/0b/480xauto-q80-bg238238238/0123_0125_0151_Tangente_33_GB_1Bn_PR.jpg",
      "https://cdn.nomos-glashuette.com/media/image/41/9a/16/320xauto-q80/125-tangente-33-grau-wrist-still.jpg",
      "https://cdn.nomos-glashuette.com/media/image/fb/6b/24/480xauto-q80/125-tangente-33-grau-detail-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/fb/6b/24/125-tangente-33-grau-detail-1.jpg",
      "https://cdn.nomos-glashuette.com/media/image/fb/6b/24/1024xauto-q80/125-tangente-33-grau-detail-1.jpg"
    ],
    "untreatedUrls": [
      "https://cdn.nomos-glashuette.com/media/image/41/9a/16/125-tangente-33-grau-wrist-still.jpg",
      "https://cdn.nomos-glashuette.com/media/image/41/9a/16/1024xauto-q80/125-tangente-33-grau-wrist-still.jpg"
    ]
  }
}

Process of creation of Image Tinder

At the time, SolidJs was very new, so I decided to give it a try. I had enough of garbage Python documentation and Solid’s was promising.

After 30 minutes of trying to get ChatGPT to make me what I needed, I realized that it didn’t have the context or my prompting was bad. Anyway I wrote code.

Now, an issue that came about was accidentally leaving the page, therefore losing the “swiping” I had done. I could have done some wicked localStorage manipulation, or even spun up a supabase DB instance, but I did not. Here is how I solved my issue:

const alertUser = (e: any) => {
    alert("STOP");
    e.preventDefault();
    e.returnValue = "";
  };

onMount(() => {
    window.addEventListener("beforeunload", alertUser);
  });

I count 8 lines of code. 8 lines that saved me hours of my precious time as an undergrad. Embrace fast, simple solutions, especially if they solve a problem only you will encounter once in your lifetime.