Week 6 (May 14 - 18): Building an AI

How does one build an AI?

After creating several onboarding screens for my app, This week I want to start researching how to train an AI model to generate floor plan layouts. I have no idea where to start; which AI models should I use, how do I train a model, and what does that whole process look like? 

The first thing I did was research what kinds of models are used for image generation. I know that I want my AI model to be able to generate a floorplan for a room, which meant it had to generate an image. I would then be able to put this image onto the app once the user clicked a button. From Google searches, I found t hat popular image generation architectures include neural networks and GANs. I learned that the GAN architecture actually incorporates 2 parts: a generator and a discriminator. In my case, the generator tries to generate new images based on the training data, while the discriminator tries to pick out the images generated among a collection of generated images and the sample size. The generator gets points if it can generate good enough images that it ‘fools’ the discriminator, while the discriminator gets points if it can successfully pick out the generated images. In this way, the GAN ‘learns’ how to generate better images over time as the generator improves how well it creates images trying to fool the discriminator, while the discriminator improves at picking out the generated images.

After learning about GANs, I found them very interesting and have decided to start by training a GAN on floor plan images.

The AI Hungers

Of course during this research process I’ve also realized I need training data for the AI. Since I was simply testing how successful GANs could be, I found a dataset of 170 interior design images that were used for general use on Kaggle. Kaggle is community for AI enthusiasts meant for public use of datasets; there are thousands of free datasets of images for whatever type of training you need. I found it very useful when searching for datasets of floorplans that I could feed into my AI.

Kaggle’s dataset homepage, providing many different types of datasets free to use

Here are 4 of the 170 floorplans from the dataset I downloaded:

The AI Grows

Now that I have the dataset needed to start training a GAN model, I’m eager to start training. I did a YouTube search to look for a GAN tutorial, and found an amazing video ‘Implement and Train DCGAN From Scratch for Image Generation’ by ‘Uygar Kurt’ detailing how to train a GAN using a Python Notebook.

Uygar Kurt explaining the code for the discriminator AI

From following his detailed video on how to train a GAN using this Python Notebook, I was able to start training the same GAN model using my own dataset of images.

At the start of the training process, the AI starts with generating random noise:

This is the 1st and 2nd training step of the GAN. EPOCH = training attempt, so here EPOCH 1 means the 1st training attempt. The GAN starts by generating random noise.

At around 160 training attempts, the AI has understood how to generate a white background, similar to the images its being trained on.

EPOCH 159, or the 159th training attempt, it’s now able to generate images of a blank background which the dataset of images does have as well

At around 250 epochs, the GAN is starting to understand how to generate lines, similar to the borders of each room. At this point, the GAN has been training for around 4 hours on my laptop as I left it idle in my bedroom, with my laptop its been taking around 2 hours per 100 epochs.

250 epochs in, or 250 training attempts, the GAN is starting to understand how to generate lines.

At around 350 training attempts, the GAN is starting to attempt to generate multiple rooms, similar to how the dataset I’ve given it has multiple rooms

350 epochs in, the AI is generating many lines, similar to multiple rooms

But after 370 attempts, for some reason the generator AI started adding more noise into it’s outputs, which eventually destroyed it’s training attempt.

At 370 epochs, the GAN is just generating a complete mess, which basically ended the whole training process

This occurred over and over, and even after 1500 training cycles, it would repeat this process of learning how to generate room borders, start generating multiple rooms, then inevitably start introducing noise and ruining its training process

at 1200 epochs, it started getting promising again, but it led nowhere.

At this point, I had trained the GAN for 24 hours and I realized that this was going nowhere. I had the impression that my dataset didn’t have enough images, so I decided to try a second dataset that I had found on Kaggle. This dataset had 4004 images of floorplans, so if my assumption was correct, it would perform better than the previous dataset of images. Here are some of the images that this second dataset contains:

Images from my second dataset of 4004 images

Now, I ran the Python Notebook and starting the training process on the GAN again, but this time with a dataset with 4000+ images.

I quickly noticed that the GAN was learning at a much faster rate, at around 10 epochs it was already able to generate a white background with a shape in the middle:

at 10 epochs using the new dataset, the GAN was improving at a quick rate

At around 20 epochs, I could tell it was trying to mold the shape in the middle into a floorplan with several rooms:

I could slowly start to see some shape forming

At around 40 epochs, I could start the see borders of the room starting to form. I could clearly tell that when the dataset was larger and there was more data for the AI to take in, it was able to train better and quicker. I had not even spent an hour, and it had already reached to the point where the previous attempt peaked at!

So quickly, borders were starting to form in the floorplan shape

At around 75 epochs, the borders were getting more solid and complete, and the AI was starting to understand that in this dataset, the rooms had different colors, and was trying to mimic that in it's generated floorplans:

at 75 epochs, I could start to see colors emerging in the shape of the floorplan, mimicking how the images in the dataset had rooms with different colors

At 140 epochs, this was only emphasized further; the border of the rooms were fully connecting and the different colors representing the unique rooms were becoming more concrete

after only 2 hours, this GAN training attempt had far succeeded the previous attempt I made

However, at this point, the training started to stall, and I noticed the generator AI couldn’t seem to finetune the image to fully establish different rooms. In fact, it began a cycle of generating blurrier images for 30-40 epochs and then refining that, seemingly like it was figuring everything out, before again generating blurry images. This cycle would repeat for 300 epochs.

Finally, at around 350 epochs and 6 hours into training, improvements were starting to be made and clearly I could see well defined shapes of the rooms appearing!

although the borders of the rooms are not straight, I felt hopeful due to the sudden progress of the colored rooms becoming more defined!

However soon, at around 500 epochs for some strange reason, the generator model started to implode, and began to generate images with increasingly more noise. Similar to my first attempt at training this GAN, this completely destroyed this training attempt.

This had crushed my hopes at training a GAN. For some reason, both times it had eventually destroyed its progress and started generating noise.

Next Steps

After this testing period of attempting to train a GAN, I want to figure out why this process didn’t work, and if there are different model architectures I could try next that would yield better results.