Part 3 - PIX2PIX Introduction (Technical)

PIX2PIX can Learn a mapping from input images to output images. PIX2PIX introduction here: PIX2PIX

To train PIX2PIX model, we used Tensorflow and ran the script in Google Cloud Platform.

Before training:

Zip the dataset for training with following format: -> dataset -> train -> *.jpg

Zip the dataset for testing with following format in the same folder: -> dataset -> val -> *.jpg

Each picture should be 512pixesX256pixes and original image on the left, simplified or recognized image on the right. Be sure to line up the two images.

Upload the zip file to somewhere you want, and get the download link, example: ‘’

It should be a direct download link, not a page for downloading.

Training on Google Cloud (CPU Only):

After setting up all things (account, project, ect), click Compute Engine, then VM instances to set up new instance.

alt text

Click ‘Create’ and it shows the settings of new instance. Click ‘Customize’ to use different settings for CPU and GPU. Here at least 2-cores CPU and 8GB Memory are recommanded. 6-cores CPU equals to 6700HQ CPU. For boot disk, we use Ubuntu 16.04 LTS and 20GB disk. Then click ‘Create’.

alt text alt text

After that, it shows:

alt text

Click SSH to connect. A window like this will be opened.

alt text

Type ‘sudo apt-get update’ to update first.

Type ‘sudo apt-get install python-pip python-dev’ to install pip first.

Type ‘sudo pip install tensorflow –upgrade’ to install tensorflow.

Type ‘sudo apt-get install libopencv-dev libboost-all-dev libprotobuf-dev libleveldb-dev libsnappy-dev libhdf5-serial-dev protobuf-compiler libatlas-base-dev libgflags-dev libgoogle-glog-dev liblmdb-dev’ to get other supporting libraries.

Type ‘git clone’ to download the script without data from github.

Type ‘cd pix2pix-tensorflow’ to get into the folder.

Tpye ‘python’ to start the downloading script.

Paste your link the press Enter to download and unzip the files. Please include ‘’.

alt text

The downloading script will erase the former train dataset, so be careful.

Type ‘python –mode train –output_dir dataset_train –max_epochs 200 –input_dir dataset/train –which_direction BtoA’ to run the script.

If it works, you will see:

alt text

When it’s running, you will see the time left and steps.

alt text

Do not close the window, it will shut down the script. But everything we already create will be in the disk.

After finishing all running, click ‘stop’ to close the instance. Otherwise you will still get charged.

alt text

Training on Google Cloud (GPU) (Highly Recommended):

Believe it or not, it runs 20 times faster when using GPU.

Testing in your own images (without WinSCP):

Reconnect the instance or keep the window open, type ‘cd pix2pix-tensorflow’ to get into the folder.

Type ‘python –mode test –output_dir dataset_test –input_dir dataset/val –checkpoint dataset_train’ to run the test script to get the new pictures.

When finished, type ‘python’ to zip test data.

Downloading results:

Type ‘gcloud init’ to start the Google cloud storage. You may need to log in your google account, please follow then instruction on the screen.

Type ‘gsutil mb gs://please-type-your-folder-name-here/’ to create a new folder. If you already did this, skip this line. If you get some errors return, type ‘gcloud auth login’(replace the email address with your google account) to re-log in and give authorization.

Type ‘cd pix2pix-tensorflow’ to get into the folder.

Type ‘gsutil cp gs://please-type-your-folder-name-here’ to upload the zip file.

Check the Storage -> Browser in Google Cloud Platform to download your file.

alt text

Setting up WinSCP:

If you want a more efficient way to manage your files on the cloud machine, please follow this:

Highly recommended! It allows you to manage files on the cloud in a much easier way.

Serving models locally (Ubuntu):

Download the ‘dataset_train’ folder from the cloud machine and put it inside the main folder of pix2pix.

Type ‘sudo apt-get install wget’ and ‘wget -qO- | sh’ to install docker.

Type ‘cd pix2pix-tensorflow’ and ‘cd server’ to get into the server folder.

Type ‘python ../ –mode export –output_dir models/dataset –checkpoint ../dataset_train’ to export the model. Please change the output_dir everytime.

Type ‘python ../tools/ python tools/ –model_dir models/dataset –input_file static/facades-input.png –output_file output.png’ to test whether the exported model works. (optional)

Type ‘python ../tools/ –port 8080 python –port 8080 –local_models_dir models’ to run the server locally.

Next time when you want to exchange data with the server, use the global ip address of the server.