+1
日增长
9,979
stars
1,366
forks
416
watchers
15
prs
64
issues
0
releases
README
deep-photo-styletransfer
Code and data for paper "Deep Photo Style Transfer"
Disclaimer
This software is published for academic and non-commercial use only.
Setup
This code is based on torch. It has been tested on Ubuntu 14.04 LTS.
Dependencies:
Torch (with matio-ffi and loadcaffe)
Matlab or Octave
CUDA backend:
CUDA
cudnn
Download VGG-19:
sh models/download_models.sh
Compile cuda_utils.cu (Adjust PREFIX and NVCC_PREFIX in makefile for your machine):
make clean && make
Usage
Quick start
To generate all results (in examples/) using the provided scripts, simply run
run('gen_laplacian/gen_laplacian.m')
in Matlab or Octave and then
python gen_all.py
in Python. The final output will be in examples/final_results/.
Basic usage
1. Given input and style images with semantic segmentation masks, put them in examples/ respectively. They will have the following filename form: examples/input/in.png, examples/style/tar.png and examples/segmentation/in.png, examples/segmentation/tar.png;
2. Compute the matting Laplacian matrix using gen_laplacian/gen_laplacian.m in Matlab. The output matrix will have the following filename form: gen_laplacian/Input_Laplacian_3x3_1e-7_CSR.mat;
Note: Please make sure that the content image resolution is consistent for Matting Laplacian computation in Matlab and style transfer in Torch, otherwise the result won't be correct.
3. Run the following script to generate segmented intermediate result:
th neuralstyle_seg.lua -content_image -style_image -content_seg -style_seg -index -serial
4. Run the following script to generate final result:
th deepmatting_seg.lua -content_image -style_image -content_seg -style_seg -index -init_image _t_1000.png -serial -f_radius 15 -f_edge 0.01
You can pass -backend cudnn and -cudnn_autotune to both Lua scripts (step 3.
and 4.) to potentially improve speed and memory usage. libcudnn.so must be in
your LD_LIBRARY_PATH. This requires cudnn.torch.
Image segmentation
Note: In the main paper we generate all comparison results using automatic scene segmentation algorithm modified from DilatedNet. Manual segmentation enables more diverse tasks hence we provide the masks in examples/segmentation/.
The mask colors we used (you could add more colors in ExtractMask function in two .lua files):
| Color variable | RGB Value | Hex Value |
| ------------- | ------------- | ------------- |
| blue | 0 0 255 | 0000ff |
| green | 0 255 0 | 00ff00 |
| black | 0 0 0 | 000000 |
| white | 255 255 255 | ffffff |
| red | 255 0 0 | ff0000 |
| yellow | 255 255 0 | ffff00 |
| grey | 128 128 128 | 808080 |
| lightblue | 0 255 255 | 00ffff |
| purple | 255 0 255 | ff00ff |
Here are some automatic and manual tools for creating a segmentation mask for a photo image:
Automatic:
MIT Scene Parsing
SuperParsing
Nonparametric Scene Parsing
Berkeley Contour Detection and Image Segmentation Resources
CRF-RNN for Semantic Image Segmentation
Selective SeaTop issues
#29Docker container request10 comments · 2017-03-29#24Matio can't load CSR files12 comments · 2017-03-28#56error loading module 'libcuda_utils10 comments · 2017-05-15#42Did someone figure out how to get rid of the "out of memory" issue while running Torch?7 comments · 2017-04-03#40Running on Windows4 comments · 2017-04-02#16Repository does not include license information4 comments · 2017-03-27#71Licence Problem3 comments · 2018-08-11#58Out of Memory3 comments · 2017-06-02