Real-Time Exemplar-Based Face Sketch Synthesis Pipeline
Learn about a real-time exemplar-based face sketch synthesis pipeline that utilizes photo-sketch pairs and advanced algorithms for sketch denoising, including Coarse Sketch Generation and Proposed Spatial Sketch Denoising Algorithm (SSD).
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Real-Time Exemplar-Based Face Sketch Synthesis Pipeline illustration Qingxiong Yang1 Ming-Hsuan Yang2 Yibing Song1 Linchao Bao1 1City University of Hong Kong 2University of California at Merced Note: containing animations
Our assumption: a database containing photo-sketch pairs 1. photo database 2. sketch database Aligned
Coarse Sketch Generation Step 1: KNN search Test photo patch ?? Test photo p Relative position ? Relative position ? Similarly ? ? ? [ ] ? ? = Training photo dataset ?? ?? ?? ? Matched photo patch ?? ? Matched photo patch ??
Coarse Sketch Generation Step 2: Linear Estimation from Photos Test photo patch ?? ? ? ? ? ? ? ? ? ? = +?? +?? ?? ? ? Matched photo patch ?? ? Matched photo patch ?? Matched photo patch ?? 2. Compute linear mapping function defined by ?? ?,?? ?, ,?? ?
Coarse Sketch Generation Step 3: Apply Linear Mapping to Sketches Test photo Coarse sketch Repeat for every pixel p ? ? ? ? ? ? ? ? ? +?? ?? +?? = ?? Estimation on pixel p Matched sketch pixel Matched sketch pixel ??+ ? Matched sketch pixel??+ ? ??+ ? ? ? ?
Denoising: State-of-the-art Image Denoising Algorithms Coarse sketch Nonlocal Means (NLM) q r ?(?,?) ?? p ???= ?(?,?) ?? + ?? + For all pixels in the neighbor of p: ? After NLM Little improvement Because Because: coarse sketch image is not natural. ?(?,?) is not a good similarity measurement between p and r. [NLM] A. Buades, B. Coll and J.-M. Morel, A non-local algorithm for image denoising, CVPR 2005.
Motivation BM3D BM3D groups correlated patches in the noisy image to create multiple estimations. How BM3D works Our idea for sketch denoising: group highly similar sketch estimations. [BM3D] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Image denoising by sparse 3D transform- domain collaborative filtering, IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080-2095, August 2007.
Proposed Spatial Sketch Denoising Algorithm (SSD) Estimations from pixels in local region ? Test photo r q ? ?? p p Averaging estimations to generate output sketch value. ?,?? ? ?, ?? ? ?, ?? ? ? ? ? Similarly Matched sketch ??+ ? ??+ ? ??+ ? = ? ? ? +?? ? ? ? ?? +?? ? ?? ??+ ? ??+ ? ??+ ? ? ? ? Nonlocal Means (NLM): ???= ?(?,?) ?(?,?) + ?? ?? + ?? Proposed SSD: ? ? ???= 1 ?? 1 ?? ?? + +
Robustness to the region size ? - the only parameter involved p Proposed SSD is robust to ? Input 17x17 local region ?? = ??? 5x5 local region ?? = ?? 11x11 local region ?? = ??? 23x23 local region ?? = ??? Note: When ?? is sufficient large (i.e., ??>100), the proposed SSD can effectively suppress noise while preserving facial details like the tiny eye reflections (see close-ups).
