Awesome-Sketch-Synthesis

A collection of papers about Sketch Synthesis (Generation). Mainly focus on stroke-level vector sketch synthesis.

Feel free to create a PR or an issue.

Outlines

• 0. Survey
• 1. Datasets
• 2. Sketch-Synthesis Approaches
  • 1) Semantic Concept-to-sketch
  • 2) Photo-to-sketch
  • 3) Text/Attribute-to-sketch
  • 4) 3D shape-to-sketch
  • 5) Art-to-sketch
• 3. Vector Graphics Generation (2D)
• 4. Vector Graphics Generation (3D)

---

0. Survey

Paper	Source	Code/Project Link
Deep Learning for Free-Hand Sketch: A Survey	TPAMI 2022	[code]

1. Datasets

Here Vector strokes means having svg data. With photos means having the photo-sketch paired data.

Level	Dataset	Source	Vector strokes	With photos	Notes
Characters	Omniglot		✔️	❌	Alphabets characters
	KanjiVG		✔️	❌	Chinese characters
	Kuzushiji		❌	❌	Japanese characters
Icon	FIGR-8-SVG		✔️	❌	Icons with text descriptions
Systematic Symbol	SketchIME	ACM MM 2023	✔️	❌	Systematic sketches with semantic annotations
Instance-level	TU-Berlin	SIGGRAPH 2012	✔️	❌	Multi-category hand sketches
	Sketchy	SIGGRAPH 2016	✔️	✔️	Multi-category photo-sketch paired
	QuickDraw	ICLR 2018	✔️	❌	Multi-category hand sketches
	QMUL-Shoe-Chair-V2	CVPR 2016	✔️	✔️	Only two categories
	Sketch Perceptual Grouping (SPG)	ECCV 2018	✔️	❌	With part-level semantic segmentation information
	FaceX	AAAI 2019	✔️	❌	Labeled facial sketches
	Creative Sketch	ICLR 2021	✔️	❌	With annotated part segmentation
	ImageNet-Sketch	NeurIPS 2019	❌	❌	50 images for each of the 1000 ImageNet classes
	SEVA	NeurIPS 2023	✔️	✔️	90K human-generated sketches that vary in detail
	Sketchtopia	CVPR 2025	✔️	❌
Scene-level	SketchyScene	ECCV 2018	❌	✔️	With semantic/instance segmentation information
	CMPlaces	TPAMI 2018	❌	✔️	Cross-modal scene dataset
	Context-Skecth	Expressive 2018	❌	✔️	Context-based scene sketches for co-classification
	SketchyCOCO	CVPR 2020	❌	✔️	Scene sketch, segmentation and normal images
	FS-COCO	ECCV 2022	✔️	✔️	Scene sketches with text description
	SFSD	VC 2022	✔️	✔️	Completely hand-drawn scene sketches with label annotation
Drawing from photos	Photo-Sketching	WACV 2019	✔️	✔️	ScenePhoto-sketch paired
	Tracing-vs-Freehand	SIGGRAPH 2021	✔️	✔️	Tracings and freehand drawings of images
Drawing from 3D models	DifferSketching	SIGGRAPH Asia 2022	✔️	❌	3D model-sketch paired, with novice and professional ones
Portrait	CUFS	TPAMI 2009	❌	✔️	Face-sketch pairs
	APDrawing	CVPR 2019	❌	✔️	Portrait-sketch paired
	SKSF-A	EG 2024	❌	✔️	Face-sketch pairs of seven styles
Children's Drawing	Amateur Drawings	TOG 2023	❌	✔️	With character bounding boxes, segmentation masks, and joint location annotations
Rough sketch	Da Vinci	CGI 2018	❌	✔️	Line drawing restoration dataset
	Rough Sketch Benchmark	SIGGRAPH Asia 2020	✔️	✔️	Rough and clean sketch pairs (only for evaluation)
CAD	ld3d	SIGGRAPH 2008	❌	❌	Line Drawings of 3D Shapes
	OpenSketch	SIGGRAPH Asia 2019	✔️	❌	Product Design Sketches
	SketchGraphs	ICML 2020 Workshop	✔️	❌	Sketches extracted from real-world CAD models
	Fusion 360 Gallery	SIGGRAPH 2021	✔️	❌	For 'sketch and extrude' designs
	FloorPlanCAD	ICCV 2021	✔️	❌	With instance and semantic annotations
Anime	Danbooru2021	/	❌	✔️	Anime images annotated with tags
	DanbooRegion	ECCV 2020	❌	❌	Anime images with region annotations
	Danbooru-Parsing	TOG 2023	❌	✔️	For anime portrait parsing and anime translation
	CreativeFlow+	CVPR 2019	❌	✔️	Large densely annotated artistic video dataset
	ATD-12K	CVPR 2021	❌	✔️	Animation frames with flow annotations
	AnimeRun	NeurIPS 2022	❌	✔️	Correspondence dataset for 2D-styled cartoons
	AnimeCeleb	ECCV 2022	❌	❌	Animation head images with pose annotations
	PaintBucket-Character	CVPR 2024	❌	✔️	Animation frames with region annotations
	Sakuga-42M (link 2)	arxiv 24.05	❌	✔️	Cartoon videos with text descriptions and tags
	Anita	online 2024	❌	✔️	Professional hand-drawn cartoon keyframes, with 1080P sketch and color images
	MangaSeg	CVPR 2025	❌	✔️	Manga segmentation annotation

2. Sketch-Synthesis Approaches

1) Semantic Concept-to-sketch

Level	Paper	Source	Code/Project Link
Instance-level	A Neural Representation of Sketch Drawings (sketch-rnn)	ICLR 2018	[Code] [Project] [Demo]
	Sketch-pix2seq: a Model to Generate Sketches of Multiple Categories		[Code]
	AI-Sketcher : A Deep Generative Model for Producing High-Quality Sketches	AAAI 2019	[Project]
	Stroke-based sketched symbol reconstruction and segmentation (stroke-rnn)	CGA 2019
	BézierSketch: A generative model for scalable vector sketches	ECCV 2020	[Code]
	Pixelor: A Competitive Sketching AI Agent. So you think you can beat me?	SIGGRAPH Asia 2020	[Project] [Code]
	Creative Sketch Generation	ICLR 2021	[Project] [Code]
	Computer-Aided Design as Language	arxiv 2105
	DoodleFormer: Creative Sketch Drawing with Transformers	ECCV 2022	[Project] [Code]
	SketchKnitter: Vectorized Sketch Generation with Diffusion Models	ICLR 2023	[Code]
	Self-Organizing a Latent Hierarchy of Sketch Patterns for Controllable Sketch Synthesis	TNNLS 2023	[Code]
	Scale-Adaptive Diffusion Model for Complex Sketch Synthesis	ICLR 2024
	StrokeFusion: Vector Sketch Generation via Joint Stroke-UDF Encoding and Latent Sequence Diffusion	arxiv 25.03

---

2) Photo-to-sketch

• vector image generation

Data type	Paper	Source	Code/Project Link
Facial	Style and abstraction in portrait sketching	TOG 2013
Instance-level	Free-Hand Sketch Synthesis with Deformable Stroke Models	IJCV 2017	[Project] [code]
	Learning to Sketch with Shortcut Cycle Consistency	CVPR 2018	[Code1] [Code2]
	Learning Deep Sketch Abstraction	CVPR 2018
	CLIPasso: Semantically-Aware Object Sketching	SIGGRAPH 2022	[Project] [Code]
	SwiftSketch: A Diffusion Model for Image-to-Vector Sketch Generation	SIGGRAPH 2025	[Project] [Code]
Scene-level	Sketch Generation with Drawing Process Guided by Vector Flow and Grayscale	AAAI 2021	[Code]
	CLIPascene: Scene Sketching with Different Types and Levels of Abstraction	ICCV 2023	[Project] [Code]
	Learning Realistic Sketching: A Dual-agent Reinforcement Learning Approach	ACM MM 2024
Technical Drawings	Deep Vectorization of Technical Drawings	ECCV 2020	[Project] [code]

• raster image generation

Type	Paper	Source	Code/Project Link
Facial	ArtLine	Online demo	[Code]
	APDrawingGAN: Generating Artistic Portrait Drawings from Face Photos with Hierarchical GANs	CVPR 2019	[Code] [Demo]
	Unpaired Portrait Drawing Generation via Asymmetric Cycle Mapping	CVPR 2020	[Code]
	Line Drawings for Face Portraits From Photos Using Global and Local Structure Based GANs	TPAMI 2020	[Code]
	Quality Metric Guided Portrait Line Drawing Generation from Unpaired Training Data	TPAMI 2022	[Code]
	Human-Inspired Facial Sketch Synthesis with Dynamic Adaptation	ICCV 2023	[Code]
	Stylized Face Sketch Extraction via Generative Prior with Limited Data	EG 2024	[Code] [Project]
Instance-level	Deep Factorised Inverse-Sketching	ECCV 2018
	Synthesizing human-like sketches from natural images using a conditional convolutional decoder	WACV 2020	[Code]
Anime	sketchKeras	online demo	[Code]
	LineDistiller	online demo	[Code]
	Anime2Sketch	online demo	[Code]
	Reference Based Sketch Extraction via Attention Mechanism	SIGGRAPH Asia 2022	[Code]
Scene-level	Photo-Sketching: Inferring Contour Drawings from Images	WACV 2019	[Code] [Project]
	Learning to generate line drawings that convey geometry and semantics	CVPR 2022	[Code] [Project]
Arbitrary	Im2Pencil: Controllable Pencil Illustration From Photographs	CVPR 2019	[Code]
	Semi-supervised reference-based sketch extraction using a contrastive learning	SIGGRAPH 2023	[Code] [Project]
	MixSA: Training-free Reference-based Sketch Extraction via Mixture-of-Self-Attention	TVCG 2024
	One-Shot Reference-based Structure-Aware Image to Sketch Synthesis	AAAI 2025	[Code]

---

3) Text/Attribute-to-sketch

• raster image generation

Type	Paper	Source	Code/Project Link
Facial	Text2Sketch: Learning Face Sketch from Facial Attribute Text	ICIP 2018
Scene-level	Sketchforme: Composing Sketched Scenes from Text Descriptions for Interactive Applications	UIST 2019
Scene-level	Scones: Towards Conversational Authoring of Sketches	IUI 2020

• vector image generation

Type	Paper	Source	Code/Project Link
Arbitrary	Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts	arxiv 21.09	[code] [project]
Arbitrary	CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders	NeurIPS 2022	[code]
Arbitrary	StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translation	IJCAI 2022	[code]
SVG	VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models	CVPR 2023	[project]
Arbitrary	SketchDreamer: Interactive Text-Augmented Creative Sketch Ideation	BMVC 2023	[code]
Arbitrary	DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models	NeurIPS 2023	[project] [code]
Icon	IconShop: Text-Based Vector Icon Synthesis with Autoregressive Transformers	SIGGRAPH Asia 2023	[project]
SVG	Text-Guided Vector Graphics Customization	SIGGRAPH Asia 2023	[project]
Arbitrary	Text-based Vector Sketch Editing with Image Editing Diffusion Prior	ICME 2024	[code]
SVG	SVGDreamer: Text Guided SVG Generation with Diffusion Model	CVPR 2024	[project] [code]
SVG	SVGDreamer++: Advancing Editability and Diversity in Text-Guided SVG Generation	arxiv 24.11	[code]
SVG	NIVeL: Neural Implicit Vector Layers for Text-to-Vector Generation	CVPR 2024	[project]
SVG	Text-to-Vector Generation with Neural Path Representation	SIGGRAPH 2024	[project]
Arbitrary	SVGCraft: Beyond Single Object Text-to-SVG Synthesis with Comprehensive Canvas Layout	arxiv 24.04	[code]
SVG	VectorPainter: A Novel Approach to Stylized Vector Graphics Synthesis with Vectorized Strokes	ICME 2025	[code] [webpage]
SVG	Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models	CVPR 2025	[webpage]
Arbitrary	SketchAgent: Language-Driven Sequential Sketch Generation	CVPR 2025	[code] [webpage]
SVG	SVGBuilder: Component-Based Colored SVG Generation with Text-Guided Autoregressive Transformers	AAAI 2025	[webpage]
SVG	SVGFusion: Scalable Text-to-SVG Generation via Vector Space Diffusion	arxiv 24.12	[webpage] [code]
SVG	Empowering LLMs to Understand and Generate Complex Vector Graphics	CVPR 2025	[webpage] [code]
SVG	StarVector: Generating Scalable Vector Graphics Code from Images and Text	CVPR 2025	[webpage] [code]
SVG	NeuralSVG: An Implicit Representation for Text-to-Vector Generation	ICCV 2025	[webpage] [code]
SVG	LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformer	arxiv 25.02	[code]
SVG	OmniSVG: A Unified Scalable Vector Graphics Generation Model	arxiv 25.04	[code] [webpage]
SVG	Style Customization of Text-to-Vector Generation with Image Diffusion Priors	SIGGRAPH 2025	[code] [webpage]
SVG	LayerPeeler: Autoregressive Peeling for Layer-wise Image Vectorization	arxiv 25.05	[webpage]
SVG	Reason-SVG: Hybrid Reward RL for Aha-Moments in Vector Graphics Generation	arxiv 25.05
SVG	SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation	arxiv 25.06	[webpage] [code]

---

4) 3D shape-to-sketch

Paper	Source	Code/Project Link
DeepShapeSketch : Generating hand drawing sketches from 3D objects	IJCNN 2019
Neural Contours: Learning to Draw Lines from 3D Shapes	CVPR 2020	[project] [code]
Cloud2Curve: Generation and Vectorization of Parametric Sketches	CVPR 2021	[project]
Neural Strokes: Stylized Line Drawing of 3D Shapes	ICCV 2021	[code]
Learning a Style Space for Interactive Line Drawing Synthesis from Animated 3D Models	PG 2022
CAD2Sketch: Generating Concept Sketches from CAD Sequences	SIGGRAPH Asia 2022	[project]

---

5) Art-to-sketch

Here we list sketch synthesis based on other image types, like Manga, line art, rough sketch, etc.

a) Line art

• Vector-to-Vector

Paper	Source	Code/Project Link	Deep learning?
Closure-aware Sketch Simplification	SIGGRAPH Asia 2015	[Project]	No
StrokeAggregator: Consolidating Raw Sketches into Artist-Intended Curve Drawings	SIGGRAPH 2018	[Project]	No
StrokeStrip: Joint Parameterization and Fitting of Stroke Clusters	SIGGRAPH 2021	[Project] [code]	No
StripMaker: Perception-driven Learned Vector Sketch Consolidation	SIGGRAPH 2023		No
Region-Aware Simplification and Stylization of 3D Line Drawings	EG 2024		No

• Raster-to-Vector (a.k.a. Vectorization)

Paper	Source	Code/Project Link	Deep learning?
Topology-Driven Vectorization of Clean Line Drawings	TOG 2013		No
Fidelity vs. Simplicity: a Global Approach to Line Drawing Vectorization	SIGGRAPH 2016	[Project]	No
A Delaunay triangulation based approach for cleaning rough sketches	C&G 2018	[Code]	No
Semantic Segmentation for Line Drawing Vectorization Using Neural Networks	EG 2018	[project] [code]	Yes
Deep Line Drawing Vectorization via Line Subdivision and Topology Reconstruction	PG 2019		Yes
Inertia-based Fast Vectorization of Line Drawings	PG 2019		No
Vectorization of Line Drawings via Polyvector Fields	TOG 2019	[Code]	No
Integer-Grid Sketch Simplification and Vectorization	SGP 2020	[Project] [Code]	No
Deep Vectorization of Technical Drawings	ECCV 2020	[project] [code]	Yes
General Virtual Sketching Framework for Vector Line Art	SIGGRAPH 2021	[project] [code]	Yes
Keypoint-Driven Line Drawing Vectorization via PolyVector Flow	SIGGRAPH Asia 2021	[project]	Hybrid
End-to-end Line Drawing Vectorization	AAAI 2022		Yes
Vectorizing Line Drawings of Arbitrary Thickness via Boundary-based Topology Reconstruction	CGF 2022		No
Singularity-Free Frame Fields for Line Drawing Vectorization	SGP 2023	[code]	No
Deep Sketch Vectorization via Implicit Surface Extraction	SIGGRAPH 2024	[project] [code]	Hybrid

b) Rough sketch simplification / cleanup

• Datasets and benchmark

Paper	Source	Code/Project Link
A Benchmark for Rough Sketch Cleanup	SIGGRAPH Asia 2020	[Project] [Code]

• Methods

Paper	Source	Code/Project Link
Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup	SIGGRAPH 2016	[Code] [Project]
Mastering Sketching: Adversarial Augmentation for Structured Prediction	SIGGRAPH 2018	[Code] [Project]
Real-Time Data-Driven Interactive Rough Sketch Inking	SIGGRAPH 2018	[Code] [Project]
Perceptual-aware Sketch Simplification Based on Integrated VGG Layers	TVCG 2019

c) Manga (Comics)

Paper	Source	Code/Project Link
Deep extraction of manga structural lines	SIGGRAPH 2017	[Code]
Manga Filling Style Conversion with Screentone Variational Autoencoder	SIGGRAPH Asia 2020	[Project] [Code]
Generating Manga from Illustrations via Mimicking Manga Workflow	CVPR 2021	[Project] [Code]
MangaGAN: Unpaired Photo-to-Manga Translation Based on The Methodology of Manga Drawing	AAAI 2021
MARVEL: Raster Gray-level Manga Vectorization via Primitive-wise Deep Reinforcement Learning	TCSVT 2023
Manga Generation via Layout-controllable Diffusion	arxiv 24.12	[code] [webpage]
DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generation	CVPR 2025	[code] [webpage]

3. Vector Graphics Generation (2D)

Here we focus on learning-based vector graphics generation without depending on vector training data, and traditional vectorization algorithms.

• Learning with external black-box (non-differentiable) rendering simulator

Paper	Source	Code/Project Link
Synthesizing Programs for Images using Reinforced Adversarial Learning	ICML 2018	[Code]
Unsupervised Doodling and Painting with Improved SPIRAL	arxiv 1910	[Project]

• Learning with built-in differentiable rendering module

Paper	Source	Code/Project Link
Layered Image Vectorization via Semantic Simplification	arxiv 24.06	[webpage] [code]
Inverse Painting: Reconstructing The Painting Process	SIGGRAPH Asia 2024	[code]
ProcessPainter: Learning to draw from sequence data	SIGGRAPH Asia 2024	[code]
Segmentation-guided Layer-wise Image Vectorization with Gradient Fills	ECCV 2024
Towards High-fidelity Artistic Image Vectorization via Texture-Encapsulated Shape Parameterization	CVPR 2024
SuperSVG: Superpixel-based Scalable Vector Graphics Synthesis	CVPR 2024	[code]
Vector Graphics Generation via Mutually Impulsed Dual-domain Diffusion	CVPR 2024
Optimize and Reduce: A Top-Down Approach for Image Vectorization	AAAI 2024	[code]
Segmentation-Based Parametric Painting	arxiv 23.11	[code] [project]
Editable Image Geometric Abstraction via Neural Primitive Assembly	ICCV 2023
Stroke-based Neural Painting and Stylization with Dynamically Predicted Painting Region	ACM MM 2023	[code]
Intelli-Paint: Towards Developing More Human-Intelligible Painting Agents	ECCV 2022	[project]
Towards Layer-wise Image Vectorization	CVPR 2022	[code] [project]
Paint Transformer: Feed Forward Neural Painting with Stroke Prediction	ICCV 2021	[code]
Combining Semantic Guidance and Deep Reinforcement Learning For Generating Human Level Paintings	CVPR 2021	[code]
Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes	CVPR 2021	[code]
Im2Vec: Synthesizing Vector Graphics without Vector Supervision	CVPR 2021	[Project] [code]
Stylized Neural Painting	CVPR 2021	[Code] [project]
Painting Many Pasts: Synthesizing Time Lapse Videos of Paintings	CVPR 2020	[Code] [project]
Learning to Paint With Model-based Deep Reinforcement Learning	ICCV 2019	[code]
Strokenet: A neural painting environment	ICLR 2019	[Code]
Neural Painters: A learned differentiable constraint for generating brushstroke paintings	arxiv 1904	[Code]
Learning to Sketch with Deep Q Networks and Demonstrated Strokes	arxiv 1810
Unsupervised Image to Sequence Translation with Canvas-Drawer Networks	arxiv 1809

• Vectorization

Paper	Source	Code/Project Link
Depixelizing pixel art	SIGGRAPH 2011
Perception-Driven Semi-Structured Boundary Vectorization	SIGGRAPH 2018	[Webpage]
PolyFit: Perception-aligned Vectorization of Raster Clip-art via Intermediate Polygonal Fitting	SIGGRAPH 2020	[Webpage] [Code]
ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis	TVCG 2021
TCB-Spline-Based Image Vectorization	TOG 2022
Image vectorization and editing via linear gradient layer decomposition	SIGGRAPH 2023
Image Vectorization via Gradient Reconstruction	EG 2025

4. Vector Graphics Generation (3D)

Paper	Source	Code/Project Link
3Doodle: Compact Abstraction of Objects with 3D Strokes	SIGGRAPH 2024	[code]
Diff3DS: Generating View-Consistent 3D Sketch via Differentiable Curve Rendering	ICLR 2025	[webpage]
Recovering Dynamic 3D Sketches from Videos	CVPR 2025	[webpage]
Empowering Vector Graphics with Consistently Arbitrary Viewing and View-dependent Visibility	CVPR 2025	[code]
ViewCraft3D: High-Fidelity and View-Consistent 3D Vector Graphics Synthesis	arxiv 25.05	[webpage]

• Wire Art

Paper	Source	Code/Project Link
Wired Perspectives: Multi-View Wire Art Embraces Generative AI	CVPR 2024	[code] [webpage]
Fabricable 3D Wire Art	SIGGRAPH 2024
WIR3D: Visually-Informed and Geometry-Aware 3D Shape Abstraction	arxiv 25.05	[webpage] [code]