Phenomena such as fluidity, diffusion, and absorption are often observed when a picture is drawn. As paints and solvents are types of liquids, fluidity plays an important role in determining the outcome of any drawing. When two fluid components are adjacent, diffusion caused by the different concentrations always occurs between them. As time elapses, the liquid is absorbed to leave pigments and a large portion of binder on the paper. Therefore, it is important to consider these characteristics, observed during the drawing of a picture, in the design of a paint simulation system. However, there has not been much effort to consider these factors simultaneously in the field of computer graphics research.
Many previous researches studies have focused on the representation of specific types of paint effects [1-4] in the domain of non-photorealistic rendering (NPR). These studies have tended to consider each paint effect in isolation. However, paint effects are naturally generated by complicated physical laws in the real world. Moreover, many works rely on flat representation [5, 6]. Although the final result of painting lies on an almost flat surface, the paint and solvent are not flat during the process of drawing. Therefore, the creation of realistic painting effects requires the incorporation of three-dimensional information. Addi-tionally, previous works have usually defined specific types of painting, such as oil painting or water painting, and have represented them differently. However, there are interactions that are universal, regardless of the type of painting, between the elements of paint and paper.
There have been many research efforts to develop physically correct fluid dynamics [7, 8]. Some studies specifically deal with miscible simulation [9-11]. The approaches in these studies have successfully modeled the mixing behavior of multiple fluids. However, such methods do not account for the relationship or the interaction between painting components. Moreover, the process of absorption of a fluid by a paper has been largely neglected, despite the importance of interaction with paper in the creation of paint effects [3, 5, 6].
In this paper, we propose a new paint simulation method that takes the characteristics of painting process into account. First, we separate the component materials of painting. These include the binder, the pigment, and the solvent. Here, the binder and the pigment can be combined into paint. Second, we employ smoothed-particle hydrodynamics (SPH) to govern the motion of the paint and the solvent. Third, we include the pigment diffusion effect of the paint into the solvent. This diffusion is caused by the differences in pigment concentration between liquids. The mass transfer method is adopted for the modeling of the diffusion effect. Fourth, the absorption of the fluid into the paper is incorporated. Depending on the degree of absorption, the resulting painting can be flat or thick, which is one of the determining characteristics of painting, that is, of water paint or oil paint. In Figure 1 that is a sample image created by our paint system, the important features of paint are naturally shown.
2 PREVIOUS WORK
In the field of NPR, there have been several studies that have considered the characteristics of specific fluid simulation. To simulate ink dispersion on a piece of absorbent Chinese paper for the purpose of art creation, Chu and Tai presented a fluid flow model on the basis of the lattice Boltzmann equation . On top of implicit physics simulation, they coupled an image-based approach to render high quality outputs. Similarly, Sun et al.  proposed a method for artistic Chinese drawing. They calculated the amount of diffusion of water–ink particles according to Brownian motion and the capillary effect. The study of Curtis et al.  is another combination of NPR and fluid simulation. They defined actual water color effects such as edge darkening, flow effects, and glazing, to implement the corresponding artistic effects. For the simulation of watery paints, Laerhoven and Reeth  utilized physically based but heuristic rules in a layered canvas design. Baxter et al.  developed a viscous paint model that can be used in an interactive painting system; this system demonstrated a thick and impasto-like style of painting.
One of the main characteristics of painting is diffusion between mixing substances. To handle miscible and immiscible fluids, Kang et al.  combined a distance function and a volume fraction. Park et al.  adopted the Cahn–Hilliard equation to trace the evolution of a diffused concentration field on the basis of an advanced lattice Boltzmann approach. Liu et al.  presented a liquid–liquid mixing simulation, which is known as LLSPH. A mass transfer mechanism employed by Shin et al.  handled diffusion of liquids at the interface of those fluids. Especially, the use of Darcy's law reproduced the viscous fingering effect, which generated fractal-like shapes that are typically observed in ink diffusion. Similarly, Cha et al.  reproduced the viscous fingering effect using a diffusion-limited aggregation model. Recently, Gregson et al.  reproduced turbulent fluid mixing behaviors by capturing a fluorescing fluid mixed with a transparent fluid. Huber et al.  simulated wetting and wicking appearances in fabrics on the basis of Fick's Law for a liquid diffusion state. As Fick's Law is one of the most important equations in the field of mass transfer, many chemical physics papers have utilized the theory [19-21].
Fluid simulation is one of the popular research areas in computer graphics. SPH is the main branch of Lagrangian approaches in determining the motion of fluids. SPH was mainly utilized in astronomy [22, 23] first. Desbrun et al.  introduced SPH into the Computer Graphics (CG) field to animate highly deformable bodies. Muller et al.  developed SPH further for water simulation with free surfaces. Becker and Teschner  suggested the weakly compressible SPH method, which avoids solving the Poisson equation. Adams et al.  used adaptively sampled particles to dynamically reduce the simulation speed. Solenthaler and Pajarola  enforced the incompressibility of SPH using a prediction correction scheme, which is known as PCISPH.
Viscoelastic fluid dynamics has also been actively developed. Carlson et al.  developed a Eulerian solver for highly viscous liquids and melting objects by modifying the marker-and-cell algorithm. Goktekin et al.  modeled the viscous, elastic, and plastic behaviors of fluids by including an additional elastic stress term in the basic Navier–Stokes equations. Steele et al.  presented a particle-based method for simulating viscous liquids. There are approaches that utilize springs to create viscoelasticity. Exchanging radial impulses determined by differences in particle velocity produces a viscoelastic effect . Chang et al.  presented a particle-based method for viscoelastic fluid simulation by introducing an additional elastic stress term. Batty et al.  introduced the dynamics of thin sheets of incompressible viscous liquids. They demonstrated several interesting viscous sheet behaviors, including stretching, buckling, sagging, and wrinkling.
3 OBSERVATION IN REAL WORLD PAINTING
We observed the process of real painting before designing a paint simulation system. To paint a picture, paints and solvent are first prepared. The paint typically requires one type of solvent, except in special cases such as the creation of a marbling effect.
Paint is composed of pigments and a binder. Pigments are tiny granular particles that determine the color of the paint. These particles are combined by the binder. The binder is a colorless material and is commonly referred to as a vehicle that binds the pigments together into paint. The properties of the binder determine the characteristics of the paint such as type, level of hydrophilicity, exterior durability, flexibility, and toughness. Before the invention of commercial tube-type paint, artists produced a paint by mixing the pigments and the binder by themselves. (Figure 2). The mixture can be decomposed into the original constituents and move individually if certain conditions are met. During drawing, decomposition always arises when the paint meets the solvent.
The solvent is another liquid element that carries non-volatile components. For example, water is the main diluent for water borne paints. Oil-based paints require petroleum distillate, alcohol, or other various types of oils as the main diluent. The important role of the solvent is to dilute the stiff paint. When the paint is mixed with the solvent, the connection of the pigments and the binder becomes loose. The pigments are separated from the paint, they flow according to the movement of the solvent, and shuttle on the surface of a paper or a canvas.
Our observations identified common physical phenomena that occurred during the process of real painting. The first common phenomenon is fluidity. As the paint and the solvent are liquids, they are moved by fluid dynamics. However, different from the solvent, the paint is a viscoelastic material . Therefore, the motion of the paint should be computed accordingly. The second common phenomenon is diffusion. When the paint is mixed with the solvent, the pigments in the paint are diffused into the solvent, driven by the differences in pigment concentration. Figure 3 shows the diffusion process observed during real painting. The third common phenomenon is absorption. As time elapses, the binder and the solvent are absorbed into the paper, and most of the pigment remains on the surface. This happens because the pigment particles are bigger than the diameter of a pore in the paper. However, as the absorbability of the binder is smaller than that of the solvent, some of the binder is also left on the paper. These remaining amounts of binder give the painting a rough texture. In the next section, we explain our paint simulation more technically.
4 MODELING OF PAINT SIMULATION
We designed our paint simulation to reflect the real process of painting. We utilize the PCISPH method for basic fluid simulation. We also include the concept of viscoelasticity for the paint movement. When paint is mixed with solvent, the binding between the binder and the pigments becomes loose, and the pigments diffuse from the paint to the solvent. To handle the mass transfer phenomenon, we employ Fick's law. Each fluid particle in our system contains the individual pigment concentration value. At the initial stage, the paint particles have the initial value of pigment concentration, and the solvent particles have their values set to zero. Following the diffusion law, the pigment concentration is updated. In the last step, we calculate the penetration distance, which is caused by absorption. Figure 4 provides a graphical illustration of our system.
The SPH interpolates the variable quantities carried by each particle. The value of a particular quantity A at any position r can be evaluated according to the weighted sum of the contributions from a neighbor particle j:
where j iterates over all neighboring particles, m is the mass of the particle, ρj is the particle density, rj is the position, and W(r − rj,h) is the smoothing kernel with radius h.
4.1.1 Governing Equation
To simulate the motion of the paint, we add the viscoelastic force [29, 32] to the SPH momentum equation. The governing equation is modified as follows:
where u is the velocity field, ρ is the density, p is the pressure, μv is the viscosity, μe is the elasticity of the fluid, ε is the elastic strain tensor, and f represents the external forces. Because the particles move with the fluid, the convective term (u ⋅ ∇ )u is meaningless and can be omitted from the equation. Equation (1) is then applied to the remaining terms of Equation (2).
We apply Equation (5) to the binder particles because paint is a classic example of a viscoelastic fluid. Figure 5 shows the different motions of fluids moved by different forces. Red paint is affected by viscoelastic force, whereas green paint is moved by viscous force only. Yellow paint shows the results of excluding both forces. For an additional description of the computing of the elastic strain ε in Equation (5), see Appendix A.
The pigments in the paint are diffused in the solvent by mass transfer. Unlike momentum transfer, which is driven by velocities, mass transfer is mainly driven by concentration differences. Therefore, paint diffusion can occur even in a stationary fluid. The transport of one constituent in a region of higher concentration to one of lower concentration minimizes the concentration difference within a system.
To handle the mass transfer phenomenon, we employ Fick's second law as follows:
To utilize Fick's second law, the system has to be under conditions of no fluid flow and no chemical creation. Our system satisfies these conditions. The detail derivation is written in Appendix B. We convert the equation to SPH form as follows:
The final concentration of each pigment in each particle is updated at each time step.
4.2.1 Diffusion Coefficient
The speed of diffusion can vary depending on the media. For instance, as a solvent usually has a higher diffusion coefficient than a binder, the pigments are diffused faster in the solvent. Furthermore, water-based solvents have higher diffusivity than oil-based solvents. In Equation (6), D is the diffusivity that controls the diffusion speed.
During the diffusion, the pigments spread out irregularly in the fluid. A constant value of diffusivity may be enough for ideal diffusion. However, as shown in Figure 3, the diffusion of real paint requires a variable coefficient for the diffusivity to achieve realistic results. We utilize the curvature-based diffusion coefficient. We calculate the curvature of each pigment concentration and apply it to the diffusivity as follows:
To control the magnitude of the diffusivity, we add a weight value, ωD.
When paint and solvent make contact with paper, they are absorbed into the paper. Many studies have dealt with highly absorptive materials such as Xuan paper, which is a very thin and highly absorbent Chinese paper, or fabric [5, 18]. These types of paper handle absorption and diffusion simultaneously within one material although the fluid spreads out widely. However, as shown in Figure 3, the diffusion usually occurs in the liquid, and the absorption occurs only in the humidified area of the paper. Therefore, we divide the two stages into the pigment diffusion into the liquid and the absorption of the liquid by the paper.
To calculate the penetration of liquid into a porous material such as paper, we adopt the Lucas–Washburn equation :
The magnitude of the velocity of penetration, in length of the capillary, l, is calculated by using a small capillary of radius, rc, the hydrostatic pressure, Ph, the capillary pressure, Pc, and the viscosity, μ. For the capillary pressure, Pc, we use
where γ is the surface tension of the liquid and θ is the angle of contact.
As Equation (9) computes the velocity explicitly, it can be integrated well with our SPH system. Moreover, the capillary radius controls the paper absorptiveness. Canvas, which is used for oil painting, has lower absorptiveness than paper, which is used for watercolor painting. Therefore, our unified system can generate different painting effects such as those of oil painting, which has thick textures with many remaining paint particles, or those of watercolor painting, which has thin and flat textures.
We develop a unified rendering algorithm to express both the surface of the liquid particles and the pigment concentrations. Although we have divided the fluids into two types, binder and solvent, as the binder and the solvent are miscible fluids, they only interface toward air. The individual pigment concentration of each particle is represented as the density. We render these value at the same time.
5.1 Liquid Surface Reconstruction
There have been many studies that have addressed the construction of a surface from particles [25, 35-37]. We utilize Zhu and Bridson's surface reconstruction method. A signed distance field can be computed from the particles. The implicit function is as follows:
where is the weighted average of the nearby particle positions and is the weighted average of the radii. We exclude the absorbed particles in this computation as only the remaining liquid should be visualized.
5.2 Pigment Rendering
We express pigment concentrations as pigment densities. This is because it is very hard to visualize individual particles, whose typical size is about 0.001 ∼ 0.002 mm. Specifically, we utilize a grid-based density field. In each cell, the pigment concentration is gathered within the radius . The values of the pigment concentration are rendered.
After fluid particles are absorbed into paper, many particles of pigment are left on the paper. We allow the paper to have pigment density when the liquid particles are in contact with the paper. Finally, the geometry of liquid and the pigment density are rendered simultaneously using mental ray which is 3D rendering software plugged in Autodesk Maya.
We implemented the proposed simulation framework in C++ with OpenMP to parallelize the computation. The simulations were run on an Intel Xeon 2.80GHz CPU with 8GB memory for the creation of all the scenes. See the accompanying video for an animation of the results.
Figure 6 shows an image drawn as an acrylic painting similar to the work of Jackson Pollock, who was an influential American painter. The painting was also studied by Lee et al.  from an NPR point of view. As Pollock's technique was to pour and drip paint, known as action painting, three-dimensional information must be considered. Our paint simulation can generate a dynamic fluid motion occurring in a three-dimensional space. In addition, the incorporation of the viscoelastic force makes the particles move like acrylic paints. We set the value of μv equal to 80.0 and μe equal to 1000.0. Because of the high values of viscosity and elasticity, the simulation time step was small, 0.0002. The simulation took 57seconds per frame on average when computing 29263 particles for paints and 20716 particles for paper. Extracting surfaces and pigment concentration fields took an additional amount of time of 61seconds per frame on average.
Our system effectively produces various types of painting. Figure 7 shows an image that appears to be a watercolor painting. Utilizing a solvent that has a low viscosity makes the simulation results thin and flat. Furthermore, the pigment concentration diffuses faster than the case shown in Figure 6 because the solvent has higher diffusivity than that of the binder. The diffusivity weight of the solvent is 0.038. Absorption is also applied because the paper has high absorptiveness compared with that of canvas used for oil or acrylic paintings. For the Jackson Pollock painting, we set rc equal to zero, which means no absorption. For the flower image, we set rc equal to 2.5 e − 5. We tabulated the parameters used in Figures 6 and 7 in Table 1.
Table 1. Parameters used for the two examples.
2.5 e − 4
Figure 8 shows a series of images in which the paints of the CASA logo fall into a solvent sequentially. This simulation was performed with 9006 paint particles, 37180 solvent particles, and 23028 paper particles. As the paints have only viscous force, μv = 40, the motion in this image is smoother than that in the image in Figure 6. The diffusivity weight value is set to 0.01 for the solvent and 0.002 for the binder. The simulation took 130seconds per frame. Setting rc equal to 2.5 e − 4 for the solvent causing the absorption.
7 CONCLUSION AND DISCUSSION
In this paper, we have attempted to combine the NPR features of painting with fluid simulation. On the basis of the observation of a real painting, we have taken into account the components of painting, such as the pigment, binder, solvent, and paper, in the design of our general painting system. The incorporation of viscoelasticity into the SPH fluid simulation faithfully reflects the property of fluidity of painting. The application of Fick's law handles the diffusion caused by different concentrations. The absorption is also included to model the interaction of paint with paper. This approach allows our paint simulation to generate various types of painting.
In the future, we would like to include the inherent properties of paper or canvas. Information on interaction with paper will allow for more detailed and natural expressions of paint effects. Moreover, information on absorption that consider the paper characteristics will create more realistic painting simulation. Interactive applications such as brush interfaces are suitable for our system. A new interface that allows easy drawing in three-dimensional space may be very useful in practice.
We appreciate the anonymous reviewer's valuable comments. We are grateful to Shiguang Liu, Jinho Park, and Hyunggoog Seo for discussions and help. This work was supported by KOCCA/MCST (R2010050008, software development for 2-D to 3-D stereoscopic image conversion) and MKE (10040959, development of compositing software supporting 4K images).
We explain the particle-based elastic strain tensor used in Equation (5). The total strain tensor is decomposed into a plastic and an elastic component, and the elastic strain is evaluated by the integration of time derivative:
The total strain rate is given by
where ∇ u is computed as
To determine the occurrence of the plasticity, von Mises's criterion is used.
where α is the material's elastic decay rate, γ is the yield point, and ∥ ε ′ ∥ f is the Frobenius norm of the elastic strain deviation tensor ε ′ . Finally, the elastic strain tensor is
We present Fick's second law from the continuity equation. The continuity equation for mass transfer can be described as follows:
where c is the local concentration of chemical species quantity and R is the creation or the termination of the quantity by a chemical reaction. The molecular flux, N, is given by Fick's first law :
where D is the diffusivity for a specific component and u is the velocity. Plugging Equation (B.2) into Equation (B.1) yields
We assume that there is no chemical creation, R = 0. Note that ∇ ⋅ cu = c ∇ ⋅ u + u ⋅ ∇ and all of the particles conserve the mass, ∇ ⋅ u = 0. The formulation becomes ∇ ⋅ cu = u ⋅ ∇ . The advection–diffusion equation is as follows: :
As the pigment concentration is advected with the SPH liquid particles in our system, the advection term can be omitted. Therefore, Equation (B.4) becomes as Fick's second law,
Mi You is currently a Ph.D. student in the Graduate School of Culture Technology at KAIST, where she received M.S. degree in 2009. She received Fine Arts B.F.A. degree and Digital Animation B.F.A. degree in 2007 from Hongik University, Korea. Her research interest includes fluid simulation, non-photorealistic rendering, and realistic rendering.
Taekwon Jang received his B.S. degree from Ajou University in 2002. He has been a game programmer at several game companies from 2002 to 2005. He earned his M.S. and Ph.D. degree in Culture Technology from KAIST in 2007 and 2013, respectively.
Seunghoon Cha is currently a Ph.D. student in the Graduate School of Culture Technology at KAIST, where he received his M.S. degree in 2012. He received his B.S. degree in Computer Science and Engineering in 2010 from Dongguk University, Korea. His research interest includes scientific visualization, physically-based simulation, and parallel processing.
Jihwan Kim is currently a M.S student in the Graduate School of Culture Technology at KAIST. He received his B.S. degree of Science in Electrical Engineering in 2012 from Ajou University, Korea. His research interest includes scientific visualization, image processing and physically-based simulation.
Junyong Noh is an Associate Professor in the Graduate School of Culture Technology at KAIST. He earned his computer science Ph.D. from the University of Southern California (USC) in 2002 where his research focus was on facial modeling and animation. His research relates to human facial modeling/animation, character animation, fluid simulation, and stereoscopic visualization. Prior to his academic career, he was a graphics scientist at a Hollywood visual effects company, Rhythm and Hues Studios. He performed R&D for movie post productions including Superman Returns, Happy Feet, The Chronicles of Narnia, Garfield, Around the world in 80 days, and The Chronicles of Riddick. He has been doing consulting for or collaborative work with many visual effects and animation companies.