Online 3D model reconstruction and virtual display for garment

An online garment model reconstruction method is proposed using the axial deformation to simulate the various garment models to display the wearing effect under different sizes. The garment model is classified into three categories according to the structural features. The main sections and key points are determined based on the existing dimensional features. Inverse operation and radial weight are used in the garment deformation. Texture mapping is applied to the model reconstruction result to obtain a new model with clothing patterns. An online system is developed and garment models with multiple scales are applied to adapt to various devices. The reconstruction results are compared and analyzed, which proves that the method proposed can realize real-time 3D reconstruction of garment models with the changing sizes on the web page.


Introduction
With a remarkable rise in people's living standards and aesthetic understanding, mass-produced garments have become unable to meet the individual needs of consumers. The customized garment has broken new ground for the fashion industry. All garment designs, from size to fabrics including style, color, body shape material, and so on, should be determined by the individual requirements of consumers. However, personalized customization will be restricted by some factors in the process of implementation. The fact is that no part is the same length on the different bodies even if they have the same height. Thus, it is inevitable to have a face-to-face measurement for size details. And it is time-consuming to modified the garment until satisfactory results are obtained. Therefore, it is highly necessary to realize the remotely operated garment size design in an efficient and real-time way.
Numerous researchers have studied garment modeling. The shapes of garments dressed in human change with body shape and postures. So many studies related to garment modeling are supported by human body driving. Detailed grids were arranged on the body surface based on the laser-scanned 3D unorganized points. The body model is deformed with the input body sizes. 1 Initial body feature parameters covering section rings and contour lines were extracted and adjusted and the garment framework was obtained by surface interpolating. 2 Chen et al. 3 proposed the main control points and auxiliary control points of the body sections and the constraint relationships among surface points, control points, and central curves were established. The quadrilateral surface interpolation generation method was used to generate the clothing models. A radial basis function (RBF) was proposed by Dong et al. 4 to transform the control points on the human model. And the interpolation function was used to calculate the displacement of each point so that the parametric human body modeling was realized. The feature size and control points of the human body needed to be extracted in this method. Similarly, the garment modeling method via anthropometry was presented by Sun and Xu. 5 Cubic spline interpolation curves in human body axial deformation were used on parameterized mannequin modeling, which made the computational process more complicated. This method was optimized by Peng et al., 6 in which the axis curves were redefined as axis lines to improve computing efficiency. Nevertheless, the variable step length still took quite some time. Analogously, the variable step length contour extraction was also applied in the parametric human modeling. 7 Three main plains from three perspectives were defined and the body was divided into four parts in total. The main body and the four limbs were analyzed separately and the surface of the adaptive mannequin was generated according to the given size. Four key points of the section contour were defined by Wang. 8 When the body circumference changed, the deformation of front and rear sides were not the same and different ratios were given to four points and other points were calculated by angle interpolation. The researches above can change the size effectively, which provided a good foundation for garment modeling. But the common features of them are that the modeling method is applicable for the whole body and the design of the tight clothing rather than the daily wearing. And although the size of the human body has been changed, the garment mesh model needs to be constructed primitively, which increases the complexity.
Given the research for garment modeling, mapping from 2D (2-dimensional) pattern to 3D (3-dimensional) model was used in basic garment pattern generation. 9 The garment designed in this paper fitted the body but the style was single whose size and fashion look were unable to be changed. The initial garment model was built by constructing the layered grid model. The multi-layer and hybrid dimension drove by several factors such as size, human body, and garment restraint was introduced to treat the garment model deformation 10 . Both of the methods above used 2D to 3D mixed dimensional mapping but the mesh model needed to be constructed which is complex. Vertical proportions were considered in garment modeling. 11 The vertical length of the given garment model was measured and each segment was assigned by a vertical scale factor. In the same way, the horizontal multiplication factors were assigned to the horizontal circumference for size deformation. The new garment model was generated by adjusting those two scale factors. The actual size of the garment was measured and the comparison between the model and the actual garment was taken, but the detailed modeling method was not given in this research. Price and Zamkoff 12 and Lim and Istook 13 considered the different garment types that were taken into consideration when making garments of different sizes. In this paper model reconstruction is used for the garment model deformation with different styles based on size measurement. Firstly, the section information of key parts is obtained from the given garment model and the control points are calculated. The connecting lines of adjacent control points are defined as the center axis, whose function is to control the basic length of each part. Then mapping constraints between surface points and the center axis are established and the new surface points after deformation. The weight factor is introduced into the calculation of the circumference deformation for the sake of smooth surface curves. In light of the input actual size, the real-time deformation of the garment model is implemented in an online system.

Structure classification
Loose clothing has obvious drapability, and its shape and fold degree change greatly when wearing, so it is unable to accurately control the size parameters. The drapability of the loose clothes is so obvious that it is impossible to control the size parameters accurately. Distinguish from the loose clothes, the tight clothes are infinitely close to the body when dressed on the human body. The change of clothing size is based on the human body size, so the method proposed in this paper is only suitable for the tight clothing. In this paper, the tight clothes are simply classified into three categories as follows: (1) Single tubular structure (Figure 1(a)). In this category, the garment style is simple, which only has a single tubular without branches. The typical garments include the vest, the skirt, undergarment covering the chest, and so on. (2) Upper branching structure ( Figure 1(b)). The main body of the garment remains the tubular structure but the garment bifurcates under the armpit. Two smaller tubes extend from the upper tubular, representing two sleeves. Most of this type is upper garment such as the T-shirt, the sweater and other clothing with sleeves.
Lower branching structure (Figure 1(c)). Opposite the upper branch structure, two branches are generated under the perineum and the upper part is a single tubular, which is defined as the lower branching structure. The pants are the most common structure that belongs to this type.

Parametric definition
There are mainly the following three steps for garment parameterization. The main feature of circumference and height can be obtained from the analysis of garment model information. The model is layered and the garment sections are defined. The key points are found through the section information and each model segment is associated with two sections upper and lower. The parametric model is acquired based on the model operation.

Section extraction
To facilitate the calculation of key points and size-based deformation, it is important to determine the main parameter sections, which are used for model layering. In the traditional method, the main sections are obtained by searching with the fixed steps or variable steps. The smaller the step is, the longer the time takes. The garment models used in this paper are created by external garment design software in obj format, from which the size information can be got directly. The 3D garment model is set in a spatial coordinate system using WebGL technology, which results in the coordinate unconformity between the external software and the system. Even so, the size proportion of each part remains unchanged. So the height of the main section can be derived from Formula (1).
Where, Y i is the y-coordinate of the i-th section from above down, k i is the ratio of height to the total length of the garment at the i-th section and H is the total length of the garment. i∈ [1, N] and k i ∈[0, 1]. N is the total amount of sections a garment can be defined. y min is the y-coordinate of the lowest point in the WebGL spatial coordinate system. The pants shown in Figure 2 are layered by seven sections. The ratio of each section height to the total length of the model is k i . k 1 = 1, representing the top layer, while the k 5 = 0 of the bottom layer is 0. The ratios of the middle sections change from 1 to 0. The pants are divided into several segments, each of which is associated with two sections as shown in Figure 2(b).

Control points
Intersections of sections and meshes are the basics of control point calculation. Suppose that P 1 P 2 is one side of a triangle as Figure 3 shows, where P 1 (x 1 , y 1 , z 1 ) and P 2 (x 2 , y 2 , z 2 ). Plane α i is the i-th section, whose y-coordinate is Y i . The intersection of P 1 P 2 and α i is P c .
Referring to the similar triangle principle, it can judge whether there is an intersection between the plane and the side. As Figure 3(a) shows, O is the projection of the P 1 on the plane α i . Extending P 1 O to construct two similar triangles ∆P 1 P c O and ∆P 1 P 2 O'. The length of P 1 O is | y 1− Y i | and the length of P 1 O' is | y 1− y 2 |. The ratio λ of those two is shown in Formula (2). If 0 ⩽ λ ⩽ 1, P 1 and P 2 are on each side of α i , which means there is an intersection between the mesh edge and the section. Conversely, if λ < 0 (Figure 3(b)) or λ > 1 (Figure 3(c)), P 1 and P 2 are set on the same side of the α i , there is no intersection. The x-coordinate and z-coordinate of Pc can be derived using the similarity ratio as Formula (3).
In this paper, the triangles are traversed as the index order to judge the intersection. Once the triangle has an intersection with sections, the judgment loop goes on to deal with the next triangle. The advantage of this method is that each triangle is processed only once, which reduces running time.
The geometric center of the section is defined as the control point, which can be calculated as the average of the coordinate on the closed curve. Thus, the control points (geometric center of α i ) are denoted as Mα i in Formula (4).

Initial size
Because of the good coating property of the tight clothing, the body's natural curve is well depicted.
It means that body circumference can be regarded as the garment circumference. The cross section of body is irregular. When calculating the space occupied, the circumference of the convex hull is considered, ignoring those concave parts. However, the disordered triangles lead to the irregular intersections at sections, which need to be reordered as shown in Figure 4. From the top view, a simple orthogonal plane coordinate system x-P 1 -z is established, taking the point with minimum z-coordinate P 1 as the coordinate origin. It is guaranteed that P 1 must belong to the convex hull. Connecting with P 1 separately, all points are sorted anticlockwise by the angle β with the square of the x-axis from small to large. If there are the same angles of two points at least, they are sorted by distance from near to far. Traversing all lines in order, the straight-line Formula (5) can be derived. The coordinates of all the remaining points are substituted into the formula to get the value V n (Formula (6)). If V n = 0, P n is in this line; if V n > 0, P n is above this line while V n < 0, P n is under this line. The nonzero value is put into a judgment set V. If both of positive and negative numbers are included in V, the points are on both sides of the line such as the line P 2 P 3  and P 3 is not the point on the convex hull, which is excluded from the remaining points. If all points in set V are positive or all are negative, they are proved to be on the same side of the line. And both two points of the baseline are on the convex hull such as P 2 and P 4 . The convex hull points are connected in order.
Take the human chest section as an example. Figure 5(a) is the initial cross-section and Figure 5(b) is the convex polygon derived from the calculation method mentioned above. The radial dimension is the space distance between adjacent control points.

Garment deformation
Three different types of clothing can be controlled by several line segments, which are defined as centerlines as Figure 6 shows. The number of control points in the main   on v 2 ur u is the distance between S and O 1 (shown in the second formula in Formula (7)). Through those two simultaneous formulas, t is presented in Formula (8). S can be denoted as the first and the second formula in Formula (9) Based on the projection points calculation, the new point P' after deformation can be derived through inverse calculation. As shown in Figure 7(a), the distance from the surface point to the projection point remains the same as before, such as |P'S'| = |PS|. Therefore, the new point P' = S'−S+P and the result is shown in Formula (10).
Weighting factors need to be introduced into the circumference deformation because the deformation of two sections controlling the same garment segment will not always be the same. As shown in Figure 7(b), the rates of change of α 1 and α 2 are represented as R 1 and R 2 , where R 1 = L 1 /l 1 and R 2 = L 2 /l 2 . l 1 and l 2 are the initial circumferences and L 1 and L 2 are the new circumferences. Weighting factors are obtained by the ratio of SO 1 and SO 2 to O 1 O 2 . Thus, the weighting factor of SO 1 is w 1 = |SO 1 |/|O 1 O 2 | and the weighting factor of SO 1 is w 2 = 1−w 1 . The deformation coefficient can be calculated as ε P = R 1 × w 1 + R 2 × w 2 = w 1 × (R 1 −R 2 ) + R 2 . So the new circumference points are derived from Formula (11).

An online system
An online garment model 3D reconstruction system is carried out based on the web technology in Windows10 with a processor intel core i7-8750H and 8 GB of memory. In VisualStudio2015, a computer programming language named C# is used to do data operations such as data collection and data computation. Data is passed to the frontend page by another programming language, JavaScript, which realizes the interaction and data transmission between browser and server. Figure 8 is an example of the size-inputting page. Using this online system, garment model reconstruction can be implemented without extra software installation. Initial garment models are stored in the cloud in advance and they wait for a call from the clients. Conveniently, the model documents and the software are unnecessary to be carried when the location and equipment are changed. Just open the web page and users can work.

Personalized model reconstruction results
In this paper, pants are taken as an example to illustrate garment deformation. Figure 9 shows the deformation of the same pants with different sizes. The size details are expressed in Table 1. Figure 9(a) is the initial input pants model and Figure 9(a) and (b) are models after deformation from circumference and length separately. For the sake of better results, texture mapping is applied in this paper. Clothing bitmaps are mapped to models with different deformation degrees according to the 3D parameters as shown in Figure 9(d) to (f). Also, texture bitmaps can be replaced following the user's need to manifold models.  Two other examples are taken as Figures 10 and 11 show. Figure 10 shows the size deformation for a T-shirt. The left one is an initial T-shirt model without deformation while it is obvious that the waistline is noticeably smaller. Figure 11 shows the deformation of a skirt. In Figure 11(a), the length of the skirt has changed. And Figure 11(b) shows that the change in length will cause a slight change in circumference. The skirt becomes longer and the hipline is a little smaller.
The method proposed in this paper is beyond the limitations of gender, in other words, using this method can realize the deformation of clothing not only for ladies but also for gentlemen. Figure 12 shows the deformation for a men's vest. Figure 12(a) is the initial vest model and Figure  12(b) is the model contracted at chest and waist. Results express that the method proposed in this paper can realize the deformation in real-time.

Multi-mode selection
In this paper, the browser is a necessary medium to realize the garment model deformation. The garment can be refactored not only on the computer but also on the mobile phone, iPad as well as other mobile devices. Because of the parameter differences among diverse equipment such as the screen size, rendering efficiency and so on, garment models in appropriate sizes are automatically matched to adjust the different equipment. For instance, the garment model with a proportion of 1 to 1 to the real clothing is displayed on the computer while the 3.33% of the real one is applied on the mobile phone with a smaller screen and lower rendering performance ( Figure 13).

Conclusion
In this paper, a method for garment 3D reconstruction based on the size-changing is proposed to realize the 3-dimensional simulation for different body sizes of the same garment style. The clothing is divided into three categories, which are the basis for the option of the corresponding parameter algorithm. Multiple control points are introduced to control the transverse circumference and the longitudinal length. The garment model is separated into several segments by controlling sections. The constraint relation between the center line and the surface points is established to realize the overall zooming process of the circumference while the  longitudinal length is adjusted by the weighting factor of each point in a segment. In order to verify the effectiveness of this method, the garment model reconstruction with circumference and length changing is made on an initial garment model. In addition, texture mapping technology is applied in garment model reconstruction, by which the pattern bitmaps are fitted well with models of different sizes for a better display effect. Moreover, an online garment model reconstruction system is developed to facilitate users to use without space and software restrictions. A variety of initial garment models with different proportions are packed into one group, which are uploaded to the cloud as basic elements of the garment model reconstruction to adapt to the diversification of equipment parameters and performance. The results show that it is available to realize the model reconstruction of the garment models dressed in different body sizes well using the method proposed in this paper. In this paper, circumference is changed on its central axis, which is ideally regarded as a uniform variation from the axis. But in fact, when the human body becomes far or thin, variations are not the same in different directions at the same section. It follows a complex and detailed rule, which will be conducted an in-depth study in further work.

Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors acknowledge the financial support from National Nature Science Foundation of China (61772238 & 61902150), the Fundamental Research Funds for the Central Universities (JUSRP52013B) and Taishan Industry Leading Talents (tscy20180224).