Exploring product design quality control and assurance under both traditional and crowdsourcing-based design environments

Small and medium-sized enterprises face the challenges that they do not have enough employees and related resources to produce high-quality products with limited budget and time. The emergence of crowdsourcing provides an opportunity for them to improve their products by leveraging the wisdom of a large community of crowds, including their potential customers. With this new opportunity, product design could be conducted partially in a traditional design environment (in-house design) and partially in a crowdsourcing environment. This article focuses on product design stages to investigate what key factors affect product design quality and how it can be controlled and assured. First, we define the concept of product design quality and then identify its attributes and sub-attributes. Second, we separately survey key factors affecting product design quality in traditional and crowdsourcing-based design environments, quality control approaches/theories and quality assurance policies in traditional design environment. Third, a comparison of product design quality issues between the traditional and crowdsourcing-based design environments is progressed focusing on various aspects influencing product design activity quality. Finally, we discuss product design quality control approaches and quality assurance policies, quality control challenges and corresponding solutions in crowdsourcing-based design environment.


Introduction
Product design is an essential activity in the modern life as it can generate and develop ideas through a process that leads to new products to meet the ever changing user needs and expectation [1,2].
Products with high quality will not only meet needs of both manufacturing enterprises and various users along the product lifecycle but also bring good user experience and better social and environmental benefits to our society, thus helping the enterprises remain globally competitive in the fierce competition. Therefore, the control of product quality through the whole lifecycle is a vital activity to enterprises. The aim of product quality control is to make a compromise between cost and product quality. However, product quality is a nebulous concept and it may be viewed in different perspectives [3]. And the existing literature usually pays too much attention to quality control of manufacturing and activities after the product is manufactured. For example, Literature [3] shows that quality control of product quality is controlling production, carrying out repairs and warranty costs through defect discovery and maintenance, which ignores the importance of product design quality.
From the incline of product quality through the whole lifecycle (see Fig. 1) in literature [4,5], it is clear that product design quality is the key factor determining product quality. Fig. 1 Incline of product quality through the whole lifecycle [4,5].
As a stand-alone product is not sufficient to fulfil customer requirements, product service systems (PS2 or PSS) are usually combined with the product to provide product-related services, such as maintainability, repair, update and quality warranty throughout the whole lifecycle to better fulfil user requirements [6,7]. Therefore, we define product quality as the sum of product design quality, manufacturing quality, maintainability and recycling. Correspondingly, we consider that the product lifecycle consists of four stages, i.e. design, manufacturing, maintenance and recycling. The relationship among these terms is shown in Fig. 2 Fig. 2 The relationship among quality-related terms through product whole lifecycle.
In manufacturing aspect, many studies have been conducted to ensure the product quality from various perspectives, such as manufacturing methods and modelling approaches [8], the influence of human factors [9], key technologies of intelligent design for customized products [10], challenges and future of manufacturing in engineering [11], etc. Key factors affecting manufacturing quality can be classified into two categories: hard and soft factors [12]. After the product is manufactured and sold out, after-sale service, such as maintenance and recycling, plays a critical role in the successful marketing of many products as it enables customers to get the full value from its products [13].
After-sale services attempt to resolve problems met by a customer, e.g. product failure restoration and problem with using the product, which will cause dissatisfaction if the problem is not well resolved [14]. Due to the increasing importance of after-sale service, Rolstadaas et al. [15] discussed various aspects of after-sale services with regards to business model, service-delivering methodology, performance metrics, service portfolio and product planning and control. While Takeuchi et al. [16] summarized the measures that are used for ensuring the quality of after-sale services. Tore et al. [14] considered that the key factors affecting maintainability includes cost consideration, technological consideration, human factors, statutory requirements and accidents, etc. While product collection method, local authority facilities, charging structure and support, geographical location [17] are considered to influence recycling quality. Compared to manufacturing, at maintenance and recycling stages, there are relative less studies focusing on how to integrate product design quality into product after-service qualities despite that design for manufacturing, design for assembly and design for disassembly are proposed as design guidelines.
In conventional design environments, product design activities are usually performed by an in-house design team consisting of individuals with different expertise and experience. This kind of design collaboration mode is effective for large-scale enterprises, but not for SMEs who are lack of enough skilled employees and related resources to support their product design activities. Benefited from the advantages of crowdsourcing, such as cost-effectiveness and global participation of crowds, SMEs show great interests in leveraging crowdsourcing-based platforms to perform their product design activities or improving their products as crowdsourcing has the potential to overcome their shortcomings in employees and related resources. In parallel to traditional product design processes, more and more SMEs [18][19][20] are moving part of their product design processes on to a crowdsourcing platform, benefiting from the participation of a large number of crowds including their potential customers to the fast speed and cost-effectiveness of the solutions generated. Although crowdsourcing has shown great potential to create more values in product design domain, there are no crowdsourcing platforms that fully support product design activities [18,21]. Thereby, SMEs just rely on these platforms to perform some specific product design activities, such as idea generation and selection. Until now, many researchers have devoted to exploring how to support product design processes in crowdsourcing context [18,[20][21][22][23]. Since product design research over a crowdsourcing platform is still in its early stages, there is few studies on how to control the product design quality over a crowdsourcing-based platform.
This paper intends to give a definition of product design quality, survey the key factors affecting it in both traditional design environment and the crowdsourcing-based design environment and conduct a comparison between them. It also identifies product design quality control challenges over a crowdsourcing platform and investigates possible solutions to deal with these challenges.
Our main contributions in this paper are four fold: (1) Giving a hierarchy of product design quality attributes and defining their sub-attributes.
(2) Surveying the key factors affecting product design quality, quality control models/approaches and quality assurance policies in conventional design environments, and summarizing product design quality control studies in terms of four research focuses: the management of information, the sharing of information, quality control approaches and quality assurance policies.
(3) Analysing key factors affecting product design quality in crowdsourcing context and Comparing the traditional design environment and the crowdsourcing context in terms of various aspects affecting product design quality.
(4) Discussing product design quality control challenges over a crowdsourcing platform and proposing corresponding solutions to deal with these challenges.
The rest of this paper is organised as follows. Section 2 gives a definition of product design quality, in which key quality attributes and sub-attributes of product design quality are given. Section 3 presents our literature searching rules, the analysis results of found literature and the classification results of some literature that are conducted to ensure quality of different design quality attributes. The main contributions of this paper are described in Section 4 and Section 5. Section 4 presents research finding in traditional design environments, including key factors affecting product design quality, research findings in terms of product design quality attributes, key factors affecting design quality, quality control approaches and quality assurance policies. Section 5 mainly describes the research finding in the context of crowdsourcing, including key dimensions and factors affecting product design quality, and discusses quality control and assurance policies in crowdsourcing context and the product design quality control challenges in crowdsourcing context. Final section concludes the paper and indicates the benefits of crowdsourcing in PDD.

Definition of product design quality
At the product design stages, quality control is one of the most important activities as it ensures to achieve the design goal [4]. The aim of product design quality control is to check the key aspects of design quality against a set of standards or specifications. The quality control process can be achieved by a 3-step iterative loop of design process in its simplest terms [24]: ideate, prototyping and evaluate phase, which is shown in Fig. 3. In this process, design idea/concept is first generated, then it is prototyped, after that the design team would evaluate the design via prototypes. Finally, the design feedback generated from evaluation via prototypes will be fed back to corresponding designers for design improvement. The iteration of the design process helps designers discover unknown variables and their interrelationships [25] and this iteration process is helped by idea prototypes with different fidelities ranging from 2D sketches, 3D CAD models, 2D/3D mock-ups, and printed 3D models. Lim et al. [26] have explored the role and characteristics of prototypes in the domain of design. And impact of prototyping on design results has been investigated by Dow [25] and he found that prototyping can initiate a conversation with the space of design possibilities. By integrating design feedback generated from prototypes that represent a class of a generalized heterogeneous grouping of elements derived from alike design cases that provides the basis for the start and continuation of a design into their designs [27], the designers can improve their current design concept. However, since the interpretation of product design quality varies from person to person, it is hard to give it a specific definition. Nevertheless, many studies have tried to define it. The various definition of product design quality can be found in Table 1 Fig. 3 Product design quality control process. Table 1 The definition of product design quality. Source Definition or arguments of product design quality Zhu et al. [5] Design specification should conform to the requirement of customers.
Zhu et al. [28] ; BBC Design requirements reflect the voice of the customer or the demands of the market Aas et al. [29]; Mrugalska et al. [30] Design object satisfies its specification.
Salimun et al. [31]; ISO 9000: 2005 The degree to which a set of inherent characteristics fulfils requirements. Spacey et al. [32] The value of a design to customers.

BusinessDictionary
Level of effectiveness of the design function in determining a product's operational requirements (and their incorporation into design requirements) that can be converted into a finished product in a production process. IBM; Hermans et al. [33] The degree in which customer requirements are met.
According to the relationship between Maslow's hierarchy of needs and emotion design [34,35], Bradley [36] proposed a five levels design hierarchy of needs including functionality, reliability, usability, proficiency and creativity. Most of the existing product design quality control approaches are used to ensure the lower levels of user needs, i.e., functionality, reliability and usability.
Nevertheless, in the domain of industrial design, the maintenance of product is an important aspect affecting quality as well. Therefore, the maintainability should be taken into consideration when designing the product and it is treated as a key quality attribute in our product design quality hierarchy.
In addition, we treat proficiency as a sub-attribute of functionality. Fig. 4 shows the product design quality hierarchy of this research. Taking both Table 1 and Fig. 4 into consideration, this paper defines product design quality as follows: the degree in which a set of inherent design characteristics of product meet requirements of hierarchical design quality of functionality, reliability, usability, maintainability and creativity, commercial requirements and the required standards. In this definition, we consider not only the requirements of target audience (users), but also the requirements of other role players, such as workers involved in the manufacturing, maintenance and recycling stages. From the perspective of product whole lifecycle, all involved workers in the product lifecycle are customers of the product.
In Fig. 4, the highest level of design quality, creativity, is the last but very important aim of product design. Philips [35] and Schutte [37] have showed that making customers feel satisfied and happy could bring more benefits both in monetary and emotional aspects. Although a lot of research has been conducted on the controlling of one or two combinations of these five quality attributes, the ultimate goal of product design is to ensure the product quality in these five aspects. The five levels of design quality hierarchy are treated as design quality attributes in this research. Each design quality attribute consists of several sub-attributes. Referring to product quality characteristics defined by ISO 25010 standards [38] in Software Engineering, the common types of design quality in Table 2 indicated by John [32] can be treated as sub-attributes of these five design quality hierarchies. The aim of quality control at product design stages is to ensure the achievement of these five aspects.
There are already related studies focusing on ensuring single and specific design quality attribute (see Table 2). In reliability design, there is already a review paper about analysing the research progress based on axiomatic design theory [39]. The extent to which the functions cover all specified tasks and objectives.

Functional correctness
The extent to which the product provides correct results with the needed degree of precision.

Functional
The extent to which the functions facilitate the accomplishment of appropriateness specified tasks and objectives.

Time behaviour
The extent to which the response and processing times meet requirements.

Resource utilization
The extent to which the amounts and types of resources used by a product meet requirements.

Capacity
The extent to which the maximum limits of a product meet requirements. Sustainability/env ironment friendly It is a requirement to the designer. The product must be friendly to environment. Safety & Security The extent to which a product is safe for its users. [30,39,42,43] [44]

Maturity
The extent to which a product meets needs for reliability under normal operation.

Fault tolerance
The extent to which a product continues in a reasonable way when errors occur.

Availability
The extent to which a product is operational and accessible when required for use.

Learnability
The extent to which a product can be used by specified users after learning.

Operability
The extent to which a product has attributes that make it easy to operate and control.

User interface aesthetics
The extent to which a user interface enables pleasing and satisfying interaction for the user.

Accessibility
The extent to which a product is useful for everyone in the context of use.
Maintainability [50,51] Analysability The extent to which the product behaviours and performances are diagnosable and predictable.

Testability
The extent to which a product facilitates the establishment of acceptance criteria and supports evaluation of its performance.

Modularity
The extent to which a product is composed of discrete components.

Reusability
The extent to which a product or its components can be reused after disposal.

Modifiability
The extent to which a product can be modified without introducing defects or degrading existing product quality.

Emotional durability
A design that people value at an emotional level such that they do not easily throw it out.
Since reliability, usability, maintainability and creativity must be relied to functionality, more attention is often paid to the creation of functions and their combinations that satisfy established needs rather than the other four design quality aspects at earlier product design stages [42]. In order to obtain product designs with high quality, the design process has always been controlled. However, what factors affect design quality at different product design quality levels and what aspects should be controlled is still not clear. Therefore, the following method is adopted to extract research focuses from the existing literatures on quality control of product design at product design stages in both traditional design environment and crowdsourcing context.

Research Method
All reviewed literature is searched from the following databases: Web of Science, ScienceDirect, ACM Digital Library, and IEEE Xplore Digital Library. The keywords used to search articles are organised in three descriptor groups with rules listed below. In this review, Boolean operators 'AND' and 'OR' are adopted to make logical searches. 1 , 2 and 3 use the rule of 'OR' to represent the three descriptor groups, respectively. 0 is the sum of 1 and 2 or 1 and 2 and 3 with the rule 'AND'. For example, several keywords combinations, such as 'industrial product design' with 'quality control', 'conceptual design' with 'design for functionality' and 'crowdsourcing' have been examined.
3 )) As a final retrieval result, a total number of 125 literatures from six categories of resources, i.e. journal articles, conference proceedings, book sections, related webpages, standard and thesis, are found and classified. With the literature searching rules, the found studies could be classified into two categories according to the context: the traditional design environment and the crowdsourcing context.
After getting the data set, all found literatures are screened manually to select out studies that are conducted at product design stages. As a result, 118 literatures from journal articles and conference proceedings are selected as most closely related to our research objective. The number of studies in these two different contexts are 100 and 18, respectively. Since we have reviewed key crowdsourcing technologies for PDD regarding to organization structure, solution evaluation, workflow management and quality control in [21,55], here only 18 literatures (10 journal papers, 7 conference papers and 1 Book section) are mentioned to analyse the factors affecting product design quality in crowdsourcing context. Tables 3 shows literature distribution of studies in the traditional design environment. For these research papers appearing in journals and conference proceedings at product design stages in traditional environments, the TagCrowd was applied to extract keywords distributions to find out the research focuses. During this process, the searching keywords were excluded. The rest 258 words or word groups were collected and analysed. Fig. 5 illustrates the top 50 words with the highest frequency. The result shows that the top-ranking keywords are management, collaborative, process, information, model, data, sharing, communication, knowledge, lifecycle, systems and conflict. Based on this result, the extracted key research focuses in controlling product design quality in traditional design environments includes the management and communication of relevant information and quality control approaches. After reading through these studies, another research focus, quality assurance policies, is found. The detailed analysis against each term is presented in the following sections. the studies about these research focuses have been analysed in, here we no longer describe them.
Please refer to [21,55] for more details about these research focuses.

Research findings of quality control studies in traditional design environments
Traditionally, a successful product is achieved by the effort of a team [56]. A product design task is performed by a well-designated team which consists of employees from different functional departments, such as marketing, finance and technical department [57]. The budget for design, the capability of designers and the tools designers used definitely influence the product design quality at every design stages. Except these, there are many other factors [56] that are outside the direct control of designers affecting at least one attribute of product design quality, e.g. product development strategies, market orientation, technology, top management support, etc. For a product design team, the support of design process, team environment and other assistance is to support team members to access the required knowledge and to utilize the knowledge for product design [58]. All these factors [56] have an important role to product design quality. They could be classified into three key aspects in terms of design process, team management and information management. The factors affecting product design quality in terms of these three aspects (design process [59][60][61][62][63][64][65]  Knowing key factors affecting design quality in design process, team management and information management, we further investigate factors affecting product design quality at different quality levels, key quality control theories/principles/methods/tools, and key quality assurance policies, which is shown in Fig. 8. Some factors [70][71][72][73][74][75] affect single and specific design quality attribute and the others influence one or more attributes of product design quality [4]. In order to better understand how to ensure the product design quality, this paper takes into consideration control theories [39], approaches [41,43,44,47,48,52], principles [4] and tools [76]. Quality assurance is a set of policies through product lifecycle to make sure the product can systematically meet the quality standards and thus leave an impression to the customers that companies provide high-quality products that are reliable, durable, dependable [14]. The quality assurance policies on both supply chain [77] and after-sale service [14,78] have an influence on final product quality. It enables the quality of products according to Plan-Do-Check-Act ethos [79]. Case studies from Dale [80] found that nearly all companies have quality assurance policies that usually exist in the form of written documents or verbal communications. [71] Design object The designer [70] Health conditions Safety conditions Worker motivation Worker satisfaction [72] etc.  Basic principles of QFD and HOQ [4]  Fujitsu s conceptual model of kansei quality [52]  Robust design method based on axiomatic design and fuzzy clustering algorithm [44]  QQ-Dplatform (top-down and bottom-up strategies and rules) [76]  Interview [41]  Axiomatic design theory [39]  Analytic hierarchy Process and entropy method [43]  Qestionnaire-based usability evaluation method [47]  Modified free modulus method [48] usability Cultural differences User characteristics goals Contexts of use etc.
[73] [49,74] Documentation for specification Documentation for use Maintenance & service procedures Maintenance strategy [14] Maintenance times Maintenance frequency Maintenance cost etc.
User characteristics goals Contexts of use [49,85] Operating environment [14] [14]  Operator control  In order to ensure the quality of these five design quality attributes, the design process needs to be well controlled. Therefore, TagCloud (a data visualization tool) is utilized to analyse keywords from related literatures and four research focuses are identified. Fig. 9 shows the classification of related  Fig. 9 The classification of literatures according to research focuses.

Information management
Information management is to identify, capture, evaluate, retrieve, maintain and share all of the information assets in an enterprise [81]. The effective management of information can help the enterprise find out valuable information, thus earning more economic benefits. Whatever scale the enterprises are, they should pay attention to the management of relevant information. For SMEs, they usually collaborate with other companies that are geographically distributed, thus effective information management is needed to achieve a specific market objective [82]. For large companies, they need to well manage the information to support the collaboration of employees.
With the objective of knowing what kind of information is interchanged during a collaborative design project, Giannini et al. [82] analysed the design activities currently carried out in the technical offices of typical SMEs and found that designers always needed to communicate a subset of the whole information describing the product to be designed to co-designers. In this process, it is necessary to effectively and securely track, control, manage and share the rich information [51]. The management of information at design stages is vital to the success of final product and it is often achieved by PLM (product lifecycle management) systems. PLM systems are gaining acceptance for managing all information about products acting as a collaboration result of designers from different departments or enterprises throughout their lifecycle. The existing studies have proposed some information modelling framework trying to identify design information and product knowledge. The proposed information frameworks/models and the information managed are shown in Table 4. Product information, design rationale, assembly and tolerance information, the evolution of products and product families.
Sudarsan et al. [83] Product manager Product specific knowledge, concepts pertaining to the description of the product whose design is in charge of the node and its lower level co-designers.
Giannini et al. [82] Process manager The description of the processes: needed activities, dependencies between them, rules to follow, time schedules and constraints, planning of work, participant organisation and synchronisation of all of them.
Giannini et al. [82] Cloud Zhang et al. [87] In this part, the following questions need to be considered for design quality control: the representation of information, the quality of gathered information and the management of conflict information.

4.1.1The representation of information
During a product design process, there is a large amount of information available, such as product information and knowledge (knowledge is extracted from the analysis results of information), process information, the version and status of design solutions and customer feedback, etc. These information is often stored at designated servers and is organized through basic elements so that it can be easily handled and monitored. In order to better represent these information, Chandrasegaran et al. [88] defined it in three dimensions: Formal vs. Tacit, Product vs. Process, and Compiled vs. Dynamic (see in Table 5). The aim of information representation is to communicate the attributes and benefits of the design concept and to help users judge whether the design concept matches his/her needs and interests [89]. Table 5 The dimensions of Information available during a product design process [88]. The most common and classical representation of design concept is storyboard. Such a representation usually consists of a sequence of sketchy pictures with captions [90]. This representation is considered to be effective in communicating the role that the design would have in the lives of people. Another typical representation of design concepts is prototype. It helps minimize design errors that may otherwise occur both early and late in the process. Prototypes are often help designers identify design issues and learn from failures and they support both design concepts evaluation and design exploration [26,91]. Feedback elicited from prototypes often frames subsequent actions around the existing design solution [25]. However, new technologies, such as virtual reality, make it possible for designers to create more advanced representations through utilizing animations and videos, even early in the design process. These media integrate sounds, motion and light effects into the representation to enable an immersive feeling that is an important element of good interaction design [89].
In user-centred design, the users should be taken into consideration as they can help to evaluate the design concepts holistically. Therefore, their feedback on design concepts is meaningful to corresponding designers. Buskermolen et al. [89] and Van den Hende et al. [92] have explored the effect of different representations on user's responses to early design concepts. Van den Hende et al. [92] found that the presentation format of design concepts did not have significant effects on perception and comprehension of concept, but it has distinctive effects on absorbing the participants in the narrative world. After analysing the effects of visual quality and animation of concept representation on users' responses to early design concepts, Buskermolen et al. [89] suggested that sketchy representations provided more elaborate feedback and suggestions grounded on past experience while visually refined representations were more helpful in eliciting definite judgements.

The quality control of gathered information
During a product design process, high quality product information is essential as low quality often makes product development delay and can negatively impact the overall quality of the final product [51]. The gathered information may be in many forms, such as document, dialog, audio and video, etc.
In order to quantify the quality of a document, Culley et al. [93] have explored the existing tools for gathering and analysing information and summarized information assessment criteria in a comprehensive list of 94 criteria, the detail of these criteria can be seen in [93]. However, the gathered information in the product design process is not only presented in documents. The quality of information in other forms, such as web reviews and user feedback, needs to be measured as well. In addition, providing feedback to the corresponding designer is a key step in the product design process, and it promotes the improvement of product designs. Therefore, this part mainly concerns the control of feedback quality.
Although there are a lot of ways to gather feedback from customers, such as on-site observations, questionnaires and interviews [52], the proper measures need to be adopted to ensure the feedback quality as the quality of gathered feedback relies on many different elements, for instance, the gathering criteria and process, the reliability of participants involved in this process, information quality checks, etc.
Boess et al. [41] found that how people use products is quite different from the expectations of designers, so the users need to be involved into the design process. Their feedback is potentially valuable for designers. Additionally, user review is an important part of information at product design stages. Both the designers and manufacturers should pay attention to these reviews as they contain information about product and service experiences and they have an influence on buyers making their purchase decisions [94]. It can be treated as feedback from users about use experience. For product designers, these potentially valuable reviews could help them to identify customer likes, dislikes and desires. Yagci et al. [95] introduces the design-level information quality measure to evaluate the content, complexity and relevancy of the product-related reviews. The number of reviews, sentences, words, noun words and feature matching noun words in a review database are found to be key determinants in measuring information quality.

The management of conflict information
When designing a product, designers need to take into consideration a lot of interdependent aspects, such as functional requirements and geometrical, behavioural and structural features [96]. Each aspect of these has its own set of constraints in which conflicting or unsatisfied requirements may be contained and it is hard for designers to oversee the various alternatives and constraints all the time.
The final product design is a compromised result of such constraints. In collaborative design, due to the participation of many designers with different technical background and expertise and their interactions, they may have disagreements about proposed designs [96,97]. In addition, the product design data may change frequently until the final product is manufactured as more than one designer works on the same product project. The ever-changing data versions and status should be well managed. All these information is treated as conflict and the management of them is critical in collaborative design [97].
Barclay [98] argues that there were three common types of conflict identified in product design: a) imagined or perceived conflict; b) latent or substantive conflict; c) affective conflict. All these three conflict types are accompanied by process-related conflicts. With better understanding of the conflict types and causes that lead to conflicts, the management of them can make the outcomes of conflicts constructive.
The conflict management process includes five phases: conflict detection, forming the conflict resolution team, negotiation management, solution generation, solution impact evaluation [96,97]. A lot of research about conflict management has been conducted in collaborative design, but most of them have proposed methods to support the conflict detection [99,100] and the conflict resolution [101]. Quertani et al. [97,102] also developed solutions to manage the negotiation process by tracking product specification dependencies. In order to assess the impact of a selected solution on the product as well as on the design process organization, Quertani et al. [96] proposed a process organisation framework based on data dependencies network.

Information sharing
PDD always involves many participants with different professional knowledge and background, such as marketing and engineering design. All these participants work together towards a specific design objective. Collaboration is found to be a key to enhance competitiveness [103] and to improve product quality [5]. Communicating and information sharing is the foundation for collaborative product design [86]. Information sharing disseminate information with a community, which plays a crucial role in information management in product design process [81]. Effective information sharing drives organisational and individual learning, which in turn speeds up and improves the quality of product. Due to its importance to product design, here we present it in a separate part.
From the existing literature available, information sharing is found to be useful in helping individuals, teams and organizations to improve their work performance [104]. At the same time, effective information sharing can help short the time taken to introduce the products to the market, which will give the company an edge in the fierce competition [81]. In order to help designers work together effectively, certain measures must be adopted to enable the exchange of design information. ISO 10303 has provided an ISO standard for the computer-interpretable representation and exchange of product manufacturing information. However, there is still no similar standard for the exchange of product information at product design stages.
During the collaborative design process, the design-related and process-related information can be classified into two categories: public and exchanged information [82]. Public information describes and indicates the main characteristics of the product and can be treated as a set of technical requirements, to which all participants involved in the process can get access. While the exchanged information can only be seen by certain participants and their versions depends on the sender, e.g., customer, co-designer or supplier, and the states of the project development. In information sharing, only exchanged information is considered.
Based on collected product and process information, proper information sharing and communication mechanisms enable designers and their partners to collaborate effectively, which is a critical determinant of collaboration [83,105]. The main contents of communications are the product data and the process data [82]. In traditional collaborative design scenarios, the designers usually have to spend much time on communicating with their partners through emails or phones. As for the design-related data stored at designated servers, they have to be authorized to have access to them. In the work of Zhang et al. [86], a data access mechanism from different perspectives, functional views, personal workspace, work table and personal storage space, is provided, and even from the same perspective, different users have different data access authorizations. However, the proposed product model can only support static product information, which cannot satisfy the needs of real-time collaborative design. As for the systems on which all information is public, the protection of intellectual property is threatened [106]. Mun et al. [106] suggests just share the essential data with their collaboration partners.
In collaborative environment, the sharing of information is usually achieved through various design representation formats, such as 3D models, images, videos and XML files [107]. One problem that needs to be considered in this process is that the users should exchange their ideas as instantly as possible. Hasby et al. [108] proposed a CAD system that can facilitate the conceptual-embodiment design stage in a collaborative manner. Through the system, the designers can communicate their opinions and ideas freely. In addition, the system must avoid the possibility of losing some information [82]. In order to prevent this possibility, the system should automatically create a list of changes that the user has made and notify relevant users to update their information.

Quality control models/tools
Quality control is an activity throughout the whole product design and development process. The aim of quality control is to satisfy the customer needs as well as to decrease the design and development cost in terms of time and money [109]. In this process, customer requirements and product design specification (PDS) are benchmarks of the controlled quality of product design [76]. In order to control the quality of product design, many researchers linked customer requirements with quality characteristics. For example, Tang et al. [76] argued that quality characteristics are the key control factors in the whole product design and development process and the focus of quality control is on how to translate customer requirements into product quality characteristics. Similarly, Zhang et al.
[110] developed a vector-based mapping tool that can provide reasonable mapping among PDS, behaviour parameters and structure parameters. Chu et al. [4] converted user requirements into the relevant technical requirements of design using QFD and HOQ. Although customer requirements provide benchmarks for the designated product, one problem that cannot be neglected is that the customer requirements changes over time. In order to understand how requirement changes propagate in the design of complex product systems thus helping to select best options to guide design, Zhang et al. [109] proposed a PDS-Behaviour-Structure-based design change model that can systematically analyse and search change propagation paths.
The quality control models can be classified into computer-oriented and human-oriented. Among computer-oriented quality control models, Tang et al. [76] translated customer requirements into product quality characteristics and achieve their quality control aim by controlling these quality characteristics. As for human-oriented quality control models, a typical one is Fujitsu's conceptual model of kansei quality [52]. The model has six elements [52]: product/service usage situations, stimuli produced in usage situation, somatic sensations (perception/cognition), personality and past experience, impressions/emotions, behaviour. The somatic sensations is the contact point between the user and the outside world. These elements are controlled in the Plan-Do-Check-Act cycle.

Product design optimization
During product design and development process, the initial design may be functional, but it may be far from optimal in terms of quality and cost [44]. Hence, it is necessary to optimize the initial product design to make it meet the design requirements. Product design optimization is an effective way to improve the quality of a product through minimizing the effect of the causes of variation [44]. The optimization involves product design optimization and design process optimization. Product design optimization means improving the design in terms of one or more performance aspects of a specific type of products [111], while design process optimization could benefit nearly all corresponding product designs produced by adopting the optimized design process.
When a product design is finished, it is expected to maximize performance as well as be less sensitive to variation in practical situations, such as environmental changes [30,44]. Design optimization usually achieved by mathematical optimization techniques [111]. The mathematical model allows to choose the optimal values of parameters of the model that accurately reflect customers' expectations [30]. In the mathematical model, experiments based on fractional factorial designs and orthogonal arrays can be applied to improve it. However, there are still some challenges in design optimization, such as mathematical challenges, topologies and configurations, systems design, controlled artefacts, enterprise-wide design (indicated by Papalambros [111]). Whatever the optimization technique is adopted, the aim is to make the product design as robust as possible. In order to achieve this goal, robust design method can be considered during product design process. Additionally, different sources of uncertainties and variations in design and manufacturing process, such as model uncertainty, parameters uncertainty and noise, should be taken into consideration [30].
Chen and Li [112] proposed a computerized team approach for process design optimization. In their work, they treat design teams as game players in a multi-player game and classify different types of team interactions from the view of game theory. Finally, the satisfaction metric is utilized to evaluate the effectiveness of the proposed approach. Different from their work, Taguchi [113] divides the design process into three stages: system design, parameter design and tolerance design and the design focuses of these three stages are basic functional prototype design, controlling parameters to make the design insensitive to variations and the upper limit of the number of variation or noise factors allowed in the design, respectively. Compared to research work in system design, more work is conducted in parameter design and tolerance design [44]. For example, Cheng et al. [114] took any deviations of design parameters into sensitivity analysis and robust optimization design of suspension system so that the system would have better performance. Liu et al. [115] utilized goal programming approach that incorporated analytic network process and cost budget limitation to determine the variant components to be focused on redesign. However, system design determines the attainable level of product robustness in the parameter stage, argue by Andersson [116]. Due to the importance of system design, Cheng et al. [44] proposed a frame on the basis of system modelling, cluster analysis and design of experiments for the development of robust system.

Quality assurance policies
Quality assurance policies are adopted to satisfy the customers by supplying products that fully comply with customer, statutory and regulatory requirements. Most enterprises have their quality assurance departments to ensure the quality of their products and they have their own product or service quality rules, such as customer first, priority on quality, safety and confidence and global quality accreditations [78].
In a questionnaire survey conducted by Dale et al. [80], they found that the majority of companies have quality assurance policies, and most of them make their policies in the form of written documents and statements, although some companies utilize verbal communications via the management structure to communicate quality policy information. The attainment level of quality objectives relies on constant examination, improvement and effective communications and the way of quality assurance policies affects the communications and decision making processes [80]. In order to obtain the correct quality products, all participants in the production process should be well controlled and should be motivated into the process. Therefore, operator control method can be combined with job satisfaction schemes, bonus schemes, suggestion schemes and certification schemes in production [80].
However, the attainment of the desired quality level not only relies on the production enterprise, but also relies on its suppliers [77,80]. In the game theoretic model proposed by Xiao et al. [77], they investigated how an enterprise coordinates the supply chain with a quality assurance policy via a revenue-sharing contract and found that the optimal service quality first decreases and then increases as the defective rate of the final product increases. After the product is launched on the market, the enterprises will have in place telephone helplines and online/Internet support or will provide field service for helping customers solve problems in the product use process fast [14]. In addition, they will use many databases and information systems to manage customer feedback, complaints and product problem resolution for future improvement of corresponding products.

Research findings and quality control challenges in crowdsourcing context
In the traditional design environment, it is hard to overcome the shortcomings of SMEs in employees and resources, sometimes in budget. Extending the design environment to crowdsourcing context, these shortcomings may be well addressed by leveraging the globally distributed crowds and the costeffectiveness of crowdsourcing. Therefore, this part mainly analyses the factors affecting product design quality in crowdsourcing context, discusses quality control models and quality assurance policies and investigates quality control challenges over a crowdsourcing platform.

Research findings of quality control studies in crowdsourcing context
When extending the traditional design environment to the crowdsourcing context, the possible design process over a crowdsourcing platform (see Fig. 10) has been explored by Niu et al. [21]. They indicated four challenges (please refer to [21] for more detail) when performing product design tasks in the crowdsourcing context.
During this process, there are more factors affecting product design quality. In the crowdsourcing context, three more factor dimensions can be added, i.e., requester, task and platform. Fig. 11 shows key factor dimensions and some of the key factors affecting the corresponding dimension. More factors affecting product design quality in terms of key elements of crowdsourcing process are shown in Table 6.  Fig. 10 A simple design process over a crowdsourcing platform [21].
The explanation of each factor dimension is shown as follows: (1) Crowds. It is one of the key elements of a crowdsourcing process. It is extended from the traditional design team, so it can be treated as a virtual team. In traditional design environments, the capability of their employees is well mastered by their team leader. Thus, the team leader can assign tasks to the most right individuals. While in the crowdsourcing context, the truthfulness and reliability of these information provided by the crowds themselves are usually doubtful. In order to assign the task to right persons, these information must be verified.
(2) Requester. It is one of the key elements of a crowdsourcing process. The requester is responsible for proposing task requirements, incentives, timelines, etc.
(3) Task. The task is proposed by the requester and crowds perform it to achieve expected outcomes. In order to better assign the task to crowds, it usually needs to be decomposed either by the requester or by the crowdsourcing platform runner.
(4) Platform. The crowdsourcing platform is a workplace which provides interfaces for its users including the crowds and the requester to interact with it. All information related to design process, information management, task, the requester and the crowds are controlled by the crowdsourcing platform.
(5) Design process. In the crowdsourcing context, the product design process is controlled by the crowdsourcing platform. The process consists of many sub-processes, such as task decomposition, task assignment, the selection of crowds, and the synthesis of task results, etc.
In order to ensure the collaboration of crowds, the process should have the capability to involve enough qualitied crowds by participation motivations and proper selection rules and support the communication among them [117].  Fig. 11 Factors affecting product design quality in crowdsourcing context. (6) Information management. In the crowdsourcing context, all information including product-, process-, crowds-, tasks-related information are stored in database. The users only need to pay attention to the interactions between them and the crowdsourcing platform. Since the information management is controlled by the crowdsourcing platform, the factors affecting software system quality, such as platform performance and data structure, will influence the crowdsourcing platform quality, thereby influencing the quality of work conducted on the platform [118]. Table 6 Factors affecting product design quality in terms of key elements of crowdsourcing process.
Task [119][120][121] task definition, user interface, granularity, and incentives and compensation policy, instruction, task description, target audience, complexity, type of action, modularization, nature of the reward, type of the reward, latency, submission time, closure time, duration, visibility, confidentiality, human requirement, technical resource.
As product design activity quality control studies in terms of four aspects, i.e. crowds, platform, task and workflow, have been reviewed by Niu et al. [21,55], here we do not review them repeatedly.
Here, we mainly compare traditional design environment and crowdsourcing context in terms of the following aspects influencing design activity quality. The comparison is shown in Table 7.

Discussion of quality control models and quality assurance policies
Literature [18,21,55] found that the existing crowdsourcing platforms can only partly support PDD activities. And the existing studies about product design quality control is relatively less. In crowdsourcing context, a crowdsourcing process with high quality makes it more likely to achieve better product designs. Therefore, many researchers have devoted in investigating crowdsourcing quality control. For example, Daniel et al. [117] have analysed the quality attributes, assessment techniques and assurance policies of in crowdsourcing quality control. They mainly control the quality from the perspective of individual, group and computation. The corresponding assessment methods can be found in [117]. However, when PDD activities are performed in the crowdsourcing context, both the traditional product design quality control approaches and crowdsourcing quality control approaches should be considered. In literature [21,55], we have reviewed some studies about product design quality control approaches (see Table 8). However, most of them focus on controlling the quality of a specific stage of product design process. The quality control models/approaches through the whole product design stages will be investigated in the future. As for the crowdsourcing quality assurance policies, Daniel et al. [117] have identified 6 strategies: (1) improve data quality; (2) select people; (3) incentivize people using extrinsic and intrinsic motivations; (4) train people; (5) improve task design; (6) control execution. These strategies aim to improve quality as first-order goal. In addition, some assessment measures have positive side effects on quality, especially when the assessment object are people. For example, reviews have been found by Zhu et al. [123] that it can improve the performance of both crowds and reviewers. And rating the performance of crowds has similar positive side effects [124]. Together with the product design quality assurance policies in traditional design environments, such as job satisfaction schemes, certification schemes, free warranty policy, telephone helplines, remote monitoring, and customer first, etc., the crowdsourcing quality assurance strategies should make it possible to obtain high-quality product designs in the context of crowdsourcing.

The quality control challenges over a crowdsourcing-based platform
This part mainly focuses on discussing the challenges of product design quality control over a crowdsourcing-based platform. Based on the previous analysis results, it is found that most of design work is finished by the collaboration of many designers as collaboration is found to be effective in improving product quality and work efficiency [5]. However, it is hard for SMEs to organize enough designers to perform the design task unless outsourcing parts of their work to related companies [106].
In addition, it has been demonstrated that crowdsourcing can enable the scaling-up of design and manufacturing operations and improve design performance and quality [18,125].
As a result, a lot of enterprises have utilized crowdsourcing to achieve specific goals, such as to increase customer engagement and to choose better ideas. However, there are still some challenges on controlling the quality of generated ideas or solutions when product design activities are performed over a crowdsourcing-based collaborative platform. In our previous work, we have analysed the gaps and challenges in adopting crowdsourcing in PDD process from the perspective of crowdsourcing technologies [21,55] and have indicated the necessity of developing relevant tools to support product design and development activities. Except the problem that the existing crowdsourcing platforms and tools cannot fully support PDD activities, there are still other challenges in controlling the product design quality over such a crowdsourcing platform. The possible challenges includes: (1) The management of information.
On a crowdsourcing platform, there are more participants in the product design project than in conventional settings. The participants are globally distributed and have various cultural background and different levels of expertise, which may lead to more product design conflicts.
(2) The representation of product design.
On the crowdsourcing platform, the participants have various levels of design capabilities and they may not know how to present their designs as expected.
(3) The communication of designers.
Effective communication plays an important role on improving product design quality, which can help designers have a better understanding of design requirements and work done by their colleagues.
(4) The protection of intellectual property.
In order to support the collaboration of crowds, the platform has to support the information sharing and communication. Since crowdsourcing process is open to the crowds registered on the platform, the intellectual property protection faces more risks than in traditional environment.
Aimed at the above challenges, the following corresponding solutions are proposed.
(1) The platform should be cloud-based. With the support of cloud technology, the distributed participants of product design process can easily get access to product and process related information to effectively perform design activities.
(2) The platform should provide a presentation tool to guide designers to present their designs and other users can pose queries about the design such as rationale and purpose, or the causality between physical and functional elements.
(3) The platform should provide APIs to common social medium because it not only helps designers to communicate with other designers more freely and in real-time, but also makes them have a connection with their friends.
(4) The platform should integrate with blockchain technology as it supports the encrypted transmission of information.
As there is no such a crowdsourcing platform that fully support product design process, these solutions cannot be verified until the platform is developed. However, these solutions would provide some guidelines for developing such a platform in controlling product design quality in the future.

Conclusion
This review paper mainly analysed the key factors affecting product design quality both in traditional design environments and in the crowdsourcing context. And based on the analysis results, this paper indicated product design quality control challenges over a crowdsourcing-based platform and proposed corresponding solutions.
In order to better understand key factors affecting product design quality, this paper first gave a definition and sub-attributes of product design quality. Then TagCloud is adopted to analyse the keywords to find out the research focuses in these two design environments. In the traditional design environment, four research focuses including information management, information sharing, quality control approaches and quality assurance policies are found. Through analysis, we found that there are more factors affecting product design quality when the design environment is extended to the crowdsourcing context from the traditional design environment. While in the crowdsourcing context, we mainly analysed the key factors affecting product design quality, compared these two design environments and discussed quality control and assurance policies. Based on that, the paper indicated four product design quality control challenges over a crowdsourcing-based collaborative platform and proposes corresponding solutions to these challenges.
If these challenges can be well addressed, SMEs will benefit a lot as they can get access to a large pool of crowds with various skills and experience, which will effectively relieve their pressure resulting from the lack of skilled employees and related resources. The research direction in the near future is to develop such a crowdsourcing platform to support PDD activities. This paper can provide some guidelines for the platform development.