Influence of Playability on Perceived Game Software Quality: A Comparison Between Chinese and South Korean Gamers

The usability of ISO/IEC 25000 series SQuaRE and other existing software quality models seems unsuitable for unique software products such as game software. This study revealed that playability has a higher predictive capacity than usability for evaluating game software quality using a survey methodology involving 344 Chinese respondents and 142 South Korean respondents. Specifically, dimensions of playability in terms of enjoyable, exciting, fun, interesting, relaxing, novel, creative, convenient, curious, fantastic, stimulating, splendid, effective and attractive positively influenced both Chinese and South Korean game software users’ satisfaction. In addition, results show that there is a significant difference between Chinese and South Koreans in playability dimensions such as exciting, relaxing, novel, creative, convenient, curious, stimulating, splendid and attractive. This study extends the existing knowledge on how playability influenced game software users’ satisfaction and software quality, and limitations were also highlighted.


Introduction
According to the 2017 Global Games Market Report from Newzoo (2017), the global revenues of game software reached $107 billion in 2017, and Chinese consumers dominated the global game market.For example, China contributed $24.4 billion in revenue to the mobile game market in 2016, followed by the USA ($23.6 billion), Japan ($12.4 billion) and South Korea ($4 billion) (Newzoo, 2016).The studies on game software are not new since they can date back to the early 1980s; nevertheless, in recent years, there has been a rising interest in game software studies in developed nations such as the USA, Japan and many more (Perani, 2012).Ullmann et al. (2023) stated that game studies should be considered similar to the study of the field of architecture which include elements of the esthetic aspects, media studies, and sociology.However, game studies should also need to be treated as an independent academic structure because they cannot be reduced to a single entity entertainment (Aarseth, 2001).
The ISO/IEC 25000 SQuaRE is one of the series of ISO/IEC which has been considered an important model for evaluating the quality of all types of software products (Nakai et al., 2017).Many software-related researchers considered the SQuaRE model suitable for evaluating the quality of all software products (Estdale & Georgiadou, 2018).However, recent scholars argued that the SQuaRE model is ineffective for testing the quality of certain types of software, such as games software (Koh & Jiang, 2020).Specifically, usability as one of the eight characteristics of SQuaRE's product quality model, plays the most important role in determining software quality because it is considered the ultimate goal of each software product (Koh, 2017b).However, the definition of usability, or the effect of characteristics of usability on game software quality seems to be outdated (Koh & Jiang, 2020).Moreover, playability is similar to usability which can be considered as the psychometric measurement for evaluating software quality for game studies.Although studies on usability have been growing explosively over the past 10 years, the research on the effect of playability on game software is still at its preliminary stage.
Additionally, teachers and students seek educational software to enable an improved and more recognized approach to learning (Ahmad, 2019).This is because educational games serve as an educational system that enables leaners to learn and expand concepts while providing a learning experience, boosting motivation, and engaging critical thinking to improve cognitive skills (Sungkaew et al., 2022).However, educational games have demonstrated positive and occasionally negative effects on educational learning.Specifically, the influence of educational games is still unresolved as to how to guarantee quality learning outcomes are achieved through educational games and what game designers can do in their development processes (Ahmad et al., 2014).It is critical to comprehend how educational games and software quality assurance from a software engineering perspective are providing potential customers with suitable products (Ahmad, 2019).Thus, the current study attempts to apply both usability and playability for measuring their effect on the quality of game software.

Usability
According to the ISO/IEC 25000 SQuaRE, usability refers to the degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.However, there are various definitions of usability that can be found in past studies (Gonza´lez-Sa´nchez, Zea, & Gutierrez Vela, 2009;ISO/IEC 25010:2011, 2011).Some researchers demonstrated that usability is linked to the product itself (Nielsen, 2012;Z.-X. Wang et al., 2022) while other studies have shown that usability is connected to users' experiences (Gonza´lez-Sa´nchez, Zea, & Gutierrez Vela, 2009;Gonza´lez-Sa´nchez, Montero Simarro et al., 2009;Lim et al., 2019).
Nevertheless, certain studies showed that it had been difficult to extend traditional usability methods for evaluating game software.For instance, traditional software product studies could examine usability in terms of the twin goals of ease of learning and ease of use, but Song et al. (2007) indicated that game software is different from other software products as the nature of games was utilized during leisure time and to provide fun.Hence, the main goal of game usability is to reduce fun obstacles and enhance the enjoyment of game participation.On the other hand, software productivity strives for consistency, but game software needs to provide a slightly different user experience each time (Esposito, 2005).Therefore, there is a significant difference between game software and utility-based application software.

Playability
Certain researchers criticized the applicability of ISO/ IEC 25000 SQuaRE's product quality model for evaluating all types of software due to the incompatible relationships among internal views, external views as well as inuse views (Koh & Jiang, 2017).In other words, the concept of ISO/IEC 25000 SQuaRE is ambiguous when it comes to defining some characteristics and the causality between these views.For example, researchers indicated that usability can be a fundamental characteristic of quality, which has a direct or indirect effect on other characteristics in the quality models (Jaiswal et al., 2022).Playability is a set of properties that can describe the player experience using a specific game system whose main objective is to provide enjoyment and entertainment (Gonza´lez- Sa´nchez, Zea, & Gutierrez Vela, 2009;Gonza´lez-Sa´nchez, Montero Simarro et al., 2009).The game software quality focus on how players interact with basic game elements (user interface, games, and gaming platforms) and playability in game software is a broad term because it needs to cover multiple aspects of the game (Korhonen, 2016).ISO 9241-11 in ISO/IEC is the reference to this definition and demonstrated that playability is the extension of ISO's usability or quality in use for the player-centered video game.Playability is a term that depicts the player's individual experience about function, interaction, storyline, and the audio and visual effects of a certain game (Lee, 2015).Gonza´lez- Sa´nchez, Montero Simarro et al. (2009) indicated that playability is mainly about joy, emotion, fun and usefulness of the game software, and specify playability as the property of a user's experience.
As part of the literature review process, the Scopus database for advanced search was accessed and the search keywords were entered in the following order: game software, playability, game software usability, game software quality.A total of 677 articles were retrieved; 353 articles related to game software, 83 articles related to playability, 201 articles related to game software usability, and 40 articles related to game software quality.
Based on the number of papers issued, the research in this field has been relatively active in recent years, and the number of relevant papers issued each year is more than 20 (See Figure 1).Among them, the number of papers issued in 2016 was the largest, 33 and there were at least 23 articles in 2017, 2019, and 2020, and it is expected that this research field will remain popular in 2022.
A keyword co-occurrence analysis of relevant articles in the above research fields was conducted, and diagrams on the co-occurrence was created (see Figures 2-4).It can be seen that the research foundation and research bridge in the field of game software are video game, game theory, gaming, virtual validity, and other research fields.The focus of this research is game software playability and game software usability, which are also reflected to some extent (see Figure 2).Through in-depth analysis, it can be seen that the relevant research in the research direction of game software playability is mainly reflected in the field of user experience.Research in the field of game software usability is mainly reflected in the field of ''ease of use'' and video games.In order to conduct a deeper analysis of the two related research fields, the keyword co-occurrence atlas of metadata of research papers related to game software playability and game software usability retrieved from the Scopus database were made available for analysis (See Figures 3 and 4).
It can be seen from the co-occurrence analysis highlighted in the above figures, that research on game software playability related fields has strong relevance, mainly reflected in video, engagement, user experience, educational video game, game metrics, player experience, digital game, mobile gaming and other areas of research.In the research direction of game software, the content is relatively comprehensive, the attention is high, and most of them are in the research field of emerging things, with strong practical characteristics.However, according to the graph of game software usability research field, the relevance of the research is weak, and most of them stay at the theoretical level.It is related to game theory, gaming, and online game.
In order to analyze the research evolution process of game software playability related fields, a key word time zone map of the research field was created to explore the evolution process and research frontier content of research hot spots in this field (See Figure 5).It can be seen that previous research in this field mainly focused on fields such as serial game, educator game authoring,    In order to clarify the content category of the research field, the keyword clustering analysis is carried out from 677 retrieved articles again (See Figure 6).Through clustering, it is found that there are six effective clusters, including #0 playability, #1 serious game, #2 artificial intelligence, #4 playing style, #5 mobile game, and #7 software process improvement.The content under each cluster mainly includes: #0 playability (playability; serial game; massively multiplayer online role playing game; game metrics), #1 serious game (serious game; jeu se´rieux; e´motion; domain specific visual language; algorithme automatise´), #2 artificial intelligence (artificial intelligence; prodecural content generation; game), #4 playing style (playing style; adaptation; personality type; assessment; affective serious game), #5 mobile game (mobile game; herb related knowledge; herbopoli), and #7 software process improvement (software process improvement; online game; process assessment; game development methodology; game performance).
Playability is more suitable for evaluating game software than usability by definition due to playability is more focused on the users' emotions and feelings, which plays an important role in being attractive to users.Nacke (2009) indicated that good playability should be considered as a prerequisite for evaluating game experience as it refers to the game and game design.Thus, playability should not separate the players' actions and the game concepts because the concept of playability is associated with fun, flow, fulfillment, players experience, satisfaction, engagement, and pleasure (Wong et al., 2010).Such attributes are related to the players' perspective should be included in the playability models.
Furthermore, most utility-based application software focused on how useful it can be to its users, thus, usability had always been the main area of focus by scholars.However, when individuals play with game software, they always choose the game based on enjoyment resulting in a more emotional feeling attached to the game.They will not select a game software based on whether a game software is efficient or effective or even useful.In addition, it is reasonable to ensure that efficiency is not ruled out by playability (Gonza´lez-Sa´nchez, Zea, & Gutierrez Vela, 2009) because minimizing time may not be an objective related to playing video games while effectiveness is defined to be almost the same as typical efficiency (Koh & Jiang, 2017).Therefore, based on the arguments above, it is believed that the general software quality models (e.g., SQuaRE) are unable to meet all game studies' expectations, and for most game software studies, it is quite rare for researchers to focus on the playability aspect.

Methodology
A purposive non-probability sampling technique was adopted to collect data at Binzhou University, China and Chungbuk National University in South Korea, from 19th November to 3rd December 2019.Varsity students were chosen as samples for multiple reasons: (1) a large proportion of Chinese and South Korean students are game software users (CNNIC, 2015); (2) they are advanced users of information technology (L.Wang et al., 2021); and (3) this demographic segment from both countries is a major contributor to of game software revenues (Newzoo, 2016).All questionnaire items related to playability were adopted from past relevant studies (see Table 2 & Appendix: Questionnaire) and evaluated by field experts to ensure respondents' understanding of the items.The questionnaire items were translated into Korean and Chinese respectively with the back-translation method using three bilingual experts to reduce translation ambiguity.A pre-test was performed with a total of 33 respondents to ensure instrument validity and understanding and to prevent potential data quality issues.A five-point Likert scale ranging from ''strongly disagree'' to ''strongly agree'' was used to evaluate the questionnaire items.
A total of 650 Chinese language version questionnaires and 253 Korean language version questionnaires were distributed to each country's respondents.After eliminating invalid responses, 344 Chinese language questionnaires and 142 Korean language questionnaires were analyzed for this study.Among the 344 Chinese respondents, most of them (36%) reported that they utilized game software twice in the last week.Of the 142 South Korean respondents, the majority of the respondents (46.5%) reported that they utilized game software once in the previous week (see Table 1).

Data Analysis and Results
Exploratory factor analysis (EFA) with the principal component method and varimax rotation was performed using SPSS and Python.EFA is a multivariate statistical method that has become a fundamental tool in the development and validation of psychological theories and measurements (Watkins, 2018).Both the Kaiser-Meyer-Olkin (KMO) and Bartlett's test of sphericity should take into account in the EFA procedure assuring the suitability of the data for EFA.A KMO value more than 0.5 is considered to be acceptable (Hair et al., 2010;Kaiser, 1974), KMO value between 0.5 and 0.7 is mediocre and between 0.7 and 0.8 is good (Hair et al., 2010).The KMO and Bartlett's test of sphericity results of Korean data showed that the KMO measure of sampling adequacy is 0.692, the approximate Chi-Square is 1,922.951,df = 496, p \ .05; the KMO and Bartlett's test of sphericity results of Chinese data showed that the KMO measure of sampling adequacy is 0.887, the approximate Chi-Square is 4,108.061,df = 496, p \ .05.Thus, the results showed that the data were appropriate for EFA test.
The characteristic of satisfaction is used to measure playability.After the EFA test, for Chinese respondents, dimensions of enjoyable, exciting, fun, interesting, pleasant, relaxing, dynamic and intellectual belonged to joy; dimensions of the novel, creative, original, convenient, curious and fantastic belonged to novelty; dimensions of stimulating, splendid and realistic belonged to the stimulus; dimensions of useful, educational, effective, efficient, practical, beneficial and trustworthy belonged usefulness; dimensions of attractive, beautiful, appropriate and comfortable belonged to attractiveness; dimensions of easy and difficulty belonged to easiness; and emotion and harm was considered as a single characteristic based on EFA results.
There are a few differences among dimensions in the variables between Chinese and South Korean respondents based on the EFA results.For South Korean respondents, dimensions of enjoyable, exciting, fun, interesting and attractive belonged to joy; dimensions of the novel, creative and original belonged the novelty; dimensions of curious, fantastic, stimulating, and splendid belonged to stimulus; educational, effective and efficient belonged to usefulness.In addition, dimensions of pleasant, relaxing, realistic and practical were defined in group 1; dimensions of dynamic, easy and difficult included in group 2; dimensions of beneficial, beautiful and appropriate belonged to group 3; dimensions of intellectual, convenient, comfortable and harmful belonged to group 4; and dimensions of useful, trustworthy and emotional were belonged to group 5 (See Table 2).

Discussion and Conclusion
This study focused on the influence of playability on Chinese and South Korean game software users' satisfaction.An extended scale of playability incorporating a usability scale was developed and empirically tested.The findings showed that, for Chinese game users, most variables (i.e., joy, novelty, stimulus, usefulness, attractiveness, easiness and emotion) positively correlated to satisfaction while harm negatively influenced satisfaction.However, for South Korean game users, there are no relationships found between dimensions of pleasant, dynamic, intellectual, original, realistic, useful, educational, efficient, practical, beneficial, trustworthy, beautiful, appropriate, comfortable, easy, difficult, emotional, harmful and satisfaction respectively.In other words, most of the traditional dimensions of usability models had no relationship with satisfaction for South Korean game software users.
The existing usability models are too specific to be generalizable to game software; thus, these models lack validity and reliability for various software products and services (e.g., game software).The results of this study showed that the playability scale is more suitable than usability for evaluating game software products.
The usability model is important but deceptive for evaluating all software quality.Certain researchers have tried to define general usability models to apply to most kinds of software.Playability, however, is generally regarded as a special type of measurement model that had more predictive capacity on game software products.In summary, this study's results showed that dimensions of playability in terms of the enjoyable, exciting, fun, interesting, relaxing, novel, creative, convenient, curious, fantastic, stimulating, splendid, effective and attractive positively influence game software users' satisfaction.
The results of this study also show that it is useful for users and practitioners to develop a game software playability model with sufficient validity and reliability.The classification of game software and game software playability model should be designed mutually.The user evaluation should be the final standard for evaluating game software quality.The results show that the playability scale is more accurate than the ISO/IEC 25000 SQuaRE and makes it easier to explain users' performance.Thus, it can be used as a better-quality model to help software engineers solve development problems.Furthermore, structured, organized students activities for enjoyable learning, liked by students to help goal attainment, and increased understanding by young students are all positively impacted by educational games (Ibda et al., 2022).However, the question of how to guarantee the quality of educational games and what game designers may do in their development processes still remains a barrier (Ahmad et al., 2014).The findings of this study offer a fundamental understanding of the quality's characteristics (i.e., enjoyable, exciting, fun, interesting, relaxing, novel, creative, convenient, curious, fantastic, stimulating, splendid, effective, attractive) that can affect users' pleasure with game software.These findings show promise for software developers to create engaging educational games of the highest quality for students to increase their motivation for learning as well as their experiences, abilities, and critical thinking.
There were several limitations in this study.According to playability and usability research, it can be found that many definitions of usability and playability are based on the ISO/IEC series of standards.Even though ISO/ IEC 25000 SQuaRE has been considered by software developers as a relatively complete and accurate software quality assessment model, studies conducted by Koh and others have discovered many problems in using ISO/IEC 25000 SQuaRE.For example, the definition of usability is particularly complicated and ambiguous, and so is the definition of playability.This has caused a lot of questions on whether the ISO/IEC series of standards including ISO/IEC 25000 SQuaRE can be used as a reference for the correct quality model, and whether the usability and playability defined under the existing framework are correct.
This paper only focuses on one type of software (i.e., game software), and there is a difference between game software platforms, game software types and game software genres.Further studies about playability should focus on the perspective of game developers and stakeholders separately.Indeed, only university students from China and South Korea made up the study's sample population.The results are therefore not generalizable, and it is recommended that playability factors be investigated in other places, such as the Arab world or Western countries, in order to determine how they affect game participants, including game users (e.g., students), developers, and stakeholders.Third, this study is an exploratory one, and further research should use confirmatory factor analysis or other methods to confirm and validate this study's results.Furthermore, the questionnaire's items were designed based on limited available literature with 33 dimensions about game software experience.Specifically, there is no unified and identified groupings for dimensions (e.g., attractive belongs to the attractiveness factor for Chinese respondents but it also belongs to the joy factor among South Korean respondents).Further studies should replicate and test those dimensions' effect on either users' satisfaction or stakeholders' expectations to further confirm its validity and robustness.
h.What did the most memorable game feel from the games you played last week?Please indicate the degree of relevance in the table below.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Figure 1 .
Figure 1.Annual number of documents issued in game software quality.

Figure 2 .
Figure 2. Co-occurrence analysis of game software.

Figure 3 .
Figure 3. Co-occurrence analysis of game software playability.

Figure 4 .
Figure 4. Co-occurrence analysis of game software usability.

Table 1 .
Result of Descriptive Statistics.

Table 2 .
Comparison of Factor Analysis Results Between Chinese and South Korean Respondents.