Assessment of ICT in Tertiary Education Applying Structural Equation Modeling and Rasch Model

Assessment studies using structural equation modeling and Rasch Model have long been prominent in all areas of educational psychology. Their emphases have covered, but have not been limited to, the improvement of quality in teaching, learning, and research as well as the pursuit of research productivity. However, such methodological approaches have rarely taken place in the arena of information and communication technology (ICT) in tertiary education. As such, this study assesses Chinese lecturers’ expression of ICT satisfaction through the technology satisfaction model (TSM). Data from 196 Chinese lecturers were collected using stratified random sampling and analyzed applying structural equation modeling and the Rasch model. The findings suggested that Chinese lecturers’ satisfaction around ICT use was explored by the three underlying antecedents of computer self-efficacy, perceived usefulness, and ease of use. Computer self-efficacy was evidently a powerful exogenous variable in assessing direct and indirect causal associations among the dimensions of TSM. In future research, it is recommended that assessment of Chinese lecturers’ ICT use for teaching and research purposes should be considered through structural equation modeling and the Rasch model. This would support future researchers, academicians, and practitioners in performing technological measurements in all the arenas of tertiary education.


Introduction
The procedures of identifying, gathering, and interpreting information about learning outcomes are central to what "assessment" means in educational contexts. Therefore, measurement directed toward what is important in such processes is critical. In general, assessment is an indispensable aspect of teaching, training, and learning. Moreover, it is widespread within the systematic support of learning, both in training and in formal education (Farrell & Rushby, 2016). According to Bennett et al. (2017), assessment is an important factor in student engagement, as it has a critical impact on both students' learning and its certification. Furthermore, it could be highly beneficial to consider assessment design as a procedure supporting formative development rather than multiple iterations. Doing this could make the initial stakes lower, achieve a gradual roll out over time, predict possibilities for collecting evidence, and facilitate the management of resourcing and workloads. Such strategies are familiar to large educational projects or instructional design work but, comparatively, are much less frequently used in routine design work.
Assessment studies have long been prominent in all aspects of educational psychology (Baars et al., 2018;Graham, 2018;Halberstadt et al., 2018;Lee & Vlack, 2018;Martin & Lazendic, 2018;Mok et al., 2017;Zhu et al., 2018). Its emphases have covered, but have not been limited to improvement in the quality of teaching, learning, research, and the pursuit of research productivity. Contemporary psychologists have conducted numerous assessment investigations that apply computer-based analytical tools such as structural equation modeling (SEM; Gibbons et al., 2018;Lee & Vlack, 2018;Scherer et al., 2017;Zhu et al., 2018), hierarchical linear modeling (Areepattamannil & Khine, 2017), the Rasch model (Campbell & Bond, 2017;Mok et al., 2015), partial least squares (Onn et al., 2018), and computerized adaptive testing (Buuren & Eggen, 2017;Huebner et al., 2018). However, despite a decade of practice, the literature suggests that researchers or psychometricians may need to better comprehend and apply these analytical tools to improve the quality of practice measurements in higher education. In particular, the combination of SEM and the Rasch model has not been adequately pursued in modern assessment strategy. Moreover, assessment studies have rarely taken place in the arena of information and communication technologies (ICTs) as effective resources for tertiary education. In the arena of educational technology, researchers frequently use exploratory factor analysis (EFA) and SEM to perform assessments that focus on the reliability and validity of items rather than on the individuals being studied. The Rasch model considers both the items and the respondents in developing and validating the instrument using dichotomous and polytomous data. In such a case, we assumed that combining these two analytical tools would provide a great opportunity for conducting meaningful measurements, which could potentially be generalized and used in other situations. Coniam and Yan (2016) claimed that the widespread development of ICT has had progressively powerful effects on numerous aspects of education, including assessment. Hence, it is assumed here that Chinese university lecturers' ICT skills would be valuable for performing their measurement activities, if they frequently used ICT for teaching, research, and learning purposes. Despite the ubiquity of ICT applications, there has been a scarcity of investigations into the contributing dimensions of lecturer satisfaction with ICT as encountered in the tertiary education environment (Islam, 2015). According to Wu et al. (2010), satisfaction is the most recognized measure of the quality and usefulness of teaching and learning. Along these lines, Islam et al. (2019) identified that teachers' satisfaction depends on the benefits of using digital technologies. For instance, the effect of teachers' perceived usefulness (PU) of and satisfaction with new technologies in teaching and research manifested in teachers increasing their research productivity and, enhancing their research skills, making information easier to find, and providing the latest information on specific areas of research.
In the Chinese educational setting, explorations related to the pedagogical use of technologies have grown exponentially (Teo et al., 2018). However, the majority of studies have considered preservice (Teo et al., 2018) and inservice (Teo & Zhou, 2017) teachers' use of technology. On the other hand, considering ICT's underlying benefits, like promoting educational and academic achievement, China aims to speed up the digitalization process in the field of education. China intends to do this, through engaging educational digitalization within a developmental strategy of national digitalization conceived in a holistic perspective, and it is currently implementing an educational information network. Hence, it is necessary to determine what facets of the current situation are affecting Chinese educators' uptake of technologies for educational purposes (Teo et al., 2018). Although technology plays a crucial role in promoting efficient instruction, there is still evidence that teachers do not always apply technology in a way that maximizes its effect on teaching and learning (Teo & Zhou, 2017). In such a case, satisfaction could be one of the reasons why teachers or lecturers are not willing to use ICT frequently in their teaching and research activities in China. Therefore, this study aims to assess the components of ICT satisfaction for Chinese lecturers through the technology satisfaction model (TSM) by applying SEM and the Rasch model.

Literature Review
In the contemporary literature on the acceptance or adoption of ICT application, the most recognized approach is the technology acceptance model (TAM) claimed by Davis et al. (1989). Technology acceptance studies have been globally conducted using TAM (Al-Azawei et al., 2017;Fathali & Okada, 2018;Holzinger et al., 2011;Sánchez-Prieto et al., 2017;Yim et al., 2019). The TAM was generated from the theory of reasoned action (TRA) as first presented by Fishbein and Ajzen (1975), which is shown in Figure 1. The TAM was designed to measure the causal associations among six dimensions such as external variables, PU, perceived ease of use (PEOU), attitude toward using, behavioral intention to use and actual system use of computers. TAM predicted that PU and PEOU are the two specific beliefs that are most relevant to computer usage behaviors. On the other hand, the TRA is a well-known and often-used theory of human behavior that consists of beliefs and evaluations, normative beliefs and motivations to comply, attitude toward behavior, subjective norms, behavioral intention, and actual behavior, as indicated in Figure 2. However, these theories or models (TAM and TRA) are poorly theorized in educational psychology. This means that these models did not have psychological constructs such as computer selfefficacy (CSELE) and satisfaction.
Thus, this investigation has adopted an earlier model, the TSM. This is because, the TSM (Islam, 2014) has combined the TAM as well as educational psychology theories, which have been extensively applied by educational psychologists (Liu et al., 2018;Lodewyk, 2018;Phan et al., 2018), such as social cognitive theory (Bandura, 1986). Self-efficacy is the key component of social cognitive theory, a theory that has been contributing widely in various arenas of education. The TSM has also included an intrinsic motivation component, namely "satisfaction." The TSM posits that users' satisfaction with technology is driven by three factors: the technology's PU, its PEOU, and the user's CSELE.
The TSM has proven "strong predictive power" in gauging students' satisfaction with new technology (Islam, 2014). However, it is not yet clear whether this model could be viable to assess factors underlying Chinese university lecturers' ICT satisfaction. On the other hand, the TSM literature indicates that it previously did not measure how lecturers' PEOU could have a direct impact on the PU of ICT. Subsequently, the TSM also did not explore how lecturers' CSELE would have an indirect impact on PU mediated by their PEOU of ICT. As such, the present research includes these associations within the model as displayed in Figure 3. The associations among the features of the TSM are described in the next section to generate the present hypotheses, along with operational definitions of the terms that have been used with the model.

Hypotheses of the TSM Model
"Self-efficacy" refers to the extent to which a person has the belief that he or she can master a specific skill. Self-efficacy beliefs function as a vital set of proximal determinants of human action, motivation, and affect, influencing action through cognitive, affective, and motivational intervening processes (Bandura, 1989). In line with this definition, modern psychologists (Cheema & Skultety, 2017) have claimed that self-efficacy defines the belief of a person in his or her own capability and ability to succeed in accomplishing a given assignment. Thus, subject-specific self-efficacy means a measurement of self-confidence in a personal capacity to accomplish tasks associated with that subject. It is not necessary for this confidence to mirror actual ability in the subject: it can be an under-or overestimation of true ability. As a result of emerging educational technology, researchers have introduced the idea of "computer self-efficacy," which refers to lecturers' confidence in their ability to use new technologies (Islam, 2015). CSELE was found by many researchers to be associated with students' PEOU and PU of various ICT applications such as computer simulation (Liu & Huang, 2015), wireless internet (Islam, 2014;Islam et al., 2018),  research databases (Islam & Sheikh, 2020), and electronic textbooks (Chiu, 2017). However, this association has not been frequently demonstrated in Chinese university lecturers' ICT use. On the other hand, recent studies have claimed that self-esteem (Cheng, 2018) and social presence (Huang, 2017a) are significantly associated with PEOU and PU. Due to these mixed associations, this investigation predicts the following: Hypothesis 1 (H1): Chinese university lecturers' CSELE will be directly associated with their PEOU of ICT applications. Hypothesis 2 (H2): Chinese university lecturers' CSELE will be directly associated with their PU of ICT applications.
The TAM model (Davis et al., 1989) explored the idea that PEOU and PU are the main components of user technology acceptance. Moreover, the TSM (Islam, 2014) recently discovered that these two antecedents are solely powerful in measuring learner satisfaction. To explain these two factors, the present study reports their recent definitions as related to ICT. For instance, Islam (2015) suggested that "perceived usefulness" refers to the benefits perceived by lecturers through using ICT, while "perceived ease of use" means the ease of using ICT. Studies have identified these two influential dimensions as key in estimating learner satisfaction (Chen et al., 2020;Islam, 2015;Joo & Choi, 2016;Masrek & Gaskin, 2016). However, the majority of these investigations have still claimed an association with either attitude (Cheng, 2018;Chiu, 2017;Huang, 2017b;Liu & Huang, 2015) or intention to use new technology (Poong et al., 2017). Against this background, the present research will validate similar associations, which were generally in line with the findings of TSM in the case of Chinese lecturers' ICT satisfaction and predicts the following: Hypothesis 3 (H3): Chinese university lecturers' satisfaction in using new technology will be directly associated with its PEOU. Hypothesis 4 (H4): Chinese university lecturers' satisfaction in using new technology will be directly associated with its PU.
Research related to technology acceptance has shown that learners' PEOU is associated with the PU of new technologies (Cheng, 2018;Chiu, 2017;Huang, 2017b;Poong et al., 2017;Yuan et al., 2016). Except for a few current studies (Huang, 2015;Huang, 2017b), such associations were generally in line with the results of TAM (Davis et al., 1989), while the TSM did not include this relationship (see Figure 1). Therefore, the current research incorporates this association into the original TSM and predicts the following: Hypothesis 5 (H5): Chinese university lecturers' PEOU of the new technology will be directly associated with their perception of its PU.
Above all, the hypotheses have been constructed based on the direct associations of exogenous, mediating, and endogenous variables of the TSM as discussed throughout extensive literature. Nevertheless, prior investigations have not adequately explored how mediating variables, such as PEOU and PU, can establish indirect associations between either students' or lecturers' CSELE and their satisfaction with new technologies in higher education. For instance, Bin et al. (2020) found that technical and vocational college teachers' perception of self-efficacy and subsequent satisfaction with new technologies were indirectly influenced by the technologies' PU and PEOU. Chen et al. (2020) claimed that the online database adoption and satisfaction (ODAS) model confirmed that the PEOU and PU of databases are indirectly associated with postgraduate students' CSELE and satisfaction with the databases. According to Islam (2015), "lecturer satisfaction" is the degree to which the use of new technology is in line with lecturers' present values, needs, and experiences. However, the TSM validated the mediating relationships among its constructs based on a student sample. Thus, the present research anticipates that these mediating variables will play the same role for Chinese lecturers and so assumes the following: Hypothesis 6 (H6): Chinese university lecturers' PEOU and PU will together mediate the indirect association between CSELE and satisfaction in using ICT. Hypothesis 7 (H7): Chinese university lecturers' PEOU will mediate the indirect association between CSELE and PU of ICT. Hypothesis 8 (H8): Chinese university lecturers' PU will mediate the indirect association between PEOU and satisfaction in using ICT.

Method
This study was undertaken at one public university in China. Before collecting the data, the researchers gained ethical clearance and informed consent from the participants. The investigation adopted a 56-item survey, originally developed by Islam (2015), which was administered to 200 lecturers. The authors selected lecturers from seven colleges of a university where 811 faculty members were employed. However, four respondents were not included in the final study due to their incomplete feedback. Before administering the survey, the questionnaire had gone through the translation and back-translation technique to assure that the meanings of the adapted English version of the instrument were not different in the translated Chinese questionnaire. Next, the questionnaire was systematically tested for face validity, and a pilot study using the Rasch model rating scale was carried out. In this empirical research, we adapted the Rasch model for validating the psychometric properties of the items and persons involved. Through using the Rasch model, we obtained a large number of valid items due to the minimum likelihood estimation. However, the majority of the studies related to educational technology have been ignored to validate the persons, who are important for the real measurement. A 56-item survey contained four domains, including the background information of Chinese lecturers, as shown in Table 1. The data from Chinese lecturers were collected using a stratified random sampling procedure and were analyzed by applying SEM and the Rasch model.
In total, 196 useable Chinese lecturers' responses were collected from seven colleges of a public university in China, and they were divided into two groups according to gender. In this study, female lecturers (52%) were slightly more represented than male lecturers (48%). Most of the Chinese lecturers were aged between 30 and 44 years, and their years of experience ranged from 6 to 10 years, as indicated in Table 2. The majority of Chinese lecturers had master's degrees (48%), and they also held the position of lecturer (53%) during this study.
The Rasch model has been frequently applied in education, business, psychology, health, and other social sciences. However, researchers have not yet developed and validated the instrument using the Rasch model, especially in ICT assessment in Chinese higher education. Hence, this research validated the technology satisfaction scale for investigating Chinese lecturers' satisfaction with new technologies. The scale's reliability (e.g., summary statistics and item polarity map) and validity (e.g., item fit order, item map, and principal components) were tested using Rasch analysis through Winsteps version 3.49, and the detailed Rasch outputs are reported in Table 3. The results of this study supported the theoretical structure of the psychometric properties as involving CSELE, satisfaction, PEOU, and PU of ICT. In total, 56 items fitted the model, and the scales had good psychometric properties. The proportion of variance explained by the measures was 65.8%, which indicated that the items were able to reflect the Chinese lecturers' satisfaction with new technologies. As exhibited in the item map (see Figure 4), Chinese lecturers are allocated on the left side of the scale, and items are allocated on the right side. Lecturers with perceived higher ability in using ICT in terms of their CSELE, satisfaction, PU, and PEOU are located higher on the scale. On the other hand, the items that were more complicated for the lecturers to agree on are located higher on the scale. The item map indicates that overall, Chinese lecturers' abilities were higher on the scale than the items' difficulty, and thus, in general, lecturers tended to agree that providing ICT facilities was satisfactory for teaching and research purposes in their university. Comparably, few lecturers (located below Items peu4 and peu6) did not agree that their university provide adequate ICT facilities to support their teaching and research activities. Therefore, some difficult items were required to measure those lecturers with strong satisfaction. For this reason, inclusion of more difficult items was recommended for future studies.  Meanwhile, the outfit mean square score for Item peu11 and the infit mean square score for Item grat5 were slightly larger than the suggested range of 0.5 to 1.5 (Bond & Fox, 2001) as displayed in Table 3. As such, these two items were excluded when conducting further analyses (see the Supplemental Appendix).

Findings
A 54-item instrument was validated through the Rasch model and was considered valid to assess four facets of the measurement model, namely, CSELE, satisfaction (SF), PU and PEOU using confirmatory factor analysis (CFA) in regard to their convergent and discriminant validity. First, a four-facet measurement model was evaluated using few fit statistics such as chi-square/degree of freedom (χ 2 /df ≤ 3), root mean square error of approximation (RMSEA ≤ .08 or .1), comparative fit index (CFI ≥ .90), and Tucker-Lewis index (TLI ≥ .90) based on the recommendation of statisticians who suggested these indices (Hu & Bentler, 1999). However, according to the results of CFA, the measurement model did not fit the empirical data as indicated in Figure 5: χ 2 = 4349.551, df = 1371, p = .000, RMSEA = .106, CFI = .706, and TLI = .693. The output of AMOS version 18 for the measurement model showed that several indicators were loaded on each other as contradicting with the measurement criteria. Against this backdrop, the model needs to be respecified, while all the indicators were significant.
All these parameters had to be excluded due to the model's fitness, multicollinearity as well as convergent and discriminant validity. Furthermore, the correlation matrix confirmed that there is no indication of multicollinearity among the indicators as none of coefficients exceeded the cutoff point of 0.85 (Fornell & Larcker, 1981). The fourfacet re-specified model revealed a significant level of loadings ranging from 0.72 to 0.89 for all the constructs and their interrelationships, as shown in Figure 6.
This research obtained the values of correlations between the dimensions and standardized regression weights through AMOS output to calculate the convergent and discriminant validity. Table 5 exhibits the composite reliability (CR) and average variance extracted (AVE) scores related to convergent validity (Fornell & Larcker, 1981;Hair et al., 2010) where all the CR and AVE scores are above 0.853 and 0.592, respectively.
In addition, as shown in Table 7, bootstrap provided evidence of a 95% confidence interval for the significant hypothesized direct and indirect paths. The significant direct path from PEOU to PU had a large effect size, whereas the direct paths from CSELE to PEOU and PU to SF had medium effect sizes. The direct paths from CSELE to PU and PEOU to SF had small effect sizes. On the other hand, the significant indirect paths from CSELE to SF (mediated by PEOU) and PEOU to SF (mediated by PU) had large effect sizes. Moreover, the indirect path from CSELE to PU (mediated by PEOU) had a small effect size.
The standardized direct and indirect effect sizes of the TSM along with its variance and summary of hypotheses are reported in Table 8.

Discussion
The outcomes of assessment in regard to ICT presented in this study have theoretically, practically, and methodologically contributed to the body of scientific knowledge related Note. RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker-Lewis index; CSELE = computer self-efficacy; SF = satisfaction; PU = perceived usefulness; PEOU = perceived ease of use.
to various aspects of education. The results of the present research prove that Chinese university lecturers' computer ability is the main factor in perceiving ICT as useful, and lecturers do not face difficulties in using it for their academic purposes. Hence, these findings suggest that academics and researchers should consider advancing lecturers' CSELE. Thus, they can achieve more benefits through ICT applications and confront less complexity in accomplishing their work, as it could be performed through the effective training courses or workshops. In line with the results of this research, investigations found that learners' CSELE was associated with the PEOU and PU of various ICT applications (Chiu, 2017;Islam, 2014;Liu & Huang, 2015). However, Cheng (2018) and Y. M. Huang (2017a) asserted that learners' self-esteem and social presence were associated with PU and PEOU. This research confirmed that the mediating variables of TSM (Islam, 2014), such as PEOU and PU, are the important  direct determinants of Chinese lecturers' satisfaction, while the TAM (Davis et al., 1989), along with other researchers, stated that they are valid antecedents of technology acceptance (Joo & Choi, 2016;Masrek & Gaskin, 2016). These results provide a strong recommendation for academicians and service providers. As an example, lecturers' satisfaction greatly depends on their experiences while using new technology. As such, to extend lecturers' satisfaction, the authority of service providers should certify that high-quality ICT facilities are available within the academic environment so that lecturers can gain desirable advantages and will not perceive difficulties in performing their academic related jobs.
Along these lines, academicians should also conduct ongoing assessment in regard to the quality of ICT services so that better teaching, learning, and research environments will be implemented in tertiary education. However, researchers have still identified that the TAM variables are associated with not only attitude (Cheng, 2018;Chiu, 2017;Huang, 2017b;Liu & Huang, 2015) but also intention to use technology (Poong et al., 2017). Meanwhile, the relationship between PEOU and PU has been widely expounded upon in the literature. This study also explored how Chinese university lecturers' perception of the usefulness of ICT is affected by their experiences in terms of accessibility, such as how easy it is to use: an outcome that is consistent with the TAM-based (Davis et al., 1989) result including new studies (Cheng, 2018;Chiu, 2017;Huang, 2017b;Poong et al., 2017;Yuan et al., 2016). On the other hand, in the previous study, the TSM (Islam, 2014) did not recognize the significance of this result, which can provide insights for future researchers to identify the strategies of getting more benefits through ICT.
Finally, the results related to mediating effects of PEOU and PU generated a new platform for future researchers. For instance, the TSM (Islam, 2014) previously claimed that these mediating variables are helping learners to increase satisfaction because of their CSELE, which is applicable for this study in measuring Chinese lecturers' satisfaction. These findings showed multiply-mediating associations between lecturers' computer ability and their satisfaction in regard to ICT use and acknowledged that lecturers' satisfaction is also dependent on their beliefs in ICT ability. So, it is imperative for future researchers to take necessary initiatives to determine the various ways of improving their ICT skills. Due to the new relationship between PEOU and PU, this research also indicates the indirect associations between lecturers' self-efficacy and usefulness along with their ease of use and satisfaction.

Conclusion
Assessment is an ongoing process in identifying problems and proposing possible solutions for studies concerned with extending the quality of teaching, learning, and research on applying appropriate psychological or analytical tools. The combination of SEM and the Rasch model has been acknowledged as the real measurement for this study. The joint findings of these models suggest that the Rasch model explores a total of 54 valid items for validating all the constructs of the TSM due to the likelihood estimation and recommends them to be adopted for future research, although SEM analysis excluded several indicators for validating the measurement and structural models due to the maximum likelihood evaluation. Subsequently, advanced statistical analyses of the results revealed that the TSM has "strong predictive power" in gauging Chinese lecturers' satisfaction with ICT. For instance, Chinese lecturers' satisfaction with ICT use was explored through the three underlying antecedents of their CSELE, the technology's PU and PEOU, whereas CSELE was evident as being a powerful exogenous variable in assessing direct and indirect causal associations of the dimensions of TSM.

Implications for Policy and Practice
Assessment of Chinese lecturers' ICT use for their teaching and research purposes as well as applying SEM and the Rasch model will be essential for future researchers, academicians, and practitioners in performing technological measurements in all arenas of tertiary education. Nonetheless, the TSM found that Chinese university lecturers' CSELE can only explain 16% of the variability of their ease of use of ICT. This result strongly recommends that policymakers prioritize the extension of lecturers' required ICT skills before providing extensive new technologies. Interestingly, the relationship between PEOU and PU was found to be strongest in TSM. This was addressed extensively in the literature. Based on the current findings, it could be concluded that the research appears to be well constructed and sound and that the results will likely be beneficial for future researchers who wish to assess Chinese lecturers' or Chinese college and university students' satisfaction with technology. These findings may encourage policymakers to establish short courses or ongoing workshops for faculty members who will be involved in hands-on practice. These courses or workshops would focus on analytical tools such as the Rasch model and SEM with the goal of helping them to improve the productivity of their research and publication. To provide a better understanding of TSM and to examine its viability, regional or cross-cultural comparative investigations are recommended. This study assumes that there may be an interaction of the results and the participants' age and gender. Due to the small sample size, researchers did not explore the moderating effects of these variables, but they suggest that these effects be explored in future research. This study contributes to relevance in advancing knowledge in the field of ICT assessment. "Satisfaction" is a complicated concept to define, and this helps provide a model for doing so.