Relations between Students’ Study Approaches, Perceptions of the Learning Environment, and Academic Achievement in Flipped Classroom Learning: Evidence from Self-Reported and Process Data

This study examined the extent to which relations between students’ perceptions of the learning environment, academic achievement, and study approaches measured by both self-reported and process data were consistent amongst 323 engineering students in a flipped classroom course. A hierarchical cluster analysis on four self-reported scales identified deep and surface study approaches. Students reporting deep approaches obtained significantly better marks than those reporting surface approaches. An agglomerative sequence clustering on sequences of students’ online interactions found four observed study approaches: two focused on content and two focused on assessment. Students adopting content-focused approaches performed significantly better than those employing assessment-focused ones. Two cross-tabulations revealed consistency of relational patterns between perceptions of the learning environment and study approaches by self-reported or process data. Amongst students with better perceptions, a significantly higher proportion of them reported deep approaches than surface approaches. Amongst students using content-focused approaches, significantly higher proportions of them held positive perceptions than negative perceptions. The study results suggest to teachers that fostering a better learning environment, helping students understand how online and in-person components are integrated, and equipping them with knowledge and principles of flipped classroom learning would be useful to enhance students’ learning experiences in flipped classroom courses.

perceptions, a significantly higher proportion of them reported deep approaches than surface approaches.Amongst students using content-focused approaches, significantly higher proportions of them held positive perceptions than negative perceptions.The study results suggest to teachers that fostering a better learning environment, helping students understand how online and in-person components are integrated, and equipping them with knowledge and principles of flipped classroom learning would be useful to enhance students' learning experiences in flipped classroom courses.
Keywords study approaches, self-reported and process data, perceptions of the learning environment, academic achievement, flipped classroom, Australian engineering students Over the last few decades, learning in higher education has undergone significant transformations and innovations.In particular, flipped classroom learning design has gradually gained popularity worldwide (Cho et al., 2021).Flipped classroom learning design is a specific type of blended learning design and requires students to engage in "interactive content focusing on key concepts prior to class thus allowing class time for collaborative activities that clarify concepts and contextualize knowledge through application, analysis, and planning and producing solutions" (Karanicolas et al., 2018, p. 1).Accordingly, university students' experiences of learning in flipped classrooms are becoming increasingly complex as students are often required to move back and forth between in-class and on-line learning environments.In these experiences, students not only interact with the human elements (teaching staff and fellow students), but also with the non-human or material elements (Fenwick et al., 2015).For instance, students spend a significant proportion of time navigating online learning platforms where they interact with a variety of technology-enabled learning tools, engage in online discussion forums, and move across physical and online learning spaces (Fenwick & Dahlgren, 2015).Transforming traditional learning and teaching into flipped classroom learning has an urgent need to understand how students perceive their learning environment (perceptions of the learning environment), how they approach learning (their study approaches), and the relations between their perceptions, approaches, and their academic achievement, in flipped classroom learning (Karabulut-Ilgu et al., 2018).
Traditionally, investigations into student learning in higher education from research areas in educational psychology, learning sciences, and pedagogy and curriculum studies, have predominantly relied on collecting students' self-reported data.Recent developments in the areas of learning analytics and educational data mining have the capacity to collect and analyse comprehensive process data in technology-mediated learning, which not only provides relatively objective descriptions of students' online study approaches, but also reflects dynamic and nuanced differences in how students approach online learning (Baker & Siemens, 2014).
However, both self-reported and process data have limitations.The self-reported data has received criticism for lacking objectivity (Matcha et al., 2020;Zhou & Winne, 2012).In addition, the responses to self-reported questionnaires may be affected by careless answering and item nonresponse (Hitt et al., 2016;Zamarro et al., 2018).Reliance on process data without guidance from educational theories has also received criticism for being data-centric, which can result in erroneous interpretation due to a lack of meaningful theory and context (Buckingham Shum & Crick, 2012).
To address the above-mentioned drawbacks, researchers have started to examine consistencies between self-reported and process data to describe students' learning (Gašević et al., 2017;Han & Ellis, 2020;Han et al., 2020;2022).Such investigations not only enable a triangulation between different data sources to describe student learning, but also provide complementary information on students' reporting of their motives for using certain study approaches and their actual use of study approaches (Reimann et al., 2014).The current study aims to extend this area of research by investigating the relations between students' perceptions of the learning environment, their academic achievement, and their study approaches measured by self-reported data collected from a questionnaire and process data collected in the Learning Management System (LMS) in a flipped engineering course.

Students' Perceptions of the Learning Environment and their Self-Reported Study Approaches
How students perceive the situational characteristics of their learning context, including the curriculum, course structure, and teaching and delivery methods, have been consistently identified as an important aspect in students' learning experience (Ellis et al., 2006;Guo et al., 2017;2022).To describe the interrelatedness of the key aspects of students' learning experience and outcomes, Biggs (1989) have proposed a Presage-Process-Product model (known as 3P model).
The Presage stage consists of both students' personal attributes and the situational characteristics of their current learning environment.The Process stage may include aspects such as, how students perceive the learning environment (their perceptions) and how they go about learning (their study approaches).The Product stage is concerned with various measures of students' learning outcomes, such as the course marks and students' post conceptions of the subject matter.The elements in the 3P model are relational and coexisting rather than indications of causal or linear relations.Past research has shown that in some learning contexts, the elements in the Process stage can mediate the relations between the Presage stage and the Product stage, whereas in other contexts, the elements in the Presage stage are directly associated with elements in the Product stage (Trigwell & Prosser, 2020).
The two key factors which have been recognized in the Process stage are perceptions of the learning environment and study approaches.Two broad categories of approaches (deep and surface study approaches) have been consistently identified in various academic disciplines.Deep study approaches are motivated by an attempt to understand the deep meaning of subject matter, and to link theoretical concepts in learning with contextualized applications.Deep study approaches are typically characterized by taking initiative, while also being independent and reflective of learning.On the other hand, surface study approaches are motivated by the aim to satisfy course requirements and learning tasks.They use strategies which are simplistic and mechanistic, rely heavily on textbooks and course notes, and are dependent on peers and teachers (Nelson Laird et al., 2014).The two categories of study approaches have also been extended to the domain of using online learning technologies.The deep approaches to using online learning technologies employ technologies with an aim to facilitate learning and to deepen understanding of the subject matter; whereas surface approaches to using online learning technologies adopt technologies mostly to fulfill practical purposes, such as downloading files (Ellis & Bliuc, 2016;2019).
Study approaches are not personal traits and can be consciously chosen according to different learning contexts and situations (Entwistle & Peterson, 2004;Joshi & Lau, 2021;Rajaratnam, D'cruz, 2016;Ramsden, 2003).In particular, past research has consistently shown that students' perceptions of the teaching and learning environment are associated with the study approaches they adopt.Deep study approaches tend to be adopted by students who perceive teaching as being of good quality and with clear aims, and assessment tasks being appropriate and coherent with the course aims.Students who perceive that the workload is inappropriate, the learning objectives are unclear, and teacher-student interactions are lacking, are more likely to employ surface study approaches (Guo et al., 2017;Lizzio et al., 2002;Wilson & Fowler, 2005).
Research has also shown that in blended course designs, when students perceive that face-to-face and online elements are well integrated, the online learning workload is appropriate, and online contributions are of value, they are more likely to adopt deep approaches to study and to using online learning technologies.Comparatively, when students do not see the relevance between face-to-face and online learning, and do not appraise the online learning, they tend to adopt surface approaches and limit their use of technologies in learning (Ellis & Bliuc, 2016;2019).Coherent associations have also been found between students' perceptions of the learning environment, study approaches, and their academic learning outcomes, such that: positive perceptions, deep approaches, and higher academic performance tend to be related; negative perceptions, surface approaches, and poorer academic performance tend to be associated (Entwistle & Peterson, 2004;Guo, 2018;Lonka et al., 2004).
However, the existing research on the relations between students' perceptions of the learning environment and their study approaches has mainly employed self-reported measures to identify students' study approaches.Apart from the above-mentioned limitations of lacking objectivity and issues with students' careless responses, selfreported measures also lack capacities to represent the complex and dynamic nature of students' study approaches in the blended course designs where students traverse back and forth between physical and virtual spaces and integrate ideas across learning activities, resources, and technologies (Hadwin et al., 2007;Han, 2022;Matcha et al., 2020).To improve insights into understanding contemporary university students' experiences of learning, suggestions have been put forward to expand the current selfreporting methods by including other types of measurements to study student learning (Vermunt & Donche, 2017).In this regard, the process data frequently employed in learning analytics research seems to be promising as such data "provides both researchers and practitioners with the opportunity to monitor students' strategic decisions in online environments in minute detail and in real time" (Richardson, 2017, p. 359).

Observed Students' Study Approaches Measured by Process Data
The recent development of educational technology has produced prolific studies using process data, which enables researchers to collect rich and detailed digital traces of students' interactions with a variety of online learning resources and activities.The process data has the advantages of not only offering descriptions of student learning behaviors more objectively but also in more granular details than using self-reported methods (Gašević et al., 2015;Siemens & Gašević, 2012).Process data has been increasingly employed in various domains in the higher education sector to fulfil multiple purposes, such as advising students' career choice (Bettinger & Baker, 2014); detecting at risk students to reduce dropout rates (Krumm et al., 2014); providing personalised feedback in learning (Gibson et al., 2017); facilitating collaborative learning (Kaendler et al., 2015); monitoring students' affect (Ocumpaugh et al., 2014); identifying patterns of study strategies and approaches (Chen et al., 2017); and predicting students' academic learning outcomes (Romero et al., 2013).
Earlier research in learning analytics predominantly used frequency data to describe students' online learning experiences.More advanced LMSs are able to collect many different types of process data, such as: what types of online learning actions students generate; the proportions of each type of learning action; the total duration of students' online access; the time spent on a certain online learning task; and the sequential order of the learning actions with precise timestamps (Chen et al., 2017;Matcha et al., 2019;2020).Using data mining techniques, such as Hidden Markov Model (HMM), agglomerative sequence clustering, and process mining algorithms; the complexity of students' study approaches can be represented (Dawson et al., 2017;Fincham et al., 2019;Jovanović et al., 2017;Matcha et al., 2019;2020).
In a number of studies, similar approaches have been taken to analyse the sequences of time-stamped online learning events by using a two-step method.While the first step focused on identifying common study strategies shared by all the students, the second step focused on identifying study approaches by categorizing sub-groups of students who shared similar study strategies.In the first step, the types of online learning events were processed by HMMs to generate a limited set of learning sequence types by similar sequential distributions of the online learning events, which represented students' study strategies.In the second step, the generated learning sequences were processed by agglomerative sequence clustering on student population sample to identify students' study approaches.
For instance, using the above-mentioned two step method, Jovanović et al. ( 2017) identified five different types of online study approaches amongst 290 computer science undergraduates: • "the intensive learners": they used a variety of study approaches; • "the strategic learners": they prioritized summative and formative assessment tasks; • "the highly strategic learners": they emphasized summative; • "the selective learners": they focused on summative activities with few reading activities; • "the highly selective learners": they only performed summative activities.2017) also compared students' academic performance by their study approaches.Results showed that "the intensive learners", "the strategic learners", and "the highly strategic learners" obtained higher marks than "the selective learners" and "the highly selective learners" on both mid-term and final examinations.

Jovanović et al. (
Adopting a similar method, Han et al. ( 2022) investigated Australian' engineering students' study approaches using process data and identified four distinct approaches, namely: intensive theory application; moderate theory application; weak theory application and moderate theoretical testing; and weak reading and weak theory application.Similar to Jovanović et al. (2017), Han et al. found that students who used different study approaches also differed on their academic achievement.Students who employed intensive theory application strategy achieved the highest examination scores, whereas those who used weak reading and weak theory application scored the lowest examination scores.This study further examined the relations between students' perceptions of the learning environment and their study approaches and found logical relations between perceptions and approaches.Amongst students who adopted the intensive theory application strategy, the proportion of students who self-reported better perceptions was significantly higher than those reporting poorer perceptions.In contrast, amongst students using the weak reading and weak theory application strategy, the proportion of students having poorer perceptions was significantly higher than those holding better perceptions.
The study by Han et al. (2022) addressed the limitation of predominant use of selfreports to examine students' study approaches in the investigations of the relations between perceptions and approaches.However, it was not able to provide evidence as to whether the students' study approaches measured by self-reported and process data are similarly or differently associated with students' perceptions of the learning environment.The current study aims to fulfil this purpose.Moreover, the current study will also extend previous investigations in general blended course designs into a specific type of course designthe flipped classroom design.The current study addresses three research questions: 1. What are students' study approaches in flipped classroom learning identified by self-reported and process data? 2. To what extent are students' perceptions of the learning environment in flipped classroom learning are associated with their study approaches by self-reported and process data? 3. To what extent do students' academic achievement in flipped classroom learning differ based on their study approaches by self-reported and process data?

Method
Participants and Recruitment Procedure The participants of the study were 323 undergraduates recruited from a large metropolitan research-intensive Australian university.All the participants were first-year students who majored in Bachelor of Computing Engineering.The range of their age was from 17 to 31 years old, with an average of 20 years old.Of 323 students, 170 were males and 153 were females, accounting for 52.6% and 47.4% respectively.The percentages of male and female students showed a relatively equal gender distribution.
The recruitment of the participants strictly followed the ethical requirements of the researchers' institution.All the participants signed a written consent form.

Course Design
This engineering course, which lasted a full semester, was offered to first-year computing engineering students.The course covered foundational concepts of a computer system: including hardware and software components of a computer, principles of computer architecture, digital logic design, the Internet and servers, steps in program development and design, and programming style and documentation.Apart from content learning, the course also aimed to develop students' graduate skills of inquiry and research competence in areas such as formulating appropriate questions, searching and synthesizing ideas, and evaluating and critically reflecting on evidence.
The course adopted a flipped classroom learning design.The online course site was hosted in a bespoke learning management system (LMS), which was comprised of the following learning activities: • pre-lecture videos: covered the learning and teaching in the forthcoming week, including: 1) theoretical concepts and topics to be explained in the lectures; 2) the sample problem-solving sequences to be demonstrated in tutorials; and 3) the laboratory instructions and the tasks to be completed in the laboratory sessions; • pre-lecture quizzes: consisted of quizzes for students to check their learning and understandings of the pre-lecture videos; • post-lecture quizzes: consisted of quizzes of problem-solving sequences, which required students to apply theories to solve practical problems.
The in-person part of the course consisted of the following elements: • 2-hour weekly lectures: recapped and expanded the concepts from pre-lecture videos.
• 2-hour weekly tutorials: theories to solve practical issues were first demonstrated by tutors; students were asked to work in pairs or in groups to go through problem-solving sequences, including calculations, model configurations, and mini case studies.The tutors facilitated the process by providing timely feedback and answering questions raised by the students.• 3-hour weekly laboratory sessions: students worked in pairs or in groups to gain hands-on skills through completing design projects, such as design and configuration of an electric circuit.

Data and Instruments
Self-reported Data Collected by a Questionnaire.A 5-point Likert scale questionnaire was used to collect self-reported data.The questionnaire consisted of six scales.Two scales assessed students' perceptions of the learning environment, two measured students' approaches to learning through inquiry, and two evaluated students' approaches to using online learning technologies.These scales have been used and validated in previous studies that investigated university students' learning experiences (Ellis & Bliuc, 2016;2019;Han & Ellis, 2020;Han et al., 2020).The descriptions of the six scales and coefficient H reliability are reported below: • perceptions of the integrated learning environment scale (7 items, H 1 = .89):examined students' perceptions of the level of integration between the online learning and the face-to-face learning components (e.g., The online activities in this course help with the work we do in class).• perceptions of the online contributions scale (6 items, H = .89):assessed students' perceptions of the value of online contributions in the course (e.g., Online contributions in this course motivated me to think about things more).• deep approaches to learning through inquiry scale (4 items, H = .86):described approaches that had characteristics of taking initiative, being proactive, and undertaking reflection (e.g., I often take the initiative when pursuing a line of questioning in research.).• surface approaches to learning through inquiry scale (7 items, H = .87):described approaches that were formulaic, highly dependent upon teaching staff, and without much reflection in the learning process (e.g., I only use the directions my teacher gives me when researching something for a task).• deep approaches to using online learning technologies scale (6 items, H = .83):evaluated using online learning technologies in meaningful ways to facilitate learning, such as to deepen their understanding of the concepts in the course, or to facilitate their learning through inquiry process in the course (e.g., I spend time using the learning technologies in this course to connect key ideas to real contexts).• surface approaches to using online learning technologies scale (5 items, H = .84):evaluated using online learning technologies in formulaic and mechanistic ways, such as downloading documents or fulfilling course requirements (e.g., I use learning technologies in this course mainly to download files).
Process Data Collected by LMS.The process data was collected by the learning analytic functions in the LMS, which recorded students' unique identification codes and students' interactions with different online learning activities with timestamps.The three types of timestamped online interactions were: 1) interactions with pre-lecture videos; 2) interactions with pre-lecture quizzes; and 3) interactions with post-lecture quizzes.
Academic Achievement.Students' academic achievement was represented by the course mark with the highest possible score being 100.The course mark was an aggregated score of lecture attendance, performance in the tutorials, and a close-book examination.
Lecture attendance was 10 best results out of 13 quizzes at the end of each lecture.The quizzes used a short-answer format, which tested important concepts covered in each lecture.Each quiz was scored between 0 and10.The maximum raw score of lecture attendance was 100, which was given 10% weights in the course mark.Performance in the tutorials was 10 best results of 13 quizzes at the end of each tutorial.The quizzes assessed students' abilities to apply theories in tackling problems through problemsolving sequences.Each quiz was scored from 0 to 10.The maximum raw score of performance in the tutorials was 100, which was weighted 40% in the course mark.The close-book examination evaluated both students' understandings of theoretical concepts using multiple-choice questions and their competence to use theories to solve practical problems using mini case studies.The maximum score of the close-book examination was 100, which was given 50% weights in the course mark.

Data Analysis Methods
To answer research question 1students' self-reported study approaches, we performed a hierarchical cluster analysis using the means of deep and surface approaches to learning through inquiry scales; and deep and surface approaches to using online learning technologies scales.Because cluster analysis is exploratory and does not offer a single best solution, the appropriate number of clusters needs be determined by the researchers.We used the values of the Squared Euclidean Distance between different cluster solutions and the meaningfulness of the results to decide upon the number of clusters (Murtagh & Legendre, 2014).We then conducted one-way ANOVAs on the four approaches scales using the cluster membership as the independent variable and the Ms of the four approaches scales as the dependent variables.To provide robust statistical analyses of the data, we calculated and reported 95% robust confidence intervals for one-way ANOVAs.The results of cluster analysis and one-way ANOVAs were used to describe students' self-reported study approaches.For students' study approaches by process data, we used agglomerative sequence clustering to cluster students based on the sequences of the three types of timestamped online interactions.We used clustering quality measures calculated in TramineR (Gabadinho et al., 2011) and the interpretability of the results to determine number of clusters.
To answer research question 2the association between students' perceptions of the learning environment and their study approaches by self-reported and process data, we first calculated the mean of the two perceptions scales and categorized students into better (above the mean) or poorer (below the mean) perceptions groups.Then we conducted two separate cross-tabulation analyses using groupings of perceptions of the learning environment and using either groupings of study approaches by self-reported data or groupings of study approaches by process data.
To answer the last research questionstudents' academic performance based on study approaches by self-reported data or by process data, we carried out two separate one-way ANOVAs using either the cluster membership of study approaches by selfreported data or the cluster membership of study approaches by process data as the independent variable and students' final mark as the dependent variable.We calculated and reported 95% robust confidence intervals for ANOVAs.

Descriptive Statistics
The descriptive statistics of the six scales in the questionnaire and students' course mark are presented in Table 1, including minimum and maximum values, Ms and SDs.

Results for Research Question 1 -Students' Study Approaches by Selfreported Data
The hierarchical cluster analysis produced a range of two-cluster to four-cluster solutions.The values of Squared Euclidean Distance measure revealed a relatively large increase in the value of a two-cluster solution compared to three-cluster and four-cluster solutions, suggesting a two-cluster solution was more appropriate.The labelling of the two clusters was in accordance with the literature of students' study approaches (Trigwell & Prosser, 2020).

Results for Research Question 1 -Students' Study Approaches by Process Data
Consulting with the clustering quality measures, we retained the four clusters used in the agglomerative sequence clustering.The four clusters represented four distinct study approaches by process data, which are visualized in Figure 1. Figure 1 schematically demonstrated the learning sequences, which consisted of the three types of timestamped online interactions with varying proportional distributions.Red colour represented students' online interactions with pre-lecture videos, green colour represented students' interactions with post-lecture quizzes, and blue indicated students' interactions with post-lecture quizzes.The four observed study approaches are described below: • observed study approach 1 (n = 51): extensive pre-lecture videos watching, moderate pre-lecture quizzes, and few post-lecture quizzes; • observed study approach 2 (n = 115): extensive pre-lecture videos watching, few pre-lecture quizzes, and moderate post-lecture quizzes; • observed study approach 3 (n = 128): moderate pre-lecture videos watching, few pre-lecture quizzes, and moderate post-lecture quizzes; • observed study strategy 4 (n = 29): few pre-lecture videos watching, few prelecture quizzes, and extensive post-lecture quizzes.

Results for Research Question 2 -Association between Students' Self-reported Study Approaches and Their Perceptions of the Learning Environment
A 2 (deep vs. surface study approaches) x 2 (better vs. poorer perceptions) crosstabulation analysis was conducted and the association was significant and moderate, χ 2 (1) = 22.83, p < .01,φ = .27.The proportions in Table 3 revealed that amongst students who held better perceptions of the learning environment, a significantly higher proportion of them adopted deep study approaches in the course (68.9%) relative to using surface study approaches (31.1%).In contrast, of students who held poorer perceptions, a significantly higher proportion of them used surface study approaches (59.4%), compared with employing deep approaches (40.6%).

Results for Research Question 2 -Association between Students' Observed Study Approaches and Perceptions of the Learning Environment
A 4 (four study approaches) x 2 (better vs. poorer perceptions) cross-tabulation analysis was conducted and the results show a significant and moderate association between students' observed study approaches and their perceptions of the learning environment, χ 2 (3) = 22.48, p < .01,Cramer's V = .26).The proportions in Table 4 show that amongst 51 learners who adopted observed study approach 1 (extensive pre-lecture videos watching, moderate pre-lecture quizzes, and few post-lecture quizzes), a significantly higher proportion of them had better perceptions of the learning environment (41.0%) than poorer perceptions (24.5%).A similar pattern was also observed amongst 115 students who adopted observed study approach 2 (extensive pre-lecture videos watching, moderate pre-lecture quizzes, and moderate post-lecture quizzes), where the proportion of students having better perceptions (77.4%) was significantly higher than poorer perceptions (22.6%).In contrast, of 128 students who adopted observed study approach 3 (moderate prelecture videos watching, few pre-lecture quizzes, and moderate post-lecture quizzes), a significantly higher proportion of them held poorer perceptions of the learning environment (60.2%) than better perceptions (39.8%).The same pattern was also found amongst students who adopted study approach 4 (few pre-lecture videos watching, few pre-lecture quizzes, and extensive post-lecture quizzes), where the proportion of students having poorer perceptions (62.1%) was significantly higher than those having better perceptions (37.9%).

Results for Research Question 3 -Students' Academic Achievement by Their Observed Study Approaches
The results of the one-way ANOVA showed that students adopting different observed study approaches differed significantly on their academic achievement, F (3, 319) = 30.71,p < .01,η 2 = .22.To identify the differences, we conducted post-hoc analyses and the results are displayed in Table 5.According to Table 5, students using observed online learning approach 1 (extensive pre-lecture videos watching, moderate prelecture quizzes, and few post-lecture quizzes: M = 72.87),were followed by students adopting observed online learning approach 2 (extensive pre-lecture videos watching, moderate pre-lecture quizzes, and moderate post-lecture quizzes: M = 67.27).Students using observed study approach 3 (moderate pre-lecture videos watching, few prelecture quizzes, and moderate post-lecture quizzes: M = 56.88)were ranked third by their course marks and those employing observed study approach 4 (few pre-lecture videos watching, few pre-lecture quizzes, and extensive post-lecture quizzes: M = 42.66)scored the lowest for their academic performance.

Discussion
This study investigated Australian first-year engineering students' study approaches in flipped classroom learning using both self-reported data and process data.It also examined the relations between students' perceptions of the learning environment, their academic achievement, and their study approaches measured by self-reported and process data.

Students' Study Approaches Identified by Self-reported and Process Data
Consistent with previous findings in the general blended learning context, students studying in flipped classrooms could also be clearly distinguished by either adopting more deep approaches or adopting more surface approaches, which covered the ways they learned and used online learning technologies in the learning designs (Han et al., 2020;Ellis & Bliuc, 2016;2019).When students' motives for learning was to achieve understanding of subject matter, the ways they approached learning was proactive, reflective, and analytical, and used online learning technologies in meaningful ways to help them understand, make inquiry, and research.On the other hand, when their learning intents were to merely satisfy the course requirements, they approached learning through relatively simplistic activities of rote learning, reproducing, and duplication; and used online learning technologies in simplistic ways (Biggs, 1989;Biggs & Tang, 2011;Ramsden, 2003;Trigwell & Prosser, 2020).The results showed consistency in how students implemented inquiry-based learning strategies and how they used online learning technologies to facilitate them to inquire.
Apart from emphasizing pre-class or post-class activities, study approaches 1 and 2 seemed to be content-focused approaches, whereas approaches 3 and 4 were assessment-focused.The former two approaches seemed to study the course to gain understandings and to focus on learning through watching pre-lecture videos.Despite this, the two approaches differed in whether the testing of understanding occurred before (approach 1) or after (approach 2) the in-person lectures.This kind of contentfocused approach could be an in-depth approach rather than an assessment-focused one, which places greater emphasis on post-lecture quizzes.As the process data were not able to reveal students' intents and motives behind what they actually did, students could complete post-lecture quizzes to help them consolidate their learning, while they could also undertake quizzes to fulfill the assessment requirements.If their objective was the former, clearly post-lecture quizzes seemed not to be as effective as pre-lecture quizzes, as reflected by the poorer academic achievement results of students who used approach 3 and 4.However, students' study motives and intents need to be verified via interviews.
The study approaches identified by self-report and process data provided complementary information.Specifically, while the self-reported measures offered the information of students' motives and purposes of using a particular strategy, the process data showed the distribution of different online learning activities and the stages (e.g., before or after the in-person lectures) in which these activities occurred in flipped classroom learning.Hence, the advantage of employing both self-reported and process measures is the capacity to equip teachers and researchers with richer information on the study approaches of students (Han, 2022;Ober et al., 2021;Reimann et al., 2014).

Patterns of Relations between Perceptions of the Learning Environment, Academic Achievement, and Study Approaches Measured by Self-Reported and Process Data
The results of the relations between students' self-reported study approaches, perceptions of the learning environment, and academic achievement, largely corroborate previous findings that deep study approaches, better perceptions, and higher academic performance tend to be related, whereas surface approaches, poorer perceptions, and lower academic performance are more likely to be associated with traditional in-person delivery (Asikainen & Gijbels, 2017;Asikainen et al., 2022;Guo, 2018;Guo et al., 2017) and general blended course designs (Ellis & Bliuc, 2016;2019;Han et al., 2020); but extend these patterns of the relations in the flipped classroom learning context.In our study, of students who reported using more deep study approaches in the course, a significantly higher proportion of them also perceived that the online learning materials and activities, including pre-lecture videos, and pre-and post-lecture quizzes, were well integrated with what they did in lectures, tutorials, and laboratory practice, while the online learning was also an important part in the whole course.In contrast, amongst students who employed more surface study approaches, a significantly higher proportion of them perceived that the online learning was fragmented from what they did in class and did not value the online learning.At the same time, students adopting more deep study approaches also obtained higher course marks than their peers using more surface study approaches.
As explained above, content-focused approaches involved learning at a more indepth level than assessment-focused approaches, and for the observed study approaches we found that amongst students who adopted two content-focused approaches (approaches 1 and 2), there were higher proportions with better perceptions of the learning environment; whereas amongst students using assessment-focused approaches (approach 3 and 4), it was found that there were higher proportions of students holding poorer perceptions.Also, resembling the patterns observed for the relations between students' self-reported study approaches and academic achievement, students who adopted content-focused approaches also performed more highly than their peers who used two assessment-focused approaches.These results suggest that the relations between students' observed study approaches and perceptions of the learning environment were consistent with the relations between students' self-reported study approaches and perceptions of the learning environment.

Implications for Teaching
The findings of our study may offer teachers a number of strategies to improve Australian engineering students' learning in flipped classrooms.The significant associations between students' perceptions of the learning environment and their use of study approaches suggest to teachers that fostering a better learning environment would be helpful for students to develop desirable study approaches.For instance, teaching staff may design online learning activities and assessment tasks which can reinforce the learning and teaching of disciplinary contents in classroom activities (Rotellar & Cain, 2016).Teachers may also explicitly signal to students how the online learning materials, activities, and assessment tasks, complement examples and demonstrations in lectures and tutorials in order for students to see how the online and in-person components of the course are integrated.
Furthermore, teachers could equip students with sufficient knowledge about, and principles of, flipped classroom learning at the commencement of the course, as research has shown that many students do not possess much understanding about learning in a flipped classroom upon entering college (Hao, 2016;Yilmaz, 2017).As our study results indicated the importance of pre-class learning in flipped classrooms, teachers may emphasize activities in the preparation phase, such as watching the pre-lecture videos and completing the pre-lecture quizzes (Long et al., 2017;Schwarzenberg et al., 2020).Inclusion of the pre-lecture quizzes as a compulsory assessment may also be useful in flipped classroom learning design.This would ensure that students achieve a certain level of conceptual understanding to be discussed and expanded in classroom teaching (Cho et al., 2021).The learning analytic functions built in the LMS may offer teachers tools to identify students' study approaches early in the semester so that teachers can remind those who do not prepare them well before the classroom learning to change their preparation strategies and approaches (Viberg et al., 2018).

Limitations of the Study and Directions for Future Research
The limitations of the study should be noted and addressed in future research.First, the current study only investigated some important aspects in the Process (study approaches and perceptions of the learning environment) and Product (academic performance) stages in the 3P model.Future studies should also include important aspects in the Presage stage, such as students' conceptions of learning and their motivation in learning, as these aspects have also been reported to relate to study approaches in previous research (Trigwell et al., 2013).
Moreover, the current study adopted a cross-sectional design, which did not reflect changes of students' study approaches and perceptions of the learning environment in the course during the semester.Hence, fluctuations and changes of students' study approaches and perceptions were not able to be reflected in the current study.Future research should use a longitudinal design to measure these aspects multiple times throughout the semester, which may help reveal dynamic relations between study approaches, perceptions, and academic performance.
Last but not least, while the perceptions of the learning environment covered both face-to-face and online learning components in this flipped classroom course, the study approaches measured by process data only demonstrated what students did online.Future studies should employ other data collection methods, such as video recording, to capture information on how students approach their learning tasks in-person, so that a more comprehensive picture of students' observed study approaches can be captured.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Figure 1 .
Figure 1.The four observed online study approaches.

Table 1 .
Descriptive Statistics of The Six Scales and Students' Course Mark.

Table 2 .
Self-reported Online Study Approaches.
Note: DAI = deep approaches to learning through inquiry, SAI = surface approaches to learning through inquiry, DAT = deep approaches to using online learning technologies, SAT = surface approaches to online learning technologies.

Table 3 .
Association between Self-Reported Study Approaches and Perceptions of the Learning Environment.

Table 4 .
Association between Observed Study Approaches and Perceptions of the Learning Environment.

Table 5 .
Post-hoc Analyses of Students' Academic Achievement.