The role of collaborative argumentation in future teachers' selection of online information

Criteria guiding the evaluation of online information

Empirical studies have identified several kinds of criteria that are generally used by information seekers, to evaluate whether they can rely on online information (Choi & Stvilia, 2015; Sundar, 2008). These criteria often relate to meta information – that is, information about the individuals and organizations that create and provide content, or information about when, where, in what context, and for what purpose the content was created and provided (also called source information: Bråten & Braasch, 2018). In online contexts, the criteria often refer to three levels: 1) the information itself (e.g., comprehensibility); 2) the providers of information (e.g., their expertise); and 3) the online media (here: any media digitally encodable and connected to the Internet).

Given the diverse and ever-changing manifestations of media offerings online, it seems worthwhile to also consider media affordances when aiming to understand how teachers may evaluate information found on the Internet. Media affordances represent the dynamic relations among users, how users perceive an online environment that is accompanied by certain tools, and how and why users typically interact with these tools (e.g., comment or click ‘like’ below a video) (Evans, Pearce, Vitak & Treem, 2016). Hence, affordances – such as interactivity, navigability, or modality – do not just represent the tools available to users (e.g., a button on a website), but rather represent users' typical use of the online media. In considering the affordances of search engines and how teachers typically navigate therein, teachers often use only one search engine (Bougatzeli et al., 2017). Furthermore, teachers – like other users – may also be attracted to the results shown at the top of the search results page and thus by a search engine's affordance (Haas & Unkel, 2017). They may hence risk being guided by a search engine's algorithm, which pre-selects information. Table 1 lists exemplary studies on criteria at the levels of information, provider, and media. These studies indicate certain types of criteria as being influential, when it comes to information seekers' evaluation of online information in general.

Table 1 Research examples alluding to the relevance of diverse criteria for credibility evaluations of online information

Note: This table represents an excerpt of research studies on criteria shown to be relevant in sourcing online information.
Criteria related to the information	Correctness (Macedo-Rouet et al., 2019) Appropriateness (Zimmermann & Jucks, 2018) Readability and Recency (Macedo-Rouet et al., 2019) Scientificness (Thomm & Bromme, 2012) Two-sided arguments (Mayweg-Paus & Jucks, 2018)
Criteria related to the information provider	Competence/Expertise/Authority/Reputation (Iding & Klemm, 2005) Benevolence & Integrity (Hendriks, Kienhues & Bromme, 2015)
Criteria related to the media	Different media per se (e.g., layperson or expert forum) (Zimmermann & Jucks, 2018) Rank of search result in search engine – a search engine’s navigability (affordance) (Haas & Unkel, 2017)
Other aspects not related to the online information	Experiential knowledge sources, such as own teaching experiences (Bråten & Ferguson, 2015)

Table 1 Research examples alluding to the relevance of diverse criteria for credibility evaluations of online information

View as image HTML

The benefits gained from relying on such criteria can be manifold. For example, if teachers cannot exclusively rely on their prior knowledge (as it might be out of date) or do not have the time to critically evaluate the information they have found, it may be helpful for them to draw on certain criteria for their sourcing. This allows them to effectively decide whether they can rely on the information – based, for instance, on its provider's expertise or the trustworthiness of a website (Bromme & Goldman, 2014; Choi & Stvilia, 2015). In this sense, drawing on such criteria can help teachers to decide which sources are more trustworthy or provide more credible information (Bråten & Braasch, 2018; Bromme & Goldman, 2014; Rousseau & Gunia, 2016).

The importance of critical reflection around any use of criteria

While drawing on certain criteria can be useful, it can also influence a teacher's evaluation of information in a dysfunctional way. Highly ranked search results can be misleading, for example, since the ranking of a piece of information within the search results does not necessarily indicate the quality of that information (Bougatzeli et al., 2017). Similarly, teachers' familiarity with a certain website may lead them to select information that only confirms their existing knowledge (Iding et al., 2008). The use of criteria thus poses some risks. Critical reflection – regarding how, when, and why we make use of certain criteria – is considered an essential competency that can help to source online information effectively and efficiently (Hendriks et al., 2020).

While studies on how to promote sourcing competencies indicate that individuals do use certain criteria – at first – when checking for trustworthiness, they generally do not critically reflect on the results of this ‘trustworthiness check’ (Brante, 2019). Researchers have, however, successfully conducted trainings to improve individuals' critical reflection about their use and justification for the use of certain criteria (Bråten, Brante & Strømsø, 2019; Pérez et al., 2018). These trainings have mainly addressed individual measures to foster sourcing competencies. Implementing collaborative learning might allow such interventions to foster additional and deeper critical reflection, when reflecting on one's sourcing activities. To follow, we outline the potential of collaborative argumentation and why it might help (future) teachers to reflect critically on their use of any criteria, when sourcing online information.

The potential of collaborative argumentation to foster critical reflection

Educational research on collaboration points to several benefits of learning with others. It helps individuals gain knowledge and skills (e.g., writing skills within a Wiki environment) and can positively affect their self-efficacy, especially at a tertiary education level (Chen, Wang, Kirschner & Tsai, 2018). In a study focusing on the potential of collaboration for synthesizing evidence, for example, future teachers who engaged in collaboration analyzed pedagogical problems in a more reflective and evidence-based manner (Csanadi, Kollar & Fischer, 2020). Collaborative engagement also seems to provide a promising setting for sharing, interpreting, and critically examining scientific information in online contexts (Hendriks et al., 2020). From an educational perspective, collaboration is thought of as a “co-elaboration of conceptual understanding and knowledge” (Baker, 2015, p.4), accompanied by communicative activities – such as explaining and understanding ideas, representing knowledge and concepts, getting multiple perspectives from others, and arguing collaboratively (Asterhan & Schwarz, 2016; Chinn & Clark, 2013).

The latter, collaborative argumentation, occurs in dialogic debates in which two or more people engage – thereby usually exchanging statements and questions, making claims, supporting their claims with reasons and evidence, and critically questioning the others' arguments. This may result in agreement or disagreement among the learners (Chinn & Clark, 2013). The dialogic setting can thus stimulate the explicit articulation of dialog partners' ideas by asking them for clarification and critical elaboration on the reasons behind these ideas. It thereby promotes high-quality cognitive elaboration processes that are assumed to elicit changes in knowledge structures (Csanadi et al., 2020; Van Boxtel, van der Linden & Kanselaar, 2000). Accordingly, explicit forms of cognitive elaboration processes are often considered indicators for the depth of one's reflection processes (Csanadi et al., 2020; Felton & Kuhn, 2001; Thiebach, Mayweg-Paus & Jucks, 2016). In this vein, a critical reflection of teachers' own sourcing could be characterized by their manner of elaboration. That is, for instance, whether they give reasons for why certain criteria had guided the selection, whether they support their own or the others' perspectives with further explanation or arguments, and/or whether they report uncertainty about whether their criteria are appropriate.

A meta-analysis on studies within the field of argumentation-based computer-supported collaborative learning (ABCSCL) – as a means of learning and collaboratively debating with others, using a variety of technological and pedagogical strategies – indicates that engaging in collaborative argumentation can positively affect domain-related knowledge construction, acquisition of argumentation skills, and elaboration of materials (Noroozi, Weinberger, Biemans, Mulder & Chizari, 2012). Furthermore, among other functions such as finding consensus or justifying knowledge claims, an important function of argumentation is critical reflection on one's own reasoning (Hoffmann, 2016). The positive outcomes of collaborative argumentation may be a result of its allowing the partners to successfully integrate multiple perspectives (Veerman, Andriessen & Kanselaar, 2002); interact transactively by challenging the partners' knowledge and arguments (Felton & Kuhn, 2001; Thiebach et al., 2016); and use high-quality argumentation strategies, such as critically questioning one's own and others' arguments (Mayweg-Paus, Thiebach & Jucks, 2016). The aims of collaborative argumentation may be either to persuade or to reach consensus. Yet a consensus-oriented discourse may increase the above-mentioned positive outcomes as the partners may become more interactive, using the arguments originally introduced by their partners and critically challenging their own arguments (Felton, Garcia-Mila, Villarroel & Gilabert, 2015; Felton & Kuhn, 2001). Nonetheless, collaborative learning also introduces certain challenges – for example, that individuals learn at different rates. There is thus a risk that some partners may move more quickly through the collaborative phases, moving on to the next sub-task before everyone is ready (Mullins, Rummel & Spada, 2011).

Collaborative argumentation may supercharge the individual learning process and has the potential to support teachers' critical reflection on both their sourcing of online information and their use of criteria that may have guided this sourcing. Collaboration may also impact another important part of any successful sourcing of online information – that is, one's self-efficacy when seeking information.

Future teachers' ISSE

Individuals' ISSE is considered an essential part of competencies related to successfully sourcing online information (Andreassen & Bråten, 2013; Caena & Redecker, 2019; Hendriks, et al., 2020). Research indicates that a teacher's perceived ISSE is also related to his/her actual sourcing behavior: Education students with high ISSEs, for example, used online library databases rather than Google to search for information (Tang & Tseng, 2013). Similarly, elementary teachers' information seeking standards (e.g., judging the accuracy of online information using multiple sources) were positively related to their confidence in using the Internet for advanced sourcing strategies (e.g., searching with keywords) (Wu & Wang, 2015).

According to Bandura (1997), self-efficacy refers to one's belief in one's own capabilities to organize and execute the courses of action required to attain particular goals. Thus, teachers' ISSE reflects their interpretation of their own competencies around sourcing online information (Kurbanoglu, 2003). It is likely that a teacher's confidence in sourcing online information is influenced by his/her interpretation of his/her own performance, and also by social comparisons to others' successes or failures (Bronstein, 2014). In collaborative settings, the partners' search approaches come into play while collaboratively elaborating on both discourse partners' sourcing strategies. Thus, the use of one's own strategies may become more evident in comparison to those of others. However, there is not enough evidence to derive assumptions on whether collaboration – which allows for social comparison – may increase or decrease one's perception of one's own seeking skills. The exchange with others may lead to more awareness of one's own competencies, and hence to higher perceived self-efficacy; or it may instead reveal deficits in one's own search strategies and thus lead to lower perceived self-efficacy.

The present study

When (future) teachers search the Internet to inform themselves on educational topics, they should critically reflect upon their sourcing of online (scientific) information – including reflecting upon any criteria that may have guided their selection. In this study, we first investigated whether future teachers select science-related information about an exemplary educational topic (i.e., the use of students' mobile phones in classes) at all when searching for it online.

Since collaborative argumentation is accompanied by the underlying mechanisms of exchanging and critically questioning one's own and others' perspectives and arguments, it may support future teachers in critically reflecting upon their sourcing of online information. Hence, secondly we investigated whether future teachers' critical reflection on how they selected online information differed, according to whether they were asked to reason their guiding criteria individually or collaboratively. In this context, we focus on elaboration in future teachers' argumentative reasoning behavior, since it serves as indicator for cognitive reflection processes. Informed by the literature – which indicates that drawing on diverse criteria affects how people evaluate the credibility of information – this study further examined whether and how future teachers critically question their own use of different criteria, when sourcing educational information online. We assumed that, in a collaborative reasoning task, future teachers' argumentative reasoning behavior would more frequently show elaborated reasoning and they would more frequently refer to criteria that guided their choice of online information (i.e., criteria related to the online information, provider, and media; see Table 1), than in an individual reasoning task.

Collaboration may also impact a teacher's ISSE, as it allows for comparison with others and thus may influence a teacher's interpretation of his/her own searching competencies. Thus, thirdly we exploratively investigated whether collaborative or individual reasoning affected participants' self-reported ISSE. Since feeling confident in sourcing information online is likely influenced not just by how someone interprets his/her own performance, but also by social comparison to others' successes or failures in the search task, we assumed that participants who reasoned their choice either collaboratively or individually would show a different self-reported ISSE after the reasoning task than before. However, we had no assumption as to whether collaborative or individual reasoning would increase the degree of difference (or whether they would lead to either higher or lower self-reported ISSE) after the reasoning task.

Since we know from the literature that one's epistemic beliefs (i.e., one's beliefs about knowledge and knowing) affect the sourcing of information (Hendriks et al., 2020), we assessed participants' epistemic beliefs regarding educational knowledge from the Internet, to control for potential differences between the collaborative and individual settings.

Participants

Ninety-one future teachers participated voluntarily in the study and were reimbursed with 20 Euros. Participants were studying secondary-school teaching, at either the bachelor's or the master's level. We excluded data from six participants whose Internet connectivity failed during the investigation. We likewise excluded data from two participants whose time of actual chatting during the common reasoning task deviated more than one standard deviation from the overall mean duration for chatting time, among all participants in the discourse group (M = 23.77 min.; SD = 7.47). Hence, we finally analyzed data from N = 83 participants (56 female and 1 diverse) aged 18–41 (M = 25.34, SD = 5.14), with n = 33 participants in the individual group (group_in) and n = 50 participants in the discourse group (group_coll; grouped in 25 dyads).

Of these 83 participants, 73 indicated German as their first language. At the time of the investigation, participants had been studying for an average of 3.83 years (SD = 3.10). Eighteen of the participants in the group_coll and 9 of the group_in were studying at the master's level; 31 and 25 of the participants, respectively, were female (differences between experimental conditions were not significant; study level: χ ²(1) = .690, p = .406; and gender: χ ³(2) = 3.86, p = .145). The average duration of participation for all participants was 103.67 minutes (SD = 31.81) and did not differ between the experimental conditions, Wald χ ²(1) = 2.54, p = .11.

Participants reported that they used a computer, notebook, or tablet for an average of 4.02 (SD = 2.47) hours per week (hrs/wk) and spent an average of 4.94 (SD = 2.96) hrs/wk on the Internet. Participants reportedly sought general online information for an average of 1.99 (SD = 1.51) hrs/wk and searched specifically for educational online information for an average of 1.54 (SD = 1.2) hrs/wk. Participants rated their self-perceived prior knowledge on the ‘use of mobile phones in classes’, based on four items, as neither very low nor very high (M = 2.50; SD = .60). With respect to participants' prior opinions, they were neither for nor against mobile phones in classes, based on one item (all items ranged from 1 = ‘I strongly disagree’ to 5 = ‘I strongly agree’) (M = 2.96, SD = 1.03).

Design

In a 2 × 2 mixed design with the between-participants factor reasoning (individual vs. collaborative) and the within-participants factor self-reported ISSE (pre vs. post measure), all participants answered the questionnaire in terms of self-perceived ISSE, before and after the search and reasoning task. Participants were instructed to individually search for pedagogical information on ‘mobile phone use in classes’ online. Based on their search results, participants were asked to select four pieces of online Web content (in the following: Web items [WI]) that they perceived to be relevant for reasoning an opinion. Participants in the group_in individually reasoned their choice of WI, while participants in the group_coll engaged in a collaborative discourse via chat and reasoned their four WI together. Each discourse partner contributed two of the selected WI. Participants were randomly assigned to one of the two experimental conditions. In the group_coll, they were randomly paired into 25 dyads.

Procedure

Participants performed the experiment on-site at the university and, hence, had access to the university network (i.e., including scientific sources such as scientific journal articles). Each participant sat individually in front of a computer. All participants worked at their own pace and were guided through the experiment by the online survey (unipark.com by Questback EFS Surveys), without verbal instructions from the examiner. From the beginning of the study, each computer displayed an open Web browser window (i.e., Mozilla Firefox) showing the same university website. Additionally, participants in the group_coll saw an open window of the open source chat application (i.e., https://discordapp.com/).

Participants first answered items according to demographic and preparatory variables. They were then asked to rate their self-perceived ISSE (pre-measure). Afterwards, participants received a fictional scenario: They were asked to imagine themselves as teachers, searching for information on the topic of ‘mobile phone use in classes’. All participants were instructed to use this search for preparation of a fictional school conference on the topic. The group_coll was further instructed to subsequently discuss these search results with a fellow teacher. After the individual search for educational information was finished (M = 25.05 min; SD = 6.1 in the group_in and M = 23.09 min; SD = 10.16 in the group_coll), participants were asked to give reasons – either individually or collaboratively – for how, and based on what criteria, they had selected their four online sources. Participants in the group_coll were also asked to commonly select the two most relevant WI. Similarly, participants in the group_in were asked to choose two of their initially selected WI as being the most relevant, which should help to increase their reasoning motivation (see electronic supplementary material [ESM] 1 for all the experimental instructions). Participants in the group_coll communicated only via chat. Reasoning their choices took M = 20.47 minutes (SD = 7.54) in the individual group and M = 24.48 minutes (SD = 6.69) in the discourse group. Afterwards, all participants were asked to state their view on the topic ‘mobile phone use in classes’ and rate their self-perceived ISSE (post-measure) again. Finally – and only to ensure the possibility of future exploratory investigations – the participants judged trust-related measures for each of their four initially selected WI and attitudinal measures, in terms of their opinion towards the topic after the reasoning task (see ESM 2). We had no assumptions regarding these measures. Each participant's computer screen was recorded, via screen video, throughout the entire study.

The educational topic

The use or ban of students' mobile phones in classes is a highly relevant educational topic – one that is regulated differently in schools throughout Europe and even within each country. In Germany, each of the federal states applies different regulations. Educational science research on this topic describes the advantages and disadvantages of using mobile phones in classrooms – regarding students' attention and learning outcomes, as well as students' social and digital competencies (Sung, Chang & Liu, 2016). Thus, regardless of whether a teacher supports or opposes mobile phone use in classes, a variety of educational evidence is available to support either view. In this study, participants were asked to search for pedagogical reasons for any use of – or for a ban on – student's mobile phones in the classroom. They were given no further instructions on specific aspects within this topic (that is, they were allowed to search for any opportunities or challenges mobile phones might present – for example, opportunities for students' learning or challenges regarding students' distraction) (see ESM 1).

Measurements

Preparatory analyses

A multivariate ANOVA – with experimental conditions as the independent variable and the demographic variables, as well as participants' ISEB as dependent variables – yielded no significant differences (all, F (1, 79) ≤ 2.41, p ≥ .13). We further tested whether the relative frequency of science-related WI – among all of the initially selected four WI – differed between the experimental groups, since differences in these relative frequencies may have affected the length and depth of participants' elaboration during the reasoning task (Salmerón, Fajardo & Gómez-Puerta, 2019). There were no significant differences between experimental conditions in the relative frequency of the science-related WI initially selected by participants, χ 4(1) = 2.58, p = .63, see ESM 4. Thus, we did not include these variables in our main analyses, as any difference between the experimental conditions is not caused by these variables.

Main analyses

To consider any variance in differences regarding the dependent measures at the level of the discourse dyads, we conducted a hierarchical model. The intra-class correlation of the dyadic level (ICC = .01) was not significant (F(1, 108) = 0.78, p = .85), meaning that only 1% of variance was caused by the dyads. Thus, three generalized linear models were conducted to test whether participants' pre- and post-ISSE and the relative frequencies of (un)elaborated reasoning in terms of the types of criteria differed between the experimental conditions (Cress, 2008). We set an α error of α = .01.

Results of science-relatedness of WI

Of the 232 selected WI (100 for the 50 participants in the group_coll and 132 for the 33 participants in the group_in), 111 WI were unique. While most of the WI were only selected once or twice by all participants, there were six WI that were selected very often; see ESM 5. Of these 111 WI, only 38 (34.2%) were determined to be science-related.

Results of argumentative reasoning behavior

From participants' individual and collaborative ARB, we determined the relative frequencies of the reasoning criteria that guided participants' selection of WI – in relation to the overall frequencies of on-task units (i.e., comments made about the reasoning task). These relative frequencies thus represent the relative numbers regarding all task-related comments and not the total number of comments made (there were also off-task comments – e.g., related to task management). As expected, participants in the group_coll more often expressed elaborated reasoning (M = 90.2%, SD = .10) and less often engaged in unelaborated reasoning (M = 9.7%, SD = .10), compared to the group_in (elaborated comments: M = 62.2%, SD = .26; unelaborated comments: M = 37.8%, SD = .26), both F(1,56) = 26.01, p < . 001, η ² = .32. Unexpectedly, participants in the group_coll did not more often cite criteria related to 1) the information (M = 59.1%, SD = .21); 2) the provider (M = 4.2%, SD = .07); or 3) the media (M = 7.6%, SD = .08), compared to the group_in (comments related to information: M = 61.2%, SD = .20; provider: M = 4.3%, SD = .07; media: M = 14.3%, SD = .12); all: F(1,56) ≤ 5.69, p ≥ . 02, η ² ≤ .09). Furthermore, the group_coll significantly more often made comments related to the management and coordination of the task (i.e., relative frequencies of off-task units in relation to overall units) (M = 48.2%, SD = .13), compared to those in the group_in (M = 4.5%, SD = .07), F(1,56) > 256.21, p < . 001, η ² = .82 (see Tables 2 and 3). Interestingly, only one participant mentioned that s/he used a specific search engine. Furthermore, none of the participants considered whether the WI was generated or filtered through algorithms. See ESM 3 for an extract of the individual and collaborative ARB.

Table 2 Multivariate ANOVA to test for differences between the experimental conditions, regarding the relative frequencies of unelaborated and elaborated reasoning of the criteria guiding the selection of WI

Source	Dependent variable	Type III sum of squares	df	Mean square	F	p	η 2part
Experimental conditions	Freq. of criteria related to information/Freq. on-task units	0.01	1	0.01	< 1	.70	< .01
	Freq. of criteria related to provider of information/Freq. on-task units	3.42	1	3.42	< 1	.99	< .01
	Freq. of criteria related to media/Freq. on-task units	0.06	1	0.06	5.69	.02	.09
	Freq. of criteria other but related to media/Freq. on-task units	0.06	1	0.06	1.98	.17	.03
	Freq. of criteria other but no relation to media/Freq. on-task units	0.01	1	0.01	< 1	.48	.01
	Freq. of elaborated comments/Freq. on-task units	1.11	1	1.11	26.02	< .001	.32
	Freq. of unelaborated comments/Freq. on-task units	1.11	1	1.11	26.02	< .001	.32
	Freq. of off-task units/Freq. units	2.72	1	2.72	256.21	< .001	.82

Table 2 Multivariate ANOVA to test for differences between the experimental conditions, regarding the relative frequencies of unelaborated and elaborated reasoning of the criteria guiding the selection of WI

View as image HTML

Table 3 Descriptive statistics of the relative frequencies of unelaborated and elaborated reasoning around the criteria guiding the selection of WI

Frequency of	Experimental condition	M	SD
Note: IR = individual reasoning; CR = collaborative reasoning; N total = 58, NIR = 33, NCR = 25.
criteria related to information/ all on-task units	IR	.61	.20
	CR	.59	.21
	Total	.60	.20
criteria related to provider of information/all on-task units	IR	.04	.07
	CR	.04	.07
	Total	.04	.07
criteria related to media/ all on-task units	IR	.14	.12
	CR	.08	.08
	Total	.11	.11
criteria other but related to media/ all on-task units	IR	.11	.13
	CR	.17	.21
	Total	.13	.17
criteria other but no relation to media/ all on-task units	IR	.10	.13
	CR	.12	.14
	Total	.11	.13
elaborated comments/ all on-task units	IR	.62	.26
	CR	.90	.10
	Total	.74	.25
unelaborated comments/ all on-task units	IR	.38	.26
	CR	.10	.10
	Total	.26	.25
off-task units/all units	IR	.05	.07
	CR	.48	.13
	Total	.23	.24

Table 3 Descriptive statistics of the relative frequencies of unelaborated and elaborated reasoning around the criteria guiding the selection of WI

View as image HTML

Results of information seeking self-efficacy

A generalized linear model, including the between-participants factor reasoning and the within-participants factor time of measurement of ISSE, revealed a significant main effect of time but no significant main effect of experimental conditions (group_in: M_in = 3.52, SE= .01; group_coll: M_coll = 3.64, SE_coll = .08; F(1, 81) = .96, p = .33, η ² = .01), as well as no interaction effect between time and experimental conditions (group_in: M_{pre in} = 3.51, SD_{pre in} = .61; M_{post in} = 3.53, SD_{post in} = .59; group_coll: M_{pre coll} = 3.57, SD_{pre coll} = .56; M_{post coll} = 3.71, SD_{post coll} = .52; F(1, 81) = 4.18, p = .04, η ² = .05). Overall, participants reported higher ISSE after the reasoning task (M_pre = 3.55, SD_pre = .58; M_post = 3.64, SD_post = .55; F(1, 81) = 6.59, p = .01, η ² = .08).

Summary of findings

The findings of the present study shed light on how future teachers select online educational information, when carrying out an authentic search. With respect to the participants' selection of types of educational information, their selected WI were often not science-related. This is in line with previous findings on teachers' preferences for sourcing evidence (Bråten & Ferguson, 2015; Duke & Ward, 2009; Williams & Coles, 2007). Regarding participants' critical reflection around their selection of educational information, this study investigated how they reasoned their selection (elaborated vs. unelaborated) and how often they referred to the criteria that may have guided this selection (i.e., meta information about the information, provider of information, and media; see Table 1). As expected, participants more often gave elaborated and less often gave unelaborated responses, when reasoning their selections collaboratively (e.g., they more often supported their own or their partner's argument with further explanations or evidence). Accordingly, the setting wherein future teachers collaboratively engaged in argumentation seemed to support their critical elaboration of how they selected online information. This is in line with approaches on collaborative learning and argumentation (Baker, 2015; Noroozi et al., 2012).

Yet, unexpectedly, participants in both the collaborative and the individual reasoning groups referred equally often to the three theoretically and empirically derived types of criteria (Table 1). In this context, criteria regarding the provider and the media itself was mentioned relatively rarely – even though research indicates that others often use these criteria (Choi & Stvilia, 2015) and that drawing on associated cues related to the providers can facilitate the efficient selection of online information (Bromme & Goldman, 2014). With respect to cues related to the media, only one participant questioned his/her own use of search engines; no other participants mentioned criteria related to algorithm-based content or the type of search engine (Guzman & Lewis, 2020). This is in line with other findings, indicating that teachers tend to use only one type of search engine (Bougatzeli et al., 2017).

Furthermore, participants in the two experimental conditions did not differ in terms of their self-reported ISSE. This was contrary to our assumption that the possibility to compare to the others′ search approaches could impact one's own ISSE. Both groups reported higher ISSE scores afterwards, possibly in part because the combined seeking and reasoning task may have led both groups' participants to feel more competent around seeking information in general.

Limitations and future research

In this study, future teachers' self-reported confidence in seeking online information – as well as their elaboration on their argumentative reasoning behavior – may function only as indicators of how deeply they reflect on their sourcing of online educational information. Yet, even though neither group was shown to frequently discuss criteria related to the provider or media, we do not know whether the participants reflected internally on these aspects without discussing them in the reasoning task.

Due to the sample size of N = 83 participants, determining the variances caused by the individuals within the dyadic levels is impossible. Yet achieving a sufficiently large group-level sample size to conduct multi-level analyses is challenging in general, for studies investigating collaborative learning processes (Cress, 2008). Future research could investigate the underlying collaborative processes – and, for instance, whether different kinds of paired dyads result in different qualities of argumentation (e.g., when both collaborators are experienced teachers versus student teachers, or when they have high versus low ISEB or ISSE).

While the topic of using mobile phones in class is highly relevant for (future) teachers – who must establish practices grounded in educational research – our study's focus on this topic means that the findings cannot necessarily be generalized. Therefore, future research could investigate whether (future) teachers' critical reasoning of how they select online information differs for other topics. Similarly, since the participants in this study were future teachers, the findings cannot be generalized completely towards in-service teachers and their sourcing of online information. Although participants in this study had access to scientific research articles on campus, they rarely selected science-related WI. Thus, future research – as well as teacher trainings on teachers' sourcing of online information – should consider the different circumstances under which in-service teachers, who lack such access to scientific articles, source scientific evidence.

Another limitation refers to the coding scheme that addresses future teachers' argumentative reasoning behavior when sourcing online educational information. This was determined with regard to 1) whether participants referred to types of criteria that have been empirically and theoretically found to impact information seekers' evaluation of online information; and 2) whether they referred to the criteria in either an elaborated or an unelaborated way. This coding scheme thus addresses critical reasoning – not just regarding the analysis of evidence within the information, but also regarding the criteria participants used to select online information, including criteria specific to online sources and media types. Nonetheless, the types of criteria addressed in the coding scheme are non-exhaustive. With respect to the coding of the reasoning behavior, it is important to note that our scheme merely focused on the structure of participants' reasoning behavior – for instance, whether and how participants provided reasons for their selection of information – without any coding of whether these reasons were adequate for their intended function (e.g., ‘I chose this WI because the author is a prominent vs. an experienced teacher’). Thus, in this study, we coded participants' reasoning as critically elaborated if they gave any reason or argument for their selection that was plausible to them. This was done because the intention of this study was to better understand the role of collaborative argumentation in stimulating critical reflection on one's own preferred and established sourcing strategies. In this sense, we were interested in whether participants were able to provide any reasons and to elaborate on them, since such elaboration processes are essential for deeper reflection (e.g., Felton & Kuhn, 2001). Future research should build on these findings and investigate whether a given argument is not just plausible to the (future) teachers themselves, but whether it is a promising strategy for selecting correct, complete, and appropriate information about online educational information (Bromme & Goldman, 2014).

Lastly – in considering the complexity of processes for using scientific knowledge, as well as when sourcing for such knowledge on the Internet (Rousseau & Gunia, 2016) – this study focused on those processes related to the critical reflection of sourcing after the participants had conducted their realistic searches. Future research may additionally investigate teachers' critical reflection prior to selecting information, or may explore their decision not to select particular information found online.

Implications for teacher education

This study has potential implications for the development of trainings to support future teachers' sourcing of online educational information. Besides preparing future teachers to effectively use research evidence for professional practice in general, teacher education programs should focus explicitly on the characteristics of the Internet as a meaningful source of scientific information (Caena & Redecker, 2019; Duke & Ward, 2009; Rousseau & Gunia, 2016). Teachers need to know what information they can rely on and how to find it. Especially since the Internet serves as an easy, fast, and accessible resource of information that can be used by teachers for a lifetime – long after their university years – trainings should consider the research indicating the influence of certain criteria, such as information about the information, the provider of information, and the media. Effective training for educators may include, for example, not just the use of certain technologies and media (e.g., search engines and websites) but also how to critically reflect on the potential influence of the corresponding affordances (e.g., search engine algorithms that can lead to different search results). Teachers should become more aware of the various types of criteria that may influence their selection of online educational information (Choi & Stvilia, 2015). The ability to critically reflect on whether and how to draw on these criteria is, likewise, a crucial competency for sourcing evidence appropriately – given the fact that, while often useful, criteria can also inadvertently guide teachers toward a rather biased evaluation of the information (Bråten et al., 2019; Bromme & Goldman, 2014; Rousseau & Gunia, 2016 ).

Trainings that aim to encourage (future) teachers to critically reflect on any use of criteria may also benefit from adding a collaborative component (Bråten et al., 2019; Pérez et al., 2018). Collaborative argumentation may particularly help future teachers to discuss information (whether science-related or not), as well as to critically reason whether and how they have selected online information. In this study, the group_coll more often referred to the criteria that had guided their selection using elaborated reasoning – even though they had not received previous training on key argumentative activities, such as critically questioning others (Mayweg-Paus et al., 2016; Noroozi et al., 2012). Apparently, the dialogic setting itself stimulates deeper elaboration as a person is subject to the interlocutors' scrutiny of his/her preferred selection strategy. Future research could shed light on whether adding interventional trainings that promote the use of argumentative strategies more systematically (e.g., prompts on how to question other arguments) could help future teachers to more critically reflect on whether their choices of online information are guided by criteria that they currently inadvertently overlook. In addition to that, future research may investigate whether the benefits of collaboration, when reflecting about online sourcing, occur only if teachers actually collaborate – or even when they simply imagine discussing the sourcing with another teacher. This might become relevant, since actual collaboration also poses challenges (e.g., effort to manage the task). In this study, participants who worked collaboratively more often uttered phrases related to managing and coordinating the reasoning task. Collaboration, therefore, likely increased participants' investment of resources like time, effort, and communication in managing the task (Mullins et al., 2011). This additional effort needs to be considered – particularly in terms of time constraints. In fact, it may be due to time constraints that teachers are searching for information on the Internet in the first place; they may thus have no extra time to discuss their search with another colleague.

Overall – while critical reflection when sourcing online information may not replace formal criteria for scientific practices (e.g., knowledge about how scientific evidence emerges) – collaborative argumentation appears to be a promising approach to increase critical reflection as an important element of any successful sourcing of educational information on the Internet. As the online context will continue to rapidly evolve over time, teachers are required to continuously reflect on and engage with their own strategies in this specific environment. In this sense, future research – as well as teacher trainings – could help encourage teachers to critically question both the background evidence and how they acquire the information. This should include considering the types of criteria that may guide this process in online contexts. Such critical questioning could also be promoted by collaborative learning components – allowing participants to reflect on their own, as well as on others' methods of choosing online content.