Delft University of Technology Characterizing conceptual understanding during design-based learning Analyzing students' design talk and drawings using the chemical thinking framework

Design-based learning is considered a powerful way to help students apply and develop understanding of science concepts, but research has shown that the success of this approach is not a given. Examining students' understanding of science concepts in various design-based learning contexts has thus continued to be an important field of research. To help advance such work, we explored the affordances of a novel analytic approach for studying data gathered in design-based learning classrooms. We used the “ chemical thinking framework, ” specifically its “ conceptual sophistication ” dimension, to analyze 10th-grade, chemistry students' design talk and drawings. We gathered this data during small-group design planning and drawing activities in the classrooms of two teachers whose students were designing a product that harnesses chemical energy to change the temperature of a beverage or snack. The findings demonstrate that this analytic approach was able to reveal that students (implicitly) drew on their understanding of several chemistry concepts while

small-group design planning and drawing activities in the classrooms of two teachers whose students were designing a product that harnesses chemical energy to change the temperature of a beverage or snack.The findings demonstrate that this analytic approach was able to reveal that students (implicitly) drew on their understanding of several chemistry concepts while This article was originally based on a chapter of the PhD thesis of Hanna Stammes (2021).
designing.Moreover, it showed that students could use everyday as well as more sophisticated understandings regarding a given chemistry concept while designing.This study furthermore unveiled differences in what and how students' design talk and drawings may reveal use of conceptual understanding, and it showed that different student teams may use a unique combination of understandings during design planning and drawing.We describe how this study's analytic approach complements existing approaches in design-based learning research, and how our findings provide implications for research and practice.
chemistry education, conceptual understanding, design drawings, design talk, design-based learning

| INTRODUCTION
In the last decade, design-based learning has been finding its way into more and more science curricula and classrooms across the world (e.g., Dutch Board of Tests and Examinations [CvTE], 2014; NGSS Lead States (NGSS), 2013).Implementing a design-based approach to learning entails engaging students in solving real-world challenges through design practices such as identifying constraints, generating possible solutions, and testing prototypes (Fortus et al., 2004;Kolodner et al., 2003).While design-based learning can serve multiple purposes, it is frequently highlighted as a powerful way to help students apply and develop understanding of science concepts (incl., Apedoe et al., 2008;Fortus et al., 2004;Kolodner et al., 2003;National Research Council, 2012).Trying to solve a design challenge can offer students a meaningful purpose for using and developing conceptual understanding (J. S. Brown et al., 1989;Kolodner et al., 2003).For example, drawing on understanding of science concepts may help students explain and justify their design ideas to each other (English et al., 2017).But, design-based learning research has also stressed that this is not a given.Students may, for instance, opt for a trial-and-error approach when constructing a design solution instead of apply understanding of targeted science concepts (Kolodner et al., 2003).Because of the potential of design-based learning on the one hand, and challenges impacting its effectiveness on the other, investigating students' understanding of science concepts in various design contexts continues to be an important area of research (see, e.g., the recent collection of studies in Henze & de Vries, 2021).
To gain insight into students' conceptual understanding in design contexts, researchers have been studying different kinds of student information.One of these concerns analyzing "designauthentic" information.This information arises from design activities, and includes sources such as students' design talk and design drawings (e.g., English et al., 2017;Roth, 1994).While engagement in design can render this information "readily visible for research purposes" (Kelly & Cunningham, 2019), there seems to be a need for exploring new ways to meaningfully analyze design-authentic information.For example, some existing analytic approaches rely heavily on students' use of scientific vocabulary (e.g., Valtorta & Berland, 2015).Cognitive research has, on the other hand, stressed the importance of paying attention to students' everyday ideas when seeking to grasp their understanding of science concepts (e.g., DiSessa, 1993;Vosniadou & Brewer, 1992).Other existing analytic approaches for design-authentic data evaluate, for instance, the "(in)correctness" of students' understanding of science concepts (e.g., Fortus et al., 2004).However, science education researchers have since argued that looking at what students mean rather than whether they are correct provides deeper insight into students' disciplinary thinking (Coffey et al., 2011).To help advance research into designbased learning, we will therefore explore the affordances of a novel analytic approach.
In this study, we will use the "chemical thinking framework" (Sevian & Talanquer, 2014) to examine design-authentic information from 10th-grade chemistry classrooms.The chemical thinking framework was developed to support, among other things, characterizations of students' understanding of chemistry concepts in line with the nature of the chemistry discipline, and research on how students learn chemistry (Sevian & Talanquer, 2014).Rather than focusing chiefly on, for instance, scientific vocabulary or (in)correctness of ideas, the framework also recognizes students' everyday science ideas.These are seen as relevant because they may be used productively in certain situations, and could provide stepping stones toward more sophisticated understandings (Sevian & Talanquer, 2014;Talanquer, 2006).As such, this framework offers a perspective uncommon in design-based learning literature, but one that might support developing new insights into students' application of conceptual understanding in design contexts.Despite researchers having called for such work (Sevian & Talanquer, 2014), this has not yet been realized using design-authentic data gathered in real classrooms.
Using the chemical thinking framework, we seek to characterize 10th-grade students' understanding of chemistry concepts applied during small-group design planning and drawing activities.For our analysis, we focus on the so-called "conceptual sophistication" dimension of the framework (Sevian & Talanquer, 2014), and consult three types of design-authentic information (students' design talk within their team, talk with the teacher engaging in the conversation, and annotated design drawings).We collect this data in the classrooms of two teachers whose students (15-16 years old) are designing a product that uses a chemical reaction to change the temperature of a self-chosen drink or food item.In addition to characterizing students' conceptual understanding in the design context, we will also compare what each of the three sources of information reveals.While engaged in design activities, students may convey what they mean through multiple forms of communication (Roth, 1994).Different sources may have different affordances.For example, in design thinking research, students' design talk has been described as more revealing of cognitive resources than students' design products (Guzey & Jung, 2020).Such knowledge can guide the data collection and interpretation efforts of researchers as well as teachers (e.g., Hiebert et al., 2007).Different types of design-authentic information still remained to be compared with a focus on students' understanding of science concepts.
The outcomes of our study provide new insights into how researchers, and possibly also teachers, can gauge students' understanding of science concepts during design-based learning.Moreover, whereas design-based learning studies are often conducted in elementary and/or physics educational settings, and chemical thinking research in "lab" situations, this study yields a comprehensive characterization of students' understanding of chemistry concepts as applied during design-based learning in secondary school, chemistry classrooms.
In this section, we first review literature regarding the collection and analysis of designauthentic information with the goal of characterizing students' understanding of science concepts in design-based learning contexts.We then describe the chemical thinking framework, including its "conceptual sophistication" dimension.Finally, we review literature regarding what students' design talk and drawings may reveal of students' conceptual understanding, and present this study's aim and research questions.

| Design-authentic sources of information
When aiming to characterize student understanding, we tend to rely on information in the form of observable student behavior, such as what students say, write, make, or gesticulate (Griffin et al., 2010;Taber, 2013).Two main approaches are used for obtaining this information in research on design-based learning in science education.
One approach relies on the implementation of instruments which are archetypical of school and educational research cultures (also see J. S. Brown et al., 1989).Cunningham et al. (2020), for example, studied the impact of design-based units on students' understanding of science concepts using multiple-choice tests.Conducting interviews is also a common method (e.g., Marulcu & Barnett, 2013;Schnittka & Bell, 2011).We can contrast this approach with one where students' design activities are seen as giving rise to potential sources of information on students' understanding of science concepts.English and King (2019), for instance, found evidence of elementary students' application of science concepts in students' annotated design drawings.We define behavioral information arising from students' engagement in design activities as "design-authentic" (as in, e.g., Peterman et al., 2017).
We focus on design-authentic information in this study because it means having an opportunity to characterize students' application of conceptual understanding during design activities.Moreover, studying this type of data has provided researchers new insights into students' understanding as compared to the analysis of more traditional data.Doppelt et al. (2008), for instance, remarked that classroom observations and design portfolios showed that "low achieving" students "reached similar levels of understanding scientific concepts despite doing poorly on the pen-and-paper test" (p.34).Reviewing existing literature does suggest, however, that designbased learning research may benefit from exploring new ways to interpret such information, as we will see next.

| Interpreting design-authentic information
Students' understanding of science concepts can be studied from a variety of perspectives, which we also see in research that studies design-authentic student information.
Researchers in the field of design-based learning have examined, for example, whether students are able to connect certain science concepts to a design context (e.g., Siverling et al., 2019;Valtorta & Berland, 2015).An analysis of high school students' design talk, for instance, led Valtorta and Berland (2015) to conclude that there were strikingly few episodes in which students integrated science concepts into their design process.The accuracy of students' application of conceptual understanding has also been a focus of interest.Fortus et al. (2004), for instance, evaluated students' design artifacts using a scoring list to determine whether students had applied particular science concepts correctly (e.g., identifying the anode and cathode of a battery design).Another prominent analytic perspective entails characterizing students' misconceptions, alternative conceptions, and/or scientific conceptions in design contexts (e.g., Schnittka & Bell, 2011;Wieselmann et al., 2020).Such work showed, for example, that middle school students who were solving a design challenge and experienced demonstrations targeting common alternative conceptions had more scientifically accurate conceptions about insulation than students in more traditional science classrooms (Schnittka & Bell, 2011).
On the one hand, design-based learning research has thus shown that studying designauthentic information from these perspectives can provide interesting insights into certain aspects of students' understanding of science concepts in design-based learning contexts.On the other hand, we see that consulting the broader science education literature suggests that developing additional analytic perspectives for studying this data could lead to new insights.For example, research on cognition has highlighted the importance of paying attention to students' implicit and intuitive understandings of their everyday world (e.g., DiSessa, 1993;Vosniadou & Brewer, 1992).Characterizing these ideas can, among other things, support the identification of cognitive resources that may help students progress toward more sophisticated understandings (D.E. Brown & Hammer, 2008).Some of the existing approaches for interpreting designauthentic information, however, rely heavily on students' use of scientific vocabulary, sometimes even excluding data that involves everyday understandings (e.g., Valtorta & Berland, 2015).Also, while inferring misconceptions or alternative conceptions does draw attention to the relevance of students' everyday science understandings (Schnittka & Bell, 2011), this may inadvertently feed the common belief that teaching science entails fixing or preventing a list of common mistakes (Talanquer, 2006).Rather than evaluating student understanding against a "body of correct knowledge," science education researchers have argued for employing ways of interpretation that reflect how ideas are assessed in a discipline (Coffey et al., 2011).
A framework that appeared to offer an approach addressing the above-mentioned issues for use in chemistry education contexts is the "chemical thinking framework" (Sevian & Talanquer, 2014).This framework has been receiving particular attention in recent chemistry education research, but has not yet been applied to design-authentic data collected in design-based learning classrooms.We describe the framework and its key principles in more detail in the next section.

| The chemical thinking framework
This study draws on the "chemical thinking framework" (Sevian & Talanquer, 2014) to analyze design-authentic sources of information.This framework was developed to support the creation of curricula, instruction and assessments that are better aligned to the nature of the chemistry discipline, relevant to students, and informed by research on how students learn chemistry through time (Sevian & Talanquer, 2014).Achieving the type of meaningful education that is envisioned by the framework, requires students to be actively engaged in typical chemistry activities, such as the design of new synthetic pathways or new ways to harness the chemical energy of substances."Chemical thinking" is then taken to entail "the development and application of chemical knowledge and practices when analyzing, synthesizing, and transforming matter for practical purposes" (p.10).In their presentation of the framework, Sevian and Talanquer (2014) also refer, often interchangeably with "thinking," to the framework's usefulness for evaluating students' "conceptual understanding" as students are engaged in chemistry activities.They describe, for instance, how the framework's six selected crosscutting chemistry concepts seek to offer "lenses through which to analyses students' conceptual understanding" (p.13; also see Table 1).
In fact, the core of the work on the framework's (continuous) development is shaped by examinations of student understanding regarding the crosscutting concepts presented in Table 1 while students are engaged in various chemistry activities (Sevian & Talanquer, 2014).This conceptual understanding can be studied and described along two dimensions: "conceptual sophistication" and "modes of reasoning."The first dimension seeks to characterize students' use of "underlying assumptions," a type of (implicit) cognitive element that supports or constrains students' chemical thinking (e.g., the assumption that matter has the same properties at macroscopic and submicroscopic scales; Talanquer, 2009).The second dimension measures the complexity of students' thinking (e.g., students' ability to connect ideas and build justifications; Sevian & Talanquer, 2014).
While its developers have argued the importance of applying this framework to characterize students' conceptual understanding in various contexts, including design-based learning contexts (Sevian & Talanquer, 2014), this has actually not yet been investigated using data gathered in real classrooms.Previous chemical thinking research used data collected in, for instance, one-on-one research interviews where students were asked specific, design-related questions (Cullipher et al., 2015;Sevian & Talanquer, 2014;Weinrich & Talanquer, 2015).Design-based learning research conducted within classrooms has long stressed, however, that it is by no means a given that students use their understanding of science concepts while designing, particularly when the teacher is not present to support this (see, e.g., Kolodner et al., 2003).Whether the framework also facilitates analyzing design-authentic, real-classroom student information, and what understanding regarding the crosscutting chemistry concepts students apply in a design-based class thus remained to be examined.The present study addresses this, and for this initial investigation we limit our study to the "conceptual sophistication" dimension of the chemical thinking framework (as in, e.g., Ngai et al., 2014;Weinrich & Talanquer, 2015).Analyzing students' conceptual understanding from this perspective could already complement common analytic approaches in design-based learning research, as we describe next.

| Conceptual sophistication
Characterizing students' conceptual understanding according to the conceptual sophistication dimension of the chemical thinking framework entails focusing on the, often implicit, assumptions about chemical entities and processes that support as well as constrain student thinking T A B L E 1 The six crosscutting chemistry concepts and associated core questions of the chemical thinking framework (Sevian & Talanquer, 2014) Crosscutting chemistry concept Core question

Chemical identity
How do we identify matter?
Structure-property relationships How do we predict the properties of matter?

Chemical causality
Why do chemical processes occur?
Chemical mechanism How do chemical processes occur?
Chemical control How can we control chemical processes?
Benefits-costs-risks How do we evaluate the impacts of chemically transforming matter?(Sevian & Talanquer, 2014).When engaged in a chemistry task, students may, for example, state that atoms or molecules expand when heated, or that atoms or molecules have the same density as the actual substance.Both these statements can be explained by students relying on the underlying assumption that the granules or particles that comprise a substance have the same properties as a macroscopic sample of the substance (Talanquer, 2009).This analytic perspective could complement common approaches used in design-based learning research in several ways.For one, it would allow researchers to study not only which chemistry concepts students make use of while designing (as in, e.g., Siverling et al., 2019), but also enable descriptions of their level of sophistication.Whereas some underlying assumptions can be traced back to everyday experiences (such as the example above), others involve more academic knowledge (i.e., are more sophisticated; Sevian et al., 2018).Yet, focusing on students' use of assumptions can still allow one to take into account the dynamic and contextual nature of student understanding (also see D. E. Brown & Hammer, 2008).For example, expert chemists have been found to rely often on more normative assumptions, but they may still make productive use of more everyday assumptions in certain situations (Sevian et al., 2018).This is also relevant in design contexts, where the quality of ideas may be evaluated based on, for example, their usability for solving a certain design problem (Kolodner et al., 2003).
A focus on assumptions is furthermore seen as contrasting with another perspective we had come across in design-based learning research, namely that of zooming in on students' misconceptions or alternative conceptions (as in, e.g., Schnittka & Bell, 2011).Characterizing misconceptions or alternative conceptions can be perceived as resulting in vast and loosely connected inventories of mistakes that instruction needs to fix (Talanquer, 2006).Research on assumptions, however, aims to describe the cognitive elements underlying such alternative conceptions (Talanquer, 2006), and which may actually be leveraged as resources or stepping stones toward more sophisticated understandings (Sevian & Talanquer, 2014).Work conducted from this perspective also seeks to provide coherence by mapping the landscape of assumptions that "commonly" guide student thinking during chemistry tasks (Sevian & Talanquer, 2014).In order to provide such coherence, researchers have also combined one or more interrelated assumptions into so-called "conceptual modes" (Weinrich & Talanquer, 2015).This approach additionally seeks to accentuate that a student can express different understandings regarding a given concept, depending on the context (Weinrich & Talanquer, 2015; also see Mortimer, 1995).In one study, for instance, asking students to design a synthesis process during an interview elicited less sophisticated conceptual modes than when students were asked to compare the easiness of given reactions (Weinrich & Talanquer, 2015).
A final feature of the chemical thinking framework that we deem important to highlight here, is that it appeared to be potentially suitable for studying different types of student information.Researchers have, for example, been able to characterize students' use of underlying assumptions based on student talk (e.g., Cullipher et al., 2015), and students drawings and associated writings (e.g., Stains & Sevian, 2015).The use of instruments that aid blending inferences from multiple data sources can lead to more robust characterizations of student understanding (e.g., Griffin et al., 2010).Such instruments can furthermore facilitate investigations into what different types of sources may be able to reveal of students' conceptual understanding.Whether the chemical thinking framework also offers affordances for the analysis of design-authentic sources of information, specifically design talk and annotated design drawings, is a topic of investigation in this study as we will describe next.

| Design talk and annotated design drawings
Small-group design planning and drawing activities are the classroom activities framing the present study's data collection.These design activities play an important role in many educational models for design-based learning in science education (including, Chusinkunawut et al., 2020;Fortus et al., 2004;Kolodner et al., 2003).Moreover, previous research, predominantly conducted in elementary and physics educational settings, indicates that students may make use of understanding of science concepts during design planning and drawing (e.g., Chusinkunawut et al., 2020;English et al., 2017;Roth, 1994;Sung et al., 2019).For example, productively generating potential design solutions in a team requires students to explain and justify design ideas to each other for which they may draw on their conceptual understanding (English et al., 2017).
Our study zooms in on three sources of design-authentic information that tend to arise during small-group design-planning and drawing activities: students' talk within their design team, students' talk with the teacher participating in the team's conversation, and students' annotated design drawings.Students' use of underlying chemical assumptions has been characterized using talk-, drawing-, and writing-based information before (e.g., Cullipher et al., 2015;Stains & Sevian, 2015), making these design-authentic counterparts a good starting point for this study.Also, while design-based learning researchers have hinted at the potential richness of students' talk and annotated design drawings, there are indications that these sources may differ in what they reveal.For example, design-thinking research has described students' talk as more revealing of knowledge resources than students' design products (Guzey & Jung, 2020).Still, design drawings accompanied by annotations (e.g., labels, dimensions, narratives, arrows) have provided certain insights into students' conceptual understanding (English et al., 2017), and offer a source of information accessible once students have left the classroom.Also, on the one hand, formative assessment research tells us that teachers may leverage conversations with students to unobtrusively yet purposefully elicit understanding (Ruiz-Primo, 2011), which may also occur in design-based learning classrooms (Guzey & Aranda, 2017).On the other hand, however, teachers might only hear students' superficial retellings during such interactions, while students may be more effective in drawing out conceptions among each other when engaged in authentic tasks (J. S. Brown et al., 1989).
These examples suggest that there may be differences in what different design-authentic sources of information reveal of students' conceptual understanding, but this has not yet been thoroughly examined.Such work could, however, inform the data collection and interpretation efforts of researchers and teachers in design-based learning contexts (e.g., Hiebert et al., 2007).This study's inclusion of multiple design-authentic sources of information allows us to explore this.As such, we will study both what combining the three sources reveals, and how these sources compare in the variety of student understandings which they reveal.We will investigate this at a small-group level rather than at an individual level.Student understanding can be studied at different space scales (e.g., Levin et al., 2018), and the small-group level suits our explorative aim and research context (e.g., assumptions inferred from an annotated design drawing may be group products).

| Aim and research questions
In this study, we aim to characterize students' understanding of chemistry concepts as applied during design planning and drawing activities by analyzing three design-authentic sources of information using the conceptual sophistication dimension of the chemical thinking framework.This analytic perspective entails focusing on students' use of underlying assumptions and conceptual modes in order to provide coherent characterizations of students' conceptual understanding as used during engagement in a chemistry activity (Sevian & Talanquer, 2014;Weinrich & Talanquer, 2015).Contrary to previous work conducted with the chemical thinking framework, we will include all six crosscutting concepts in our analysis, namely chemical identity, structure-property relationships, chemical causality, chemical mechanism, chemical control, and benefits-costs-risks (also see Table 1).We deem this important as previous designbased learning research suggests that the multifaceted nature of design challenges may cause students to involve a variety of concepts (Siverling et al., 2019;Watkins et al., 2018).The sources of student information we direct our investigation at in our study are students' talk within their team, talk when the teacher participates in the conversation, and annotated design drawings.We collect these sources in the context of small-group design planning and drawing activities during a Dutch design-based learning project for 10th-grade chemistry education (15-16 years old), implemented by two different teachers.
We formulated two research questions to guide this investigation: RQ1.What do students' talk within their team, talk when the teacher participates in the conversation, and annotated design drawings combined reveal about the conceptual modes and underlying assumptions which students use while planning and drawing designs?RQ2.How do students' talk within their team, talk when the teacher participates in the conversation, and annotated design drawings compare in terms of the variety of conceptual modes and underlying assumptions they reveal?

| METHODS
We examined the research questions using qualitative research methods.In the following sections, we describe this study's design-based learning project, participants, and approach to data collection and analysis.

| Design-based learning project
In design project the "Thermo Challenge," 10th-grade chemistry students iteratively design a product which harnesses energy from an exo-or endothermic chemical reaction to change the temperature of a self-chosen drink or food item.The project's aims include stimulating students to apply and develop understanding of chemistry concepts.Students' understanding of reaction energy, reaction heat, and reaction rate is specifically targeted (which relate involve the crosscutting concepts of chemical causality, chemical control, and benefitscosts-risks; Table 1).Drawing design ideas and annotating drawings as a team is framed as a key design practice throughout the nine-lesson project.Drawing-related student activities include playing an introductory drawing game to get more familiar with drawing and to experience its relevance; formulating success criteria for design drawings with the class by discussing examples; and making annotated drawings as part of the "instructable" of a teams' final design solution.The present study focuses on lessons 3 and 7 of the project, when students plan and draw designs in their design teams before constructing and testing prototypes in subsequent lessons.An overview of activities for each of the nine project lessons is provided in Table S1.Design canvasses were developed to support students' activities, activate students' understanding of chemistry concepts, and offer potential sources of information for teachers (also see Puntambekar & Kolodner, 2005).Students use one or two new canvasses each lesson, and have access to their canvasses of previous lessons.Canvasses contain prompts and empty spaces for teams' responses (e.g., "sketch three different design ideas," lesson 3; "How did you incorporate your understanding about reaction rate, colliding particles, and activation energy in the design?"lesson 7).They have a large, A3 paper size to support collaboration within a team.The main design canvas for lesson 3, which includes example drawings of potentially useful materials and objects, is provided as Figure S1.During the design planning and drawing activities, students also have access to construction materials, and any prototypes already constructed.The teacher guide included presentation drafts, information on proposed classroom and lab activities, examples of student work, and questions for eliciting student understanding in class drafted by teachers.

| Participants
Two chemistry teachers and twelve 10th-grade students (15-16 years old) participated in this study.The teachers were voluntary members of a professional learning community on designbased learning and formative assessment in chemistry education.They had implemented the Thermo Challenge design project a year before, and had experience with other design-based chemistry projects.The teachers were selected because of these experiences, and their interest in participating.Teacher 1 had about 7 years of experience in teaching secondary school chemistry, and Teacher 2 about 3 years.Both held a master's degree in (bio)chemistry, were qualified for teaching upper-secondary school chemistry, and had worked as (bio)chemical engineers.They taught chemistry in the same urban secondary school in the Netherlands, and had engaged the students participating in this study in design before.
Before the start of the Thermo Challenge design project, each teacher divided their 10th-grade chemistry class into teams of three students with the aim to promote learning.Teachers were asked to suggest two teams per class to form the center of our data collection; teams of students who would want to participate in the design activities and our research.Focusing on two teams per class would allow an in-depth analysis of their design-authentic sources of information.Student teams A and B were part of Teacher 1's class (a general secondary, so-called "havo" class), and teams C and D of Teacher 2's class (a university preparatory, so-called "vwo" class).The chemistry curricula in the Netherlands at the havo and vwo level prescribe the same set of design practices to be learned, and the curricular requirements regarding the chemistry concepts targeted in the Thermo Challenge project are very comparable.Generally speaking, though, the havo curriculum has been characterized as more application oriented, and the vwo curriculum as more research oriented, and as expecting a higher competency level (National Institute for Curriculum Development, 2017).
Students and teachers were informed about our general research aim (understanding how teachers can gain insight into student learning), and research process (e.g., approach to data collection), and gave consent.Due to other obligations or illness, focus teams were not always complete.In lesson 3, two students of team B were absent, and one student of team C.

| Data collection
We collected design-authentic sources of information in lessons 3 and 7 of the Thermo Challenge project, when students were engaged in small-group design planning and drawing.To gather students' talk arising from the design activities (both within a design team and when the teacher participated in the conversation), we used video and audio recorders.We positioned a small-size action camera with a wide angle (showing, e.g., who was talking), and an audio recorder (providing better audio quality) at each team's table.To gather students' annotated design drawings, we took photographs of their design canvasses after each lesson.The photographed drawings thus represent a source of information available when students have left the classroom.We collected secondary data by filming the class as a whole, recording teachers' talk, and taking field notes.We tested the video and audio setup in each class during the first lesson of the design project, also as a way for teachers and students to get acquainted with the approach to data collection.

| Data analysis
Drawing on ethnographic analysis approaches (as laid out by Miles et al., 2013), we analyzed the collected design-authentic sources of information in three main phases.First, we prepared the data.Then, we examined the three design-authentic sources of information using the chemical thinking framework, characterizing which underlying assumptions and conceptual modes students used during design planning and drawing (RQ1).Finally, we compared the different sources of information in terms of the variety of conceptual modes and assumptions they revealed (RQ2).
In the following sections, we describe these analysis phases in greater detail.

| Preparing data
We transcribed students' and teachers' talk ad verbatim (in Dutch), and uploaded the transcripts and photographs of teams' design canvasses into nVivo.As we planned to examine what students' talk within their design team ("talk within team") revealed compared to students' talk while their teacher was participating ("talk with teacher"), we coded the transcripts for these two conditions.We consulted the video data to help us distinguish this.Regarding the design canvasses, which consisted of several spaces, we marked teams' design drawings and any annotations (e.g., labels, narratives, arrows; English et al., 2017) for subsequent analysis.

| Characterizing conceptual modes and underlying assumptions
Next, we qualitatively analyzed the transcripts of students' talk and photographed annotated design drawings in nVivo using the conceptual sophistication dimension of the chemical thinking framework.The aim of this analysis was to characterize the conceptual modes and underlying assumptions which students used while planning and drawing designs (RQ1).We commenced with identifying assumptions, and then characterized conceptual modes (which involve one or more interrelated assumptions; also see Weinrich & Talanquer, 2015).
To identify and code underlying assumptions in the data, we used the six crosscutting concepts (Table 1), and previous characterizations of underlying assumptions as lenses (including, Banks et al., 2015;Cullipher et al., 2015;Ngai et al., 2014;Weinrich & Talanquer, 2015;Yan & Talanquer, 2015).For example, previous research regarding the crosscutting concept of chemical identity (how do we identify matter?) suggests that students may use the assumption that matter belongs to distinct classes of stuff with different perceivable properties, usages or origins (Ngai et al., 2014).We also encountered this assumption while examining our data, such as when a student of team C said: "if you just make the sides of that thing heavy, really like with iron or metal, you know, or with wooden blocks."To this student, iron, metal and wooden blocks belonged to the class of heavy matter.However, through constantly comparing data excerpts among each other and against previous findings, we noticed that two assumptions could be distinguished regarding classes of matter.Students assigned properties and usages differently to matter categorized as belonging to a "normal," daily life matter class (e.g., wood, iron and metal being heavy) than matter belonging to a "chemistry" matter class (e.g., metals conducting heat).For instance, from the viewpoint of chemistry matter classes, students often assigned characteristics based on chemical names and formulas (e.g., anything called a substance might react).We thus developed two codes: "normal matter classes," and "chemistry matter classes."So, while previous characterizations informed our analysis, new codes could emerge which were better grounded in this study's dataset (also see Miles et al., 2013).Like others before us, we also considered that a given crosscutting concept can have multiple relevant aspects to it (also referred to as "progress variables"; Sevian & Talanquer, 2014).For instance, the concept of chemical identity (how do we identify matter?) can involve assumptions regarding the aspect of "what types of matter are there?"and the aspect "what cues can be used to differentiate matter types?" (Ngai et al., 2014).
We additionally used source-specific coding strategies in this analysis phase.For interpreting students' talk and the annotations accompanying students' design drawings, we relied on generally-employed strategies for inferring implicit cognitive resources.We paid attention to, for instance, the entity or phenomenon that students were talking or writing about, the predicates they used, and the nature of students' claims (see, for detailed explanations, e.g., DiSessa, 1993;Slotta et al., 1995;Weinrich & Talanquer, 2015).For interpreting students' annotated design drawings, we additionally turned to work on analyzing visual data (particularly Freeman & Mathison, 2009).We paid attention to a drawing's physical features (e.g., thick vs. thin lines could indicate different types of matter), design features (e.g., product consisting of several compartments could suggest that students considered that different types of matter may react), and relationship to classroom activities (e.g., similar shape drawn by students as drawn among canvas examples or during whole-class session suggesting choice for a certain type of matter).As others have noted (incl., Chusinkunawut et al., 2020), we experienced difficulties in inferring conceptual understanding, in our case use of underlying assumptions, when annotations were scarce.For example, the arrows in the design drawing shown in Figure 1 appear to signify heat transfer, suggesting that students considered thermal effects of using matter (concerning the concept of benefits-costs-risks).However, we could not identify a concrete assumption, because what these arrows meant to students was unclear as was how students would evaluate this effect (which is relevant to benefits-costs-risks; Cullipher et al., 2015;Sevian & Talanquer, 2014).Annotations in the form of labels and/or narratives, on the other hand, allowed us to stabilize inferences (also see Freeman & Mathison, 2009), or were informative in themselves.As well as critically examining each source, we continuously went back-andforth between the different design-authentic sources of information to develop a code list that could be used across sources.On a few occasions, we also referred to secondary data (e.g., videotapes showing student gestures and facial expressions).Making use of this information helped us understand what inferences might reliably be drawn from students' talk and annotated design drawings.
The above-described coding process led to the identification of a variety of underlying assumptions which formed the basis for subsequent characterization of conceptual modes.Conceptual modes describe the (different) understandings students express concerning a given concept, and rely on one or more interrelated assumptions (also see Weinrich & Talanquer, 2015).For example, several of the identified assumptions concerning the concept of chemical identity involved students paying attention to the explicit properties of matter as a way to identify matter.We thus combined these interrelated assumptions, and created the conceptual mode "explicit properties" to describe them.This process of characterizing conceptual modes also involved distinguishing modes (and related assumptions) according to their level of sophistication.These decisions were based on previous research, and involved judging whether modes and assumptions were based more on everyday experiences or on school chemistry knowledge (Sevian et al., 2018; also referred to as having a greater explanatory power; Weinrich & Talanquer, 2015).For example, focusing on explicit properties of matter in the identification of matter is typical for novice chemistry learners, and relates to how types of matter are identified in daily life (Ngai et al., 2014).Considering how matter changes under certain (experimental) conditions, however, is seen as a step toward more sophisticated understandings about chemical identity (Ngai et al., 2014).We thus ascribed a higher level of sophistication to the conceptual mode "explicit change" than to the mode "explicit properties."In the findings section, we describe the identified conceptual modes and their associated assumption(s).Table S2 also offers a tabular overview.
We had started this analysis phase with examining the design-authentic sources of teams A and B. Subsequently including team C's data resulted in refining previously identified assumptions, and characterizing one new conceptual mode with associated assumption.Finally adding team D's data resulted in no new codes.

| Comparing sources of information
In the last analysis phase, we focused on comparing the three sources of design-authentic information in terms of the variety of conceptual modes and underlying assumptions they revealed (RQ2).To enable analysis-at-a-glance (Miles et al., 2013), we mapped the conceptual modes characterized in the previous analysis phase on a circle (see Figure 2).Modes were organized according to their level of sophistication; ones placed farther away from the center of the circle represent more sophisticated modes.Using this basic map, we created diagrams displaying which conceptual modes had been identified in which source of information.Diagrams were developed to enable comparison of data sources across teams (i.e., across the four cases), and per team (i.e., per case; also see Figure S2).Using these diagrams, we looked for patterns and anomalies describing the variety of conceptual modes revealed by the different design-authentic sources.This process also involved examining which underlying assumptions had been identified in which sources (diagrams at assumption level are available upon request).We checked our interpretations against the coded data, analytic memo's and field notes (Miles et al., 2013).In the findings section, we describe our observations.
Throughout the three analysis phases we sought to develop trustworthiness by employing strategies proposed by Miles et al. (2013;building on Lincoln & Guba, 1985).In addition to above-mentioned approaches (including, using multiple data sources, connecting to prior research, and testing emerging patterns against data), this entailed regularly discussing data, codes and patterns within this study's research team (i.e., the authors of this paper).The first author did the bulk of the analysis, and the discussions with the team served to consider new and alternative interpretations, and settle on interpretations through consensus (also see Saldaña, 2016, pp. 37-38).An additional educational researcher, who was also familiar with the thermo challenge design project as a chemistry teacher, was consulted as an additional analysis reviewer.

| FINDINGS
In the following sections, we present the analysis outcomes per research question.First, we describe what examining chemistry students' talk within their team, talk when the teacher participates in the conversation, and annotated design drawings using the chemical thinking framework revealed about the conceptual modes and underlying assumptions which students used while planning and drawing designs (RQ1).Second, we describe how the three design-authentic sources of information compared in terms of the variety of conceptual modes and assumptions they revealed (RQ2).

| Conceptual modes and underlying assumptions (RQ1)
Analyzing the three design-authentic sources of information led to the characterization of ten conceptual modes which students used during small-group design planning and drawing.Each of the identified conceptual modes describes students' use of one or more interrelated assumptions about the nature of chemical entities and processes.Before we describe these in detail, we turn to the visual overview of the identified conceptual modes presented in Figure 2. In this diagram, modes relying on more everyday experiences are placed toward the center of the circle while more sophisticated conceptual modes are placed further toward the outside of the diagram.As indicated by the diagram, we found at least one conceptual mode for each of the six crosscutting chemistry concepts of the chemical thinking framework.We also saw that students relied on a single conceptual mode regarding some crosscutting chemistry concepts (e.g., the mode "active agents" in the case of chemical causality), while students applied multiple modes regarding other concepts (e.g., "explicit properties," "explicit change", and "components of matter" for chemical identity).A description of each mode can be found in the findings section, and is available in tabular form Table S2.
We describe each of the identified conceptual modes and their associated assumptions in the following sections.Table S2 also provides a tabular overview.

| Chemical identity
The crosscutting concept of chemical identity concerns the question of how matter can be identified (also see Table 1).Analyzing the design-authentic data revealed that students used multiple assumptions regarding chemical identity while engaged in design planning and drawing activities.These could be described according to three major conceptual modes: explicit properties, explicit change, and components of matter.

Explicit properties
Students who applied this conceptual mode focused on the explicit properties of matter as a way to identify and distinguish matter types.Explicit properties that students turned to concerned how people use matter in daily life, what matter looks, feels or tastes like, and what names and formulas are used to label matter.For example, students of team C concluded among each other that aluminum was a type of insulating matter, because people feeling cold would get wrapped in aluminum blankets.In this example, students were cuing on how matter is used in daily life as a way to identify matter.Students were also observed assuming that what matter looks, feels, or tastes like was a way to identify matter.Students of team B, for instance, were wondering whether magnesium powder was the same as magnesium ribbon, as these types of matter had looked differently during an experiment they had conducted.While in conversation with their teacher one of them asked: "With that magnesium powder … When you make that ribbon smaller do you then get magnesium powder?"We also observed students relying on the assumption that chemical names and formulas could be used to identify matter.See, for instance, the labels of team C's design drawing in Figure 3 where students differentiate "H 2 O" from "Na 4 Cl" using chemical formulas.Paying attention to explicit properties could also help students classify matter.Students of team C, for instance, discussed needing matter they would use at home, like cardboard or plastic cups, for their squishy-cup design idea.These students implicitly classified cardboard and plastic cups as belonging to the squishy matter class.Matter could also be identified as belonging to a class of chemistry matter, where certain labels suggested certain characteristics (e.g., "metals which can conduct coldness").

Explicit change
This conceptual mode entailed students focusing on how matter's identity or its properties change or not in certain conditions in order to identify and distinguish matter.This mode typically still involved a focus on explicit properties of matter (hence the label "explicit" change).Application of this mode could involve students identifying matter based on its expected or observed response to certain (experimental) conditions.Team D students, for example, talked about needing metal for their design because it would conduct coldness well.In some cases, students applying this mode implicitly considered that the identity of matter could change in certain conditions, such as when a salt dissolves in water or when a plastic melts at high temperature.In other cases, students assumed that matter's identity would not change while the properties of that matter could change (e.g., turned on or off, shared, or used up).Students of team A, for example, discussed their idea of placing ammonium chloride in a filter that would allow water to pass through to induce the salt's cooling ability.The filter would retain the unchanged salt, which could then be used again and again to cool down beverages (also see their design drawing in Figure 4).In this example, students assumed that the identity of ammonium chloride would not change while its cooling ability could be turned on and off.

Components of matter
The third, most sophisticated conceptual mode we distinguished entailed students considering that the components of matter were suitable cues to identify and differentiate matter.Students using this mode used the assumption that a homogenous looking mixture could in fact be comprised of several substances.For example, while one of team A's students said that it would not matter whether they used water or coffee for their chosen reaction as they were both liquids, another student of the team highlighted that water was actually a component of coffee.He said, for instance, that when a reaction takes place "there will go water out of the coffee, I think."This second student focused on the components of matter to differentiate matter whereas his teammate had focused on matter's explicit properties.

| Structure-property relationships
The concept of structure-property relationships concerns the question of how we can predict the properties of matter, and depends on students' understanding of relationships between F I G U R E 3 Annotated design drawing (team C, lesson 7).The Dutch labels read: "Straw," "watertight lid," "small bottle with a drink," "plastic insulator and cooler," "anchor," and "catalyst."The narrative reads: "We use this 'anchor' to keep the water and the substance apart.Pull out the small anchor and let the water cool down.After a few minutes the small bottle is cold and you can drink your drink!" structure and properties at different scales.Students rarely considered matter at multiple scales while designing, but one conceptual mode did emerge: inheritance.

Inheritance
Use of this conceptual mode entailed students assuming that the "particles" of matter would have the same properties as the matter at a macroscopic level.Matter at a particulate level was seen as inheriting the properties of a macroscopic matter sample.This conceptual mode was applied by students of team C who had been discussing wanting to use an endothermic reaction to freeze a drink, thus creating a "slush puppie."When talking among each other, they expressed that particles could similarly be cold or get frozen (italics added for emphasis): Student C3: Particles stand still when they are totally cold, right?I mean Student C1: Yes.So, when they are frozen they stand still

| Chemical causality
The crosscutting concept of chemical causality concerns the question of why chemical processes occur.Data analysis revealed one conceptual mode regarding this concept: active agents.

Active agents
Application of this mode entailed students expressing that chemical change happens because of active agents acting on passive substances.These leading agents could be external ones, namely a person or change in surrounding temperature, but also other substances.
F I G U R E 4 Annotated design drawing (team A, lesson 3).The Dutch label "vloeistof" reads "liquid" in English.
This conceptual mode was applied in, for example, the written narrative that accompanied a design drawing of team C (see Figure 3).In the narrative, a reaction is taken to be caused by an external agent pulling out "the small anchor" (italics added for emphasis): We use this "anchor" to keep the water and the substance apart.Pull out the small anchor and let the water cool down.After a few minutes the small bottle is cold and you can drink your drink!Students could also assume substances to be the active agents.For instance, a student of team A told her team mates that water would induce the "stuff" (i.e., ammonium chloride) to start cooling.Water was considered to be the active agent in this example.

| Chemical mechanism
Chemical mechanisms concerns how chemical change processes occur.For this concept, we found one major conceptual mode: mixing.

Mixing
Application of this mode entailed students referring to reactions as something happening simply through mixing reactants.References to specific types of interactions were not made, other than that different starting substances needed to be in contact for a reaction to occur.A student of team D, for example, told his team that he had noticed during an experiment that the contents in a reaction compartment had to be stirred to get the temperature to go down.And, while students of team A were explaining their teacher their drawn design idea (see Figure 4), one of them highlighted that water ("it") needed to get mixed with ammonium chloride ("ammonium") in some way to achieve the intended temperature change (italics added for emphasis): Student A1: Yes, no but we have a filter and you can remove that filter, open up the small cap [Teacher: Nice], and then you put the ammonium in it, and then you put that back in, and then you shake, or well that is not even necessary [Student A3: Yes] because when you start drinking it already passes through, and then it is cold.

| Chemical control
The crosscutting concept of chemical control concerns how one can control chemical processes.We identified two major conceptual modes regarding this concept, both of which typically involved students considering influencing reaction rates.These modes were: substance presence and substance closeness.

Substance presence
This conceptual mode entailed students referring to changing the presence of substances as a way to control a reaction.Students using this mode typically considered the presence of a (somewhat mystical) substance called a "catalyst" to be influencing reaction rates.One of the labels of their resulting design drawing also refers to this presence of a catalyst (see Figure 3).Others referred to changing the amounts of starting substances to influence the reaction rate.We did not observe students going into detail about, for instance, why a catalyst could increase reaction rates.

Substance closeness
Application of this mode entailed students considering not just the presence of a certain substance, but rather the level of contact between different substances to be a way to influence reactions.For example, a student of team A told her team mates that ammonium chloride should be "constantly touching" water to get a better reaction process going than the one that had occurred during a previous experiment.One of their ideas for achieving this was shaking the container.Students of team C, for instance, also considered improving substance interaction and reaction rates through finely graining one of the starting substances.

| Benefits-costs-risks
The crosscutting concept of benefits-costs-risks concerns how to evaluate the impacts of chemically transforming matter.The types of effects that students typically referred to concerned energetic benefits (or the lack thereof), health risks (e.g., people consuming poisonous matter), safety risks (e.g., people getting hurt by touching hot matter), and sustainability (e.g., striving to design reusable containers).Data analysis furthermore showed that students relied on two conceptual modes when considering impacts and ways to control them: personal values and evidence-informed.

Personal values
Using this conceptual mode entailed students assuming that the benefits, costs, and risks of transforming matter could be evaluated or mediated based on personal values.A student of team D, for example, told his team mates: "citric acid does not seem to me like something poisonous."As well as basing judgments on their own personal values, we observed students referring to values of an imagined future user of their product.Use of the conceptual mode of personal values could involve selectively accepting or dismissing effects, data, information, calculations, or chemistry knowledge.For example, when students of team D had calculated the amounts of starting substances required to change the temperature of a certain drink volume, they thought their designed product would become too heavy to carry for its users, and started generating reasons for why their calculation was incorrect.One student said, for instance: Student D2: But actually, let me tell you, you have got, we definitely need less of this actually, because you only need that outside water, euhm, only the outside part you need to cool down.

Evidence-informed
The second conceptual mode students applied entailed assuming that some form of evidence needed to inform their considerations and decisions regarding the benefits, costs, and risks of using matter.This mode concerned students referring to data, information, calculations, and/or chemistry knowledge.For instance, a design drawing of team A showed students using their understanding of insulating and conducting matter types to be able to efficiently harness the energetic benefits of an endothermic reaction.Their drawing (see Figure 5) shows two containers, one is labeled "you put your water in here [↲] from metal," and another "where reaction takes place [↲] from plastic."And, the team D student who did not think citric acid to be poisonous, looked up its toxicity online to be sure.Use of this conceptual mode could also involve asking a teacher for more information before making decisions.A team B student, for example, asked her teacher which materials (which she referred to as "substances") would not melt under the high temperatures caused by an exothermic reaction.Part of their conversation reads: Student B1: Euhm, sir, suppose you take euhm […], for example, if you use very high temperatures.Which substances would be best for that?If you mix these two?Teacher: I do not understand your question.Which substance is best?Student B1:Yes, for the outside, for example, or for the inside.So that it does not melt, for instance when you … The second research question involved examining how students' talk within their team, talk when the teacher participates in the conversation, and annotated design drawings compared in terms of the variety of conceptual modes and underlying assumptions they revealed.In the following sections, we present our findings.First, we compare what each source of information revealed when looking across the four teams' datasets.Next, we compare sources of information per team.Please refer back to the map of Figure 2 to aid interpretation of the diagrams presented in the following sections.

| Comparing sources across teams
Looking across the data of the four teams shows that students' team talk was, overall, most revealing in terms of the variety of conceptual modes that were identified in this source of information.All 10 of the identified conceptual modes, which span six crosscutting chemistry concepts, had been observed in the team talk of one or more teams (see the left diagram in Figure 6).The various underlying assumptions we characterized (see Table S2) had also all been apparent in at least one of the teams' team talk.While students' team talk was thus overall most revealing, students' talk with their teacher and annotated design drawings also revealed the use of a variety of conceptual modes and underlying assumptions.In students' talk during conversations with their teacher, conceptual modes regarding the concepts of chemical identity, causality, mechanism, and benefits-costsrisks had been apparent (see Figure 6; middle diagram).But, we did not observe students using modes regarding structure-property relationships or chemical control when conversing with their teacher.In students' annotated design drawings, modes regarding chemical identity, causality, control, and benefits-costs-risks had been apparent, though not always all modes per concept (see Figure 6; diagram on the right).Regarding chemical identity, for instance, we did observe students focusing on "explicit properties" and "explicit change" in annotated design drawings as a way to identify matter, but we did not observe a focus on "components of matter"  2 for a map of which fields correspond to which conceptual modes).Gray fields represent conceptual modes identified in a source in at least one team's case; white fields represent modes not identified in a source in any of the teams' cases.
in this data source.Similarly, we did notice use of the mode "substance presence" in annotated design drawings, but not that of "substance closeness" (both concerning the concept of chemical control).
Comparing the three diagrams in Figure 6 furthermore shows that students' application of some conceptual modes had been apparent in only one source type, whereas others had been observed in two or three source types.Use of the modes "inheritance" (concept of structureproperty relationships), and "substance closeness" (chemical control) had only been observable in students' team talk.Use of the conceptual modes "components of matter" (chemical identity), and "mixing" (chemical mechanism) had been recognizable in students' team talk and talk with their teacher, but not in any of the teams' annotated design drawings.Students considering "substance presence" a way to control chemical processes was observed in team talk and annotated design drawings, but not in teams' conversations with teachers.The other conceptual modes, such as those regarding the crosscutting concept of benefits-costs-risks, had been observable in all three source types.At the level of underlying assumptions we similarly saw that some assumptions had been apparent across source types, whereas others had been observable in one or two of the studied source types.For instance, only students' team talk had revealed students using the assumption that matter could be identified based on how the matter is used in daily life (associated with the mode of "explicit properties"; also see Table S2).

| Comparing sources per team
We also compared the three sources of information per team which showed that the extent of the variety of conceptual modes and underlying assumptions identified in a certain source type differed between teams (also see the diagrams presented in Figure S2).
The annotated design drawings of teams A, B and C had revealed students' use of a variety of conceptual modes (see Figure S2).However, we could not satisfyingly identify a single mode or underlying assumption in team D's design drawings.Their design drawings did not have labels or narratives which we relied upon for our analysis (see, e.g., Figure 7).Analysis of team B's and C's design drawings, on the other hand, had even revealed the use of underlying assumptions that had not been observable in any of these teams' other sources of information.For instance, analysis of team C's annotated design drawings had revealed them assuming that change processes are driven by an active substance acting on a passive substance (concept of chemical causality; also see Table S2).Use of this assumption had not been observed in team C's team talk or talk with their teacher.
We saw that the team talk of teams A, C, and D had revealed the use of all or many of these teams' conceptual modes, whereas the team talk of team B had been relatively less revealing (also see Figure S2).In team A's case, the complete variety of conceptual modes and underlying assumptions that we had found for this team had been observable in their team talk.In team B's team talk we had identified a relative smaller variety of conceptual modes and assumptions.Still, their talk did reveal the use of a conceptual mode that had not been apparent in their annotated design drawings or conversations with the teacher (namely "substance presence" for the concept of chemical control).
Comparing what students' talk with their teacher revealed for the different teams, highlights that team A's and B's conversations (part of Teacher 1's class) were relatively more revealing than team C's and D's conversations with their teacher (part of Teacher 2's class).In team A's and B's talk to their teacher we had identified a larger variety of conceptual modes and F I G U R E 8 Diagrams showing per team which conceptual modes were identified in that team's sources of information (see Figure 2 for a map of which fields correspond to which conceptual modes).Gray fields represent conceptual modes identified in at least one of the team's studied sources (i.e., team talk, talk with teacher, and/or annotated design drawings); white fields represent modes not identified in any of the team's sources.
underlying assumptions than in the other teams' cases.In team B's talk with their teacher, we could also identify use of assumptions that had not been apparent in any of the team's other sources (e.g., assuming that matter's identity can transform in certain conditions).Conversations between teams C and D and their teacher had been relatively less revealing.
Although not a focus of this study, we also saw that comparing teams' data revealed a unique profile per team describing the variety of conceptual modes they used (see Figure 8).Five of the ten identified conceptual modes were used by all teams: "explicit properties" and "explicit change" (chemical identity), "active agents" (chemical causality), "personal values," and "evidence-informed" (benefits-costs-risks).The other five modes were not used by all teams.Rather, each team relied on a unique combination of conceptual modes during the design planning and drawing activities (see Figure 8).For example, in addition to the commonly used modes, team A's profile of modes also included "components of matter" (chemical identity), "mixing" (mechanism), and "substance closeness" (control).And, students of team C, for instance, were the only ones we found to be using the mode "inheritance" (structure-properties), in addition to both modes concerning the concept of chemical control ("substance presence" and "substance closeness").

| CONCLUSIONS AND DISCUSSION
Design-based learning is frequently highlighted as a meaningful way to help science students apply or develop understanding of science concepts (including, Apedoe et al., 2008;Fortus et al., 2004;National Research Council, 2012).Yet, researchers have also stressed that it is not a given that students will use their understanding of science concepts when they are engaged in solving design challenges (see, e.g., Kolodner et al., 2003).Examining students' understanding of science concepts in different design-based learning contexts has thus continued to be an important topic of research (see, e.g., Henze & de Vries, 2021).
To help advance research in this area, this study explored the affordances of a novel analytic approach to characterizing students' conceptual understanding in a design context.Using the conceptual sophistication dimension of the chemical thinking framework (Sevian & Talanquer, 2014), we characterized 10th-grade students' understanding of chemistry concepts in a design-based learning context.This analytic approach entailed identifying and describing students' use of so-called "underlying assumptions" and "conceptual modes" regarding six crosscutting chemistry concepts while designing (also see Sevian & Talanquer, 2014;Weinrich & Talanquer, 2015).This approach appeared to offer an analytic perspective uncommon in existing design-based learning literature.For example, analysis using the chemical thinking framework allows for recognizing the relevance of students' everyday and implicit science understandings and their context sensitivity (Sevian & Talanquer, 2014;Talanquer, 2006).Some existing approaches for analyzing data collected in design-based learning classrooms, on the other hand, focus on, for instance, students' scientific understandings or (in)correctness of ideas (e.g., Fortus et al., 2004;Valtorta & Berland, 2015).The chemical thinking framework furthermore seemed to offer opportunities for studying different types of student information.Designing students can convey what they mean through different forms of "design-authentic" information (e.g., design talk and drawings; Roth, 1994), with potentially different affordances (see, e.g., Guzey & Jung, 2020).
To guide this study's investigation into affordances of analyzing design-authentic data using the chemical thinking framework, we developed two research questions.First, we asked what three sources of student information (namely students' talk within their team, talk when the teacher participates in the conversation, and annotated design drawings) revealed about students' use of conceptual modes and underlying assumptions while planning and drawing design ideas (RQ1).Second, we asked how these three sources of information compared in terms of the variety of conceptual modes and underlying assumptions they revealed (RQ2).We gathered data as chemistry students from two classrooms in the Netherlands were planning and drawing design ideas for a product harnessing chemical energy to change the temperature of a beverage or snack.
In the following sections, we draw conclusions and discuss our findings.We also consider limitations of this study, and propose avenues for future research.

| Affordances of studying conceptual modes and underlying assumptions
This study's findings regarding RQ1 demonstrate that analyzing design-authentic information using the chemical thinking framework indeed offers a way to characterize students' understanding of chemistry concepts as applied during design-based learning activities.Use of the "conceptual sophistication" dimension of the framework (Sevian & Talanquer, 2014), resulted in the identification and description of ten conceptual modes relating to six crosscutting chemistry concepts (also see Figure 2).Each conceptual mode was associated with one or more underlying assumptions about the nature of chemical entities and processes (also see Table S2).Chemical thinking literature highlights that being able to capture the nature of conceptual modes and assumptions used by students while engaged in chemistry activities, means having a window into students' understanding of chemistry concepts (Sevian & Talanquer, 2014;Weinrich & Talanquer, 2015).While students' conceptual understanding may be studied from additional perspectives still (e.g., modes of reasoning; Sevian & Talanquer, 2014), focusing on students' use of (implicit) cognitive resources in the form of assumptions and conceptual modes already provided some interesting insights.
The findings reveal that students (implicitly) drew on their understanding of several chemistry concepts as they were planning and drawing designs.We found students using conceptual modes related to each of the six crosscutting chemistry concepts of the chemical thinking framework (incl.chemical identity and chemical control; see Figure 2).Previous design-based learning research suggests that students may involve several science concepts while designing (e.g., Siverling et al., 2019), but this had not yet been demonstrated or coherently characterized in a secondary school chemistry setting.Both design-based learning studies and chemical thinking studies typically zoom in on students' understanding of one or a few specific science concepts (e.g., Apedoe et al., 2008;Fortus et al., 2004;Maeyer & Talanquer, 2013;Weinrich & Talanquer, 2015).While such an approach may facilitate the tracking of students' understanding about targeted concepts, the present study's approach allows one to take a broader view.As such, our approach could facilitate, for instance, investigations into students' ability to connect concepts while designing (also see Apedoe et al., 2021).Moreover, comprehensive characterizations of students' understanding during design-based learning, such as presented in this study, may be leveraged to help teachers notice and respond to the range of ideas that designing students may express in class.A previously conducted study suggested that some chemistry teachers may indeed need help to achieve this (Stammes et al., 2021).
The present study's findings furthermore show that use of the chemical thinking framework can facilitate the differentiation of conceptual modes based on their degree of sophistication (i.e., involving more everyday or more academic ideas; Sevian et al., 2018).Regarding the crosscutting concept of chemical identity, for instance, we found students employing three conceptual modes: "explicit properties," "explicit change", and "components of matter" (in order of increasing sophistication).This type of work can support the identification of cognitive resources that instruction may leverage as stepping stones toward more sophisticated understandings (Ngai et al., 2014;Sevian & Talanquer, 2014).These characterizations also demonstrate that students can consider a single crosscutting concept, like chemical identity, from multiple viewpoints while engaged in design.This multiplicity of ideas is a key notion in chemical thinking research (e.g., Weinrich & Talanquer, 2015), but we do not see this emphasized in design-based learning research.Perhaps because characterizing heterogeneity in understanding is not facilitated by common analytic approaches in the field, such as analyses of students' misconceptions or alternative conceptions (e.g., Schnittka & Bell, 2011;Wieselmann et al., 2020) or correctness of understanding (e.g., Fortus et al., 2004).Having access to instruments that do facilitate mapping the landscape of understandings that students use could, however, facilitate further investigations into, for instance, the context-sensitivity of students' understanding (see, e.g., Pic on et al., 2020).

| Affordances of different design-authentic sources of information
Comparing the three types of design-authentic sources of information analyzed in this study (RQ2), showed that students' talk within their team, students' talk with the teacher participating, and students' annotated design drawings could all reveal use of conceptual modes and underlying assumptions.Earlier work involving the chemical thinking framework has tended to rely on student information collected through research interviews or surveys with targeted prompts (e.g., Sevian & Talanquer, 2014;Stains & Sevian, 2015;Weinrich & Talanquer, 2015).Our study demonstrates that the framework can additionally be used to make sense of students' understanding of chemistry concepts using data collected in the authenticity of design-based classrooms.Moreover, focusing on students' (implicit) cognitive resources in the form of conceptual modes and underlying assumptions proved to offer a single lens for studying multiple types of student information (i.e., talk and annotated drawings).This means that the approach of using different analytic lenses for different types of design-authentic information (as in, e.g., English et al., 2017) can be overcome.Designing students' communication has been characterized as multimodal (Roth, 1994), and being able to merge multiple types of behavioral information coherently aids the robust characterization of student understanding (Griffin et al., 2010).Our study showed, for instance, that students' use of a particular conceptual mode or assumption was not necessarily observable in multiple data sources.
Our findings regarding RQ2 also point to how and what different design-authentic sources may reveal students' conceptual understanding.Recognizing the limitations and affordances of specific sources of student information can support the work of researchers as well as teachers (see, e.g., Hiebert et al., 2007;Lam & Chan, 2020).Regarding design drawings, our findings highlight the importance of annotations like labels and narratives to be able to draw inferences about student understanding.Analyses across the four teams' datasets furthermore suggest that students' talk during group-wise design activities, supported by design canvasses can offer a rather rich source of information.Students' team talk generally revealed the greatest variety of conceptual modes (see Figure 6).Talk-based sources furthermore proved of interest as they had revealed conceptual modes and assumptions concerning students considering matter at multiple scales (see, in particular, the modes "components of matter" and "inheritance").These had not been identified in students' annotated design drawings.These understandings are typically of interest, however, as thinking about systems and processes at multiple scales is considered a critical aspect of chemistry, and something students often struggle with (Gilbert & Treagust, 2009).To gain access to such understandings in a design context like the one framing the present study (i.e., one concerning chemical transformation rather than, e.g., synthesis) it thus seems to be important to consult students' talk.This is also relevant to teachers, who can elicit conceptual modes and underlying assumptions as we saw in this study (particularly in Teacher 1's case).To gain access to students' use of conceptual understanding, students might additionally be asked to, for instance, create an annotated drawing at a submicroscopic scale as well as any macroscopic design drawings.
Although not a focus of this study, we saw that comparing design-authentic sources also provided some insight into differences between design teams' use of understanding of chemistry concepts.We saw that each team relied on a unique profile of conceptual modes during the design activities (see Figure 8).For example, only two of the four teams were found to use the more sophisticated mode "components of matter" regarding the concept of chemical identity.This study's analytic approach may thus facilitate investigations into differences between students' use of conceptual understanding in design contexts.Perhaps especially so when combined with analyses on the frequency of employed modes (see, e.g., Weinrich & Talanquer, 2015), or analyses conducted at an individual level (see, e.g., how students within a design team seemed to use contrasting conceptual modes; p. 17).Still, the chemical thinking framework was developed in particular to unveil differences between individuals with different years of chemistry training (e.g., from high school students to grad students to chemical experts; Sevian & Talanquer, 2014).Our study's sample, on the other hand, consisted entirely of 10thgrade students.Comparing the teams' emplyed conceptual modes and assumptions to findings of previous studies shows that these are indeed rather typical for students who begin to learn chemistry.The four teams used modes based on everyday intuitions (e.g., "explicit properties" and "mixing"; Ngai et al., 2014;Weinrich & Talanquer, 2015), as well as some modes that involved more school knowledge (e.g., "components of matter" and "evidence-informed"; Ngai et al., 2014;Cullipher et al., 2015).

| Limitations and avenues for future research
The small-scale setup of this study enabled an in-depth investigation into affordances of analyzing design-authentic sources of information using the chemical thinking framework in order to characterize students' conceptual understanding as applied in a design-based learning context.While fitting for our aim, this research design does entail that one should interpret our findings within the limits of the study's context rather than generalize findings.Also, while we took care to enhance the trustworthiness of findings when collecting and analyzing data (e.g., by building on previous research and collaboratively examining data), another mind may yet bring another interpretation.Nevertheless, we see the characterization of students' use of assumptions, and comparison of sources presented here as providing important insights into the possibilities of using design-authentic information.Moreover, this explorative study points to directions for future research.
Future research could examine what and how other, additional sources of designauthentic information may reveal of students' conceptual understanding as they design.For example, students' gestures and design prototypes may also unveil certain understandings (see, e.g., Roth, 1994;Stammes et al., 2021), and teams' annotated design drawings could be treated as a process-based source of information (e.g., by analyzing video recordings of emerging annotated design drawings).The interpretation of such sources may similarly be aided by a focus on implicit use of cognitive resources like conceptual modes and underlying assumptions.Researchers could additionally explore what insights into conceptual understanding in design contexts other analytic perspectives may yield, such as analyses focusing on interactions between implicit and conscious ideas (e.g., D. E. Brown, 2018), or the value of students' social connections (e.g., Wilson-Lopez et al., 2018).To benefit educational practice, in-depth theoretical research should be complemented with investigations into teachers' and students' (developing) use of design-authentic information.This type of research is emerging in elementary school contexts in particular (e.g., Watkins et al., 2018;Wendell et al., 2019), and we conducted a first exploration into chemistry teachers' use of design-authentic information (Stammes et al., 2021).However, more work is required to be able to fruitfully support implementation of design-based learning in secondary school science education.
Informed by the findings of the present study, we also see directions for future research involving the chemical thinking framework.For example, we may expand our collective understanding of students' use of understanding of chemistry concepts in design contexts by transferring this study's approach to other chemistry-design contexts, and comparing findings (e.g., ones involving chemical synthesis or analysis; Sevian & Talanquer, 2014).Students' use of conceptual modes and assumptions while designing should furthermore be studied at more space and time scales (also see Levin et al., 2018).For example, students' individual use of conceptual understanding is a topic requiring further investigation, as well as whether students apply their understanding productively in a design process (also see Kolodner et al., 2003;Sevian et al., 2018).Researchers could furthermore examine the evolution of students' use of conceptual modes and assumptions during a design project, such as students' potential inclusion of more submicroscopic understandings.This type of future research could collectively support formulating a clear designbased chemistry response to the long-standing call to evaluate student understanding in ways reflecting assessment practices in a discipline (Coffey et al., 2011).

F
I G U R E 1 Design drawing with arrows and patterned shapes (team B, lesson 7).
Diagram presenting the identified conceptual modes, organized per crosscutting chemistry concept and level of sophistication.Modes relying on more everyday experiences are placed further toward the center of the circle.

|
For instance, when discussing among each other what to draw and annotate, students of team C said: Student C1: Yes, and then we'll add a catalyst, because then it'll go faster Student C3: To increase the reaction rate

F
I G U R E 5 Annotated design drawing (team A, lesson 7).The Dutch labels read (from top to bottom and left to right): "watertight but you can still unfasten it," "where reaction takes place [↲] from plastic," "you put your water in here [↲] from metal." 4.2 | Comparison of sources of information (RQ2) U R E 6 Diagrams showing across teams which conceptual modes were identified in which sources of information (see Figure

F
I G U R E 7 Design drawings without annotations (team D, lesson 3).