Biology student mastery regarding the mechanisms of diffusion and osmosis is difficult to achieve. To monitor comprehension of these processes among students at a large public university, we developed and validated an 18-item Osmosis and Diffusion Conceptual Assessment (ODCA). This assessment includes two-tiered items, some adopted or modified from the previously published Diffusion and Osmosis Diagnostic Test (DODT) and some newly developed items.

The ODCA, a validated instrument containing fewer items than the DODT and emphasizing different content areas within the realm of osmosis and diffusion, better aligns with our curriculum. Creation of the ODCA involved removal of six DODT item pairs, modification of another six DODT item pairs, and development of three new item pairs addressing basic osmosis and diffusion concepts. Responses to ODCA items testing the same concepts as the DODT were remarkably similar to responses to the DODT collected from students 15 yr earlier, suggesting that student mastery regarding the mechanisms of diffusion and osmosis remains elusive. INTRODUCTIONFifteen years ago,; ) created the Diffusion and Osmosis Diagnostic Test (DODT; available in and ), designed to assess secondary and college biology students’ understanding of osmosis and diffusion. Odom and Barrow reported that performance on the DODT by college biology majors was consistently poor, and scores earned by college non–biology majors and high school students were even lower ( ). Although understanding how the fundamental processes of diffusion and osmosis work is essential to comprehending a wide range of biological functions, the research of Odom and Barrow and others has demonstrated that student mastery of osmosis and diffusion is extremely difficult to achieve. Inadequate understanding of osmosis and diffusion has been documented among high school and college students in the United States (e.g.,;;;; ) and elsewhere (e.g.,; ).The DODT contains 12 two-tiered, multiple-choice questions.

Osmosis: Osmosis is a type of diffusion in which the solvent molecules move into the solution through a semi-permeable membrane. Example: Plasmolysis of a cell when it is placed in a sugar or salt solution. The present post describes the Differences between Diffusion and Osmosis with a Comparison Table.

The first tier asks “What happens when?” (“What?”), and requires students to analyze a situation and/or make a prediction about what will happen in a situation, given specific conditions. The second tier asks the reason (“Why?”) for the first-tier response. Item distracters (incorrect responses) were drawn from 20 prevalent misconceptions held by students, as identified by the DODT authors (; ).Twelve years later, examined responses and levels of certainty among 58 high school students who responded to the DODT after completing a week of instruction about osmosis and diffusion. Responses among subjects were remarkably similar to those obtained previously. Further, the authors assessed students’ confidence levels and found that students displayed high levels of confidence that their (incorrect) responses on the DODT were accurate. Development and DesignThe ODCA items were developed and modified over a period of several years through 1) administration of developing items each term to a subset of students in multiple classes, 2) reviews by expert faculty, and 3) interviews with both students and faculty about the test items.

Incremental revisions were made between each administration of the evolving ODCA and were evaluated after subsequent ODCA administrations. In developing the ODCA, we attempted to focus on examining students’ reasoning about the biological mechanisms of osmosis and diffusion, which involves relatively high-level cognitive skills.To align with our programmatic learning goals, which emphasize different content areas within the realm of osmosis and diffusion than are addressed in the DODT, we began with the DODT, and deleted some items, modified others, and added items (; see Supplemental Material for ODCA items). In addition, we wanted to reduce the length of the instrument to encourage more faculty, even those teaching large college classes, to use it to inform their teaching, and to encourage students to pay close attention and read all items carefully and reflect on their answers. With our modifications, the validated ODCA is composed of eight paired items (16 questions), while the DODT has 12 paired items (24 questions).

To use examples that are more relevant to essential curriculum in our state, we retained items relating to categories of “kinetic energy of matter” and “membranes” (DODT items 7 and 12 ODCA items 9/10 and 1/2, respectively). To shorten the test, we reduced the total number of items assessing concepts of “the process of diffusion,” “the particulate and random nature of matter,” and “the process of osmosis,” by omitting DODT items 1a,b; 3a,b; and 10a,b. Since concepts of “tonicity and concentrations” and “the influence of life forces on diffusion and osmosis” are not specifically addressed in our learning outcomes, we omitted those items to keep the ODCA brief (DODT items 4a,b and 9a,b, and items 11a,b, respectively). New items were added to the ODCA regarding diffusion of particles, and solute and solvent movement through a membrane ( and ). We also modified one item into a question about human cells, rather than plant cells, so it would be more appropriate for our human physiology courses, where students might not have strong preparation in plant biology and cell structure. We modified the language of many of the response choices during iterations of face validation by expert faculty to: 1) reduce the length of some response choices, especially when the correct choice was the longest option; 2) increase the number of responses that would allow us to diagnose students’ conceptual understanding of diffusion and osmosis, rather than testing students’ ability to define terms; and 3) attempt to reduce the number of concepts being tested in any one response choice by shortening some responses.

Lastly, we reordered response choices to ensure correct responses were more evenly distributed among all response letters (a–d). These changes produced a shorter, validated test that addressed the processes and scientific reasoning we want our graduates to apply accurately. Seven scientifically correct ideasResponse(s)Category A: Dissolving and solutions1.

When a soluble substance is placed in water undisturbed, it will dissolve in water spontaneously, assuming its concentration is below the saturation point of the solution.5bCategory B: Solute and solvent movement through a membrane2. Cell membranes are semipermeable in both directions, allowing some substances to pass through but not others.1a, 2d, 12a3. When separated by a semipermeable membrane through which only water can pass, water particles will move from the higher concentration of water (low solute concentration) into the lower concentration of water (high solute concentration). This can raise the water level on the side to which the water migrates.7a, 8c, 14d, 17a, 18c4. If water moves into an animal cell, the cell may swell and burst because water moves toward regions where there is more solute, so if a blood cell is placed in pure water, the cell will swell and eventually burst.13bCategory C: Diffusion of particles5. Solutes and solvents move from higher to lower concentrations.3a, 5b, 6d, 11a6.

Component particles of all phases of matter are moving; molecules continuously move due to Brownian motion; molecules become evenly distributed throughout their container, and continue to move.4b, 5b, 15b, 16c7. The higher the temperatures, the faster the rate of diffusion, all other things being equal, because the individual molecules are moving faster.9b, 10c. AOrganized by category and expressed in the item responses identified in the second column.From interviews, we realized that some students responded to questions based on their recognition of specific biological terms and their recall of definitions—a low-level and often short-lived cognitive skill that sometimes led them to select incorrect responses reflecting inappropriate scientific reasoning. Thus, as much as possible, we avoided using scientific terminology in the ODCA to avoid triggering recall responses and to encourage thinking about mechanisms. We also eliminated or modified responses with low student response rates to improve the attractiveness of all distracters. Ultimately, six DODT item pairs were omitted, another six DODT item pairs were modified, and three new item pairs were created as a result of the revision process.

The two-tiered ODCA items are numbered consecutively (instead of using 1a and 1b, as Odom and Barrow did).In creating the ODCA, we did not aim to identify new misconceptions held by students. Rather, like the DODT, the ODCA assesses 1) students’ abilities to identify scientifically accurate interpretations of osmosis and diffusion events and 2) their attraction to well-known misconceptions regarding the processes of osmosis and diffusion. Specification TablesSpecification tables were developed and modified along with the ODCA. These tables were used to track the scientifically correct ideas and misconceptions captured in the various item responses to allow comparison of students’ responses to similar ideas in different contexts. The tables are similar in construct to those used in the development of the Conceptual Inventory of Natural Selection (CINS;; ) and to the lists of propositions and misconceptions identified in the development of the DODT (; ). ODCA concepts are organized into three categories ( and ): “Dissolving and solutions,” “Solute and solvent movement through a membrane,” and “Diffusion of particles,” and the item responses associated with those concepts are shown in the right columns of.

Lists the seven scientifically correct ideas examined in the ODCA and summarizes the 20 misconceptions or other types of errors presented in the ODCA alternative responses. Twenty misconceptions and other errorsODCA responsesCategory A: Dissolving and solutions1. Dissolved substances will eventually settle out of solution.5a, 6a, 6c, 16b2.

Solute particles will not dissolve in water spontaneously.6b3. Conflation of solvent and solute concentration and their interactions with solutions.7c, 17b, 17c, 18bCategory B: Solute and solvent movement through a membrane4. Semipermeability is unidirectional.2b5. Only beneficial materials pass through a cell membrane.2c6. All small materials can pass through a cell membrane.1b, 2a7. Water moves from high to low solute concentration.8a, 13a, 14a8.

Solutes do not cross a semipermeable membrane.11c9. When solutes in water-based solutions cross a semi-permeable membrane, they can alter the heights of the liquids on each side; movement of water particles across the membrane does not warrant consideration.7b, 11b, 12c10. In a system open to atmospheric pressure, water always equilibrates the levels on both sides of a membrane.8b, 12d, 18a11a. An animal cell can maintain itself in any environment.13c, 14b11b.

An animal cell cannot survive outside the body.14c12. At a given temperature, solute and solvent particles move across a membrane at different speeds.12bCategory C: Diffusion of particles13. Solvent particles will move from lower concentration of that solvent to higher concentration of that solvent.3b, 7b, 8d14.

Coolness facilitates movement.9a, 10b15. Dye tends to break down in solution.10a16.

As the temperature changes, individual particles (atoms, molecules, ions) expand or contract.10d17. Particles move through solutions or gases only until they are evenly distributed, then they stop.4c, 15a, 16a18. Particles move by repelling one another.4d19.

Particles actively seek (want) isolation or more room.4a20. Particles in gases and liquids are moving; particles in solids are not.16d. Initial Refinement and ValidationAs noted previously, initial versions of the ODCA were administered to biology students for several semesters to improve instrument validity. During this period, incremental changes were made to increase clarity and effectiveness of items, and ensure that all responses were attractive to some students. Face validation was obtained when 56 biology instructors taking part in scoring the AP Biology Examination voluntarily completed the ODCA.

Of those who responded, 33 taught biology at the high school level and 23 taught biology at the community college or university level (San Diego State University SDSU Institutional Review Board IRB Approval 113073). Each of the participants stated that they had taught osmosis and diffusion to students in the past year. In addition to taking the ODCA, these instructors were asked to provide feedback concerning the wording of the instrument items and questions, and this feedback was taken into consideration during development of the conceptual assessment.

The instructors were also invited to describe the specific areas in which their students tend to experience the most difficulty when trying to comprehend diffusion and osmosis.In addition, 16 undergraduate students (10 biology majors, four kinesiology majors, and two nursing majors) were invited to participate in semistructured interviews ( ). The interviews focused on students’ thinking about osmosis and diffusion and were conducted by two graduate students. Each interviewed student was asked to read ODCA items aloud and provide explanations for why he or she selected or rejected each response. Students were also encouraged to suggest an alternative response if they were not satisfied with the fixed choices offered. Additionally, the interviewer asked students to respond to follow-up probes designed to further explore their reasons for answering the way they did.

Manipulatives, demonstrations, and/or drawings were occasionally used to elicit additional information regarding participants’ ideas about osmosis and diffusion.To determine construct validity, we calculated the standard difficulty index (p: proportion of students who answered a test item correctly) and discrimination index (d: index that refers to how well an item differentiates between high and low scorers), and evaluated the instrument's reliability with Cronbach's alpha ( ), using PASW Statistics 17, Release Version 17.0.2 (IBM SPSS Statistics 2009, Chicago, IL). Values from the ODCA were then compared with those reported for the DODT (; ).

ODCA AdministrationOnce ODCA design was finalized, student performance was assessed in four successive semesters during two academic years, 2007–2008 and 2008–2009 (; IRB Approval 278). This study compares the performance of three groups of university students: upper-division biology majors, lower-division biology majors, and lower-division allied health majors. The upper-division biology majors were students enrolled in four upper-division biology courses (“core” courses of the curriculum) taken by most biology majors. The lower-division allied health majors were non–biology majors enrolled in a lower-division microbiology course.

These two groups were surveyed for four semesters. The lower-division biology majors consisted of students enrolled in a lower-division introductory biology course required for all biology majors; they were surveyed during the 2008–2009 year. ANumber and percentage of students who actually completed the ODCA are broken out by semester. UD, upper-division biology majors; LD, lower-division biology majors; NM, nonmajors.A random subset of students in each course was invited each semester to complete the ODCA, using the course management system, Blackboard Academic Suite. Students were asked to complete the ODCA during a 10-d window after course registration ended and enrollments were finalized.

Students were encouraged to complete the ODCA as their contribution to the biology department's annual programmatic assessment effort; three participation points could be earned by completing a “survey.” We used the term “survey” to distinguish the ODCA and other conceptual assessments from the graded tests taken by students. Approximately 20% of the students enrolled in each of the courses were randomly invited to complete the ODCA each semester. The remaining students received one of four other conceptual assessments on different topics examined for programmatic assessment. None of the courses directly addressed topics of osmosis and diffusion before the ODCA was administered, as these processes were taught explicitly in previous courses. Variation among CoursesOf the 670 students invited to complete the ODCA over the four semesters, 408 (61%) responded and gave consent to use their data. Within each level (upper-division biology major, lower-division biology major, and nonmajor), there was considerable within-class variation in student performance.

Consequently, no significant differences were found in student performance across semesters within the upper-division courses, lower-division courses, or nonmajor courses (two-way analysis of variance (ANOVA), P 0.05). Thus, for each level (upper division, lower division, nonmajor), results from all semesters were combined for these analyses. Difficulty and Discrimination IndicesDifficulty indices (p) of the items ranged from 0.27 to 0.98 , with a mean of 0.65, providing a wide range of item difficulty, similar to values reported for the DODT (Odom and Barrow, 1995). The discrimination indices (d) ranged from 0.07 to 0.67, with a mean of 0.44. The discrimination index refers to how well an item differentiates between high and low scorers; it is a basic measure of the validity of an item.

On ordinary tests, a low or negative discrimination index value generally indicates that the item does not measure what other items on the instrument are measuring. However, in the case of the paired items on this instrument, the first-tier “What happens?” item is often quite easy, and provides the necessary context for the more difficult second-tier “Why does this happen?” item. This can produce a low discrimination index for some of the first (odd-numbered) items, but a high discrimination index for the paired (even-numbered) items. For example, three odd-numbered ODCA “What?” items had discrimination indices of 0.07, 0.08, and 0.16—below the value of 0.20 that is typically considered as a minimum ( ), and the subsequent even-numbered “Why?” items had discrimination values over 0.20. Thus, on this two-tiered assessment, a correct answer on the first-tier item (content) did not predict performance on the second-tier item (reason). For purposes of this project, all ODCA test items were considered acceptable, since the item pair (and particularly the “Why?” response) is the element of interest.

A See text for further details.For evaluating the difficulty of an item, it is helpful to consider the likelihood of guessing the correct answer. For a multiple-choice item with four possible responses, there is a 25% chance of guessing the correct response. A two-tier item with two selections for the first tier and four selections for the second tier provides a 12.5% chance of guessing the correct answer combination. In the ODCA, there are two or three response options in the first tier, and three or four in the second tier. This results in chances of guessing correct answers at 8.3% to 12.5%.For the first tier of the ODCA, the percentage of students responding with correct answers ranged from means of 48.3% to 97.8% for upper-division biology major classes, 45.0% to 95.4% for lower-division biology major classes, and 35.6% to 97.3% for nonmajor classes ( and ). The mean percentage of combined content–reason responses on the ODCA that were correct ranged from 28.2% to 92.3% for upper-division biology major classes, 19.9% to 74.7% for lower-division biology majors, and 16.8% to 85.3% for nonmajors. Results from college students taking the DODT and ODCA were strikingly similar , especially for the highest and lowest scoring-item combinations.

Percentages of college students who selected the correct content response (pale bars) and the correct content plus reason combination (darker bars) on (A) ODCA items 3 and 4 compared with DODT items 2a,b, and on ODCA items 15 and 16 compared with DODT items 6a,b, and (B) ODCA items 5 and 6 compared to DODT items 5a,b, and ODCA items 7 and 8 compared with DODT 8a,b. ODCA items are shown in green, while DODT items are shown in blue. NM, nonmajors; LD, lower-division biology majors; UD, upper-division biology majors; Values represent means + 1 SE. Pummel party pc free. Sample sizes appear in Table 4. Non–biology majorsBiology majorsODCA Item no.

(DODT no.)DODTDODTODCAODCALD DODTLD DODTLD ODCALD ODCAUD ODCAUD ODCAItem pairTier 1CombTier 1CombTier 1CombTier 1CombTier 1Comb1/2 (12)95.181.397.385.395.792.395.474.797.892.33/4 (2)90.22294.224.889.723.190.723.097.343.65/6 (5)37.434.144.239.537.636.845.043.148.345.87/8 (8)51.214.635.616.864.128.246.319.949.328.29/10 (7)97.695.989.650.09492.384.865.295.072.311/1255.334.158.044.674.058.013/1464.847.662.246.177.558.415/16 (6)88.665.984.464.19476.188.858.391.772.317/1868.158.663.152.863.153.7. A The first column shows the ODCA item pairs and the corresponding DODT item numbers in parentheses. UD, upper-division courses; LD, lower-division courses.

ODCA results, in bold, represent the student performance across all courses and semesters, by course level. ODCA sample sizes appear in.Test reliability was at least 0.70 each semester using Cronbach's alpha calculation (Fall 2007: 0.70; Spring 2008: 0.74; Fall 2008: 0.73; Spring 2009: 0.70), which is considered to be an acceptable level. Test completion time ranged from 5 to 20 min per student and was not correlated with performance ( r 2 0.05; n = 71, 72, and 83 for three semesters, Fall 2007–Fall 2008). Radar graph shows the percentage of upper-division biology majors (UD), by course and by semester, who selected the correct response for each item on the ODCA.

Items are grouped into the three conceptual categories described in the text. Note the similarity of performance across semesters. S, spring semester; F, fall semester. Sample sizes appear in.Of the nine ODCA question pairs, 3/4, 5/6, and 7/8 yielded the lowest combined content–reason values ( and ), and thus reflected the most prevalent misconceptions among our students.

Excerpts from these three item pairs are shown in, along with frequencies of student responses and variance among semesters in parentheses. Prevalent misconceptionsNMLDUDCategory C: Diffusion of particles b3a: “During the process of diffusion, particles will generally move from an area of (a) high to low particle concentration ” c4a. Because crowded particles want to move to an area with more room. D0.31 (0.05)0.44 (0.01)0.27 (0.06)4c. Because the particles tend to keep moving until they are uniformly distributed and then they stop moving.

D0.37 (0.02)0.24 (0.02)0.23 (0.02)Category A: Dissolving and solution b5a: “If a small amount of table salt (1 tsp) is added to a large container of water) and allowed to set for several days without stirring the salt molecules will (a) be more concentrated on the bottom of the water ” c6a. Because salt is heavier than water and will sink. D0.25 (0.04)0.24 (0.07)0.28 (0.02)6c. Because there will be more time for settling.

D0.10 (0.02)0.24 (0.07)0.28 (0.02)Category B: Solute and solvent movement through a membrane b7a: “In, the water level in Side 1 will be higher than in Side 2 ” c8a. Because water will move from high to low solute concentration. D0.11 (0.04)0.19 (0.02)0.12 (0.02)7b: “In;, the water level in Side 1 will be lower than in Side 2 ” c8a. Because water will move from high to low solute concentration. D0.19 (0.04)0.12 (0.02)0.09 (0.02)7c: “In, the water level in Side 1 will be the same height as in Side 2 ” c8b.

Because water flows freely and maintains equal levels on both sides. D0.29 (0.07)0.27 (0.06)0.26 (0.04). A Values represent proportions of students who selected combined content–reason responses, averaged across four semesters for nonmajors (NM) and upper-division biology majors (UD), and across two semesters for lower-division biology majors (LD).

Frequencies represent mean proportions of students selecting that response by course, with variance among semesters (SE) shown in parentheses. Sample sizes appear in.bConceptual knowledge categories of misconceptions (from and ).cFirst-tier (odd-numbered) item stem.dCommonly chosen (incorrect) second-tier (even-numbered) responses, along with response frequencies for each student group.More than 90% of all students correctly indicated that particles (dissolved substances) would move from areas of high to low concentration (response 3a; and ). However, when asked to provide a reason for their answer, many faltered. The correct combined content–reason ODCA responses, 3a and 4b, were selected. Comprehending Osmosis and DiffusionThe data collected from students and biology experts indicate that the ODCA is a valid and reliable tool for assessing students’ understandings of osmosis and diffusion. Conceptual AssessmentsThere are many ways to use reliable, validated conceptual assessments such as the ODCA and the DODT.

Many (but not all) conceptual assessments, including the DODT and the ODCA, employ students’ prevalent misconceptions as distracters. When such a conceptual assessment is administered at the beginning of a course, it can provide valuable insight to the instructor regarding students’ prior knowledge and beliefs. When an instructor can build from students’ initial conceptions and demonstrate why or how scientific reasoning is better able to explain observations, students may be more prepared to grasp the scientific concepts.The project described here focused on creating an appropriate, validated conceptual assessment and establishing baseline data for an academic program. Once available, there are many ways to use such reliable, validated conceptual assessments, and two strategies are described below.The first obvious use of Conceptual Assessments in Biology is to assess pre- to postlearning gains within a course or curriculum. When students complete a conceptual assessment at both the beginning and end of a course, it is possible to assess learning gains (or losses) within a single semester or to compare different groups of students across semesters to improve course and curriculum designs.

This approach is exemplified with the use of CINS (; ), and three other genetics assessments: the Genetics Literacy Assessment Instrument (, ), the Genetics Concept Assessment ( ), and the Genetics Two-Tier Diagnostic Instrument ( ). This strategy was also used with the Introductory Molecular and Cell Biology Assessment ( ), the Photosynthesis and Respiration Assessment ( ), and Energy and Matter Diagnostic Question Clusters (; ), as well as the DODT (;; ). Several authors have demonstrated the use of conceptual assessments for broader programmatic improvement, including using the Biology Conceptual Inventory, ) using the Host Pathogen Interactions Concept Inventory, and using the Energy and Matter Diagnostic Question Clusters.When conceptual assessment tools are not used for assessing instruction or measuring student learning gains, instructors may use individual items from those assessments as “concept questions” to generate in-class discussions and facilitate biology learning, as encouraged by,. This method has been used effectively for promoting learning of a variety of topics, including, for example, physics ( ), geoscience ( ), and biology (;;; ). Thus, conceptual assessments in biology are resources that allow instructors to plan teaching strategies and help their students to move toward more scientifically appropriate reasoning ( ).Validated conceptual assessments such as the ODCA, the DODT, and others (; ), hold the potential for helping students gain the capacity to talk and think using scientifically principled reasoning, and prepare for solving problems of the twenty-first century (Wood, 2009; ).

These easily administered instruments afford biology faculty data they can utilize to generate instruction tailored to the current conceptual understandings of their students. When instructors are aware of their students’ misunderstandings ( ) and utilize interactive instructional strategies (,; ), students’ comprehension of abstract scientific concepts can be improved.

We are grateful to the students who voluntarily participated in our study. We thank Janessa Molinari-Gruby for her assistance in interviewing students. We thank the SDSU College of Sciences and Department of Biology for supporting our project. We are grateful to the SDSU Instructional Technology Services staff for their help with online deployment of the ODCA. We thank anonymous reviewers for helpful comments of an earlier version of this report. We are also deeply indebted to Louis Odom and L. Barrow, on whose shoulders we stand, for their thorough explication of students’ misconceptions about osmosis and diffusion and for their pioneering creation of the DODT.