More on Insidious Context

Statistics is (among many things) the art of using data to make decisions in the face of uncertainty. But what factors besides the data influence the decision? Recently, we did problems like these:

20% of Californians aged 65–74 have been diagnosed with diabetes (from that Kaiser site). A group of 6 friends (the protoGeezers, all age 70) is sitting around the table at Denny’s waiting for their biscuits and gravy, and the talk gets round to health complaints. They notice that four of them have diabetes.

Is this group especially sickly, or could it be just by chance? (Better: is it plausible that it’s chance?)

and

Same restaurant, and 6 of their 70-year-old neighbors are waiting for their green salads. They’re talking about health, too, and they notice that none of them have diabetes. They congratulate each other on their good health.

Based on the data, can you conclude whether this group is especially healthy?

 

The number of diabetics in a randomly-chosen group of 6. Fathom simulation, 1000 cases.

Students make the relevant simulation in Fathom (a typical result appears in the graph) and see whether it’s plausible that a randomly-chosen group of 6 from that age range would have four—or zero—diabetics. The orthodox answers are no and yes, respectively. We’re supposed to say that the biscuits group is sickly (after all, P < 0.05) but the salad group is overreaching.

 

In both cases, however, students brought up the surrounding context, as in (spelling intact…)

• P(4+diabetes)=4%. There is always ths possibility that 4 of 6 people being at the table is just chance, but I would have to say that this group of people is a family of a diabetes support group.But it could be because that they always eat pancakes that they have diabetes. (My comment: it’s biscuits and gravy, not pancakes!)
• we only got 13 out of 1000 cases where the group had 4 or more diabetics, so the biscuits are making them sick
• There is a 30% chance that this group is random, however, we know there food choice is salad, so since they are eating healthy, we can conclude that this group is really healthy.
• There is more of a chance that this could have been just by chance, but it could be explained because they only eat salads and they are especially healthy. P(no diabetics) = 26.4%.

This is related to the previous post on whether data can change your mind, but puts a different twist on it. Here, students seem to use contextual hints in the problem statement—and their own preconceptions about healthy food—to help them decide which way to go.

On the one hand, I’m pleased that they do this; we should bring outside knowledge to bear. We should allow for possibilities that are outside the narrow problem statement. The last response above seems particularly astute. After all, a group that eats salads could be fundamentally different from the 65–74 group as a whole. They may indeed be healthier. To make a better analysis, we need more data, for example, the diabetes rate broken down by eating habits.

But I’m worried too. You can’t really tell from the student response whether the student really was thinking that way or just believed that the salads had to bear on the problem.

Put another way, it could be an example of our human tendency to look for patterns and reasons when, in fact, things may be due to chance alone. We often let context and preconception sway our decisions and opinions. As a result, it’s harder (it seems) to say that we don’t have evidence for some effect. That is, it’s harder (in orthodox statSpeak) to fail to reject the null.

This could be the last math class some of these kids take. What’s the habit of mind they should take to college? To bring all of their knowledge to bear? To find numbers to justify the decision they would make anyway? To look only at the data? Or do we have time to help them balance it all?

And on the assessment front, how will I write the standard—the learning goal—that goes with this, and how will I tell if they get it?

Can Data Change Your Mind?

In our study of probability, I’ve tried to keep theoretical and empirical probability as close together as possible, using manual and computer simulation to illustrate how reality can match theory, and how it varies about it. This is all in line with my Big Plan of approaching inference though randomization rather than from the traditional Normal-approximation direction. Heck, maybe I will dump the theoretical, but probably not.

Anyhow, this leads to why we need to include the basics of inference as early as possible. Two reasons:

• It’s why we have probability in a stats course after all; so let’s make our tools useful as soon as possible
• The underlying logic of inference is hard, so it may take lots of time and repetition to get it right. And I’m willing to sacrifice lots and lots of other content for that understanding.

(I wonder, right now, if that’s such a good idea. I mean, doing less content more deeply sounds right, but what inference do we really need? It may be better to pick something else—but not this year.)

During our first encounters with inference, I came across a realization I’d never had. As usual, it’s obvious, and I’m sure many have thought of this before, but here goes. Consider this problem:

“Spots” McGinty claims that he can control dice. Skeptical, you challenge him to roll five dice and get as high a sum as possible. He rolls three sixes, a five, and a four—for a total of 27. What do you think? Can he control dice?

The tail of the (simulated) distribution of the sum of five dice.

The students are supposed to make a simulation in Fathom in which they repeatedly roll five fair dice and sum, and then look at the distribution of sums. They compare the 27 to the distribution, and should conclude that getting 27 or more with truly random dice is rare, so there’s evidence to support the notion that he does have some control.

Well. Student work rolled in, and some of it was like, “the probability of rolling 27 or more is really low, but no one can control dice, so he’s really lucky.”

How much data do you need to change your mind?

This is a really interesting response to the simulation. It brings your prior opinions into play (and may therefore support the idea of doing a Bayesian number in the introductory course, but not this year!), namely, if you basically don’t believe in the effect we’re trying to observe, it will take more data to convince you than if, you’re “in favor” of the effect. For example, if it’s the point of your research.

I’m intrigued by this notion that we need a different P value, and/or a larger effect size, to convince us of some things than others. It depends on whether we’re personally inclined to find the effect real or not.

This issue is at the center one of the more intriguing talks I’ve heard recently, the one by Jessica Utts at ICOTS last summer. Here is the link to the conference keynotes; you will also see a lesser-known talk by the astounding Hans Rosling as well at three more good ones by other luminaries. The Utts talk asks us, rather brilliantly, to confront our own preconceptions and address that question: how much data do you need to change your mind?

Problem Archetypes

I bet somebody has written a book about this, but I’m unaware of it, so here goes. Stop me if you know, and put me out of my misery.

Jason Buell just posted about how interesting it is when we (or students) don’t go to the question we expect in a given situation, and how important it is for us to break set. For example, when you have nine supreme-court justices and they start shaking hands, every math teacher in the room knows to ask, “how many handshakes altogether?”

It’s vital that we learn to ask other questions. But this post is not about that.

Rather, let us observe that the “handshake problem” is an example of what I’m gonna call a problem archetype. It’s part of our mathematical maturity (I claim) that we have a fistful of these that we can bring out and use; and we do, because they’re useful. It may be that other problems have the same mathematical structure, or that it illustrates an important principle, or some other reason I haven’t thought of.

In any case, it’s part of the shared culture. We refer to it in shorthand in order to communicate with one another or to remind ourselves. It often has a name, as in, “the handshake problem,” or, to name another, “the Monty Hall problem.” (I happen to dislike the Monty Hall problem for the classroom, but I still think it’s archetypal.)

So:

• What are these? Can we start a list?
• What role do they actually play in problem-solving?
• Are they, ultimately, a positive influence? Or do they shackle us?

Just to get things started, here are some other archetypes:

• Boat in a river. Is this actually an archetypal problem, or just a common situation in problems in Algebra texts? Does that matter? We all recognize “boat-in-a-river” problems as a particular genre.

• 

  A view of the city with bridges marked.

  The seven bridges of Königsberg. When I first saw a map of the city, I was astonished at the shapes of the rivers. Of course, topology is topology, but still!

• That problem where you cut two squares out of a chessboard, from opposite corners, and then try to cover the board with dominoes.
• Speaking of chessboards, the one where you get one grain of wheat for the first square, and then double every time.

You get the idea.

Flipping the Classroom: Exposition at Home?

Thank you, blogosphere. Here’s an example of an idea from somebody else that made a positive difference in my classroom (and gave me a topic to post about)…

A while back, @TeachingStatistics wrote about flipping the classroom: all the exposition takes place in videos that kids watch at home, leaving more class time for groupwork and actual one-on-one interaction with students. The guy she referenced—Aaron Sams, in Woodland Park, CO—has this inspiring vlurb:

This idea has lots of plusses, of which two are the time savings and the fact that the kids are watching videos all the time anyway, so why not for learning?

And one big minus: you have to make the videos. “I love Camtasia Studio,” he wrote. Well. I may not know Camtasia Studio, but I have made short instructional videos (for Fathom) using iMovie and the like, and I knew firsthand how frigging much time it takes to get it all right.

But then several things happened. One is that I did that ignite talk, for which I made my slides in Keynote, and another is that someone, maybe my chair, reminded me not to to let the perfect be the enemy of the good, or something like that. Besides, I usually regret spending exposition time in class. When I get started, I usually talk too long. And it’s just not a great way to transfer information.

So I tried it, using Keynote (Apple’s PowerPoint; I assume PowerPoint has the same features) and as little time as possible in production. Keynote makes the technology part easy; I made the slides and then just recorded me presenting them. I reviewed the slides, thought about what I wanted to say, and usually got the voiceover right in one take. Well, not “right,” but good enough. We were just starting the new semester, and a new topic, probability. So over a couple weeks, I made an eight-part series, each one about 5 minutes long. It covers some fundamental ideas in probability, area models, and tree diagrams. The videos set us up for conditional probability without actually opening that door. Think of it as a one-short-period lecture on basic probability, broken up into chunks.

Anyhow, they were a hit. Students actually did the homework (“watch these videos”) and knew, at the beginning of class, some of the things I “covered” in the vids. And we could start with what they didn’t know. Furthermore, kids who did understand some technique from the video could help others in the discussions about the homework. And one even said (without any prompting) that the videos helped because you could stop them and go back. Hooray!

It still takes me quite a while to make each one, but all that is in getting the slides to do what I want. I don’t know PowerPoint, but Keynote’s graphics capabilities, though simple, are capable enough to do useful animations.

The biggest possible improvement: have the kids make the videos. Stay tuned.

If you look at them, I know I have to redo #6. If it looks OK, that means I fixed it… 🙂 Suggestions welcome.

(Here is a link to the 8-part probability series.)

December’s Ignite Talk

I had the distinct honor and terror of giving an “ignite” talk at the Asilomar math conference in December 2010. I was much more frightened than usual. This was partly due to the company of luminaries including Phil Daro, Scott Farrand, Steve Leinwand, Gretchen Mueller, and many others.

The extra anxiety also comes from the format: You submit 20 slides beforehand and they auto-advance every 15 seconds. So that’s five minutes, and there is no way to take a little extra time on a point if you need it, no chance to slip up. My voice is about a major 6th higher than usual. Still, it went well, and I’m pleased enough to want to share.

Here is the link for the event.

And here is my talk. I don’t understand why it’s longer than the allotted 5 minutes, but they set my slides and I’m not gonna check up on them…

An Empirical Approach to Dice Probability

Why seven is more likely than ten: the diagram I want them to have in their heads

We’re starting to learn about probability. Surely one of the quintessential settings is rolling two dice and adding. I’ll try to walk that back another time and rationalize why I include it, but for now, I want students to be able to explain why seven is more likely than ten. I want them to have that archetypal diagram in their heads.

But starting with the theoretical approach won’t go very well. Furthermore, with my commitment to data and using randomization for inference, an empirical approach seems to make more sense and be more coherent. So that’s what I’m trying.

The key lesson for me for this report—related to “trust the data”—is that actual data, with the right technology, can illuminate the important concepts, such as independence. This makes me ask how much theoretical probability we need, if any.

What Happened in Class

To do the Dice Sonata (previous post), I had given each student two dice: a red one and another one. They rolled them 50 times, recording each result twice: once to do the sonata, so they could make the graph of actual results by hand, and also on the computer in a Fathom Survey so we could easily assemble the results for the entire class.

If you haven’t used Fathom Surveys, you can think of it as a Google form that you can later drag directly into Fathom. The key thing here is that they recorded the red die and the other die separately. When we were done, we had 838 pairs.

This was Thursday, the second class of the semester. After students discussed the homework, and saw that their sets of 50 rolls didn’t produce graphs with their predicted shapes, we went to the computers to see if things looked any different with more data. To make the relevant graph, students had to make a new attribute (= variable = column ) to add the two values—which they already knew how to do. Here is the bottom of the table and the graph:

The data table and the graph of the sum. BTW: notice the "13?" Someone had entered 5 and 8 for the two dice, resulting in hilarity, accusations, and a good lesson about cleaning your data.

One could stop here. But Fathom lets us look more deeply using its “synchronous selection” feature (stolen lovingly from ActivStats): what you select in any view is selected in all views.

The Dice Sonata

Intrepid readers will remember the form of a Sonata for Data and Brain: you have three parts, prediction, data (or analysis or measurement), and comparison. As a first probability activity, we were to roll two dice and sum, and then repeat the process 50 times. The prediction question was, if you graphed these 50 sums, what would it look like? Then, of course, they were to do it with actual physical dice (more on the data from that next post) and then compare their graphs of real data with their predictions.

Note that we’re starting this entirely empirically. We might expect these juniors and seniors to “know the answer” because they probably did it theoretically and with real dice back in seventh grade. We would be wrong.

A key point: in the post referenced above, I bemoaned the problem of getting the kids to make explicit predictions. What’s great about doing this reflective writing (especially in this community) is that it prompts head-smacking realizations about practice and how to improve it, to wit: have the students turn in the prediction before they do the activity. In this case, I had them predict at the end of the first day (Tuesday) and turn it in; I copied them and taped the originals to their lockers before lunch; and today (Thursday), the next class, they turned in the Sonatas as homework. (I have not looked at them yet.)

Sample Predictions

Five predictions for the graph of 50 sums of two dice. Click to enlarge.

Remember, I asked for a graph, and that’s what I got. We discussed the phenomenon briefly to establish basic understanding, e.g., that the possible numbers are [2–12]. But for your viewing pleasure, a few actual graphs appear at right.

The two outliers appear below. Even so, notice the variety. What does it say about student understandings (or preconceptions) about distributions?

In any case, my hope here was that when they plotted real data, they would be appalled by how not-following-the-pattern a collection of fifty rolls would be.

What We Did

Starting the Second Semester: Liar’s Logic

Day one of semester two. In this “regular” stats class, we’ve basically spent the first semester on issues in descriptive statistics; it’s time to turn towards inferential stats. Not that we will leave all things descriptive behind. I can’t separate them. And neither will we arm ourselves with traditional, frequentist, Normal-based tests and interval estimates.

I prepared a bunch of slides as an easy intro to the semester; my idea was to give them an overview of the big issues. One thing I did right: the first draft of these slides began with presentation of the issues and ended with some short activities to illustrate them. When I realized how wrong that was, I moved the activities and interaction into the midst of the presentation so that you never went more than about two slides without breaking to do something else.

What I would do better: some ending wrap-up that did something to cement things, such as having them write about the big ideas or at least call out a few new concepts or vocabulary words. Instead, we started the homework—not as a pad, but to make sure they knew how to use Fathom Surveys (it’s been a couple of months). We could have done both, but it was OK.

The main thing I wanted to accomplish was to give some basis for the principles of inference. A plan in an AP class—on the first day of the year—might be to do a full-fledged inference activity such as Martin v Westvaco from Statistics in Action. You’d do that using randomization (cards or chips). But here that would be too much too soon. So I did Liar’s Logic, which you might want to know about.

Liar’s Logic

This is a whole-class game in three phases. First, I don’t call it “Liar’s Logic” in front of the class. It’s Guess My Number.

Phase 1: This is the guess-my-number game you have played ever since elementary school. I choose a whole number between 1 and 100 inclusive, and you have to find it using only yes-or-no questions.

This doesn’t take too long, and we can then ask how they did it. Today, they claimed they used the process of elimination, which fits just fine.

Phase 2: We‘ll play the same game except (I explain) there is a small change; see if you can figure out what it is.

Play begins, except this time, I occasionally lie. I mostly tell the truth, but make sure that after a few turns, they are faced with contradictory information. The game would never end, so if they don’t ask if I’m lying, I stop the game and tell them. Today, they actually asked if I was lying, and I said “yes.”

Wonderful disequilibrium ensued. We made the point that this was a stupid game, because I could make it so that you would never finish.

Phase 3: Same game, but with a different change. After every question, I will secretly roll a die. If I get a six, I’ll lie. Otherwise, I’ll tell the truth.

Students rapidly developed the strategy of asking the same question multiple times. The point being that although there is still lying involved, you can finish the game. At the end, they asked “is it 45?” five times and got four yesses and one no.

Yet, no matter how confident you are, you can never be completely sure. I said that one of our tasks this semester is to put numbers on that confidence.

I’m hoping that this will become one of those “touchstone” moments I can refer back to; you know, where I can just say, “remember 45?” and use that situation to talk about probability or confidence levels.

Other topics included:

• Habits of mind, especially being skeptical
• Meaning of inference in everyday life (infer/imply) and science (going from specific to general)
• Importance of thinking about alternative explanations (we used some stats from the news for this)

Whew.