Time Series and Modeling

The second in a sequence of posts about time series. Here is the first one.

Students in traditional stats, as well as in science and math classes, learn linear modeling in the sense of finding a straight line that fits some data. Suitable data are often (but not always) time series: some phenomenon is increasing or decreasing regularly with time; you take repeated measurements; you plot the values against time; et voilà! A linear relationship.

Here is a data set I’ve used before. I put a pot of water on the stove, stuck a thermometer in the water, and turned on the flame. I recorded the time whenever the temperature passed another five degrees Celsius.

The author heated water on a stove. Graph in CODAP. We could clearly connect these dots with lines, and it would make sense.

A linear model looks appropriate here. In CODAP, we can add a movable line (or a least-squares line, but the “eyeball” values from a movable line are good enough) and get a slope and intercept.

Same data, two models in CODAP: a least-squares line (LSRL) and an “eyeball” fit using a movable line.

One key thing for students to understand is the meaning of the slope and its units: about 0.35 degrees per second. By including the units, we give students a leg up in understanding what that slope means: every second, the temperature goes up by 0.35 degrees; it takes about three seconds to go up one degree. Furthermore, the slope is (well, looks) constant: that rate is the same early in the process and later, when the water is hot.

In this case, the intercept (18-ish degrees) is also meaningful: it was the temperature of the tap water. This only works because I started timing when I turned the flame on, so that time = 0.0 is the actual start of heating.

Anyway, all this is modeling pedagogy 101. Without going into all the details, four comments:

• You can use the linear model to predict, but you have to be careful: if you go outside the range of the data, the model might not work. This is part of the point of using heating-water data: if we kept going, the temperature would stall at about 100 °C when it boils and the physics of the situation changes.
• The linear model looks good, but is it the best? It’s useful to make a residual plot, and if you do, you’ll see a pattern in the residuals that suggests that the slope is decreasing as the temperature increases. This is a higher-order effect consistent with Newton’s Law of Cooling. Learning to make and interpret residual plots is a whole ‘nother topic.
• There is nothing special about nonlinear models in time series. Use the same strategies you would use in other contexts.
• We might think of a time series of growth as being a great setting for an exponential model. Ha! It’s amazingly hard to find suitable, clearly-exponential data in the wild.
  (Note added: good old gasstationwithoutpumps comments below, wisely, that cooling water will give you terrific exponential data. True! How could I forget! Here is some cooling-water data I took some years ago for that very purpose.)

Time Series, Functions, and Rates

What’s special about time series when you do modeling?

A time series dataset is a function: there is only one value at every time. In contrast, consider the a linear-modeling task where you measure the weights of different volumes of sand. You would make a linear model, and make predictions and all that, but because it is possible to have two measurements with the same amount of sand, the data won’t pass the vertical-line test. It’s inappropriate in a graph to connect those dots with lines, whereas it is in the heating-water situation.

Because it is a function, you can always put a time series in order, and then you can compute the slope between adjacent points. This slope, as we discussed above, always has a clear meaning: it’s the (time) rate of change of the quantity, averaged over that interval. The next graph shows such a rate.

Plotting rate (the difference quotient) as a function of time. That is, the rate is generally going down because it takes more time to go up 5°.

NB: The chunkiness comes from the fact that the times are to the nearest second; each lower rate is one more second between readings. Because the jagged pattern comes from the resolution of the data, I did not connect these points with lines!

NB2: the CODAP formula for rate is (temp-prev(temp))/(time-prev(time))

Notice that the linear model suggested that the slope is 0.35 °/sec, while the graph of these individual rates—though chunky—suggests that it’s decreasing from maybe 0.38 to 0.32 over the course of a couple of minutes. This is a different way of showing that a straight line may not be the best model, an interesting alternative to making a residual plot.

This slope-as-rate idea works in the non-function contexts, where, for example, the slope of the sand line might be in units of grams per cc, which has a clear meaning. (Isn’t it cool that density is a rate?) But a time series is a little different in the sense that if you do that slope calculation for adjacent points, you’re looking only at the data; there is no model distinct from the data itself: no least-squares line, just the line segments connecting the dots.

That is not to say that the model is useless, far from it. But we often compute these more instantaneous rates, e.g., the rise or fall of the number of COVID cases each day, and these are always possible to compute in time series data.

A gotcha for students: students will sometimes get complacent and think of the rate as the difference between adjacent points. That’s why it’s great to make sure they have to cope with datasets where the points are not equally spaced (as the heating water data are not equally spaced). That is, to get a rate and not just the difference, you have to divide by the difference in times. Slope, after all, is rise over run, not just rise. Notice how this leads directly to the definition of the derivative; wouldn’t it be great if every calculus student understood this before they took that course!

Next time: folding and smoothing and all that.

Another Aside about Traditional Stats

Last time, I claimed that regular stats doesn’t go deeply into time series because (among other things) it’s not very inferential. That’s not completely fair for at least two reasons that relate to today’s topic:

• As suggested above, regular intro stats does do least-squares linear models, sometimes even including transforming nonlinear data to make it linear.
• You might judge whether something is increasing by doing inference on the slope of a graph. That is, you might ask, is the slope of this graph greater than zero? This is true whether it’s a time series or not. Also, you can use randomization techniques (e.g., scrambling) for this task.