2013: Judging Perspective

Kelly Black
Department of Mathematics and Computer Science
Clarkson University
Potsdam NY 13699-5605 U.S.A. 
May 31, 2013

Download this perspective [36K PDF]


An overview of some select observations pertaining to the 2013 Moody's Mega Math Challenge is given. The 2013 Challenge focused on the impact of plastics in the waste stream in the United States. The three questions posed include a relatively straightforward prediction of the amount of plastic that will be in American landfills ten years from now, a set of recommendations for three preselected cities, and a prediction of the impact of a nationwide expansion based on these recommendations.

1   Introduction

The problem posed in the 2013 Moody's Mega Math Challenge was an analysis of the impact of plastics in landfills in the United States as well as ways to mitigate the problem. This year the Challenge was open to roughly half of the United States' high school population. Despite the rapid growth of the event in recent years the quality of the submissions continues to improve. The coaches and sponsors continue to do an incredible job of preparing student teams for this difficult event.

I will focus on this year's Challenge. In past years the judge's commentary has included a number of general comments that will usually hold true, and we skip these comments this year rather than repeat ourselves. I strongly recommend that teams and coaches review our commentary from previous years.

There are two major sections in this report. In the first section I provide some overall impressions of the writing and presentation found in the reports. In the second section I look at each of the three questions that were included in this year's problem statement.

2   Overall Impressions

Some overall impressions are given here. These observations focus on the writing and presentation of the entries. These aspects of the solution papers are as important as the modeling itself. The best model is of little use if the authors cannot express and defend their choices.

These impressions focus on five different areas: assumptions, citations, appendices, equations, and summaries. Each area is discussed in turn.

One important part of the Challenge is for the teams to inform their readers of the context that they created for themselves. Their basic assumptions guide their efforts and provide their perspective to how they approached the problem. Unfortunately, many teams paid relatively little attention to this part of their report.

The reader needs to understand what basic principles the team thought were most important, and the reader should also know which aspects are considered to be less important. Without an explicit statement of these assumptions the reader must rely on her own viewpoint and prejudices, and when a team does not include some aspect the reader must decide if it was intentional or an oversight.

A team should not leave this to chance and should make sure that the reader is fully informed of the team's point of view. The models that are presented are viewed through the lens of the team's assumptions. These assumptions can be presented in a variety of ways. Some teams include them at the beginning of their paper in a separate section, some include them as a bulleted list, and others list them at the start of each section. There is no preference on the part of the judges as long as they are clearly stated.

Another issue that stood out this year is the use of citations. More teams make use of references, and for the most part the references are appropriate and are given in a consistent manner. Citations, on the other hand, were a rare find. References are the list of sources that a team used, and most teams provided them at the end of their document. Citations are the marks within the report that indicate which reference led the team to an idea. The best references are of little use without proper citations within the document.

Another issue that arose this year is that there were more teams that included appendices in their reports. A well considered appendix can be a welcome addition to a report, but there should be some notice in the narrative. The teams should let the reader know what to expect in the appendices and let the reader know that they have been included. Without a notice it is surprising to read the report and find an appendix after the list of references. It is nice to come across a pleasant surprise, but surprising the reader of a technical report with important insights after reading the report is not the way to make the best impression.

The writing in the reports has steadily improved each year, and we have been impressed by the considerable gains that the advisors have made in this important area. As an example, the issues raised above are more about the details in the papers and not broad comments on writing in general.  One area of improvement, in particular, has been in the use of equations within sentences.

The student teams continue to do a better job of integrating equations within the prose of the narrative. More teams recognize that an equation is a clause. Also, the punctuation associated with the equations is getting better. There is one more detail, though, that can help. All equations in a technical report should be numbered.

The numbers on the equations are important because they allow people to more easily discuss certain results. Even if the team does not refer to an equation, other people may want to refer to an equation when reading the report. For this reason it is common practice to number every equation in a report.

The last topic to discuss is the summaries. Every year the summary is given a separate, explicit weight by the judges as to its relative importance in the report. Every year the quality of the summaries has improved. Again, the advisors are doing an outstanding job of sharing the importance of the summary with their students and should be commended.

A good summary should provide an overview of the problem. It should have specific results. It should also convey a sense of the approach and principles that the team employed to get their results. This is a surprisingly difficult thing to do, especially in one page. A large number of entries this year wonderfully accomplished this difficult task.

3   Modeling

This year's modeling efforts focused on three tasks. The first task was to determine the total amount of plastics that end up in landfills. The second task was to make a recommendation about what kind of recycling program, if any, should be employed in three test towns. The third task was to extend the results to discuss the impact on a national level. Each of these three tasks is discussed separately.

3.1   Q1: Production of Plastics

The first task was to estimate the amount of plastics that will be in landfills ten years in the future. By far the most common way to approach this problem was to find how much plastic is produced in each of the past years and fit the data using regression. This is problematic in that the nature of the problem requires extrapolation to estimate the amounts of plastic that will be produced in the future.

Because the question required extrapolation into the future, it is essential to motivate the choice of the model and then use regression to estimate the parameters in the model. Many teams simply inspected the data and then decided the model to use for plastic production rather than provide an intuitive or physical explanation for their model. The majority of models were either linear, exponential, high order polynomial, or a function that saturates (levels out). The most common choice for a saturation curve was a curve that most teams called “logistic.” This caused some minor confusion for some judges as there is a formal statistical technique used for discrete data called “logistic regression.”

The teams that provided motivation for a model and provided some intuition for their result tended to be better received for this part of their papers. The data should be used to gain some insight into the situation, but in the end, the presentation should include some physical motivation as to why the trend will continue to hold when extrapolating the data.

The issue of extrapolation is especially acute when using high order polynomials as a model for the data. The resulting model may give a better R2 value, but time values outside the limited range of the data will likely see dramatic swings.

Once the amount of plastic produced was estimated the next step that most teams took was to estimate the percentage of plastic that was not recycled. A wide variety of approaches were employed. Some teams simply assumed that the recycle rate would remain constant. Other teams assumed it would change in time, and again a wide variety of regression techniques were used with the same issues discussed above coming into play.

Once a function for the amount of plastic going into landfills was derived the total amount of plastic could be found. A large percentage of the teams integrated the resulting model over the relevant time span. A smaller number of teams simply added up the total for every year. In most cases this simpler method of adding up the numbers is a more appropriate model. Most teams used data for the total amounts of plastic produced each year, and if they stayed consistent with that definition they should have simply added the amount of plastic that goes into the landfill each year rather than integrating the function.

Another important aspect associated with determining a model for plastic production is that it is strongly related to the economy. A small number of teams noted that the recent decrease in plastic production coincided with the economic slowdown in the United States. For the most part these teams adopted a simple adjustment which is entirely appropriate, and the insight that these teams demonstrated spoke well to their broader understanding of the modeling process.

In addition to lurking variables, another notable approach for a small number of teams was to break up the model for plastic production into two parts. When using interpolation some teams simply used regression to approximate the data. When it came to extrapolation, these teams devised a different model that ensured that there would not be rapid fluctuations and resulted in a more robust approximation.

Finally, another notable feature of this year's solution papers is that more teams performed a more extensive analysis of their models. Students are starting to ask questions about the stability of their models. For example, some teams recognized that an exponential model will result in growth rates that are not sustainable in the long run, and they made effective arguments that their results are appropriate only for the short time span of interest. An additional feature that is becoming more common is that more teams are including a sensitivity analysis of their model. That is, they made small changes in one or more of the parameters or assumptions and asked whether or not these small changes resulted in a large change in their results. This is an important question to ask in any modeling effort.

3.2   Q2 Modeling Specific Cities

The second task was to make specific recommendations for three test cities. The teams had to develop a reasonable metric as a way to quantify the relative costs and benefits of recycling, and the metric had to be adapted to a variety of recycling programs.

One of the most important things to be done in this part of the paper was to balance the costs and benefits of recycling. A number of teams focused only on the costs of a recycling program, and some teams looked only at the benefits. The teams that found a way to balance these two competing aspects of the problem tended to receiver higher scores for this part of their paper. Some teams simply looked at the difference in the costs and the benefits, and other teams looked at a ratio of benefits to costs. Both approaches are appropriate, and the teams were expected to provide some small justification for their choices.

The teams found a wide variety of costs to include in their models. Different teams used different combinations and calculated costs in different ways. The important thing was that the teams give details and reasons why the costs they included were the appropriate ones to examine. A sample of some of the costs that teams used is given below:

Start up costs – the cost of creating a new recycling program.
Maintenance of equipment.
Costs associated with pick up (usually fuel and time).
Costs associated with personnel.

Just as for the costs, the teams also found a wide variety of benefits. Again, each team examined different combinations and used different methods to calculate the benefits. A sample of some of the benefits that teams used is given below:

Selling recycled products.
Reduced costs associated with smaller landfills.
Smaller environmental impacts due to decreased plastics production.

Once the costs and benefits were calculated, they had to be adapted to a variety of different recycling programs. Some teams looked at several different recycling programs and other teams looked only at two alternatives. The models that the teams developed made a bigger impact on how a report was received than the number of different approaches that were examined. A sample of some of the common programs is given below:

No recycling.
Pick up (either single stream or dual stream).
Drop off (either single stream or dual stream).
Unit pricing or “pay as you throw” (including single or dual stream recycling).

When examining the different cities, the teams commonly found the costs for each community based their assumptions about the cities. They often had to find values for the parameters for the resulting models. Some teams used published results while others looked at cities that they felt would be similar to the test cities and based their estimates on findings about their substitute cities.

For example, a common parameter that many teams considered was the percent of residences that take part in recycling efforts. The teams had to determine the differences between small, medium, and large towns based on geographic locations as well as different types of recycling programs. Some teams simply used the national average or used published results for communities of various sizes. Other teams found cities that they felt would be similar to the test cities and based their parameters on those samples.

3.3   Q3: Extending Across the Nation

The last part of the solution paper required teams to interpret the impact of a nationwide adoption of their approach. This was certainly the most difficult part of the problem, and it was the part that had the largest variance in the way the teams interpreted the question.

In fact, it is difficult to succinctly express the different ways that teams approached this part of the problem. There was not one approach that was adopted by a majority of teams. Some of the teams tried to take a weighted mean based on their previous results. Based on that weighted mean and their interpretation of the demographics of the United States they tried to establish the overall benefits of adopting their approach.

Some teams selected other cities and applied their methodology to the cities that they chose. The choices varied widely; some teams picked out cities that they thought were interesting or ones for which they could find the necessary data required to implement their approach. Many teams just looked at the results for the cities, while some teams took those results and tried to extrapolate to the country as a whole.

Another approach that some teams tried was to simply restate their model in a more user-friendly rubric. In doing so, they explicitly established and stated the required information to implement their model and then gave specific instructions on how to use that information.

The majority of teams were not able to get much traction on this part of the problem in the short time allotted. The result is that this was a difficult part to evaluate in the early stages of the judging, and it was not until the latest stages of the judging that this section played a greater role in comparing the remaining entries.

4   Conclusions

The 2013 Moody's Mega Math Challenge asked students to look at three questions related to plastics in the American waste stream. Teams were asked to provide a prediction for the amount of plastic in American landfills in 2023, recommendations for which recycling programs to implement in three American cities, and a prediction of the overall impact of the team’s approach if it was implemented nationwide.

The M3 Challenge is open to roughly half of all United States high schools. The expansion of the program to high schools unfamiliar with the event has not diminished the quality of the solution papers, and we continue to see improvements in the quality of the papers. The teams' teacher- coaches continue to do a remarkable and effective job in preparing their teams.

Also, the writing itself continues to improve. Additionally, the modeling and approaches that the students are using continue to be appropriate and of high quality. More importantly, the analyses that the teams perform on their models is improving. We saw more teams take an introspective view of their models and perform key analyses on their models.

Finally, we are grateful for the hard work that teachers are giving to this event. It is their efforts, combined with those of their students, that have made Moody’s Mega Math Challenge a successful and rewarding experience for all who are involved.

5   Acknowledgements

I wish to thank Kathleen LeBlanc and Michelle Montgomery for their help in preparing this document. Their editorial aid and insights have improved this document and is greatly appreciated.