The Great Debate

In the world of slogans and sound bites, it is becoming rare to see an important question of public policy being fully explored. So it was a pleasure to listen to the debate on January 18^th between Paul Shaker and Peter Cowley. The event was billed as "How should we measure schools?" but it quickly narrowed to Dr. Shaker's criticism of the Fraser Institute's controversial "Report Card" on schools and Mr. Cowley's defence of same.

Dr. Shaker, Dean of Education at SFU, confronted both the methodology of the report and its underlying assumptions. He enumerated many statistical flaws in the analysis. Calling it "pseudo-science," he noted that, although the report uses the lexicon and outward forms of scientific study, it lacks controls, peer review and other accepted research norms. He hinted darkly at the consequences of basing analysis on a narrow set of data, which he referred to as the "Enron Effect."

Dr. Shaker also called into question the assumption that education will be improved by an increase in standardized testing which he believes is part of the Fraser Institute's approach. He noted studies from the U.S. and various other countries which seemed to indicate the reverse. He suggested that better results can be achieved by instituting a process of school inspections by seconded professionals. Time, however, did not allow for this theme to be pursued.

Mr. Cowley, for his part, put up a spirited and good-natured defence. He rebutted the technical criticisms, but he put more stress on a challenge to his critics. Briefly stated this took the form of, "If you don't like my report, provide another one, so that we may have a diversity of tools with which to compare schools." This became a recurring theme of Mr. Cowley, that more information is better. He claimed that if we had more sets of numerical data, it would help education in two ways: One, it would help parents choose an appropriate school for their children. Two, it would help educators know how they are doing, both compared to previous years and compared to their neighbours.

I can not say with certainty who won, not having exactly an open mind on the question! However, Dr. Shaker was supported by several bursts of spontaneous applause from a fairly mixed crowd. It was also interesting that Mr. Cowley was on the defensive the entire time, and often resorted to the argument of, "Well, if my data is flawed, the answer is to have more data." In a sense, the battle was never quite joined, because we never got to the core issue of "Are there some things of value to which we just can not assign a number?"

Anyway, it was a very stimulating experience and a great dinner!

Pat Truelove

PS. Later, I had the chance for a brief chat with Mr. Cowley, who was very pleasant. However, I was struck by the fact that he actually knew surprisingly little about the FSA tests on which he bases his rankings of elementary schools. He didn't know what the tests consisted of, who created them or how they were scored, and he seemed not at all curious! His background, I learned, is not in Education, but he is content to take this block of raw data and translate it into his own system of "rankings," which is then picked up by the print media and given the veneer of authenticity.

It brought home to me that, as informed professionals, we need to involve ourselves in any public discussions that affect our work.