The school system in which Traditionalist Teacher toils prides itself on being “data driven”. It is a malady shared with many other school systems – public, parochial, and private.
For those readers who are fortunate enough to be unfamiliar with the phrase, being data driven purports to mean that the school adopts the utilitarian premise that it will promote whatever research shows will best facilitate the education of its students. Methods and practices will be studied and those that work best will become the standard for all instruction in that district.
On the surface, this appears to be eminently reasonable, even scientific. A sentence that begins, “The data says that…,” sounds reassuring to parents, and is an effective club with which teachers can be bludgeoned.
Those who have read the previous postings to this site will know that Traditionalist Teacher strongly objects to the idea that there is any single set of practices that will be best in every classroom – but that is not the subject of this posting.
Our purpose is to examine the accuracy of the claim to be data driven, and then to critique the ways in which the data are being collected.
Are those schools that claim to be data driven being accurate? Traditionalist Teacher believes that the answer to that question depends on the nature of the problem at hand, but that the answer is usually negative.
Largely this happens because political considerations will usually overrule educational objectives. Certain decisions are simply off the table because they would be so unpopular that they would never be taken, even if a mountain of data existed to support them.
Let’s take for example the data that supports the idea that many students would learn better if school started later in the day. There is an abundance of evidence to support this idea, both theoretical and anecdotal. Any teacher who has tried to teach a complex lesson early Monday morning would find the idea easy to believe. However, changing the schedule would be massively unpopular with some key constituencies. In Traditionalist Teacher’s school, the day begins at 7:25 a.m. and ends at 2:00 p.m. Let’s say that the school determined that the students would best learn if the school day started at 10:00 a.m., for which there is ample support in the available data. That would mean that the school day would end at 4:35.
Who would dislike such a decision? Start with teachers themselves, many of whom like the fact that their workday ends in the mid-afternoon. Then come the supporters of the athletic program. On this schedule, practice could well last until 7:00 p.m., and coaches would have to work until 8:00 on a daily basis. How about those who employ high school students? The employees would be unable to begin their duties until 5:30 at the earliest. Parents who use their high-school age children to babysit their younger siblings until the parents arrive home from work would be inconvenienced. Then there would be issues with parents who have to leave for work three hours before their high school age children have to get up for school – and an absentee rate that would likely increase. School busses would be running – and slowing traffic – at the same time that many are returning home from work. And on it goes.
Politically, such a decision would be untenable, no matter what the data says.
When political factors are not present, the decision that is made often supports the needs or prejudices of upper administration. Usually, they will be able to cite data to back up that decision – but the method of doing educational research often means that you can find some research that will back up almost any decision.
That brings us to the primary focus of this posting.
The method of doing scientific research is well known – you probably learned it as the “Scientific Method”. One starts with a question, develops a hypothesis to answer the question, gathers data, applies the available data to the hypothesis, and then determines whether or not the hypothesis has been proven. If it is, the hypothesis can be accepted as plausible. To be accepted as factual, the experiment needs to be replicable by another scientist who independently achieves the same result.
When one is dealing with matters that touch on the human mind, the necessary criteria for proof expand in two ways:
- Double-blind testing – In this process neither the person conducting the study nor the subjects of the study know what the study is trying to prove. This is crucial. Basic to many, perhaps most, of us is the desire to please. The graduate assistant wants to please the professor by proving the professor’s pet hypothesis. The subjects develop a rapport with the person conducting the study and come to want that person to be successful. The double blind procedure prevents error by making sure that the subjects are unable to figure out what is being studied.
- The use of control groups – In this way the change being studied in used on one group and not on another. Both groups are studied. The change can only be said to be effective if the group that has been subjected to the change reacts in a certain way, and the other group does not. If neither group reacts in the desired way, the change is obviously ineffective. If both groups demonstrate the desired behavior, that behavior had to be caused by some other condition.
Even with these safeguards, psychological studies are often not replicable. Opening the recesses of the human mind is a very difficult process, and highly likely to fail.
There is one other problem with this process as viewed through the eyes of the educational reformer. Double-blind testing with a control group is both time consuming and expensive – very, very expensive. Educational reform needs to happen NOW, and in a way that is as cost effective as possible. So there comes a new process. In the world of education, everything must be called by an attractive title. This one is called “Action Research”.
Supposedly, by applying the method of Action Research, every educator can analyze and create a solution to the problems facing American education. If the researcher records the research and results in the appropriate way, the research can be published. Once it is published, other studies can cite it and it becomes a part of the educational literature.
There is one major problem with this method.
Action Research isn’t scientific research, despite the efforts of its proponents to present it as such. Its conclusions are neither valid, verifiable, nor reproducible.
Action Research simply reinforces the prejudices of the person doing the “research”. If I believe it is going to work, I will put the energy behind it to make sure that it works. If it still doesn’t work, I can manipulate the situation until it works. If it still doesn’t work, I can just fake the data.
Let us say that Traditionalist Teacher wanted to prove once and for all that traditional methods are superior to progressive ones. Miraculously, administrative permission is obtained. A series of changes would be implemented in the TT classroom, including:
- The class period is reduced to be 45 minutes long.
- Lecture becomes the primary mode of instruction. Notes are collected and graded to insure that students are paying attention.
- Tests are based upon the material contained in the lectures. Grades are posted in the classroom with students listed by name.
- All students must comply with a strict dress code.
- Violations of class and school rules will be punished by repetitive writing of a statement based on the specific infraction – for example, writing “I will not talk during a lesson until I have been called upon.” 500 times. Failure to comply will result in corporal punishment and/or work duties, followed by the demand that the original punishment be completed.
- Plagiarism will result in severe sanctions, including – but not limited to – grade reduction, a requirement that all class work be conducted under TT’s direct supervision after school, public humiliation, and – as a last resort – expulsion.
- Specific key ideas will be memorized.
Such a series of changes would give Traditionalist Teacher a substantial ego boost. TT would work very hard to insure the success of this methodology. Feeling good about the job, with a renewed work ethic, and a new source of mental energy, TT would be able to insure that the students learned.
At the end of the year, Traditionalist Teacher would be able to write a book in which the level of success would be stated and amplified – and contrary evidence deleted. This would enable TT to enter the world of the educational consultant – where the real money is. I would be able to head out to the educational conventions and trumpet my success. Perhaps I could even get a chance to be on C-Span 2’s Book TV.
Some administrators might listen. Some other traditionalist teachers would come out of the woodwork and demand to be allowed to implement traditional methods in their classrooms. Those teachers would likely enjoy success. However, the fact would remain that a teacher’s personal commitment to these methods would be an important component of any success.
At the same time, many teachers would remain loyal to the progressive methods in which they have been trained, especially if they believe that they have enjoyed success with them. To these professionals, the methods I describe would be anathema. Attempts to force those teachers to adopt this plan would be met with resistance. That resistance would prevent any success. Math, science, art, and music teachers would complain that the methods that work well in a history class would not be sufficiently flexible for their subjects. Those teachers would implement traditional methods grudgingly, if at all. If they read my book (and that is a VERY big if), they would not find it convincing, because it would be all about how I did it in MY classroom.
My “Action Research” would be no more convincing to them than that done by progressives would be to me.
Because it is not really research. It proves nothing.