The Curious Case of COMLEX

Events in this world are often a mystery. A single person can't know everything, but it sure is fun to ask questions and do our best to understand things. I want to consider a question that surrounds the recent suspension of COMLEX Level 2-PE. Here is a quick summary of the situation; the NBME suspended their in-person physical examination for many reasons. It was expensive, there was growing data concerning its lack of efficacy, and the pandemic made it impossible to administer safely. The NBOME, in contrast, begrudgingly suspended their version of the same test only after tremendous pressure from doctors and medical students. You can find a much more in-depth summary of the situation on the PRN podcast “Episode 23 - COMLEX Level 2 PE - To PE or to not PE?”.

The curious case of COMLEX has one key question that needs to be answered: why did it take so much pressure and effort to force the NBOME to make a decision that seems pretty obvious? Consider the differential diagnosis for why an organization does something that doesn’t make any sense. The most common differential is probably ignorance or laziness. In this case, though, I do not think that is the reason. I believe the most likely differential is a perverted incentive structure. In other words, what benefits the organization is not what benefits those who make up the organization and is not what makes the most sense. There are three categories of incentives working to keep the NBOME from making decisions that make the most sense. The first is money, the second is power, and the third is legitimacy.

The first incentive working against the NBOME is the one I see discussed the most on Reddit and Twitter: money. The NBOME makes a lot of money. The number of DO schools is increasing every single year, and administering these tests brings a little over 34 million in revenue every year. Every test the NBOME doesn’t administer is lost money. Every step they take towards a decision like a merger is equivalent to lost salaries. Just look around at this website which details some of the money going into the NBOME. You will be amazed at the amount of money flowing through this organization. I will move on to the next incentive because this one seems to be the most self-explanatory.

The next incentive working against the NBOME is power. This argument is a little abstract, but since most of us are doctors (or future doctors), I believe it should resonate. Think for a minute about when you were put in charge of something and how it felt. It probably felt really good. Humans love to feel like they matter, feel like they have a purpose, and feel like they are in control. The people in charge of the NBOME are human beings and subject to the incentive not to make decisions that relinquish power. You should be asking why this argument applies to the NBOME but not to the NBME. The answer is that canceling the COMLEX Level-2 is a much bigger deal to the existence of the NBOME than it is to the NBME. No sane person would argue there doesn’t need to be a body with some regulatory control over physician education. The question is, “does there need to be two of them?” Probably not for any obvious reason unless you make the case that they have a different purpose. A purpose that is “supplemental and synergistic” instead of redundant and wasteful. Interestingly, this stance is the exact one taken by the NBOME and is also the argument that most osteopathic bodies have made as well (but that is a topic for a different essay). The NBOME justifies their existence by emphasizing their role in testing “osteopathic principles.” These principles include two subjects: first is “holistic medicine,” which is not something unique to DO’s, and the second is a collection of techniques known as OMM. These techniques and testing and teaching them are the only actual justification for the existence of the NBOME. Canceling the COMLEX Level-2 PE means not having an in-person test of these OMM skills, which means a step away from power. Canceling STEP-2 PE doesn’t have the same consequences for the NBME.

The last category of incentives working against the NBOME is legitimacy. The legitimacy that Level-2 PE gives the OMM techniques. This is a very similar but distinct category from power. It is similar because it is based on the understanding that the NBOME needs OMM to justify its existence. It is distinct because it is regarding the legitimacy of the techniques, not the direct legitimacy of the NBOME. When a governing body tests something, it adds to the probability that it is legitimate. Think for a second about two hypothetical scenarios involving something like Chapman's points. One scenario is 100 independent doctors across the country who use Chapman's points, but they are not taught in school or condoned by a governing body. The other scenario is still just 100 doctors across the country using Chapman’s points, but they are tested on the licensing exam for DO’s, and every osteopathic medical student is made to memorize them. This second scenario still has just a handful of doctors using these, but without any other evidence, the fact that the techniques are tested and taught at schools adds legitimacy.

Techniques like Chapman's points, cranial, and many others are barely hanging on. They have almost no credible/believable evidence, and there are very few physicians actually using them in practice. We live in an age where it is no longer acceptable to believe something just because it is taught. The life span of these techniques is rapidly approaching, and most people know it. The NBOME understands that any step away from OMM is a step towards their demise. The NBOME, fighting against canceling the Level-2 PE, should not come as a surprise. These incentives are the exact same reasons that every year, students are forced to memorize Chapman’s points and practice cranial OMM. That is the reason I wrote this essay because I was not surprised by the NBOME’s reluctance to cancel Level-2 PE. I had already been surprised when I read the papers our OMM professors gave us as evidence for the techniques we were learning. I could not understand how something with such poor evidence was still being taught alongside modern medicine. That is why I thought about the incentive structure. It is the only rational causality for why these “techniques” are still being taught and why the NBOME doesn’t want to cancel Level-2 PE.

This essay is not meant to be some slanderous takedown of the NBOME. It is possible that the leaders of the NBOME may not be fully aware of why they are doing things. It could be a case of good and honest people subconsciously driven by incentives they do not fully understand. This is about describing what is happening, so we can do something about it. This is about charting a path for our profession that is above reproach. This is about what we’re supposed to do as doctors. Look at data and make rational decisions.