WDDTY - acunpuncture for tonsilectomy patients

Our good friends at the journal of record, What Doctors Don't Tell You, on their web page, report on a "study" that suggests acupuncture is useful for relieving pain in post tonsillectomy children.  Here is the paper that WDDTY bases its story on.  In the event, the analysis is based on 31 children and reports a significant decrease in pain.

But hold on a minute.

If you ask me, this study is a classic example of how not to do it. 

Firstly, there is no control group.  Not even a thought of one.  So we have no idea of whether the pain reduction is normal or not.  Just Ochi's word.  Sham acupuncture has been developed to act as a control and when it is used, the effects of acupuncture dribble away to placebo levels.  To be fair, Ochi does recognise this:

The study sample was a sample of convenience. The current design employed neither random assignment nor control conditions against which to compare the acupuncture procedure. The experimenter and participants were not blind and both were motivated to see pain reduced.

Secondly, the group was to some extent self selecting.  These were the children whose parents brought them back specifically because the pain was still a problem.  They came back within the author's ten day limit but it is unlikely that many returned on day ten, as shown by the discussion on how long the effects last.  To be fair, the quotation above does cover this criticism to a certain extent.

Thirdly, the group is highly heterogenous. The patients ages ranged from 2 to 17, mean of  9.23.  As anyone might be able to tell you, a nine year old is more open to suggestion than a17 year old.  Some of the pain scores were reported by the parents and not the children, but we don't know how many and for what ages (although clearly the 2 year olds would have had their pain reported).  Not all of the original 56 patients had the same treatment.  This study is starting to get very messy.

Fourthly, there is no record of controlling for the parents.  Note that 9 parents refused acupuncture which suggests at least that those that chose to accept acupuncture were amenable to it.  By how much might that have skewed the results?  I know that if I were trying to persuade my granddaughter to have pins stuck in her, I would have to tell her it would help reduce her pain.  Would that affect the result? You bet.  She will do lots of things to please me.  When she complains of a pain, I give it a rub or a kiss better and she tells me the pain has gone down.  I have no idea of whether it has or it hasn't because pain is subjective.  Using the pain score chart is useful but still subjective.  Needless to say, I'm not convinced.


Fifthly, the acupuncture given varied from patient to patient.  We're getting a lot of factors here and it could be that the length of treatment, the number of needles and so on influences the patient's response, and I am not necessarily talking the reduction of pain.  The parents will already know that Ochi gives acupuncture because it is there on his website:

As one of the only pediatric ENT specialists in the country with a double Board Certification in Otolaryngology (ENT) as well as Medical Acupuncture, Dr. Ochi offers a unique advantage of being able to provide a holistic & integrative approach to caring for & treating your child’s ear, nose or throat condition.

So even before the child gets to the examination room, the parents have an expectation that the acupuncture will achieve something.  But he usually promises conventional pain medication too, as this review suggests, but...

My child had tonsils removed from him. He promised there would be pain meds given as well as acupuncture. Then after the surgery he sprang it on us that she wouldn't be given pain meds. He is evil. 

My child went into shock and had to be taken back to the hospital, developed a deadly  infection near brain stem, all because he was stupid enough not to give her antibiotics during surgery! My child was nearly an adult with no sleep apnea. 

I think his license should be taken away! Elementary Dr. 101 stuff he missed..... like pain meds and antibiotics. 

..... You will also notice, no one else in the office with you! As well as pictures of little boys bathing in his office. Creep!     (from http://www.yelp.co.uk/biz/james-w-ochi-md-encinitas)

Sixthly, the length of time that the treatment lasted was also measured in a somewhat ad hoc way. 29.41% of the patients reported less than 3 hours benefit. That's a big number, roughly the same as those that reported more than 60 hours benefit. This latter group skews the duration results significantly because the mean duration of the effects is reported as 61.24 hours. The standard deviation is greater than the mean which suggests we can have no confidence in those results.

Having given the paper a critique, I should add the Ochi knows the paper isn't that good.  But he pushes the acupuncture line.  Here is his conclusion:

The data tentatively suggest that acupuncture decreases perceived pain in children and adolescents after tonsillectomy. Though randomized, controlled research needs to be done to confirm the trends observed in the current study, the combination  of these preliminary results with the low cost and safety of acupuncture make it a promising way to relieve tonsillectomy pain in children and adolescents.

 But he would, wouldn't he?  He is trained in acupuncture, believes in acupuncture and may believe that this actually is evidence for acupuncture.  I have my misgivings.  This, for example, gives me food for thought, and not in the way that acupuncturists would like. And this.

Now back to What Doctors Don't Tell You.  Here is the news piece from their website:

Could acupuncture be routinely prescribed by US doctors for pain relief? It could happen, according to a new study that found it works in children who have had their tonsils taken out (1). Doctors’ choices were narrowed earlier this year when America’s drug regulator, the Food and Drug Administration (FDA), banned the use of codeine in children who had had a tonsillectomy after the drug was found to cause complications. But Dr James Ochi, a pediatric ear, nose and throat surgeon in San Diego, has produced a study on the successful use of acupuncture on a group of 31 children, aged from two to 17 years, after tonsillectomy. Their pain levels fell dramatically after just 15 minutes of acupuncture, and the pain relief lasted for nearly three days afterwards (2). It was no great news to Dr Ochi who says he has being using acupuncture for years, and even before the codeine ban. An earlier study at Harvard came to similar conclusions: acupuncture reduces pain in children following ear surgery, the researchers found. (Source: International Journal of Pediatric Otorhinolaryngology, 2013; 77: 2058)

(1)  Here is what Ochi actually says about the patients:

56 juvenile patients underwent tonsillectomy during the 3-month study period. In addition: all 56 patients also had adenoidectomy; 13 had bilateral myringotomy and tube insertion; two had nasal cautery; and one had cerumen removal.

 We don't know which ones were used in the study but, and this is borne out by reviews of Ochi's practice on the internet, he does seem keen on surgery.  Very keen apparently.  But nevermind.  Sixteen of the patients had other bits of surgery too.  That's 28.6%.  Quite a big chunk.  How did these other procedures affect the outcomes?

(2)  Here is what Ochi says:

Finally, the 17 estimates of acupuncture benefit duration were examined. All patient estimates were converted to hours and ‘‘capped’’ by the tonsillectomy recovery window. Put another way, if a patient on day nine claimed that acupuncture lasted for 62 h

(three days), this estimate was restricted to 48 h (the full day nine  and the full day 10). Estimates were capped in this manner to minimize the extent to which patients conflated acupuncture pain relief with the pain relief accompanying the natural healing

process, which should produce a minimum pain around or shortly after day 10 for most patients. The mean estimated duration of acupuncture benefit was 61.24 h (SD = 64.58 h, range = 1–168 h). Five of 17 patients (29.41%) reported less than three hours of

benefit, five (29.41%) reported more than 60 h, and the remaining seven patients (41.18%) reported intermediate durations.

 As I said above, I don't think we can give too much confidence to this bit.  More than 70% of the patients reported well under 3 days worth of pain relief.  So claiming that the pain relief lasted almost three days is stretching it a bit. 

I don't like Ochi's study because I think it wouldn't merit much at high school but the most important bit is that he does accept there are big limitations.  WDDTY, on the other hand, don't worry about the niceties of doubt and uncertainty in science.  They try to sew uncertainty and doubt about science.

One of the reasons Ochi gives for using acupuncture is cost.  Apparently it costs 11 cents for his needles.  According to the Walgreens website, paracetamol (acetaminophen) costs 2.7 cents per tablet, does not require an expensive consult or trained person to administer and, well, can be taken when required, within the obvious limitations imposed by sensible use of the painkiller.  The needles might be 11 cents.  The physician costs extra.