Welcome to the big bad world of patient satisfaction scores.
Often (well... pretty much all the time), I come across an article, bulletin posting, or meeting where patient satisfaction scores are announced... but no background information to back it up. HOW many people completed the survey? HOW were the people selected? WHO administered the survey?
HOW many people completed the survey?
The number of patients who actually completed the survey is very important. Mainly, because of the variance that may occur month-to-month as well as ease in manipulating the results.
Let's assume only TWO people completed the survey. The first month, both give me a thumbs up giving me a score of 100%.
However, the following month, another TWO people completed the survey, but only one gave me a thumbs up giving me a score of 50%.
NEWSFLASH!!! "Patient satisfaction scores plummeted 50% over a 1 month period of time!"
In order for meaningful interpretation of patient satisfaction scores, enough survey results MUST be obtained, especially when comparing scores across time.
HOW were the people selected?
Let's pick on the emergency room...
If I wanted to "pad" my results, I would tend to provide surveys to patients who had simple problems and who were seen quickly.
Child with an earache? Let's seem them immediately in Fast-Track ER. [Look in the ear! There's an ear infection.] Here's some antibiotics. Can you also complete this survey?
As opposed to...
55 year old narcotic-addicted patient with fibromyalgia demanding a prescription for 500 percocet pills for his chest pain. I would probably not want this patient to fill out the survey. Neither would the patient as he would probably toss it.
Granted these are extremes, but if performance bonuses are provided to departments with the best patient satisfaction scores and the ER staff were the ones providing the surveys, one can easily manipulate the results in their favor.
WHO administered the survey?
As mentioned above, survey results can be manipulated depending on who provides or gives the survey, ESPECIALLY if money is involved based on the results of the survey.
In order for survey results to be meaningful, a third uninvolved party should be the one to administer the test.
Survey Fatigue
Now even if the survey was done dispassionately and adequate number of surveys were completed, there's still the problem of survey fatigue...
What's that?
I don't know about you all, but I get about 3-4 letters per week and at least one phone call per day asking me to complete a survey on my car, toothbrush, tater tots... etc. You name it... there's a survey for it now.
Now add how many questions are asked... Hopefully one so it's quick.
But also HOW the questions are asked (yes/no versus a scale of 1-10).
As a person completing a survey, if I had my way, there would only be one question that is a yes/no question.
The more questions there are and if they are scaled rather than yes/no, the less likely I'm going to do it.
Now add if you are sick and in pain... How likely are you going to actually complete a 50 question survey on a scale of 1-10???
Probably not...
In fact, it's probably only going to be healthy individuals who tend to give better scores giving a skewed result.
Also, given the survey fatigue I and I bet most people have, it will be a rushed completion without much thought given. The default being five stars all the way down (I may not even bother to read the questions).
A decent way to look for survey fatigue is to have 2 randomly buried questions where rather than 5 stars being the best, 1 star is the best. Surveys where everything is 4 or 5 stars INCLUDING the reversed scale questions should be thrown out.
Summary
Whether good or bad, patient satisfaction scores can be VERY misleading if not completely wrong...
I should also add that whenever money is involved, NO test is going to be 100% accurate due to cheating that WILL happen one way or another.
I predict that this problem is going to get MUCH worse because health insurances will soon base how much reimbursement hospitals and physicians get based on patient satisfaction scores. We are talking about millions and millions of dollars.
Just read about the "no-child left behind" from the Bush era where student test scores influenced teacher pay leading to teachers allowing cheating on tests.
