A Consumer Reports
Manufacturers spend billions to tell us their products are great; Consumer Reports invests time and labor to tell us the truth.
How do they do it? One product at a time.
By Nancy Claus Giles
In my family, Consumer Reports is akin to a bible. No major purchase is made without consulting it first, and I’ve never been disappointed in its guidance. For Consumer Reports, and its parent organization, Consumers Union, is a uniquely pure entity, beholden to no one but the consumer—you and me.
Consumer Reports is the one place consumers can look for the plain, unvarnished truth, and every month more than four million readers do just that, making Consumer Reports one of the top 10 largest paid circulation magazines in the country. Readers want to know: What is the safest car? What’s the best dishwasher? What’s the sturdiest baby stroller? Consumer Reports answers these and other important questions by subjecting products to grueling and often tortuous regimens of testing and retesting, then publishing the results. Consumer Reports, unlike most other magazines, accepts no advertising. As a nonprofit organization, Consumers Union is funded solely by magazine subscriptions, donations and grants. The organization buys all the products they test; freebies are forbidden.
Perhaps a million times, I’ve driven by the company’s headquarters, a nondescript, 330,000-square-foot office building that is tucked in the middle of a modest residential neighborhood just off the Saw Mill Parkway in Yonkers. But once inside, I learned quickly that Consumers Union is anything but ordinary. Indeed, I’ve never seen so many happy employees.
Maybe I was being fooled. Maybe the company had issued a memo warning of a journalist on premises. Or perhaps it was the after-buzz from a recent employee auction of wines used in a taste test. Or, just maybe, these lucky folks simply love their jobs. If that’s the case, it’s with good reason.
Most of the company’s 400 Westchester employees have offices rather than cubicles. They wear whatever they want to work. There is no dress code; they aren’t required to don a suit and tie or, for that matter, long pants, so you see people walking through the halls in suits, jeans and shorts. Hairstyles are as varied too: from crew cuts to graying ponytails. There’s an in-house gym with Pilates and aerobics classes offered, and an employee cafeteria serving homemade dishes from burgers with onion rings to the pannini of the day to chicken Marsala. One man on line in the cafeteria confided: “My wife has asked me to stop talking about the great lunches I have every day.”
Oh, but the very best part is the labs. There are 50(!) testing labs, many of which reminded me of high school science labs and home economic studios—but without the stern teacher. Some experiments are jerry rigged together, Rube Goldberg-style, pipes propped on trash cans or chairs, wires dangling from the ceilings. On the day I visited, the sounds of the Beach Boys’ “Surfin’ Safari” could be heard. You could safely assume that those folks were having fun, fun, fun. Doing, ahem, what exactly?
Testing, One Two Three
Well, the nuts and bolts of Consumers Union is testing theâ€¦nuts and bolts of products. To do that, “mystery” shoppers buy products from around the country and then send their purchases to headquarters, where the goods are analyzed and then systematically put through various contortions by the 140 researchers and technicians who are ready, eager and willing to find their weak spots.
There are environmental chambers where temperature and humidity can be controlled to test air conditioners and heating systems, and rows upon rows of washers, dryers, ovens and refrigerators, which, when I visited, were lined up like soldiers. There is a glass “stomach” that simulates gastric juices to evaluate antacids; a “digester” that chews up beef rib bones to assess garbage disposals; inflatable manikins that simulate squatting to test the resilience of denim jeans; and a “sneezer” to test tissues and paper towels. (The testing of cars is performed in the company’s 327-acre facility in East Haddam, CT.)
It’s not just the testing machines and apparatus that are mind-boggling. It’s the painstaking testing procedures and protocol that amaze, too. To assess dishwashers, researchers coated dishes with oatmeal and spinach, then left them overnight to congeal and harden.‑Then, without pre-rinsing, they put the dishes through a normal cycle. Clearly, if a dishwasher can handle that mess, it should be able to handle anything you might find in the average teenager’s room.
To test detergent cleaning power, engineers blended canned soup, corned-beef hash, chocolate pudding, cola, cheese spread, rice and powdered eggs, then smeared the goo on glass plates and added a dollop of raw egg yolk for good measure. Afterwards, they microwaved the plates, and let them sit for a few more hours. Finally, the dishes were put in identical dishwashers and washed in hard water.
Consumer Reports’ rigorous testing doesn’t stop in the kitchen. I spotted testers lounging on a sofa, watching a big-screen TV in the simulated living room lab used to mimic home electronic acoustics. All that was missing was popcorn and beer (perhaps those items were being tested elsewhere). You call this work? It is—once the professional “ears” made their assessment, the true sound output was then measured scientifically in an anechoic (that’s Greek for “without echo”) chamber. (The chamber is the eeriest room in the place; if you were accidentally locked inside, no one could hear you scream.)
How far will these men and women go to check out products? Pretty darn far. To test claims that the Orka Silicone Oven Mitt was heat-resistant to 500 degrees, one valiant staffer submerged his mitted hand into boiling water and sizzling oil. Yikes!‑How’d he do?‑He lasted ten seconds in the water, five in the oil.‑ Bottom line: the product works, but not quite as well as implied. And to determine the gentleness of a wash cycle,
an extremely dedicated employee methodically counted the number of frayed threads in the holes of hundreds of yards of fabric. To test low-flow toilets, “surrogate” waste material such as sponges and paper towels was shunted through a clear plastic tube to gauge the “oomph” of the flush.
What was the most unusual test? Frank Iacopelli, director of product research operations, recalls the time the company was testing whirlpool tubs. “We turned the lab into a spa room with nine tubs. Employees could sign up for half-hour bath sessions during the day—it was a pretty popular perk.”
The most unusual product tested? Michael Vallario, manager of product acquisition, didn’t hesitate to answer: “Condoms. We had to sample about 9,000 to see how ones in pharmacies stacked up against those sold in vending machines. Finding condoms in pharmacies was easy, but the shopper had to go to nightclubs and gas stations to find vending machines. It was kind of embarrassing for him.” Vallario was quick to note that these products were not for “home-use testing” but had been evaluated in the lab under the most controlled conditions.
Speaking of controlled conditions, they aren’t required just for condom testing. To test food products, there is a staff of permanent part-time “sensory testers” whose palates have been “trained” to determine, say, how salty a potato chip is on a scale of one to 10. Wonder what happens to all the leftovers? “The testing labs are especially popular when we are testing grills,” Iacopelli acknowledges.
Products used in testing that have not succumbed to the process are auctioned off to employees in a silent online auction several times a year, with bidding starting at one-third the retail price. Cars are offered for sale as well, but with a smaller discount. All proceeds are returned to the testing budget.
a force for change
There appears to be a pretty high correlation between what Consumer Reports says and what consumers buy—or don’t buy. This is particularly true in the auto industry. In 1988, for example, Consumer Reports found that the Suzuki Samurai rolled over easily and thus rated it “not acceptable.” Sales of the Samurai subsequently plummeted nearly 70 percent. Conversely, when the magazine ranked the Volkswagen Passat as the best family car in 1999, sales soared a whopping 32,000 vehicles over the previous year.
But it’s not just consumers who listen. “A few years ago, we were testing door locks,” recalls Jim Guest, former Vermont Secretary of State and three-time candidate for Congress and current president of Consumers Union. “Most of the locks had one-and-a-half inch screws, which failed when our machine kicked the doors in. But when we tried three-inch screws on the same locks, they held. We noted that in our article.” A few years later, when the company retested door locks, it discovered that the manufacturers had voluntarily changed to three-inch screws. It’s this kind of credibility that makes Guest feel he has “the best job in the world.” He is quick to note, too, that as a result of meticulous testing and research, Consumer Reports has never lost a lawsuit or issued a retraction on a rating.
Today, the company is working to enhance its Web-based services. “Print and Web speak in two different languages,” says Joel Gurin, executive vice president of Consumers Union. “We need to be bilingual and fluent in both.” So, he says, the company is “making the move from an organization where information goes through the magazine first and then to the Web to an organization where the same information is used in different ways for complementary services.” The site (www.consumerreports.org) is a huge success—already it’s become the largest information-based subscription site on the Net, with 1.25 million subscribers, almost double the number of subscribers to The Wall Street Journal’s Web site.
“Our organization is at the forefront of getting consumers the information they need, when needed, in the form they need it,” says Margot Slade, editor of Consumer Reports. She notes that
the company today offers Consumer Reports Online, Consumer Reports TV,
e-mail alerts, and Consumer Reports Online—extra.
What does the future hold for Consumer Resports? Guest says expect to see Consumer Reports information downloaded into PDAs for on-the-spot analyses during shopping trips. “We keep the information on a platform and whatever channel comes along, we can get the information out,” Guest says. Also in the works is a new auto-buying kit that lets consumers retrieve pertinent information about a variety of cars, and then make comparisons and choices based on the data. It also supplies dealer costs and a list of area dealers. Now, if only it could arrange the bank loan and make the payments!