Strong Marketing of a Weak Success Measure:
Charity Navigator Vital Mission Hides Flawed Rankings

Everyone wants to figure out how to evaluate nonprofits. Grantmakers, donors, volunteers, journalist, and nonprofit leaders.

Individuals who contribute and the nonprofits that use those funds to provide vital services would both benefit from ranking of effectiveness and efficiency. Such evaluations would encourage nonprofits to constantly improve their performance and allow funders to make smarter investments.

Those were the driving motivations behind the creation of Charity Navigator and other nonprofit rating / ranking services. Yet, if you believe Charity Navigator or others have found the holy grail of evaluating nonprofit organizations you’re sadly mistaken — and I encourage you to keep reading.

For-profits have one thing nonprofits do not — a clear set of financial measures of success. Across for-profits is it relatively easy to measure and compare profits. Yet, it is much tougher to measure effective and efficient service – the mission and goal of all nonprofits.

Charity Navigator’s rankings are the result of gross oversimplifications. For example, Charity Navigator’s:

• Evaluation process begins and ends with creating ratios based on almost any two numbers found in nonprofit organizations’ IRS Form 990.
• Presumption is that all nonprofits complete their IRS Form 990’s in the same manner, using precisely the same definitions of what income and expenses are reported in response to a given question on Form 990.
• Ratings do not include an “affirmative confirmation” from nonprofits’ top management to guarantee the accounting basis of specific figures or that the resulting ranking is both correct and fair.

Imagine a nonprofit institution raising capital funds for a new facility. Simply looking at the IRS Form 990 could lead one to believe the organization had a dramatic increase in revenues. That’s good news to the nonprofit — unless, for example, Charity Navigator decides to use that year as a “base year” on which to evaluate future year’s revenues. Future ratings and rankings could show the nonprofit in decline as a result of the decreasing revenue.

Or, consider how you would rate a nonprofit whose mission is to care for people during and immediately following natural disasters? Funding for the organization is like winding a clock-spring. All of the investments in infrastructure are “waste” if there are no disasters.

On the other hand, if there is a disaster and the expensive infrastructure doesn’t exist, the organization will fail to react instantly when conditions demand nothing less. Only when the spring is wound can the organization deploy resources and services when they are most needed.

These situations have me reflecting on the age-old loaded-question, “Are you still beating your wife?” Charity Navigator’s ratings, rankings, and top ten lists are all presumed to be true as published until and unless they are challenged a nonprofit that was damaged by an overly simplified ranking system that is not based on an apples-to-apples comparison.

In my view, Charity Navigator, its ratings, and its top ten lists are nothing more than great merchandising of a weak underlying product.

For example, wouldn’t any donor or nonprofit be interested in the following “Top Ten” Charity Navigator lists?

• 10 of the Best Charities Everyone’s Heard Of
• 10 Highly Rated Charities Relying on Private Contributions
• 10 Charities Routinely in the Red
• 10 Charities Stockpiling Your Money
• 10 Charities Expanding in a Hurry
• 10 Charities in Deep Financial Trouble

These lists — while attractive — are the “National Enquirer” approach to a topic that demands more substantive evaluation of nonprofits’ effectiveness and efficiency. While not as quick or easy as Charity Navigator’s overly simplistic rankings, it’s fascinating to see Charity Navigator’s own recommendation on how to evaluate nonprofit institutions that it does NOT rate (listed below) far more closely represent the time, questions, and interaction with nonprofits that are required to evaluate their effectiveness and efficiency.

Charity Navigator’s Suggestions On Evaluating Nonprofit Success

1. Can your charity clearly communicate who they are and what they do?
2. Can your charity define their short-term and long-term goals?
3. Can your charity tell you the progress it has made (or is making) toward its goal?
4. Do your charity’s programs make sense to you?
5. Can you trust your charity?
6. Are you willing to make a long-term commitment to your organization?

I give up! If these are the questions that Charity Navigator recommends you and I ask of nonprofits, why don’t they use these same questions themselves?

I can think of four answers:

• It would require an enormous investment of time and money to gather the answers.
• Even if answers to the above questions were collected, they don’t lend themselves to numeric ratings;
• Without numeric ratings, it is next to impossible to produce apples-to-apples rankings, and, finally;
• Without low-cost, easy to produce nonprofit rankings, there is no Charity Navigator.

Regardless of their size, the rating of a nonprofit’s service is complicated and highly subjective. The list of provided above by Charity Navigator is a good starting point for discussions with a nonprofit’s leadership, top management and key professionals.

But here in all this complexity and subjectivity is the beauty of making an individual decision to support a specific nonprofit organization.

Over time, you learn about those organizations dealing with the causes you care about most – and are passionate about their mission.

After all, if it were that easy to determine the most successful nonprofits, everyone could invest in nonprofit mutual funds and fund managers would make the investments in only those organizations’ services rated at the top of the list in terms of effectiveness and efficiency.

Large donors and small donors. Very well funded and not so well funded nonprofits. In all of these cases, half of the fun of investing in nonprofits – in giving away your hard earned cash – is learning about the similarities and differences between the half-dozen organizations meeting needs you believe are essential.

In the end, a significant contributor only has two good options.

• They can become involved, get engaged, and learn about the organizations they fund.
• Or, they can simply hire me (just teasing) – or any other consultant with substantial hands-on, nonprofit experience, to collect information on nonprofits of interest and provide them with a thoughtful narrative report.

My real concern is that Charity Navigator’s rankings appear to be so powerful and easy to use that:

• Individuals will fail to take the time and gather the information to determine which ratings may be solid and which are gross oversimplifications or just plain wrong.
• Potential contributors will simply discard a deserving nonprofit from their list of giving priorities, and / or;
• Donors will fail to use the rankings provided by Charity Navigator as one of many topics to discuss with the top management of the nonprofit that interests them.

Charity Navigator’s (and other data aggregators / information providers) current ratings, rankings, and practice of publishing the “truth” until proven otherwise fails potential donors, some nonprofits, and its own mission.

I’m not suggesting that every poor rating of a nonprofit by Charity Navigator is incorrect or undeserved. I am urging the nonprofit industry to create better measures and / or methods of evaluating nonprofits’ mission-driven services in terms of effectiveness and efficiency.

Until that time, I would urge all nonprofits to be open and accountable and all current and potential contributors to become more involved with and knowledgeable about the nonprofits engaged in the causes that interest them the most.

<http://www.charitynavigator.org>

If you’re a homeowner in most cities, you have choices as to what television services you receive, how, and who provides them. You’re lucky.

If you live in most apartment buildings, you have only one choice. You can subscribe to the cable company that negotiated an exclusive contract with your building owners or managers or, without the option of an outdoor antenna; you can watch whatever rabbit ears will allow. But all of this may be about to change.

Sunday’s, October 28, 2007, New York Times, reports that the Federal Communications Commission (FCC), the government agency that regulates broadcast and subscription television services is about to strike down the long-existing exclusive contracts between cable operators and apartment buildings.

Tossing those exclusive contracts may do more than give apartment residents more options. It should also make cable companies more competitive and, in the end, that could mean substantially lower monthly cable fees as those companies attempt to retain their apartment subscribers.

Kevin J. Martin, Chairman of the FCC told the New York Times, “Exclusive contracts have been one of the most significant barriers to competition. ‘Cable prices have risen about 93 percent in the last 10 years.’ This is a way to introduce additional competition, which will result in lower prices and greater innovation.”

The FCC’s plans could be of significant benefit to low-income and minority families. FCC officials and consumer groups reported that while 25% of American’s live in apartment building with 50 or more people, however, that figure rises to 40% for Hispanics and African-American households. In situations were competition was encouraged, cable subscribers’ monthly fees declined up to 30%.

Who will compete with cable companies for their current apartment subscribers? This list grows every day. Satellite television companies, small point-to-multi-point wireless television services, and ultimately companies like Verizon Communications and AT&T. Looking into the future, there will be more community based wireless services — much like cellphones today. Even your power company may ultimately provide television programming services; after all few companies have their door-to-door wired infrastructure.

With cable and satellite television programming providers continuing to increase their monthly subscription rates, it’s great to see the FCC encouraging more competition. It’s an especially important step in keeping America from further divisions based on those who can afford information and entertainment and those who cannot.

With the transition from conventional to digital television set to occur in just 478 days, it is a brave new world out there!

<http://www.nytimes.com/2007/10/29/business/media/29cable.html?hp>

Thanks to those of you who took action based on my previous post about DirecTV and The Dish Network’s failure to include local public television stations new, High Definition (HD) channels in their “local station packages” as they do for commercial TV networks, ABC, CBS, NBC, and FOX.

When will the two satellite subscription services begin carrying PBS member stations HD signals? We don’t know. But, to illustrate the challenge, this post describes what DirecTV and the Dish Network had to say when I contacted them as a loyal public television viewer.

I talked or exchanged e-mails with both companies’ customer service teams. On July 6th, DirecTV replied to my inquiry with, “Thank you for writing to inquire about receiving PBS HD in your area. I’m sorry for any frustration this causes you, and I’m happy to respond to your e-mail.”

They continued, “While I do not have any information specifically about PBS in your area, we do value programming like this and realize PBS in HD is important to our customers. Unfortunately, due to limits in bandwidth, we are not able to offer all of the local channels we provide to the Salt Lake area in HD. I have entered a request for PBS HD to be offered in your area. Our programming department take channel requests like yours into account when deciding on new programming to offer.”

And they concluded, “As mentioned in our previous e-mail, we are launching new satellites to increase the number of HD channels that we are able to offer. It is possible that PBS HD may become available at that time; however, I wouldn’t want to speculate on any dates for this or our service provided by other companies.”

The Dish Network reply was more concise, “Thank you for your e-mail. At this time, we do not have specific information when the PBS stations will be launched in HD. We are currently working on offering more HD channels in the future. Please log-on to our website for more information about new programming and services being added to Dish Network.”

Increasingly, public television stations are the only stations in that are owned, governed, and operated within the communities they serve and not by far away profit making corporations.

Yes, many satellite subscribers can receive a digital / HD public television signal over-the-air if they install an indoor or outdoor antenna (if zoning permits it) and plug it into their satellite receiver box. Frankly, I would suggest everyone consider this option if they can receive over-the-air signals.

However, in a world of vastly more choices in television programming services — what were once called “channels” — there is a difference between what people can do to receive a signal and what they will do.

DirecTV and the Dish Network are now providing the commercial network stations’ HD signals as part of their “local channel packages.” Now it is time for them to do the same for local public television stations.