Assessing ACO Performance

As more health plans utilize Accountable Care Organizations (i.e., ACOs) as part of their network operations, ACO performance assessment is becoming more important. Plans should thoroughly assess an ACO before they agree to contracting. Once operational within their network, they will also need to assess the individual ACO’s performance to understand its impact on the health plan’s overall performance and whether or not the ACO is helping the health plan achieve its strategic goals. Several entrepreneurial organizations are starting to acquire and combine ACOs as business entities and require ACO assessment as part of their due diligence process. ACOs themselves need to know how they are doing and what value they bring to the negotiating table. This paper will present an outline of what could be included in an ACO assessment.

Definition

Wikipedia defines an ACO as:

“An accountable care organization (ACO) is a healthcare organization that ties payments to quality metrics and the cost of care. ACOs in the United States are formed from a group of coordinated health care practitioners. The ACO adopts alternative payment models (e.g., capitation). The ACO is accountable to patients and third-party payers for the quality, appropriateness and efficiency of its services. According to the Centers for Medicare and Medicaid Services (CMS), an ACO is “an organization of health care practitioners that agrees to be accountable for the quality, cost, and overall care of Medicare beneficiaries who are enrolled in the traditional fee-for-service program who are assigned to it”.

In this paper we will define an ACO as a combination of hospital/health system and physician providers that agree to contract collectively with a payer/health plan for a specific population, whether assigned or attributed to the ACO. Most often the contract will involve alternate reimbursement methodologies (i.e., capitation, risk share, savings share, etc.) as incentives to provide high quality, cost-effective care.

In essence, the ACO becomes a “performing unit” within the overall health plan/payer network and its performance can be measured and compared to other ACOs, the overall organization, and/or benchmarks.

Areas of Assessment

There are several meaningful areas where the ACO’s performance can be measured, some more important than others. The most critical ones are:

• Net cost to payer/health plan: how favorable is the discount?
• Care Management Effectiveness (i.e., CME): how efficiently is care delivered within the ACO? What care management processes are in place within the ACO? How committed is the ACO to achieving optimal results?
• Breadth and balance of provider network: how many physician providers are aligned with the network? What is the mix between primary care and specialty? Are their critical missing specialties? How adequate is the network?
• Managed care maturity: how mature is the local market? What portion of the ACOs overall reimbursement is related to managed care and/or value based or alternative reimbursement methods?
• Attractiveness to the payer/health plan players: how many organizations contract with this ACO? How many choose to use this ACO in its “narrow network” offerings?
• Propensity to assume financial risk: how willing is this ACO to assume risk for the care it provides? Has the ACO created a health plan or other risk bearing organization?

We recommend completing a rigorous assessment in each of these areas and perhaps others and summarizing the results in an analytic fashion (i.e., perhaps with scores from 1 to 5, where 3 is average performance) reflecting weights of importance for each category. A sample weighting is shown below.

Category	Weighting
Net payer cost	15%
Care Management Effectiveness	35%
Breadth of ACO network	15%
Managed Care Maturity	10%
Attractiveness of ACO	10%
Propensity to assume risk	15%
Total	100%

The weighting is somewhat arbitrary, however, the weights shown in the above table are ones that we have used to assess ACOs in our consulting work. An illustrative assessment is shown in the next table based upon a hypothetical evaluation of an ACO. A more detailed table is presented at the end of this document which provides information on the scoring.

Category	Weighting	Score*
Net payer cost	15%	3
Care Management Effectiveness	35%	4
Breadth of ACO network	15%	3
Managed Care Maturity	10%	3
Attractiveness of ACO	10%	4
Propensity to assume risk	15%	4
Total	100%	3.6
* based upon 1 – 5 scale, where 3 is average, and 5 is top performance

In the above sample the weighted average score of 3.6 suggests this hypothetical ACO is performing at above average levels (i.e., score > 3). This type of valuation scheme can be used to rank ACO performance among multiple ACOs and identify the top performers.

Evaluation Suggestions

There are several ways to evaluate each of these specific areas. The following describe some examples of how one might consider completing an evaluation for each area:

• Net payer cost: Net payer cost refers to the net price a payer/health plan reimburses an ACO. This reflects the discount agreed to by the ACO. For fee-for-service payments this involves a simple comparison to an agreed upon payment rate or fee schedule. For alternative payment methods this is the comparison of effective cost to that generated for a specific payment rate or fee schedule. It is important to base such comparisons on a comparable mix of services. We find it particularly beneficial to express physician based services in terms of Medicare payment rates and then the comparison becomes a simple ratio of effective conversion factors¹. Discounts alone make a poor comparator since one ACO may have significantly higher billed charges and the discount does not appropriately compare net payment levels. We also utilize a tool known as an actuarial cost model to assist with these types of comparisons. Health system/hospital comparisons need to consider the underlying reimbursement approach (i.e., DRG vs. per diem). These can then be compared to normative information the payer/health plan has on similar blocks of business. To the extent that a particular ACO has a higher than average net payer cost, it would result in a less than average score. An example for scoring net payer cost might be:
  • 1: more than 125% of target cost
  • 2: between 110% and 125% of target cost
  • 3: within 10% of target cost
  • 4: between 75% and 90% of target cost
  • 5: less than 75% of target cost
• Care Management Effectiveness: Care Management Effectiveness (i.e., CME) is an evaluation metric that measures the efficiency of a health care system using a range of 0% – 100%. Every health care system can be placed on this continuum. The 0% suggests a system without measurable controls and significant potentially avoidable care. The 100% suggests a system that has achieved ideal performance without measureable potentially avoidable care. The highest CME we have observed in our consulting practice over the past 40 years is 90% – 95%. Since medicine is not a perfect science and requires professional judgment, some modest level of potentially avoidable care should be expected. In today’s marketplace better performers are often in the 75% – 85% range. The lower the CME the greater the opportunity for providers to reduce the cost of care. The higher the CME the lower the current cost and the more competitive is the ACO positioned to help the payer/health plan offer a competitive product.There are multiple ways to assess CME, some analytical and some clinical. The simplest way to measure an ACO’s CME is a quick analysis of discharge information from its inpatient hospital stays. We find that potential inpatient savings are consistently two-thirds of overall potential ACO savings. We accomplish this assessment by comparing actual length of stay by DRG (either APR-DRG or MS-DRG) with target or ideal benchmark lengths of stay. It is critical to use an acuity-severity based stay categorization scheme to appropriately reflect patient mix. The ACO’s results can readily be compared to benchmarks with CME calculated. This same approach can be used to supplement the hospital net payer cost analysis above.The table below shows an example of this type of analysis.
  The above table shows the following information:
  • 43.5% of the reported bed-days (i.e., 30,957 / 71,155 = .435) appear to be potentially avoidable resulting in $35.5 million potential cost savings (2.21 days per discharge)
  • Major savings opportunity with least severe patients (Code 1)
  • Greater opportunity for savings in medical vs. surgery
  • One-quarter of the potential savings occurs in 10 DRGs

  We would estimate the CME for this population to be about 20% on the 0% – 100% scale (5.08 vs. 2.93 and 5.5 continuum endpoints).

  An example method for scoring CME might be:

  • 1: less than 30% CME
  • 2: between 30% and 45% CME
  • 3: between 45% and 60% CME
  • 4: between 60% and 75% CME
  • 5: more than 75% CME
• Breadth of ACO Network: The breadth of the ACO network is critical in attracting individuals to be part of the ACO. The most critical factor is the number of primary care physicians available within the ACO. A typical primary care provider can serve 2,000 – 2,500 patients, less if internal medicine and less if for a Medicare population. This translates into 0.4 – 0.5 PCPs per 1,000 potential commercial under age 65 members. A reasonably efficient ACO requires about 0.65 beds for each similar commercial 1,000 members based upon a target 85% occupancy rate (i.e., 200 / (0.85 x 360) = 0.64²). Using 0.45 PCPs per 1,000, this suggests that the ideal ACO would be able to market a PCP panel equal to about 70% of its available beds (i.e., 0.45 / 0.64 = 0.70). Less than that provides inadequate access to a PCP. There is no downside to having a broader PCP panel. The total physician panel per 1,000 members ranges between 1.00 and 1.15 depending upon the use of physician extenders. The ratio of specialists to PCP should be no greater than 155%. Higher than this suggests too many specialists with a tendency to over refer with higher than expected costs.We recommend a two-way evaluation grid, one for size of PCP panel/1,000 and the other for ratio of specialists to PCPs. The following table provides an example of how this could be done.

	Ratio of Specialists to PCPs
PCP/Bed Ratio	>1.75	1.65 – 1.75	1.45 – 1.65	1.35 – 1.45	<1.35
< .40	1	1	1	2	3
.40 – .60	1	1	2	3	4
.60  -.80	1	2	3	4	5
.80 – 1.00	2	3	4	5	5
>1.00	3	4	5	5	5

• Managed Care Maturity: Managed care maturity describes the market that the specific ACO operates in. It is a measure of how much the provider community has been exposed to managed care principles, alternative reimbursement, care management, etc. It also reflects how much these alternative reimbursement methods have impacted the provider revenue cycle management processes. The more mature the market place the more effective the care throughout the market. It is more difficult for an ACO to demonstrate extraordinary performance in a mature market than a less mature market since more providers will have made similar advances. Some of the most useful metrics to measure managed care maturity include HMO penetration in the market, higher percentages of alternative reimbursement contracts and revenues subject to them, efficiency of the overall market’s health system, etc. As in the previous section we recommend a two-way evaluation grid for managed care maturity.

	Percentage of Revenue from Alternative Reimbursement
HMO Penetration	>10%	10% – 20%	20% – 30%	30% – 40%	<40%
< 10%	1	1	1	2	3
10% – 20%	1	1	2	3	4
20% – 30%	1	2	3	4	5
30% – 40%	2	3	4	5	5
>40%	3	4	5	5	5

• Attractiveness to payer/health plan players: The more attractive an ACO is to the payer/health plan community, the more important it is in the ACO assessment process. Whether or not there are good reasons for its attractiveness or perceived value to the community, the fact that people think they need it carries worth in the assessment process. For example, if the general public believe that a certain academic medical center must be in the network, then it must be in the market, whether or not it hurts the competitiveness of the network. Decisions are made based upon the characteristic of the providers that are in the network, and without the key providers, the ACO may not be attractive to those members signing up for coverage or enrolling in the program. The best way to measure attractiveness is to determine the extent of contractual arrangements the ACO has in the market.An example method for scoring attractiveness might be:
  • 1: less than 20% of the major payers
  • 2: between 20% and 45% of major players
  • 3: 45% – 55% of the major payers
  • 4: between 55% and 70% of the major payers
  • 5: more than 70% of the major payers
• Propensity to assume financial risk: The more risk that an ACO is willing to assume the greater the impact and potential value to the payer/health plan contractor. An ACO fearful of assuming risk suggests an ACO partner that will likely provide less value to the payer/health plan. ACOs who bullishly will assume risk suggest an ACO who might even consider establishing their own health plan or broader marketplace activity. The most desirable ACO for the payer/health plan is one that values the long term rewards of risk assumption.An example method for scoring risk propensity might be:
  • 1: majority fee-for-service contracting
  • 2: only gain-share and cost savings contracts
  • 3: prefers gain-share and cost savings contracts, but open to limited gain-sharing
  • 4: willing to take downside risk-sharing arrangements
  • 5: pursuing global payment/capitation arrangements

Assessment Worksheet

We have prepared a worksheet to help organize the assessment process based upon the above criteria and prepare consistent assessment results. Completed for multiple ACOs, this would provide a concise and objective approach for comparing multiple ACOs. This type of analysis is very useful to regularly update and report performance back to the participating ACOs in the market. Unsatisfactory results can be used to adjust the provider network and/or provide useful information for contracting adjustments. Armed with this type of information, a payer/health plan can optimize their own performance and be sure they are highly competitive in the marketplace.

Summary

The previous information will provide insights to assess the performance of ACOs and will provide valuable comparative information for health plans, business managers and ACO leadership. This will provide useful information in determining how effective the use of ACOs is and could be. Based upon some initial performance observed in the marketplace some question the effectiveness. With a robust and thorough analysis as outlined above, a credible answer can be determined.

¹Conversion factors are values multiplied by Medicare’s RBRVS unit values to determine fee levels.

²This assumes the delivery system is achieving 200 bed-days/1,000 and is striving for the optimal 85% occupancy. When hospitals operate at higher than an 85% occupancy rate, there are often shortages of beds during peak times and they are not able adequately serve their patient population.

David Axene, FSA, FCA, CERA, MAAA, is the President and Founding Partner of Axene Health Partners, LLC and is based in AHP’s Murrieta, CA office. Contact us to discuss your next project.