Presenting a live 90-minute webinar with interactive Q&A
Medicare and Medicaid Audit Sampling Strategies Developing Sampling Plans and Challenging Flawed CMS Audit Samples TUESDAY, AUGUST 14, 2012
12pm Central | 11am Mountain
Today’s faculty features: Anna M. Grizzle, Member, Bass Berry & Sims, Nashville, Tenn.
Patricia L. Maykuth, Ph.D, President, Research Design Associates, Decatur, Ga.
The audio portion of the conference may be accessed via the telephone or by using your computer's speakers. Please refer to the instructions emailed to registrants for additional information. If you have any questions, please contact Customer Service at 1-800-926-7926 ext. 10.
Sound Quality If you are listening via your computer speakers, please note that the quality of your sound will vary depending on the speed and quality of your internet connection. If the sound quality is not satisfactory and you are listening via your computer speakers, you may listen via the phone: dial 1-866-370-2805 and enter your PIN -when prompted. Otherwise, please send us a chat or e-mail [email protected]
immediately so we can address the problem. If you dialed in and have any difficulties during the call, press *0 for assistance. Viewing Quality To maximize your screen, press the F11 key on your keyboard. To exit full screen, press the F11 key again.
FOR LIVE EVENT ONLY
For CLE purposes, please let us know how many people are listening at your location by completing each of the following steps: •
In the chat box, type (1) your company name and (2) the number of attendees at your location
Click the word balloon button to send
Medicare and Medicaid Audit Sampling Strategies Anna M. Grizzle Partner Bass, Berry & Sims PLC
Patricia Maykuth, Ph.D. President Research Design Associates, Inc.
August 14, 2012
Agenda • When is statistical sampling and extrapolation used? • What is the legal basis for statistical sampling and extrapolation? • How is statistical sampling and extrapolation performed? • How can I defend against extrapolated overpayment results? 5
Use of Statistical Sampling for Overpayment Estimation • Acceptable tool in different audits: Medicare, Medicaid, tax, financial statements, etc. • Appropriate when records are too voluminous for individual review • Used in Medicare overpayment reviews since the 1970’s
Use of Statistical Sampling for Overpayment Estimation
• CMS overpayment audit • OIG self-disclosure protocol • Internal compliance audit
Legal Basis for Statistical Sampling for Overpayment Estimation “The use of statistical sampling to project an overpayment. . . does not deny a provider or supplier due process. Neither the statute nor regulations require that a case-by-case review be conducted in order to determine that a provider or supplier has been overpaid and to determine the amount of overpayment.” HCFA Ruling 86-1
Legal Basis for Statistical Sampling for Overpayment Estimation Statistical sampling does not violate due process “so long as extrapolation is made from a representative sample and is statistically significant.” Chaves County Home Health Service, Inc. v. Sullivan, 931 F.2d 914 (D.C. Cir. 1991), cert. denied, 402 U.S. 1091 (1992).
Legal Basis for Medicare Statistical Sampling and Extrapolation A Medicare contractor may not use extrapolation to determine overpayment amounts . . . unless . . . – There is a sustained or high level of payment error; or – Documented educational intervention has failed to correct the payment error 42 U.S.C. §1395ddd(f)(3)
Legal Basis for Medicare Statistical Sampling and Extrapolation • Sustained or high level of payment error can be determined by: – – – – – –
Error rate determinations by MR unit, ZPIC Probe samples Data analysis Provider/supplier history Information from law enforcement investigations Allegations of wrongdoing by current or former employees of provider or supplier – Audits or evaluations conducted by the OIG Source: Chapter 8 – Benefit Integrity; Medicare Program Integrity Manual; available at: http://www.cms.gov/manuals/downloads/pim83c08.pdf (Previously found in Chapter 3)
Legal Basis for Medicare Statistical Sampling and Extrapolation • Additional Factors to Consider – Number of claims in universe – Dollar values associated with claims – Available resources – Cost effectiveness of expected sampling results
Source: Chapter 8 – Benefit Integrity; Medicare Program Integrity Manual; available at: http://www.cms.gov/manuals/downloads/pim83c08.pdf
Legal Basis for Medicaid Statistical Sampling and Extrapolation • Dictated by state law • If no explicit authority, look to due process requirements
Numbers vs. Statistics • Numbers can readily be manipulated and outcomes understood through the use of simple math: addition, subtraction, multiplication, multiplication and division e.g., %s, differences, sums and averages. • Statistics is branch of applied math concerned with the collection and interpretation of quantitative data and the use of probability theory to estimate universe parameters e.g. correlations, t-tests and point estimates 14
MPIM Requirements • Key Rules – Obtain and properly execute “probability sample” – Keep data and records so work can be replicated
• More content and direction given in RAT-STATS Manuals and standard of care expected of statisticians under Generally Accepted Statistics Procedures and Policies (“GASPP”)
Validity Sample If a particular probability sample design is properly executed, i.e., defining the universe, the frame, the sampling units, using proper randomization, accurately measuring the variables of interest, and using the correct formulas for estimation, then assertions that the sample and its resulting estimates are “not statistically valid” cannot legitimately be made. In other words, a probability sample and its results are always “valid.” MPIM § 8.4.2 16
“Always Valid” Does Not Mean Results Cannot Be Challenged Rather the “always valid” refers to the idea that internal operation of a statistical process which, when executed, will (with respect to its mathematical assumptions) yield internally consistent results. The concept of statistically “valid” includes the understanding that there is an expectation of error. “Valid” results include expectation of error: wrong 10 times in 100, precision demonstrated inaccuracy, validly rejecting hypothesis.
Valid Outcomes Require Properly • • • • • • • •
defined universe defined the frame defined sampling units use proper randomization Accurate measuring the variables of interest using the correct formulas for estimation tests of key assumptions accurate reporting of actual findings 18
Typical Problems with Extrapolation
Sample size, not associated with precision or confidence
Incorrect use of formulas
Use of wrong formulas - choose wrong method
Use of inapplicable methodology – simple, stratified, cluster, multi-stage
Fail to meet key assumptions of statistic – math basis of statistic
Exclusion of zero paid claims
Accuracy outside of recommended range – too little precision
Reporting precision and/or confidence levels that are wrong 19
Unacceptable Departure From GASPP • too excessive a departure from even a lenient interpretation of the MPIM • major departures from methodology • non-trivial mistakes in audit definition application of method • non-sampling errors • lack of statistical oversight and quality control 20
Overview of the sampling process
Universe (who; why; what data)
Frame (dates; units; criteria)
Sample Definition (simple; stratified; multi-stage)
Seed & Random Numbers
Pick Out Sample
Sample Size (Chosen precision & confidence)
Calculated Statistics of Sample Before claim review Choice of methodology Simple Stratified Cluster Multi-stage Sample size determination based on Universe size Standard deviation or probe Chosen Precision Chosen Confidence interval
After claim review Calculate overpayment
Per claim For sample Proportion of claims in error
Calculate point estimate Mean Error rates Precision for confidence interval Upper and lower CI
Key Requirements for Use of Parametric Statistics Use a sample that: • Is made up or independent observations • Randomly selected • Normally distributed • Is representative of the frame from which it was chosen and over which it will be extrapolated
Random – … each distinct sample of the set has a known probability of selection…. – … one of the possible samples is selected by a random process according to which each sampling unit in the target population receives its appropriate chance of selection….
SEED NUMBER: 50007.27
80 Frame Sort
Selection Order Value 3 4 10 6 11 7 13 40 9 42 5 43 1 49 6 51 14 56 15 60 4 62 7 64 2 73 12 76 8 77 16 13 17 45 18 70 19 15 20 5 SUMMATION OF RANDOM NUMBERS = 858
Sample Size Statistical sampling is used to calculate and project (i.e., extrapolate) the amount of overpayment(s) made on claims. The Medicare Prescription Drug, Improvement, and Modernization Act of 2003 (MMA), mandates that before using extrapolation to determine overpayment amounts to be recovered by recoupment, offset or otherwise, there must be a determination of sustained or high level of payment error, or documentation that educational intervention has failed to correct the payment error. By law, the determination that a sustained or high level of payment error exists is not subject to administrative or judicial review. MPIM, § 184.108.40.206 29
Example Frame Frequency N Mean Median Mode Std. Deviation Variance Skewness Kurtosis Minimum Maximum Sum
70 49.99 50.00 51 11.544 133.261 0.156 0.856 20 80 3,499
Sample Size Determination Based on Chosen Precision and Confidence
1% 2% Precision 5% Level 10% 15%
Confidence Level 80% 90% 95% 75 77 78 63 68 71 29 * 39 46 10 * 15 * 20 * 5* 8* 10 *
99% 79 75 56 29 * 16 *
Probability Sample Probability theory mathematics allow the comparison of sample data to a described distribution (in this case a “normal distribution”) to describe the pattern of that data. Many inferential statistics are based on the reality that the data being analyzed were “normally distributed”
This is the standard normal distribution’s mathematical formula.
Basic statistical terminology Mean (average) the arithmetic sum of all scores divided by the number of cases Median – the middle most real score Mode the score that occurs most frequently in the data set (does not have to be unique – sometimes more than 1 value is equally likely) Measures of variability (variance, standard deviation, precision and confidence interval)
Lower Confidence Level
Upper Confidence Level
Sample Frequency in $ N Mean Median Mode Std. Deviation Variance Skewness Kurtosis Minimum Maximum Sum
15 430.93 112.00 120 525.519 76,170.210 0.816 0.580 10 1,300 6,464
Non Normal Mean and SD Sample mean +/- 1 standard deviation Mean = $430.92 + sd = 525.510 = 956.43 430.92 - 525.510 = -94.18
Posterior Distribution • Sample selected from amount paid to provider • Sample analyzed using overpayment data • Never know up front what the overpayment amount is going to be unless – There is a known history of overpayment dollar amount – Conduct a probe
• Overpayment amounts must meet criteria for using parametric statistic or the confidence levels are destroyed. 43
Sample Overpay Frequency in $ N Mean Median Mode Std. Deviation Variance Skewness Kurtosis Minimum Maximum Sum
15 339.333 100.00 0.0 511.62 261,763.810 1.197 -0.524 10 1,300 6,464
RAT-STATS Overpayment Estimation Formulae:
RAT-STATS Point Estimate & CI POINT ESTIMATE
90% CONFIDENCE LEVEL LOWER LIMIT 10,368 UPPER LIMIT 43,925 PRECISION AMOUNT 16,778 PRECISION PERCENT 61.81% T-VALUE USED 1.761310135775 Lower 27,147 + 16,778 = $43, 925 Upper 27,147 - 16,778 = $ 10,368 Confidence Interval = 16,778 + 16,778 = 33,556 46
Confidence Levels Non Normal Data Point Estimate +/- ½ Confidence Interval Lower 27,147 + 16,778 = $43, 925 Upper 27,147 - 16,778 = $ 10,368 Confidence Interval = 16,778 + 16,778 = 33,556
Lower confidence Level ??? Unknowable 47
Overpayment Calculations POINT ESTIMATE 90% CONFIDENCE LEVEL LOWER LIMIT UPPER LIMIT PRECISION AMOUNT PRECISION PERCENT T-VALUE USED
10,368 43,925 16,778 61.81% 1.761310135775
Requirements of Every Study • • • • • • • • •
Define objectives of study Define the universe to be sampled All relevant data sampled, none omitted or added Ascertain the degree of precision acceptable Specify people who conduct study with expertise necessary and documented records Define the frame Select the sample (correct size, random, independent, representative, normal) SVRS Organize field work, quality controls and assurance Summarize and analyze results of validity sample 49
Poor Audit Design & Execution Produce Only “Invalid Results”
Statistics in the hands of an inept auditor are like a lamppost to a drunk--they're used more for support than illumination.
Defending Against Extrapolation Results • Medicare Appeals Process – Redetermination – Reconsideration – Administrative Law Judge Hearing – Medicare Appeals Council – Federal District Court
• Medicaid Appeals Process – Appeal rights under state law 51
Defending Against Medicare Extrapolation Results • No administrative or judicial review of determination of high level of payment error BUT determination must be made • Failure to follow one or more requirements in Benefit Integrity Manual does not necessarily affect validity • Not sufficient to argue better or more precise methods are available
Defending Against Medicare Extrapolation Results • Can challenge validity of sampling methodology based on “the actual statistical validity of the sample as drawn and conducted” • Test: Was the sample statistically valid? • Provider has burden of establishing sample was not statistically valid
Defending Against Extrapolation Results • Procedural Challenges – Did the contractor follow the MPIM or state law requirements? – Were allowed claims included in overpayment sample calculation? – Were calculations performed correctly at each level of appeal?
Defending Against Extrapolation Results • Substantive Challenges – Likely need a statistician • Where can you find one?
– “One size does NOT fit all.” – It is not your job to explain how it should be done.
Defending Against Extrapolation Results • Examples of Substantive Challenges – Is the sample representative? – Is the sample statistically significant? • Is the sample size reliable? • Is the sample within the required precision and confidence levels?
Defending Against Extrapolation Results • Obtain all documentation related to sampling calculations – Consider provider’s prior audit history • Know appeal timelines and requirements for each level • Understand reasons for denial at each level • Present reasons in written protest or position paper • Prepare for oral testimony at hearing 57
Questions Anna M. Grizzle Bass, Berry & Sims PLC [email protected]
Patricia Maykuth, Ph.D. [email protected]