10. Strategic Planning and Data Quality Analysis
10.1 Strategic Planning Related to the Use and Benefits of Data Analytics
State of Minnesota Department of Administration Enterprise Data Analytics Program
10. Strategic Planning and Data Quality Analysis PCG proposes the following in response to Strategic Planning and Data Quality Analysis: 10.1 Strategic planning related to the use and benefits of data analytics 10.2 Data collection, cleansing, and integration These areas will be presented in two distinct sections.
10.1 Strategic planning related to the use and benefits of data analytics A. Description of Relevant Experience
PCG Human Services Overview For over 20 years PCG Human Services™ has worked to assist state and local human services agencies achieve their performance goals. Our seasoned professionals and subject matter experts in combination with our proven solutions help agencies to increase program revenue, cut costs, and improve compliance with state and federal regulations. Areas of expertise include the following: Child Welfare and Youth Services PCG Human Services™ child welfare experts offer a number of consulting services – from Title IV-E eligibility determination to Supplemental Security Income (SSI) advocacy – to ensure the efficient operation of these agencies and their programs in order to meet the critical needs of children under their care. PCG Human Services™ also helps state and local juvenile justice and youth probation agencies around the country to improve operations, increase revenue, and improve service to youths under their care. Early Education and Child Care With more than two decades of experience in management consulting for state and local government agencies and with an expertise in child care operational and program consulting, PCG Human Services™ offers an array of services to meet the unique needs of child care service providers. SSI/SSDI PCG Human Services™ offers state and county governments unique and timetested methods to expedite the process of applying for disability benefits administered by the Social Security Administration (SSA). PCG’s Social Security Advocacy Management Services or SSAMS™ division, a third party, non-representation business unit, has been offering disability benefits services since 1989, with several thousand SSI and Social Security Disability Insurance (SSDI) applications filed annually. Supplemental Nutrition Assistance Program (SNAP) PCG Human Services™ works with Supplemental Nutrition Assistance Programs (SNAP) agencies across the U.S. to conduct SNAP program reviews and implement systems innovations to help clients creatively June 17, 2011
Page 1
State of Minnesota Department of Administration Enterprise Data Analytics Program
manage increased caseloads, hiring freezes, and budget reductions. Our industry best-practice modernization efforts improve clients’ program efficiency while providing the greatest benefit and timely access to all program participants. Temporary Assistance for Needy Families (TANF) PCG Human Services™ works with state and local clients to improve their TANF (Temporary Assistance for Needy Families) programs through a variety of creative and effective consulting services to enhance compliance with federal regulations, financial performance, and programmatic efficiency. Welfare to Work and Workforce Services PCG offers a comprehensive approach to help health and human services agencies to analyze the obstacles facing these vulnerable clients, plan effective strategies for addressing the challenges, and engage clients in appropriate work and work-related activities. Within each of these areas of expertise, PCG developed a deep understanding of the specific challenges that government managers face in order to provide the services that citizens need. Our understanding of these challenges and the political and financial realities that managers face have provided us with the right background to help our clients develop strategic plans that provide actionable steps and measurable results to achieve their goals. Each of our strategic planning engagements have required data collection and data analysis in order to complete the strategic plan, and many have focused on how data analytics and information systems can help our clients achieve their goals. Some examples of our strategic planning experience include: The Massachusetts Department of Early Education and Care (EEC) contracted with PCG to perform the initial analysis and design of an Early Childhood Information System (ECIS). As part of this engagement, PCG partnered with Harvard University Graduate School of Education and University of Massachusetts Lowell to hold a two day Strategic Planning Institute conference of national, state, and local leaders, community providers, advocates, researchers, national experts and other early childhood stakeholders. The Institute combined presentations, panels, and small-group working sessions while encouraging key stakeholders to engage in a conversation surrounding 1) the questions that an ECIS should be designed to answer, 2) what data elements are needed to answer those questions, and 3) the intended outcomes of the collected data, potential challenges, and next steps. After several months of planning, over 100 people representing a diverse group of early childhood stakeholders attended the conference. In consideration of the results of the Strategic Planning Institute, PCG is performing a gap analysis of required data elements and is designing an ECIS data model for the integration of EEC extant data and external data sources. PCG worked with the Massachusetts Executive Office of Health and Human Services and the Executive Office of Education to develop a business case, data systems inventory and strategic plan for the Massachusetts Statewide Comprehensive Integrated Data Warehouse and Reporting System. This project was a direct result of Massachusetts Governor Deval Patrick’s June 25, 2008, Education Action Agenda, which called for statewide reform to close the gaps that exist in educational outcomes among students. The Agenda’s first steps involved improving June 17, 2011
Page 2
State of Minnesota Department of Administration Enterprise Data Analytics Program
communication and collaboration across state agencies. Governor Patrick then created the Child and Youth Readiness Cabinet, comprised of the Secretaries of the state agencies charged with fostering the health, education and welfare of children, youth and families. The Readiness Cabinet aimed to develop a statewide youth data warehouse (or other information technology) and reporting system that would link data and information throughout the education and social services system. PCG’s work included performing stakeholder surveys with 18 state agencies; performing three planning sessions to confirm the project’s vision, goals, and action steps; a systems inventory, including the beginning of a data dictionary, that details where human service, education, health and workforce development data resides in state systems. Our final deliverable was a Strategic Plan that included an action plan, a business case and the identification of key challenges. Our Strategic Plan was heavily utilized in Massachusetts’ successful application for Race to the Top federal funding. PCG provided strategic planning services for Wayside Youth and Families, a children’s behavioral health/child welfare services provider in Massachusetts. To inform the strategic plan, PCG solicited feedback from internal and external agency stakeholders including: youth and families involved with the agency, the agency’s board of directors and senior managers, state child welfare and behavioral health program directors, and state policy makers via surveys and interviews. PCG collected and analyzed survey and interview data, as well as agency financial data, to make recommendations for a two-year strategic plan to position Wayside as a service leader in the Commonwealth and improve their year-end financial status. PCG worked with the Department of Public Welfare to perform a study directed at determining the feasibility and alternatives for the successful design, development and implementation of an information technology solution to support Pennsylvania’s Child Welfare programs within the Office of Children, Youth, and Families, including the Bureau of Juvenile Justice. PCG completed a feasibility study, alternatives analysis, cost benefit analysis, strategic plan, and three procurements (DDI, IV&V, and QA) that determined if implementing a Statewide Automated Child Welfare Information System (SACWIS) in Pennsylvania was feasible and the best approach for implementing an IT solution. As an additional challenge to this project, sixty-seven counties carry out the child welfare and juvenile justice programs under the supervision of OCYF, and, though many of these counties had already independently implemented case management IT solutions, OCYF has struggled with collecting the data they need for management and supervision. PCG developed a strategic plan that leveraged existing IT assets across state agencies and counties that will lead to comprehensive data collection and reporting. A key component of our strategic plan was to develop an operational data store to aggregate cleansed data for analysis in a data warehouse using business intelligence data analytics. B. Description of Proposed Data Analytics Services 10.1 Strategic planning related to the use and benefits of data analytics (i.e. how can data analytics help the agency)
June 17, 2011
Page 3
State of Minnesota Department of Administration Enterprise Data Analytics Program
PCG Human Services™ has considerable experience leveraging data and data systems to develop and deliver actionable strategic plans for our state partners.
Throughout this RFP, PCG has talked about how we can help you implement data analytics projects, Here, we lay out our approach to assist you to plan data analytics projects strategically, before they start, so that you can be sure that your time and resources are invested in the right solution. Whether your goal is to improve agency services or improve data quality, PCG employs the following data analytics strategic planning approach:
Understand the problem, or the business need Identify the gap between the current state and the desired future state Consider alternative solutions and the cost/benefit associated with each Collaboratively decide on a strategy and implementation plan
1. Understand the Business Need Because we run operations ourselves, and because we work almost exclusively with government agencies, PCG is particularly skilled in this arena. While we understand that every agency is unique, we have a deep understanding of the day to day business of government and we speak the language of government operations. To that end, we are able to communicate with individuals at all levels of government to discuss and diagnose your business needs. Assessing the business accurately in the early stages of the project is critical to a successful outcome. Business requirements, accurately diagnosed, should be the key drivers of data analytics solutions. PCG can assist you to assess your business needs by meeting with and interviewing stakeholders including agency leaders, middle managers, and agency customers. PCG is skilled at conducting business requirements gathering sessions and needs assessments with individuals at all levels of government agencies. Requirements Gathering and Validation Techniques Based on prior experience conducting requirements gathering and validation activities, PCG uses a variety of proven techniques:
Survey – In some cases, data may be gathered or requirements may be further clarified through preparation of a questionnaire which contains a list of structured questions to be distributed to specific groups. Where a survey or a set of surveys are required, we can draft the questionnaire, review it with the state project team, and incorporate their feedback prior to distribution.
Interviews – Interviews are used to gather information. However, the predisposition, experience, understanding, and bias of the person being interviewed influences the information obtained. The use of context-free questions by the interviewer helps avoid
June 17, 2011
Page 4
State of Minnesota Department of Administration Enterprise Data Analytics Program
prejudicing the response. A context-free question is a question that does not suggest a particular response. For example, who is the user for this system functionality? What is the real reason for wanting to solve this problem?
Brainstorming – This is a group process that involves generation of creative ideas around a particular area of interest, and then the ideas are analyzed and rated according to their appropriateness.
Joint Requirements Management (JRM) Sessions – A JRM session is a highly focused and structured group process that is designed to encourage diverse stakeholders to generate a clear list of requirements. Facilitated JRM sessions offer a collaborative approach to requirements definition, review, and validation. JRM sessions typically involve a number of participants, a facilitator, and a scribe to record the results of the session. JRM sessions focus on achieving consensus concerning requirements.
PCG can use any or all of these individual and group techniques for Minnesota at various stages of the project. However, each of these techniques has its own set of strengths and weaknesses based upon characteristics such as the nature and type of requirements to be analyzed, the functional span of processes to be reviewed, and the experience level of the participants in requirements gathering efforts. PCG would work closely with State project managers to choose the best methods for the project at hand. 2. Current Business Process Review and Gap Analysis PCG will work with Minnesota to accurately assess the root causes of your need for a data analytics solution. With leadership changes, budget cuts, and agency reorganizations, gaps regularly occur in business process. A variety of conditions can cause these gaps: lack of data or technology, operational gaps, or policy deficiencies. It can be tempting to build a data solution to a problem that is not actually driven by a data deficiency, and PCG has seen many agencies take this unfortunate and expensive path. As a first step in the gap analysis, PCG will gather information to document the “as is” business process and inform the gap analysis using the following techniques:
Stakeholder Interviews- Stakeholders could include agency leaders, agency managers, front line staff, and even customers.
Focus Groups –PCG has experience conducting focus groups designed to identify problems in the current business process and elicit strong suggestions for improving the process going forward. Oftentimes, we find that focus groups allow participants to build on one another’s responses, develop solutions for those problems, and critically assess those solutions from a number of viewpoints.
Staff Observations- PCG has worked on a number of business process reengineering projects where observation of the process and the surrounding environment has yielded
June 17, 2011
Page 5
State of Minnesota Department of Administration Enterprise Data Analytics Program
findings that help to develop important benchmarks for pre/post implementation evaluation, and/or identify quick hit recommendations for immediate implementation and instant results. At the conclusion of this phase of our work, PCG will complete an “As-Is” process map that documents current activities, major work steps, inputs, outputs, key decision points, and the important documents and forms. Gap Analysis
Identifying gaps between the “as-is” and the future process and system design serves as the beginning point for implementation of a business process redesign and/or a data analytics solution. PCG utilizes a gap analysis so that the implementation process does not jump from identification of problem areas to proposed solutions without understanding the conditions that created the current state. As we perform the gap analysis, we illustrate relationships between key actors and process steps / decision points in order to identify potential risks or points that may need further clarity. The steps for performing a gap analysis include:
Documenting the “as-is” and future processes Identifying the reasons for the current state through interviews with staff and stakeholders and observations of current processes Determining the data, procedures, or system functionality that is needed to facilitate achieving the desired state Developing specific action steps that are required to achieve the desired state Identifying the gaps that pose the greatest risk to program integrity, such as data elements not captured or processes not supported by the current system Identifying gaps in reporting requirements Developing a timeline for achieving the desired state
Quantifying the gaps enhances the probability that the desired future “to-be” state can be achieved. The template below is taken from a different project in which we evaluated a state’s call center operations. It is highly simplified but an effective example of how we illustrate the “as-is” state vs. the future “to-be” state.
June 17, 2011
Page 6
State of Minnesota Department of Administration Enterprise Data Analytics Program
Gap Analysis Matrix Template
As‐is Process
Future Process
Example: Clients unaware of amount of time on hold
Example: Provide estimate of wait time to client once they have been on hold for 30 seconds
Example: Emails are answered in the order in which they are received, regardless of complexity
Example: Respond to priority requests within six hours, ancillary/ secondary requests completed with 24 hours
Gaps Mapping
Priority
Key Stake‐ holders
Action Steps
Reasonable response time, assessment of customer expectations for Low response; review of state/legal requirements for response
Clients, call center staff, service providers, community coalitions
Does not prioritize customer needs, drain on staff
Develop Frequently Asked Clients, Questions page OCR staff, on website, to community reduce email coalitions, volume technical “Segment out” staff, simple, agency common manage questions and respond to those first
High
Software that will provide estimate of wait time to callers, research vendors and pricing, get approval
Imp. Priority and Time Frame
III
I
As shown, this analysis has identified significant areas where the current process is lacking and what immediate next steps should be undertaken in order to achieve the desired outcome in the future process. Once the gap analysis is complete, PCG presents our work to the state in order to obtain feedback and incorporate changes into the final redesign. We anticipate a two-way discussion that will evaluate our ideas and provide constructive feedback. As state employees and key stakeholders will be responsible for managing and supporting the process going forward, it is critical that the State Project Team has this formal opportunity to review our work and buy into the proposed process and system changes. 3. Alternatives and Cost Benefit Analysis June 17, 2011
Page 7
State of Minnesota Department of Administration Enterprise Data Analytics Program
To guide the alternative analysis process we will present evaluation criteria to Minnesota for its review and approval. We will present agencies with at least five viable alternatives to be considered and will rank each alternative following the established evaluation criteria protocol which may include consideration of program, organizational, operational, and cost impacts. After presenting the rankings with the top three alternatives identified, we will solicit feedback from the State to determine the extent to which state personnel agree with our assessment or wish to re-evaluate any specific alternative. Once consensus is reached, a cost/benefit analysis will be completed. Perform Cost Benefit Analysis A Cost/Benefit Analysis (CBA) is often used by state government agencies to evaluate the desirability of a given undertaking. In its most basic form, a CBA is an analysis of the costs of various alternatives in order to see whether the benefits (typically but not exclusively monetary) outweigh the costs. Alternatives need to be assessed, relevant cost parameters and variables need to be determined and monetary and/or risk values should be assigned to less tangible effects such as the various risks that could affect overall project success. PCG’s approach to completing a CBA for alternatives involves the following steps:
Perform Cost Analysis for each Alternative o Determine Technology cost parameters o Determine Operations cost parameters o Determine Resource cost parameters o Determine Additional cost parameters Perform Benefit Analysis for each Alternative o Identify Functionality benefits o Identify Usability benefits o Identify Flexibility benefits Aggregate data and perform the Cost/Benefit Comparison Produce Cost/Benefit Analysis report and present for review and approval Reach consensus on Cost/Benefit results
The table below represents a simple alternatives ranking model, with the obvious desired outcome being in Quadrant IV.
June 17, 2011
Page 8
State of Minnesota Department of Administration Enterprise Data Analytics Program
Cost/Benefit Quadrant Model
High Cost
Quadrant I: High Cost Low Benefit
Quadrant II: High Cost High Benefit
Low Cost
Quadrant III: Low Cost Low Benefit
Quadrant IV: Low Cost High Benefit
Low Benefit
High Benefit
At the conclusion of this set of activities, PCG makes a recommendation regarding the best strategic approach that supports the most technically, programmatically and fiscally viable alternative. The recommendation is supported by a detailed description of the components of the approach with a rationale for their inclusion. 4. Collaboratively decide on a strategy and an implementation plan Following the steps above, PCG will solicit additional input and feedback from the State in order to determine the final solution. Once the solution is determined, PCG can provide the State with an implementation plan that includes next steps, timeframes, and benchmarks to measure implementation success. C. Description of Necessary Tools, Hardware, and Software PCG is skilled in a host of database development and design best practices and maintains IT staff that may assist with any project engagement that may require advanced Visual Basic or other programming skills. If necessary, to gather or analyze information to inform the strategic plan, custom-built databases may be linked to MS SQL Server to allow for increased data volume / capacity and greater complexity of data processing calculations/queries. Alternatively, smaller databases can be designed in MS Access, depending on Minnesota’s needs. PCG is also experienced in generating custom reports from databases, which may serve as updateable tools for future recalculation of data mining results. PCG always strives to provide its clients with workable solutions that can assist long after any project engagements conclude. D. Proposed Level of State Involvement At this time, it is not possible to estimate resource levels. Part of the PCG initiative assessment / “proof of concept” process is to determine the current level of resources the State is using as well June 17, 2011
Page 9
State of Minnesota Department of Administration Enterprise Data Analytics Program
as what resources will be needed. These assessments will involve interviews with agency personnel so that we understand current operations. In most cases, there are two significant resource commitments PCG will request during a project:
Access to computer files – PCG often needs to download large databases so that we can conduct our analyses. We prefer to have on-line file access but understand that many clients prefer to provide data upon request. We will identify our data requirements and existing reports or files that may have that data available. Update files are often needed on a regular basis. PCG may also request the ability to post edits to the State’s files but we can provide changes for posting by State personnel. Management commitment – PCG will request the involvement of senior agency personnel to monitor each project, be available for consultation, and to help ensure that agency personnel understand that PCG’s involvement is a priority.
When an agency is already involved in an initiative, PCG will base our resource plans on supplementing the State’s existing resources. Our goal is to expand on what is already taken place so that there is an increase in revenues or savings. In some cases, our methodologies and tools will allow the State to assign existing staff to other duties so there can be a decrease in resource commitment by the State. Each initiative is unique and the impact on State resources will be detailed in a Management Letter. The Management Letter will address PCG’s analysis of the overall return on investment (ROI) on the agency. E. Proposed Personnel The following personnel are anticipated to provide professional services under this program: Personnel Name, Title
Work Completed in Service Category Dates Massachusetts Executive Office of Health and Human Services and Executive Office of Education, Statewide April – June 2009 Comprehensive Integrated Data Jill Reynolds, Associate Manager Warehouse and Reporting System Commonwealth of Massachusetts Department of Early Education and Care August 2010 – August Analysis and Design of an Early 2011 Childhood Information System (ECIS) Narrative of Qualifications in Service Category Ms. Reynolds is an Associate Manager at PCG and has managed a variety of data analytics and advanced data analysis projects. She was involved in conducting a three‐pronged survey‐based study for the Massachusetts Department of Early Education and Care (EEC), analyzing 1) parents on the MA Subsidy Waitlist, 2) program access in MA, and 3) factors impacting continuity of services. Ms. June 17, 2011
Page 10
State of Minnesota Department of Administration Enterprise Data Analytics Program
Reynolds and her team designed and implemented three surveys to obtain necessary data: 1) Online Parent Survey; 2) Parent Phone Administered Survey; and 3) Online Provider Survey. The team ensured statistical validity of all survey samples, conducted statistical analyzes and significant findings, developed recommendations and policy implications, and presented the results to state stakeholders. In Michigan, she managed the design and implementation of a statewide survey instrument, development of participant outreach strategies, data collection, implementation of a statistically valid sample and analysis and presentation of survey results to state stakeholders. Currently, in the County of San Diego, she assists in overseeing the labor market and employment outcomes assessments. Additional project experience in data analytics include assisting in the analysis of the funding and allocation methodology for the Employment Services Program in Massachusetts. As well as coordinating a study on the effectiveness of Michigan’s employment services utilizing advanced regression analysis. Prior to joining PCG, Ms. Reynolds was the Deputy Education Advisor to the Governor of Massachusetts. In this role Ms. Reynolds worked on early education, K‐12 education and higher education policy, legislation, regulation and finance. Previously she worked for the Massachusetts Office for Administration and Finance and was responsible for budget oversight and analysis related to education and other areas. Ms. Reynolds received her Bachelor of Arts from Boston College and completed a Master’s Degree in Public Administration at the University of Massachusetts. Personnel Name, Title
Work Completed in Service Category Dates Massachusetts Executive Office of Health and Human Services and Executive Office of Education, Statewide April – June 2009 Comprehensive Integrated Data Warehouse and Reporting System Commonwealth of Massachusetts Heather Baker, Associate Department of Early Education and Care August 2010 – August Manager Analysis and Design of an Early 2011 Childhood Information System (ECIS) Commonwealth of Pennsylvania, Department of Public Welfare Office of October 2008 – Children, Youth and Families, Child October 2009 Welfare Information System Feasibility Study Narrative of Qualifications in Service Category Ms. Baker is an Associate Manager with over thirteen years of experience working in government and government consulting. Ms. Baker was a Lead Facilitator on PCG’s project with the Massachusetts Executive Office of Health and Human Services and Executive Office of Education’s to develop a Business Case and Strategic Plan for the Statewide Comprehensive Integrated Data Warehouse and Reporting Systems. Ms. Baker led a team of consultants to collect information about current data systems, perform stakeholder interviews with relevant state agencies and programs and co‐facilitated three strategic visioning sessions with members of the Governor’s Child and Youth Readiness Cabinet, resulting in a strategic implementation plan for the initiative. Ms. Baker also served as a Technical June 17, 2011
Page 11
State of Minnesota Department of Administration Enterprise Data Analytics Program
Advisor to the PCG team charged with developing a strategic plan for the Commonwealth of Pennsylvania Office of Children and Youth’s implementation of solutions for a child welfare information system. In addition, for PCG’s Analysis and Design of an Early Childhood Information System, Ms. Baker served as a facilitator for a panel presentation at a Strategic Planning Institute that was co‐hosted by PCG and Harvard University. Ms. Baker previously worked for the Commonwealth of Massachusetts Executive Office of Administration and Finance, Fiscal Affairs Division, as a Senior Fiscal Policy Analyst. Personnel Name, Title
Work Completed in Service Category Dates Massachusetts Executive Office of Health and Human Services and Executive Office of Education, Statewide April – June 2009 Comprehensive Integrated Data Warehouse and Reporting System Commonwealth of Pennsylvania, Nathan Grossman, Senior Department of Public Welfare Office of October 2008 – Consultant Children, Youth and Families, Child October 2009 Welfare Information System Feasibility Study Commonwealth of Massachusetts Department of Early Education and Care August 2010 – August Analysis and Design of an Early 2011 Childhood Information System (ECIS) Narrative of Qualifications in Service Category Mr. Grossman has both expertise in federal revenue maximization and management in multiple areas of federal funding (Titles II, IV‐D, IV‐E, XIX, XXI and SSI/SSA, TANF and Food Stamps) and he has expertise in implementing information technology systems for public agencies. Mr. Grossman also has collected business and technical requirements for public agency information systems, including implementing a case management system for the juvenile justice agency for Washington, DC, conducting a feasibility study for a new SACWIS in Pennsylvania, performing analysis and design for an early education information system in Massachusetts, developing a data‐driven performance evaluation system for therapeutic group homes in Connecticut, and implementing Microsoft SharePoint to manage information sharing among local homeland security agencies throughout northeastern Massachusetts. Personnel Name, Title Work Completed in Service Category Dates Jennifer MacBlane, Senior Wayside Youth and Families, Strategic July 2010‐ December Consultant Planning Services 2010 Narrative of Qualifications in Service Category Ms. MacBlane is a senior consultant with PCG. Ms. MacBlane has over 10 years of experience working in government and government consulting. Ms. MacBlane has led several operations and operations improvements projects at PCG including the Massachusetts Department of Children and Families Revenue Management Unit, which requires personnel to meet stringent workload performance standards, which must be tracked and managed regularly. Prior to joining PCG, Ms. MacBlane worked June 17, 2011
Page 12
State of Minnesota Department of Administration Enterprise Data Analytics Program
for the Commonwealth of Massachusetts Executive Office of Administration and Finance, Fiscal Affairs Division, as an Assistant Budget Director and for the Massachusetts Department of Children and Families as their Budget Manager. Ms. MacBlane is an instructor in Public Budget and Finance at the University of Massachusetts Boston and holds a Master’s Degree in Public Administration from Arizona State University. Personnel Name, Title
Work Completed in Service Category Dates NIH/SB RESEARCH/GDOC; Multi‐Tiered Eric Chasin, NTELX Director Hierarchal Analytical Solution to 2010 – Present Facilitate Signature of Signature Narrative of Qualifications in Service Category As Vice President ‐ Technology and leader of the NTEL∙X Analytics and Information Solutions Group (A & IS), Mr. Chasin oversees both the design and development of software and technical solutions implemented on behalf of each client. Mr. Chasin has been involved in software development and management for over twenty‐five years. Mr. Chasin has enjoyed a long history serving government at the Federal and State level designing, developing and implementing innovative solutions with game changing results. His experience spans Federal Taxation and Fraud Detection, National Security, Food Safety, Homeland Security and Defense Intelligence. He is a leader in bringing techniques developed and refined in International Intelligence to Civilian application such as Health Care Quality and regulatory compliance. Mr. Chasin has led technology development for organizations such as Ajilon Consulting, James Martin & Company, and Computer Associates. His formal education is in the field of engineering, with a degree from the University of Maryland. Mr. Chasin’s key qualifications include: 20+ years of professional experience Served as Co‐Founder/CTO of a logistics technology company, National Director of global professional services organizations, and National Account Manager of a multi‐billion dollar enterprise software company Proven track record in developing and implementing solutions in supply chain management, data warehousing, software development methodologies, and enterprise application architectures Personnel Name, Title
Work Completed in Service Category Dates NIH/SB RESEARCH/GDOC; Multi‐Tiered Chris Wheeler, NTELX Director Hierarchal Analytical Solution to 2010 – Present Facilitate Signature of Signature Narrative of Qualifications in Service Category Mr. Wheeler brings over ten years of solution‐oriented analytical experience in developing and implementing successful analytics‐based business intelligence systems and programs. He has demonstrated expertise in leveraging programmatic and algorithmic regimes to drive custom analytical solutions. He has successfully implemented exemplary solutions in the healthcare, pharmaceutical and finance verticals. Mr. Wheeler’s perspective is further enhanced by his expert knowledge in the field of intellectual property and healthcare. As an experienced attorney, he prepared patent applications in the areas of software, computer science, biomedical devices, pharmaceuticals and biotechnology, while counseling clients regarding patent portfolio strategies. His June 17, 2011
Page 13
State of Minnesota Department of Administration Enterprise Data Analytics Program
practice further included advising clients regarding regulatory compliance in healthcare. Mr. Wheeler is a member in good standing of the NY and VA Bars and has degrees from Cornell University (BS) and The George Washington University Law School (JD). Mr. Wheeler’s key qualifications include: 10+ years of solution‐oriented analytical experience Successfully implemented solutions in the healthcare, pharmaceutical and finance industries Advised clients regarding regulatory compliance in healthcare
Please reference detailed resumes for the above personnel in Section 3.1.2 in Tab 3. E. Description of Past Data Analytics Engagements The following are detailed descriptions of data analytics services that PCG has provided to public sector organizations of similar size and scope to the State of Minnesota within the past three years. Name of Project Dates Commonwealth of Pennsylvania, Department of Public Welfare, Office of October 2008 – October Children, Youth and Families; Child Welfare Information System Feasibility 2009 Study Description of Services This project was to perform a study directed at determining the feasibility and alternatives for the successful design, development and implementation of an information technology solution to support Pennsylvania’s Child Welfare programs. The Commonwealth of Pennsylvania, Department of Public Welfare (DPW), Office of Children, Youth, and Families (OCYF), is responsible for administering federal Child Welfare programs under Titles IV‐B and IV‐E of the Social Security Act. The Commonwealth’s Child Welfare program is state supervised, but locally administered at the county level. Sixty‐seven counties carry out the Child Welfare programs under the supervision of OCYF. OCYF has struggled with ensuring both compliance and quality of Child Welfare services at the county level. This is in part due to a lack of real time information that is accessible at the state level. PCG’s role is to determine the feasibility of a SACWIS or another alternative and to draft the APD and the necessary work statements to implement any recommended system. Only seven other states neither have a SACWIS nor have begun developing one, though Pennsylvania actually had developed a previous SACWIS (called PACWIS), which was decommissioned in 2000 because of lack of user‐ friendliness and inadequate software capability. This prior experience highlights the difficulty and complexity of developing a statewide child welfare information system in Pennsylvania. Work Steps: Review with DPW its current processes and document problems and opportunities for achieving efficiency. Gather and prioritize Commonwealth and County business needs and functional requirements for PA’s Child Welfare program. Assess the capabilities of existing systems used by DPW, county Child Welfare offices, and June 17, 2011
Page 14
State of Minnesota Department of Administration Enterprise Data Analytics Program
other Commonwealth agencies and other States to determine their appropriateness for inclusion in the alternatives analysis. Complete a feasibility study, alternatives analysis, and risk assessment to determine if implementing a Statewide Automated Child Welfare Information System (SACWIS) in Pennsylvania is feasible. If SACWIS is not feasible, determine the best, most feasible technology approach to meet both OCY and County requirements. If SACWIS is feasible, evaluate development and implementation approaches. Work with DPW to identify risks, conduct a risk analysis and develop a risk mitigation plan. Rank the alternatives considered. Conduct and document a detailed cost/benefit analysis of either the top three approaches for implementing a SACWIS or the top three feasible technology approach alternatives. Develop a strategic implementation plan.
Name of Project Dates Commonwealth of Massachusetts, Executive Office of Health and Human Services and Executive Office of Education; Statewide Comprehensive April ‐ June 2009 Integrated Data Warehouse and Reporting System Description of Services Public Consulting Group, Inc. (PCG) was awarded a pro bono project for the Massachusetts Executive Office of Education (EOE) and the Executive Office of Health and Human Services (EOHHS). The project aimed to develop a business case and strategic plan to support the client’s vision of a statewide data warehouse and reporting system that would integrate data from state health, social service, economic and education agencies. This project was a direct result of Massachusetts Governor Deval Patrick’s June 25, 2008, Education Action Agenda, which called for statewide reform to close the gaps that exist in educational outcomes among students. The Agenda’s first steps involved improving communication and collaboration across state agencies. Governor Patrick then created the Child and Youth Readiness Cabinet, comprised of the Secretaries of the state agencies charged with fostering the health, education and welfare of children, youth and families. The Readiness Cabinet aimed to develop a statewide youth data warehouse (or other information technology) and reporting system that would link data and information throughout the education and social services system. PCG’s work included: Stakeholder interviews with 18 state agencies to collect feedback about the integration of the statewide data system, including concerns and potential needs of each agency; A systems inventory, including the beginning of a data dictionary library, that details where human service, education, health and workforce development information resides; Three separate Planning Sessions to confirm the vision of the Massachusetts Readiness Cabinet’s work group and to begin identifying possible avenues for reaching that vision; Development of a Business Case that presents the benefits of data integration for children who move across service delivery systems; and Development of a Strategic Plan, including an action plan and a discussion of key challenges such as privacy, information sharing, and technological barriers. June 17, 2011
Page 15
State of Minnesota Department of Administration Enterprise Data Analytics Program
This work was completed in collaboration with several Massachusetts state agency leaders. Building consensus among state agencies was a main objective of PCG in support of the goal of providing every child the support he or she needs to graduate high school. Name of Project Dates Wayside Youth and Family Support Network July 2010 – December Strategic Planning Services 2010 Description of Services PCG was selected by Wayside Youth and Family Support Network to provide strategic planning services, including stakeholder interviews, an environmental scan, business model assessment, and strategic planning recommendations. PCG worked with the Board of Directors and Executive Leadership, both administrative and programmatic, to better understand the current state and ideal future state of Wayside. PCG developed a questionnaire for non‐executive staff and external stakeholders, including state agency leads, payers, families, and industry subject matter experts. All questionnaire findings and interview notes were summarized for use in the strategy planning. Additionally, PCG conducted an environmental scan to understand the effects on Wayside of recent policy decisions and current events. As part of the business model assessment, PCG developed a comprehensive, objective Profit & Loss Statement for each business unit. PCG reviewed existing documentation of billing operations and accounts receivable management, conducted a rate review, investigated cost reporting and rate setting procedures, and conducted an analysis of Massachusetts health care trends. Finally, PCG presented strategic planning recommendations based on the assessment of the current business model and a two‐year outlook of the Massachusetts landscape. Recommendations specific to operational and financial improvements were based on the current and future market needs of the Commonwealth and Wayside’s strengths. Name of Project Dates Commonwealth of Massachusetts, Department of Early Education and Care, August 2010 – August Analysis and Design of an Early Childhood Information System (ECIS) 2011 Description of Services The Massachusetts Department of Early Education and Care (EEC) contracted with PCG to perform the initial analysis and design of an Early Childhood Information System (ECIS). When fully developed and implemented, the Massachusetts ECIS will be able to: 1) track children across ages and over time, 2) include children’s demographic data, 3) include child outcomes across developmental domains that can be linked across sectors, agencies and programs, 4) link to program and fiscal data, 5) support geographic analysis useful to communities engaged in birth through age eight strategic planning, resource management, program improvement, and accountability, and 6) provide internal and external policy makers, EEC staff, researchers, and other stakeholders with EEC data in diverse formats. June 17, 2011
Page 16
State of Minnesota Department of Administration Enterprise Data Analytics Program
As part of this engagement, PCG is partnering with Harvard University Graduate School of Education and University of Massachusetts Lowell to hold a three‐day Strategic Planning Institute of national, state, and local leaders, community providers, advocates, researchers, and other early childhood stakeholders. In consideration of the results of the Strategic Planning Institute, PCG will perform a gap analysis of required data elements and design a data model for the integration of EEC extant data and external data sources. PCG’s post‐Institute work steps will include: Analysis of EEC’s extant data and current legacy systems, data/reporting tables, data definitions, and interfaces for inclusion in the ECIS and prescription of data cleanup initiatives; Identification of electronic interfaces with external systems; Development of a single reporting data model; Development of an ECIS Vision Statement (with input from data discussions at the ECIS Strategic Planning Institute) and comprehensive list of ‘critical’ data elements required for the ECIS; Definition of ECIS specifications; and Management of interagency working group to support the ECIS.
Please find complete descriptions of each project in Section 3.1.1 in Tab 3. G. Only Applicable to Section 12 H. References from Past Data Analytics Engagements Name of Organization Name & Scope of Project Dates of Engagement Contact Name Contact E‐mail Contact Phone Number
Name of Organization Name & Scope of Project Dates of Engagement Contact Name Contact E‐mail Contact Phone Number
June 17, 2011
Pennsylvania Department of Public Welfare, Office of Children, Youth and Families Child Welfare Information System Feasibility Study and Alternatives Analysis October 2008 – October 2009 Susan Stockwell, Programmatic Project Manager
[email protected] (717) 772‐6902
Massachusetts Department of Elementary and Secondary Education Statewide Comprehensive Integrated Data Warehouse and Reporting System. Development of a Systems Inventory, Business Case and Strategic Plan April 2009 – June 2009 Rob Curtin, Manager, Data Analysis and Reporting
[email protected] (781) 338‐3000
Page 17
State of Minnesota Department of Administration Enterprise Data Analytics Program
Name of Organization Name & Scope of Project Dates of Engagement Contact Name Contact E‐mail Contact Phone Number
Wayside Youth and Family Support Network, Inc. Strategic Planning Services July 2010 – December 2010 Eric Masi, Ed.D., President and CEO
[email protected] (508) 270‐1222
Name of Organization
Massachusetts Department of Early Education and Care Analysis and Design of an Early Childhood Information System (ECIS) August 2010 – August 2011 Sherri Killins, Ed.D, Commissioner
[email protected] (617) 988‐6600
Name & Scope of Project Dates of Engagement Contact Name Contact E‐mail Contact Phone Number
I. Description of Results from Past Engagements
Pennsylvania, Department of Public Welfare, Office of Children, Youth and Families, Child Welfare Information System Feasibility Study. PCG performed a study directed at determining the feasibility and alternatives for the successful design, development and implementation of an information technology solution to support Pennsylvania’s Child Welfare programs within the Office of Children, Youth, and Families, including the Bureau of Juvenile Justice. PCG completed a feasibility study, alternatives analysis, cost benefit analysis, strategic plan, and three procurements (DDI, IV&V, and QA) that determined if implementing a Statewide Automated Child Welfare Information System (SACWIS) in Pennsylvania was feasible and the best approach for implementing an IT solution. As an additional challenge to this project, sixty-seven counties carry out the child welfare and juvenile justice programs under the supervision of OCYF, and, though many of these counties had already independently implemented case management IT solutions, OCYF has struggled with collecting the data they need for management and supervision. PCG developed a strategic plan that leveraged existing IT assets across state agencies and counties that will lead to comprehensive data collection and reporting.
Massachusetts Executive Office of Health and Human Services and Executive Office of Education, Statewide Comprehensive Integrated Data Warehouse and Reporting System. Development of a Systems Inventory, Business Case and Strategic Plan. PCG delivered a strategic implementation plan that included a number of recommendations focused on coordinating information and services for children in state care through the child welfare and juvenile justice programs, as well as recommendations to further define key requirements for a comprehensive system. Massachusetts leveraged our strategic plan and the visioning work performed by the Child and Youth Readiness Cabinet to make a successful application for Race to the Top funds.
June 17, 2011
Page 18
State of Minnesota Department of Administration Enterprise Data Analytics Program
Commonwealth of Massachusetts, Department of Early Education and Care, Analysis and Design of an Early Childhood Information System (ECIS). PCG is performing visioning, planning and design work to build a comprehensive ECIS that continues the assignment of unique child IDs, educator/staff IDs, and program IDs, and is also linked with timely, accessible, useful data to improve the effectiveness of both teaching and learning. In addition, EEC expects such an information system to increase access to secondary data from multiple sources on the well being of young children throughout the Commonwealth, and that the system will enable data exchanges with other state agencies and resources seeking early childhood data for their own research and outcome measurements. When fully developed and implemented, the Massachusetts ECIS will be able to: o Track children across ages and over time, encompassing data on home and community environments; o Include children’s demographic data (such as birth date, gender, race, ethnicity, language, disability status for children); o Include child outcomes across developmental domains that can be linked across sectors, agencies and programs (e.g., infants/toddlers, preschool, Early Intervention, family child care, etc.); o Link to program and fiscal data (e.g., teacher/workforce characteristics, program quality, and service costs); o Support geographic analysis useful to communities engaged in Birth through Age 8 strategic planning, resource management, program improvement, and accountability; and o Provide internal and external policy makers, EEC staff, researchers, and other stakeholders with EEC data in diverse formats.
Wayside Youth and Family Support Network, Inc. Strategic Planning Services, PCG provided a two-year strategic plan, with action steps, to position the agency as a leader in children’s services in the Commonwealth and improve their year-end financial status.
J. Description of Calculations of Cost Savings and Compensation from Past Engagements
Pennsylvania, Department of Public Welfare, Office of Children, Youth and Families, Child Welfare Information System Feasibility Study – PCG was paid a fixed fee for successful completion of deliverables under this contract. The fixed fee was based on estimated staff time and materials.
Massachusetts Executive Office of Health and Human Services and Executive Office of Education, Statewide Comprehensive Integrated Data Warehouse and Reporting System – Massachusetts solicited this work on a pro bono basis, and PCG completed the work with no compensation.
June 17, 2011
Page 19
State of Minnesota Department of Administration Enterprise Data Analytics Program
Commonwealth of Massachusetts, Department of Early Education and Care, Analysis and Design of an Early Childhood Information System (ECIS) – PCG was paid a fixed fee for successful completion of deliverables under this contract. The fixed fee was based on estimated staff time and materials.
Wayside Youth and Family Support Network, Inc. Strategic Planning Services – PCG was paid a fixed fee for successful completion of deliverables under this contract. The fixed fee was based on estimated staff time and materials
K. Description of Dispute Resolution in Past Engagements PCG believes that the best way to minimize disputes is through careful planning of each engagement. Planning includes a “proof of concept” or assessment process so that PCG can gain a clear understanding of the State’s data sources, procedures and historical results. This will allow PCG to gain an understanding of the resources needed to improve revenue or achieve cost reductions. It will also allow PCG to estimate the relative costs and benefits of the opportunity. The State and PCG may conclude that the opportunity is not viable. If there is an agreement to proceed with the opportunity, the State and PCG need to be clear on how results will be determined. If the goal is “cost reduction” then there must be agreement on what the costs are. For example, will reductions be based on total costs or unit costs? If the goal is “increased revenue” then PCG must be clear on what the baseline is. Whether the goal is cost reduction or revenue increases, both parties must agree on what data sources will be used for documentation. Whenever possible, PCG requests that results be tied to report generated through the State’s normal tracking systems. Understandings will be documented in a management letter that will underpin each initiative. PCG will request that appropriate State personnel sign off on each management letter. There may still be disagreements during the course of the project. Invoices will be prepared using the methodology identified in each management letter. If PCG needs to deviate from the methodology, it will notify the client as soon as possible. When PCG presents an invoice, we request that client personnel review the invoice within 30 days. If the client agency disagrees with an invoice, the agency and PCG will work to resolve the issue within 30 days. If the client agency and PCG cannot resolve the exceptions within 60 days of PCG submitting the invoice, PCG and the agency will present the invoice to the project’s Steering Committee for resolution. The Steering Committee should include designees for the directors of the client agency, the Department of Administration, and a senior analyst in the Bureau of the Budget. This group should meet with PCG and the client agency to resolve the issue within 120 of the invoice submission date. L. Description of Attribution of Cost Savings in Past Engagements
June 17, 2011
Page 20
State of Minnesota Department of Administration Enterprise Data Analytics Program
PCG has worked with many states and municipalities on revenue recovery/cost containment projects. The typical goal is to generate more revenue for our clients or save costs by minimizing expenditures or identifying alternative revenue sources. Often, clients prefer to have PCG assume the risk of producing new revenues or realizing cost savings. As a result, PCG performs many of these projects on a success or performance basis so that PCG receives a contingency fee based on the net revenue or costs saved as a result of our work. Contingency Based Fees PCG’s methodology for contingency based fees includes, but is not limited to, the following: Approach for how the revenue/savings are documented; Length of time for which PCG will be paid; and The determination of a “baseline” for measuring PCG’s performance. In addition to agreeing on the contingency fee, whether it is a percentage or a fixed amount, it is important to agree on how the revenue or savings will be determined, baseline (pre-project) resources assigned by the client, or data sources used to determine revenues or savings. PCG seeks to be paid contingency fees for twelve quarters where the first quarter is the quarter in which the state first receives revenue or savings for that particular initiative. In our experience, PCG can invest significant time and resources before there is any return on an initiative. Retroactive recoveries do not affect the twelve quarters. Approach for Revenues/Savings Documentation: How the revenue and savings will be documented is a key issue. This can be difficult and needs to be carefully considered in advance. We would expect there to be different understandings for each initiative. Both the agencies and PCG would want to know what systems will be used to identify the revenue and savings and how the revenue and savings will be calculated. We often have to utilize a variety of reports and systems that are beyond our direct control. For our work with Minnesota, PCG proposes to work closely with each agency to document how the revenues/savings will determined. Specifically, we will meet with agency staff to identify available documentation that is readily available to support the calculation of revenues and/or cost savings. This documentation may include enrollment, revenue, deposit, payment, reimbursement or other reports that relate to the activity that we are trying to improve. We will identify those documents in subsequent Management Letters to be submitted to the agencies. The reports and other supporting documentation will serves as a guide for all calculations. This is why PCG believes that we must prepare an assessment / proof of concept for each initiative. The assessments will provide the agency and PCG to evaluate the viability of the initiative and reach agreement on the administrative details needed to determine success and payments to PCG. Duration/Length of Time for Fee Calculation: The length of time for which PCG will be paid is an important consideration. It can often take a year or more before a project begins to yield new revenue or savings so PCG can commit June 17, 2011
Page 21
State of Minnesota Department of Administration Enterprise Data Analytics Program
significant resources before realizing any revenue for the client or PCG. Delays can result from the time to implement new systems and procedures, negotiation of baselines, federal approval for state plan amendments, etc. Such time lags need to be reflected in the length of the project and will be memorialized in all Management Letters submitted to the agencies. Determination of a “Baseline” for Measuring Performance: Another consideration for applying contingency fees is whether a baseline will be applied. If the State is asking PCG to improve an existing process and increase enrollment or revenues, or reduce costs, it is important to agree on the current revenue and costs if PCG is to be paid a fee based on the improvements. It is also important to agree that if PCG is measured against a baseline, then the State is also responsible for maintaining a baseline level of its own resources, or the baseline needs to be adjusted. For example, if the State assigned 10 FTEs to a project that recovered $1 million per year, then PCG should be paid a contingency fee on recoveries over $1 million if the State continues to assign 10 FTEs. If the staff is transferred, then the baseline should be adjusted. Examples of Contingency Fee Methodology: The following are examples of contingency fees from actual PCG contracts. These are provided to demonstrate how contingency fees have to be adapted to each initiative. Example
Fee X% of revenues recovered
1
2
X% of revenues recovered
3
X% of savings per successful enrollment
4
X% of net savings
June 17, 2011
Methodology Applied to all revenues recovered by clients for the program. No baseline was applied since the client had never received any revenue for this particular program. PCG fees will be calculated based upon the incremental revenues received over the identified historical baseline levels. This methodology will allow PCG to pursue all eligible accounts for revenues except those revenues pertaining to individual clients. In the event that client does not maintain its level of resource commitment, PCG and client will adjust the Fiscal Year 20xx baseline to reflect the reduced level of client resources. Therefore, if the client assigns a certain level of resources, the client is expected to continue that level of resources. The cost savings is calculated as $1,476 ($123 per member per month as determined by the client), the average annual cost savings per member for each successful determination. PCG will be paid a X% contingency fee of the average cost savings of each member successfully enrolled into the program. The fee is based on the calculated State‐estimated cost savings of $123 per member per month or $1,476 annually for each successful determination. This methodology was used for an enrollment project for a state client. PCG will only be reimbursed for “Savings” for “new” segments identified through PCG’s proprietary process that were not found Page 22
State of Minnesota Department of Administration Enterprise Data Analytics Program
through the current client identification process. “Net Savings” determined as follows: o Total savings less: The amount that client would have paid if Medicare was not the primary payer The actual amount that client paid as secondary payer for a Medicare covered services Premiums paid Note that this client had its own enrollment process in place prior to retaining PCG. PCG found that the client’s system logic was not identifying all possible enrollees. The agreement provided that the client would have 60 days to determine if new Medicaid enrollees were eligible for the program. PCG would evaluate all Medicaid enrollees following 60 days of eligibility. It was in the post‐60 day group that PCG identified the additional cases that the client had failed to identify.
June 17, 2011
Page 23
State of Minnesota Department of Administration Enterprise Data Analytics Program
June 17, 2011
Page 24
10.2 Data Collection, Cleansing, and Integration
State of Minnesota Department of Administration Enterprise Data Analytics Program
10.2 Data collection, cleansing, and integration A. Description of Relevant Experience PCG Education Overview In 1992, PCG began providing education consulting services and products to Boston Public Schools to modernize school-based Medicaid billing and to provide an easy and innovative approach for clinicians to document services. In the last 18 years, PCG Education has developed considerable expertise and has achieved numerous successes working with school districts, state departments of education, and Medicaid agencies since our initial work with Boston Public Schools. Our areas of expertise include:
Data Quality Assurance Data warehousing and metadata management Education Analytics and Data Capacity Services Special Education Program Evaluation and Audit Services Special Education and At-Risk Student Data Management Response to Intervention (RTI), Positive Behavior Supports (PBS) and Education Plans Solutions Professional Development and Coaching on district and school-level use of data Literacy and Learning Strategic Planning and School Improvement School-Based Medicaid Reimbursement Services Operations Improvement and Financial Consulting Services
Working with school districts and state departments of education ranging in size from 500 to 1.5 million students, PCG Education has the knowledge and expertise to provide a full spectrum of data-based services informed by research and the practical experience of our staff. PCG Education currently has over 800 contracts in 32 states plus Puerto Rico, the US Virgin Islands, Ontario, Quebec, Alberta, Saskatchewan, and British Columbia, Canada. In addition, PCG Education currently serves twelve state departments of education (Arizona, Colorado, Illinois, Kansas, Kentucky, Louisiana, Massachusetts, Michigan, New Hampshire, New Jersey, Tennessee, and Wisconsin) and 13 of the top 25 largest school districts in the nation. In Minnesota, PCG has worked with both the Minneapolis and St. Paul public schools systems for about ten years. PCG has focused on implementing systems for monitoring special education cases. For St. Paul, PCG has processed nearly $30 million in claims for Medicaid reimbursement for special education related services. In Minneapolis, PCG redesigned the processes teachers used for tracking special education services. This process redesign made the system more efficient, therefore saving significant time for teachers.
June 17, 2011
Page 25
State of Minnesota Department of Administration Enterprise Data Analytics Program
As a result of vast experience in the marketplace and a highly knowledgeable staff, PCG Education has a near perfect client retention rate - this is largely due to understanding clients’ needs, efficiently fulfilling our contractual obligations, and continually exceeding client expectations. PCG Education’s track record demonstrates the ability to improve outcomes for clients resulting in long-term relationships, contract extensions, and re-awards. The State Core Model was chartered by Council of Chief State School Officers (CCSSO) in 2009 to bring a coherent state education agency (SEA) perspective to national standards movements. The State Core Model v1.0 was published March 3, 2010 as part of the National Education Data Model (NEDM). To produce this version 34 state data handbooks were mapped, including Minnesota. The states were: AR, CO, CT, DE, FL, GA, IA, IL, KS, LA, MA, MD, ME, MI, MN, MT, NC, NH, NJ, NM, NV, NY, OH, OK, PA, SC, TN, TX, UT, VT, WA, WI, WV, and WY.
In the first phase of work, as part of the National Education Data Model (NEDM), state mapping involved 4 to 5 hours of research per state with publicly published materials. The meetings were single, 1-hour WebEx sessions. Deep mapping involved development of comprehensive metadata workbooks. This phase also involved mapping to 79 EDFacts file specifications. State Core Model In May 2010, as part of the Common Education Data Standards (CEDS) adoption work, with funding from the Gates Foundation, CCSSO began expanding the State Core Model to serve a common technical reference model for SEAs and to help guide development of their P-20 State Longitudinal Data System (SLDS). The State Core Model is a platform that enables comparability and interoperability between states, reduced Federal collection burdens, and increased research, analysis, and intervention capabilities. The State Core Model provides many benefits to participating states, such as: Comparability and interoperability with other states Discovery of gaps in individual state longitudinal data systems Extension of states’ models into early childhood, post-secondary education, and the workforce Reduction in collection burdens Greater potential for research, analysis, and targeted interventions The cycle can be depicted as follows:
June 17, 2011
Page 26
State of Minnesota Department of Administration Enterprise Data Analytics Program
Data Mapping Steps Step 1: Contact Step 2: Discovery Step 3: Mapping Step 4: Mapping Review (Iterative) Step 5: Map and Gap Step 6: Final Presentation State Core Model Review Steps Step 7: Set-up Specific Instance in Sandbox Step 8: Register for Friday technical reviews Step 9: Customize State Core ODS Step 10: Download DDL So far, in the second phase of the work, as part of CEDS, 16 states have begun adoption through these ten steps: AR, AZ, CA, IA, IL, KS, MI, NC, ND, NJ, OR, RI, UT, WA, WY, VA. B. Description of Proposed Data Analytics Services PCG Data Quality Assurance
PCG specializes in data quality assurance. We provide special education compliance software and Medicaid reimbursement services to more school districts than any other company. Over the last ten years, we have helped secure over $2B in Medicaid reimbursements. PCG’s approach to data warehouse quality assurance begins with a decomposition of the system into testable components:
Data Sources and Extract, Transform, Load (ETL) to Staging ETL to Operational Data Store (ODS) and Entity-Attribute-Value (EAV) ETL to Reporting Data Store (RDS) and DataMarts Reports and Cubes
These components are validated using SQL and manual testing procedures. The testing procedure used is to describe positive and negative scenarios where a user is (or is not) able to get access or upload (in case of files) different sources including File Validations or Data Source Connection Strings. Usually, this validation is performed as part of the ETL’s testing procedure. Once one or different testable components passes validation (Unit and Integration Testing), they are cross checked to assure integrity of the data (System Testing). Extract, Transform, and Load (ETL) Testing
June 17, 2011
Page 28
State of Minnesota Department of Administration Enterprise Data Analytics Program
The testing process of validating ETLs consists mainly of assuring that Source and Destination data inside the data warehouse matches according to technical or business specifications. The tasks involved are:
Understand the ETL Data Source subset where data will be extracted from Understand all transformation rules that need to take place in the ETL Assess on risks and testable conditions Create Positive and Negative data test scenarios to validate data elements and the Transformation Rules o Data Elements Positive Scenarios: Acceptable values per data element or group of data elements including in-boundary conditions Negative Scenarios: Out of boundaries, Nulls (If apply), or values that should not enter the data warehouse o Transformation Rule Positive Scenarios: Acceptable data source values are transformed into the correct ones in the data warehouse Negative Scenarios: Unacceptable or unknown values are treated correspondingly (Accepted or Rejected) to the rule defined Validate totals for different data conditions (Positive and Negative scenarios) Execute and monitor the performance of ETLs Validate testing scenarios
This activity requires manual testing using SQL and is achieved by managing the data at the source level and predicting its expected result. The validation of Business Rules is mainly treated as the case of validating Transformation Rules. The Positive Scenario is defined and then Negative Scenarios are derived from it in order to make sure all values are treated (Accepted or Rejected) in the data warehouse. Reports and Cubes Testing Reports and cubes are validated on the following aspects: Cubes Metadata: Appropriate Name, Description, Language, Spelling, Order Dimensions: All dimensions are displayed: Dimension metadata validation, Hierarchies, Hierarchies’ metadata validation, Members, Members’ metadata validation Measures: For each measure the metadata is validated as well as the calculated value displayed and the value format. Cross Dimension Checking: In this test, the tester picks different combination of dimensions, populates measures and validates its corresponding calculated values. Assuming that all possible combinations of dimensions are impossible to achieve June 17, 2011
Page 29
State of Minnesota Department of Administration Enterprise Data Analytics Program
(Time Restriction), the tester will focus first on the common business scenarios and then investigate unusual combinations. Security: Based on the different security roles implemented, the tester will access the cubes with the same role to make sure that role only has access to what is defined for that particular role.
Reports
Layout: Anything related to the format of the report, e.g., Structure, Language, Spelling, Colors, Font type, Font Size, Paragraph, etc. Data Content: Anything related to Titles, Headers, Parameters, Graphics, Tables, Totals, Grand Totals, Data Values. Functionalities: Anything related to Sorting, Filters, Drill-thru scenarios, Printing (Printing, Preview) and Export Functionalities to different formats. Security: Based on the different security roles implemented, the tester will access the reports with the same role to make sure that role only has access to what is defined for that particular role.
Performance/Load Testing Many times performance and load tests are treated as a single topic. However, there is a slight difference between them. A Performance Test is the activity that will answer the question of how responsive the system is in time manners (time sensitive requirements) in relation to usual workloads. On the other hand, the Load Test is the activity of evaluating the target of the test exceeding the maximum workloads and behaving acceptably. Both measure and evaluate response time, transaction rates and other time sensitive requirements in conjunction with resource utilization. Performance tests focus the attention on how many requests per hour the system can handle when managing a normal specific business scenario (i.e., regular workload of the application). For example: We want the response time to be less than 2 seconds when running reports. On the other hand, the load test will indicate if the system is capable of managing requests under a specific load request. In order to perform a Performance or Load test, the following activities take place to create the performance/load test environment: 1. Define the Performance/Load Strategy Test Plan. Document topics such as: Performance objectives: The reasons why the performance or load tests have to be performed Scope: The system under Performance or Load Test Performance or load requirements details: This section includes definitions of how responsive should the system be under a certain scenario. For example, how many requests per hour can the system handle under a specific load? Entry/exit criteria: Which conditions the activity should meet in order to be able to run the tests or exit them? Environments needs: What is the environment where the test will take place? Mimics production? June 17, 2011
Page 30
State of Minnesota Department of Administration Enterprise Data Analytics Program
Risks: Which are the identifiable risks associated with the test and how they will be mitigated? Business scenarios: Which business scenarios should be included in order to analyze the performance or load of the application?
2. Configure the environment and the performance tool. This activity consists in configuring the tool and the environment in order to execute and meet the performance and load objectives and requirements. Tasks involved include: Creation and configuration of the Environment Data Population Configuration of the performance tool to access the environment Recording of the scenarios: record the scenarios that will be repeated upon time execution Scheduling of test scenarios: When and how the load will be injected in the test run. 3. Execute the performance test. Execute the test scenarios while monitoring resources and assessing the status of the test when needed. 4. Analyze and exhibit test results. Gather, Analyze, and expose the performance results based on the objectives and requirements that should be met. Results include information about Performance and Load goals as well as details of the resource utilization. C. Description of Necessary Tools, Hardware, and Software Excel 2007 and ER Studio are needed to support these services. D. Proposed Level of State Involvement At this time, it is not possible to estimate resource levels. Part of the PCG initiative assessment/“proof of concept” process is to determine the current level of resources the State is using as well as what resources will be needed. These assessments will involve interviews with agency personnel so that we understand current operations. In most cases, there are two significant resource commitments PCG will request during a project:
Access to computer files – PCG often needs to download large databases so that we can conduct our analyses. We prefer to have on-line file access but understand that many clients prefer to provide data upon request. We will identify our data requirements and existing reports or files that may have that data available. Update files are often needed on a regular basis. PCG may also request the ability to post edits to the State’s files but we can provide changes for posting by State personnel.
June 17, 2011
Page 31
State of Minnesota Department of Administration Enterprise Data Analytics Program
Management commitment – PCG will request the involvement of senior agency personnel to monitor each project, be available for consultation, and to help ensure that agency personnel understand that PCG’s involvement is a priority.
When an agency is already involved in an initiative, PCG will base our resource plans on supplementing the State’s existing resources. Our goal is to expand on what is already taken place so that there is an increase in revenues or savings. In some cases, our methodologies and tools will allow the State to assign existing staff to other duties so there can be a decrease in resource commitment by the State. Each initiative is unique and the impact on State resources will be detailed in a Management Letter. The Management Letter will address PCG’s analysis of the overall return on investment (ROI) on the agency. E. Proposed Personnel Respondents to this RFP must be capable of providing internal employees who have successfully completed work in the identified subject areas. Such work must have been completed within the last three years. The following personnel are anticipated to provide professional services under this program: Personnel Name, Title
Work Completed in Service Category Dates CCSSO State Core Model 2010 – Present Greg Nadeau, Manager Illinois Longitudinal Data System – Data Architecture July 2010 – June 2011 Narrative of Qualifications in Service Category Greg Nadeau is a Manager at PCG. He has over seventeen years of experience leading state longitudinal data systems, eight on the public side as Chief Information Officer of the Massachusetts Department of Education, and ten in the private sector, consulting with private companies, education organizations and states. Mr. Nadeau served on the Board of Directors of the School Interoperability Framework Association (SIFA) and as an expert consultant to the Council of Chief State School Officers (CCSSO). In 2001, Mr. Nadeau created and led the US Open e‐Learning Consortium, a 14‐state USED‐ funded project. The primary objective of the consortium was to harvest released high‐stakes, assessment items to create a pool for low‐stakes interim assessments. He currently leads CCSSO’s State Core Working Group and serves as Project Director for the Illinois Data Warehouse project. Mr. Nadeau has led statewide data warehouse implementation studies for Massachusetts, Rhode Island, Maine, Delaware, and Illinois. Through his work with USED and CCSSO he worked with most all state education agencies and visited over half. He brings unique experience to the tasks called for in this project. More generally, Mr. Nadeau has led technical teams working with state education agencies on a host of other projects, including teacher portals, teacher licensure, student technical training, statewide June 17, 2011
Page 32
State of Minnesota Department of Administration Enterprise Data Analytics Program
networks, data models, instructional models, e‐learning, and strategic planning. Mr. Nadeau’s approach to project management is tightly tailored to the needs and context of state agencies. In addition, he brings a strong research and policy background with experience managing teams to identify and advance a public policy agenda. Personnel Name, Title
Work Completed in Service Category Dates CCSSO State Core Model 2010 – Present Aaron Harte, Senior Database Illinois Longitudinal Data System – Data Architect Architecture July 2010 – June 2011 Narrative of Qualifications in Service Category Aaron Harte is a Senior Database Architect at Public Consulting Group. Mr. Harte has 17 years of experience as an IT Professional and 10 years as a data warehousing architect. Mr. Harte has designed and delivered educational data systems to the State of New Jersey and State of Tennessee. Mr. Harte is a Microsoft IT professional and prior to starting his IT career he served 8 years in the Army as a special weapons technician and combat medic. He has implemented data warehousing solutions for the manufacturing, investment and healthcare industries prior to joining PCG, most recently designing the State Core Model in conjunction with CCSSO. Personnel Name, Title
Work Completed in Service Category Dates CCSSO State Core Model 2010 – Present Anthea Medyn, Consultant Illinois Longitudinal Data System – Data Architecture July 2010 – June 2011 Narrative of Qualifications in Service Category Anthea Medyn is a Consultant at Public Consulting Group in Boston. She is working with the Illinois State Board of Education (ISBE) to develop data architecture for the state’s longitudinal data system. This process involves interviewing ISBE staff, discovering and documenting sources, and mapping to the National Education Data Model. Ms. Medyn has worked with the National Education Data Model on a gap analysis to determine ability to answer prioritized research and policy questions and for the Council of Chief State School Officers in the development of the State Core Model. Prior to joining PCG, she worked as a Research Associate with the American Institutes for Research providing technical assistance to states and districts in the areas of special education and response to intervention. Personnel Name, Title Work Completed in Service Category Dates Amy Seibel, Business Analyst CCSSO State Core Model 2010 – Present Narrative of Qualifications in Service Category Ms. Seibel, a Business Analyst with Public Consulting Group, has a background in project management, metadata management, and communication. In her current role as project manager for the State Core Model, Ms. Seibel coordinates approximately 300 stakeholders and ensures that daily operations of the project run smoothly. Ms. Seibel has experience with the financial, legal, supervisory, and internal and external communication aspects of project management. She also facilitates weekly meetings with stakeholders. In addition, through her work with the State Core Model, Ms. Seibel helped develop a metadata tool that will assist states in meeting federal reporting June 17, 2011
Page 33
State of Minnesota Department of Administration Enterprise Data Analytics Program
requirements. She also researched the special education process and created the majority of the State Core Model’s special education‐related metadata. Ms. Seibel holds a Bachelor of Arts degree from Brown University.
Please reference detailed resumes for the above personnel in Section 3.1.2 in Tab 3. F. Description of Past Data Analytics Engagements Name of Project Dates CCSSO State Core Model 2010‐Present Description of Services PCG was awarded a contract by the Council of Chief State School Officers (CCSSO) to develop the State Core Model, a common technical reference model for states implementing P20 state longitudinal data systems (SLDSs). The project was launched by CCSSO as part of the Common Education Data Standards (CEDS) adoption work. The purpose of the State Core Model is to serve as a set of best practices and standard views to support efficiency, maturity, comparability, and interoperability of SLDSs. The State Core Model enables states to align across multiple agencies and streamline data management and reporting. The Model enables comparability of data between states from early childhood through K‐ 12, post‐secondary, and workforce. This allows for improved research and analytic capabilities and targeted intervention. PCG coordinates and facilitates working meetings with different groups of internal and external stakeholders each week. Specifically, a total of five different stakeholder groups are involved: individuals who specialize in early childhood, K12, postsecondary education, workforce, and technical architecture, respectively. Moreover, each stakeholder group includes a diverse group of internal and external members, ranging from members of the project team to employees of state and federal agencies, nonprofits, universities, vendors, etc. Deliverables achieved include: Map CEDS TWIG data elements to State Core, add justifications, share through EIMAC with initiation to state to do deep mapping Complete NCES Forum Question decomposition and map to State Core Expand State Core to incorporate Special Education, RTI, PBIS, DEWIS, unified chart of accounts, early childhood, post‐secondary education, and workforce Develop a logical relational model and rigorous architecture description Add Common Core State Standards metadata and data Do deep mapping with 5‐10 states student information systems Furnish a final written report fully and completely detailing the use of contract funds (including salaries, travel, supplies, etc.) and the project activities and accomplishments
Name of Project Illinois Longitudinal Data System – Data Architecture Description of Services
June 17, 2011
Dates July 2010 – June 2011
Page 34
State of Minnesota Department of Administration Enterprise Data Analytics Program
The Illinois State Board of Education (ISBE), a recipient of three State Longitudinal Data System grants funded by the National Center for Education Statistics, contracted with PCG to develop enterprise‐ wide data architecture. Illinois had begun to deploy the building blocks of a statewide longitudinal data system to analyze education data to understand the variables that produce successful academic outcomes for students and lead to successful employment opportunities in the State and elsewhere; and reduce data redundancy and improve data quality. The Illinois Longitudinal Data System Project connects ISBE staff with PCG Education consultants to create a data dictionary, conduct a data gap analysis, and provide metadata maintenance PCG uses an Excel‐based data audit system to identify people involved in different source systems; conduct interviews and document data sources; and document data items, indicators and gaps.
G. Only applicable to Section 12 H. References from Past Data Analytics Engagements Name of Organization Name & Scope of Project Dates of Engagement Contact Name Contact E‐mail Contact Phone Number Name of Organization Name & Scope of Project Dates of Engagement Contact Name Contact E‐mail Contact Phone Number
Council of Chief State School Officers (CCSSO) State Core Model 2010 – present Melissa Johnston
[email protected] (202) 326‐8697 Illinois State Board of Education (ISBE) Illinois Longitudinal Data System – Data Architecture July 2010 – June 2011 Connie J. Wise, PhD
[email protected] (217) 782‐0354
I. Description of Results from Past Engagements So far, in the second phase of the work, as part of the Common Education Data Standards (CEDS) work, 16 states have begun adoption through these ten steps: AR, AZ, CA, IA, IL, KS, MI, NC, ND, NJ, OR, RI, UT, WA, WY, VA. J. Description of Calculations of Cost Savings and Compensation from Past Engagements N/A K. Description of Dispute Resolution in Past Engagements PCG has not encountered any disagreements as to savings generated or fees owed for any engagements. PCG believes that the best way to minimize disputes is through careful planning of each engagement. Planning includes a “proof of concept” or assessment process so that PCG can June 17, 2011
Page 35
State of Minnesota Department of Administration Enterprise Data Analytics Program
gain a clear understanding of the State’s data sources, procedures and historical results. This will allow PCG to gain an understanding of the resources needed to improve revenue or achieve cost reductions. It will also allow PCG to estimate the relative costs and benefits of the opportunity. The State and PCG may conclude that the opportunity is not viable. If there is an agreement to proceed with the opportunity, the State and PCG need to be clear on how results will be determined. If the goal is “cost reduction” then there must be agreement on what the costs are. For example, will reductions be based on total costs or unit costs? If the goal is “increased revenue” then PCG must be clear on what the baseline is. Whether the goal is cost reduction or revenue increases, both parties must agree on what data sources will be used for documentation. Whenever possible, PCG requests that results be tied to report generated through the State’s normal tracking systems. Understandings will be documented in a management letter that will underpin each initiative. PCG will request that appropriate State personnel sign off on each management letter. There may still be disagreements during the course of the project. Invoices will be prepared using the methodology identified in each management letter. If PCG needs to deviate from the methodology, it will notify the client as soon as possible. When PCG presents an invoice, we request that client personnel review the invoice within 30 days. If the client agency disagrees with an invoice, the agency and PCG will work to resolve the issue within 30 days. If the client agency and PCG cannot resolve the exceptions within 60 days of PCG submitting the invoice, PCG and the agency will present the invoice to the project’s Steering Committee for resolution. The Steering Committee should include designees for the directors of the client agency, the Department of Administration, and a senior analyst in the Bureau of the Budget. This group should meet with PCG and the client agency to resolve the issue within 120 of the invoice submission date. L. Description of Attribution of Cost Savings in Past Engagements PCG has worked with many states and municipalities on revenue recovery/cost containment projects. The typical goal is to generate more revenue for our clients or save costs by minimizing expenditures or identifying alternative revenue sources. Often, clients prefer to have PCG assume the risk of producing new revenues or realizing cost savings. As a result, PCG performs many of these projects on a success or performance basis so that PCG receives a contingency fee based on the net revenue or costs saved as a result of our work. Contingency Based Fees PCG’s methodology for contingency based fees includes, but is not limited to, the following: Approach for how the revenue/savings are documented; Length of time for which PCG will be paid; and The determination of a “baseline” for measuring PCG’s performance. In addition to agreeing on the contingency fee, whether it is a percentage or a fixed amount, it is important to agree on how the revenue or savings will be determined, baseline (pre-project) June 17, 2011
Page 36
State of Minnesota Department of Administration Enterprise Data Analytics Program
resources assigned by the client, or data sources used to determine revenues or savings. PCG seeks to be paid contingency fees for twelve quarters where the first quarter is the quarter in which the state first receives revenue or savings for that particular initiative. In our experience, PCG can invest significant time and resources before there is any return on an initiative. Retroactive recoveries do not affect the twelve quarters. Approach for Revenues/Savings Documentation: How the revenue and savings will be documented is a key issue. This can be difficult and needs to be carefully considered in advance. We would expect there to be different understandings for each initiative. Both the agencies and PCG would want to know what systems will be used to identify the revenue and savings and how the revenue and savings will be calculated. We often have to utilize a variety of reports and systems that are beyond our direct control. For our work with Minnesota, PCG proposes to work closely with each agency to document how the revenues/savings will determined. Specifically, we will meet with agency staff to identify available documentation that is readily available to support the calculation of revenues and/or cost savings. This documentation may include enrollment, revenue, deposit, payment, reimbursement or other reports that relate to the activity that we are trying to improve. We will identify those documents in subsequent Management Letters to be submitted to the agencies. The reports and other supporting documentation will serves as a guide for all calculations. This is why PCG believes that we must prepare an assessment / proof of concept for each initiative. The assessments will provide the agency and PCG to evaluate the viability of the initiative and reach agreement on the administrative details needed to determine success and payments to PCG. Duration/Length of Time for Fee Calculation: The length of time for which PCG will be paid is an important consideration. It can often take a year or more before a project begins to yield new revenue or savings so PCG can commit significant resources before realizing any revenue for the client or PCG. Delays can result from the time to implement new systems and procedures, negotiation of baselines, federal approval for state plan amendments, etc. Such time lags need to be reflected in the length of the project and will be memorialized in all Management Letters submitted to the agencies. Determination of a “Baseline” for Measuring Performance: Another consideration for applying contingency fees is whether a baseline will be applied. If the State is asking PCG to improve an existing process and increase enrollment or revenues, or reduce costs, it is important to agree on the current revenue and costs if PCG is to be paid a fee based on the improvements. It is also important to agree that if PCG is measured against a baseline, then the State is also responsible for maintaining a baseline level of its own resources, or the baseline needs to be adjusted. For example, if the State assigned 10 FTEs to a project that recovered $1 million per year, then PCG should be paid a contingency fee on recoveries over $1 million if the State continues to assign 10 FTEs. If the staff is transferred, then the baseline should be adjusted. June 17, 2011
Page 37
State of Minnesota Department of Administration Enterprise Data Analytics Program
Examples of Contingency Fee Methodology: The following are examples of contingency fees from actual PCG contracts. These are provided to demonstrate how contingency fees have to be adapted to each initiative. Example Fee X% of 1 revenues recovered
2
X% of revenues recovered
3
X% of savings per successful enrollment
4
X% of net savings
June 17, 2011
Methodology Applied to all revenues recovered by clients for the program. No baseline was applied since the client had never received any revenue for this particular program. PCG fees will be calculated based upon the incremental revenues received over the identified historical baseline levels. This methodology will allow PCG to pursue all eligible accounts for revenues except those revenues pertaining to individual clients. In the event that client does not maintain its level of resource commitment, PCG and client will adjust the Fiscal Year 20xx baseline to reflect the reduced level of client resources. Therefore, if the client assigns a certain level of resources, the client is expected to continue that level of resources. The cost savings is calculated as $1,476 ($123 per member per month as determined by the client), the average annual cost savings per member for each successful determination. PCG will be paid a X% contingency fee of the average cost savings of each member successfully enrolled into the program. The fee is based on the calculated State‐estimated cost savings of $123 per member per month or $1,476 annually for each successful determination. This methodology was used for an enrollment project for a state client. PCG will only be reimbursed for “Savings” for “new” segments identified through PCG’s proprietary process that were not found through the current client identification process. “Net Savings” determined as follows: o Total savings less: The amount that client would have paid if Medicare was not the primary payer The actual amount that client paid as secondary payer for a Medicare covered services Premiums paid Note that this client had its own enrollment process in place prior to retaining PCG. PCG found that the client’s system logic was not identifying all possible enrollees. The agreement provided that the client would have 60 days to determine if new Medicaid enrollees were eligible for the program. PCG would evaluate all Medicaid enrollees following 60 days of eligibility. It was in the post‐60 day group that PCG identified the additional cases that the client had failed to identify.
Page 38
