In 2005, Newsweek magazine ranked several Charlotte-Mecklenburg Schools (CMS) high schools among the nation’s top 100. The CMS Strategic Plan 2010 codifies the district’s dedication to boosting student performance, laying out seven goals to drive improvements over the next three years. At the top of the list: high academic achievement, effective educators, and adequate resources and facilities.
But maintaining its trajectory of exceptional student achievement required more than simply defining goals. The district needed a way to track the efficacy of its programs and initiatives – so that it could identify which succeeded and which failed to produce desired results. Rather than depending on ad-hoc reports manually produced by the central office a few times a year, the Board of Education needed a way to continually monitor and assess programs in real time. Then, it could make informed, timely decisions about continuing only those programs that showed a measurable, positive effect on student, teacher, and the district’s operational performance.
With a business intelligence solution based on Microsoft® Office PerformancePoint™ Server 2007, CMS has completely transformed the way that it collects, monitors, and analyzes data on student, teacher, and operational performance. Easy-to-understand dashboards visually present the district’s results based on key performance indicators (KPIs) that tie directly back to the goals identified in the CMS Strategic Plan. Now, the Board of Education can get a real-time snapshot of whether CMS is likely to meet, exceed, or fall short of its performance goals by 2010 – and make timely adjustments to stay on track.
Improving Board Visibility
CMS doesn’t equivocate when it comes to improving performance. The district’s Strategic Plan 2010 outlines seven high-level goals as well as objectives and strategies for reaching each goal. Also included are key performance measures.
“The overarching purpose [is] educating our students for the future,” the plan states. “We must prepare them to reach beyond today’s scores, today’s tests, and today’s challenges. If we do not, we will have sentenced our children to a lifetime of the only kind of poverty that is permanent: a lack of opportunity.”
The targets CMS has set are ambitious. For example, the goal of high academic achievement is 95% of students achieving at or above standard on reading End-of-Grade tests in grades 3-8 and 75% of students meeting or exceeding the national average on the SAT.
In the midst of rapid enrolment growth, meeting objectives like these by 2010 is even more challenging. CMS must continuously monitor and assess programs for their effectiveness. “To succeed, CMS must let go of projects and programs that do not increase student achievement in measurable ways,” the plan states. “CMS must support only programs proven to work.”
But assessing programs hasn’t been easy, as CMS relied on disparate, disconnected systems to store data. CMS had a state-mandated student information system, separate HR and financial systems, and an insufficiently maintained home-grown data warehouse – and none of these systems were integrated or provided any real visibility into program effectiveness. With this outdated infrastructure, the CMS central office wasted staff time and effort trying to keep the Board of Education and other stakeholders apprised of progress. “We had a whole team that did nothing but field ad-hoc data requests manually,” says Jonathan Raymond, Chief Accountability Officer at CMS. “We had a lot of data, but it wasn’t available in a way that could inform decisions and drive policy. We needed to automate this information so people could self-service their data requests – and we could redeploy our team to more valuable pursuits.”
Data integrity was also a real problem. “One person would produce a report and disseminate it in Excel or as a PDF,” explains David Fitzgerald, Education Practice Manager at Mariner, the IT consultancy helping CMS implement its performance management solution. “And that person might not maintain the integrity of the data on a regular basis. The CMS’s data warehouse reflected this lack of rigor and oversight. Over the same historical time period, values for the same metric would change for from one day to the next.”
In addition, the CMS Board of Education received data sporadically and in a format that was difficult to consume. “We’d dump data on them once or twice a year,” says Raymond. “And they’d just drown in it.”
Frustrated by the lack of real-time visibility into the district’s performance, the Board requested a tool that could give them a snapshot, at any point in time, of whether the district was on track.
“They asked for some basic indicators and we decided that we wanted to create a dashboard that went deeper,” says Raymond. “We needed the solution to not only align with our strategic plan, but allow us to track more KPIs. We wanted to be able to easily report: how students were tracking toward proficiency; how we trend on gap issues along economic status, race, and other aspects; whether school operations are efficient and construction projects on time and on budget. We wanted a dashboard that would help us improve our performance along many dimensions and could grow with us in the future.”
Ready for Growth
The dashboard required by the Board of Education was only the first step. In an effort to increase transparency district-wide and throughout the community, CMS plans to deploy its performance management tool internally and out to the district’s constituents – close to a million community members. These ambitious plans called for a solution that was scalable and affordable. Microsoft PerformancePoint Server delivered on both counts.
“Many BI solutions are very expensive to deploy to large communities beyond the firewall,” says Fitzgerald. “But with Microsoft’s pricing and modular components, it’ll be cost-effective and relatively easy for CMS to scale its program.”
Microsoft BI Improves Data Access
The performance management solution CMS implemented gives the district the data it needs to track progress towards its 2010 goals – and make better decisions about which initiatives to pursue. The solution leverages Microsoft SQL Server 2005, and uses Windows SharePoint Server 3.0 to deliver PerformancePoint Server dashboards and scorecards to users. It runs on three servers: one for database cube analysis, another runs PerformancePoint Server and Windows SharePoint, and the third houses a 50-gigabyte data warehouse.
Using PerformancePoint Server Monitoring and Analysis components, CMS embedded all scorecards, dashboards, and reports into SharePoint. Mariner, CMS’s consulting partner, designed a metrics repository using a relational database and cube. The repository pulls data from CMS’s data warehouse, Excel spreadsheets, departmental and organizational systems and more.
Implementing the PerformancePoint Server solution not only helped CMS manage and access its data more effectively, it pushed the district to re-evaluate the way it captured data. While CMS was clear on the metrics it needed to measure performance towards goals, it didn’t always have the business processes in place to collect the data. “For example, the Board wanted to know how often school buses arrived on time,” explains Fitzgerald. “But we discovered that CMS surveys buses one week a year – and only 15% of schools complete the surveys. Departments are now working to develop a much more consistent, defined process for reporting data.”
Measuring Progress
KPIs tie back to the objectives outlined in the district’s 2010 plan – and are measured at different points throughout the year. For example, end-of-grade test scores are done once a year, while other KPIs – such as school audits – are measured on a semester or quarter basis. CMS even monitors a few KPIs daily, such as emergency work order response time.
Perhaps some of the most helpful KPIs that CMS measures are proficiency gap KPIs that apply to all end-of-grade test scores in reading, math, writing, and science. These KPIs allow CMS to compare scores based on economic and ethnic disparity – and analyze gaps in a number of ways. “With PerformancePoint Server, we can view the gap between Hispanic kids in free and reduced lunch programs and those Hispanic kids that live in affluent areas,” says Raymond. “Or we can see how the end-of-grade reading test scores compare for kids overall in the district to those at FOCUS schools, which serve kids who need individual attention and extra support.”
One of the most helpful tools PerformancePoint Server delivers is a progress indicator that maps to the 2010 CMS strategic goals. “CMS has a visual map of whether it will meet, exceed, or fall short of goals based on the current trend,” says Fitzgerald. “An indicator can be red even though it’s meeting the goal this year. It means the trend is trailing way from the target – so that by 2010, there’ll be a shortfall. It’s a good reminder to always keep your eye on the ball.”
Achieving Value
Using the performance management dashboard, CMS can now ask the really hard questions that drive student achievement. Collaborating and working together to improve performance becomes a way of life for all of the stakeholders, and progress indicators in the dashboards enable everyone to work together to achieve the goals spelled out in the 2010 plan. Furthermore, consistency and trust in the data lead to greater credibility of the improvement efforts and those leading the initiative.
Microsoft BI Earns Top Grade
CMS is already experiencing many benefits as a result of its Microsoft BI implementation. Raymond estimates his that staff will spend 45% less time dealing with data requests and therefore can devote more time to other valuable activities.
“We’re going to be driving better decisions with our richer data,” continues Raymond. “We are on the leading edge in education. We’re using data to drive better decisions and create policy – increasing transparency and pushing ourselves to do more for our kids. Because we’re seeing how we’re performing in real time, we can make adjustments as we go and ensure student success.”
