Why Reporting in the LRS?

| Comments

All the standalone LRSs available at the moment build in some form of additional reporting beyond pure Experience API methods. This makes sense: it is hard to build reporting tools without seeing real Experience API data collected in an LRS, but an LRS with no way to work with the data inside it would be quite lackluster. If LRSs waited for basic reporting tools to exist outside the LRS, they’d be doing their customers a disservice, giving them a powerful way to aggregate data, but no good way to look into that data. LRSs pretty much needed to provide some basic reporting from the get go.

But I’m going to argue there’s a larger reason. I’m going to make the case that, in the long run, we’re going to see lots of LRS + reporting hybrids, for solid reasons. Then, I’m going to talk about why this doesn’t break the big interoperability goals of xAPI.

Of course, I’m not talking about reporting alone here, but any extra functionality that provide new ways to interact with the underlying data — anything that’s not just LRS management capabilities. There’s some resistance to this concept, and understandable resistance: LMSs lock up data and content to absurd degrees. It makes complete sense to steer far clear of any possible perpetuation of that dystopia.

First, groundwork for why LRS + reporting hybrids will happen: the Experience API’s ability to filter experiential data for retrieval focuses on the “data synchronization” problem, and doesn’t try to solve every retrieval need. That’s good, because it means there’s a focused way of thinking about how the standard is built out. But, it also means it won’t always be easy to narrow in on exactly the data used for an analysis with those retrieval methods, even if the data needed is relatively small, much less when the data needed is larger. That’s all for the groundwork, now on to imagining.

Imagine you’re building a reporting tool for the Experience API. You start by letting the user select a person from the employee database, then doing some crunching over all of that person’s data. Not too bad, even a prolific person won’t have too much Experience API data. So, for most people your tool can load the data in less than a second, for some another few seconds, and for a tiny number of people maybe a few tens of seconds at worst. Acceptable.

Then, you see needs of your stakeholders that drive you to want…

  • other things that happened around a particular slice of time, or
  • comparison to averages among many more employees, or
  • a historical baseline, or
  • faster responses so you can display parts of the results in the CRM to create feedback loops

Any of those needs, among many more, leads to a need to move the data to your reporting tool behind the scenes, beforehand, and possibly process some of it in advance there.

In some cases that’s just going to be a matter of a data warehouse plus a business intelligence (BI) tool — many questions L&D wants answered fit the BI paradigm well, though working with time series data can make it harder to frame queries.

But, like every other business function, L&D is going to have specialized reporting needs, and lots of them. Sales, marketing, operations, HR, purchasing, R&D, and so on, all have numerous specialized reporting tools to reflect their specialized needs, even in businesses with stellar data warehousing and business intelligence practices.

L&D’s specialized reporting tools are frequently going to need to synchronize data out of the LRS, for the reasons given above. They’ll need standardized ways to ingest data, and to investigate what data’s present. Even to get raw data out, to check it.

Luckily, there’s a standard for doing that with L&D data, called the Experience API. Implement all those things, and a larger data synchronizing reporting tool is suddenly an LRS.

That’s why I expect reporting-LRS hybrids to be common. And, since existing LRSs are already the places where data is local, I expect their early reporting capabilities to grow.

There’s a line of argument out there, that relying too much on the built in reporting capabilities of an LRS means you’re tied to that LRS. Yes, but no more than you’d be tied to any reporting tool you adopted. Say you decide your LRS is doing a great job (or at least, a hard to replace job) of reporting, but doesn’t scale like you need: get a new LRS, synchronizing the data into your old LRS in a more smoothed out way, and continue to do reporting off the old LRS/reporting system. Is this effortless? No, but it won’t require more effort, and likely will require less, than setting up data synchronization to a non-LRS reporting tool.

What about querying the LRS database directly? That’s a far more dangerous approach. First, LRSs can change database structures on upgrades. Suddenly that reporting code you’ve written might stop working, or become slower, or worst, look like it works but give the wrong answers. Second, it ties you just as much to the LRS in question, but now your reporting code is far more likely to cause performance issues than if it was mediated through tested functionality.

One of the first blog posts I wrote about the Experience API, I believe at the first ever “Alley”, talked about One Possible Future:

“One possible future features symbiotic specialization. A Learning Record Store has peerless insight into its internals; no tool without direct access to the data will be as fast or as capable. However, the [Experience] API makes it easy for Learning Record Stores to synchronize their data. An organization with learning needs can easily add a further LRS in order to support new insights, connections, and other capabilities.

In the future, an organization might have one LRS aimed at evaluating instructors, and another that uses certification data to monitor compliance, and another that identifies learner problem areas. Adding a new capability would be a matter of signing up or installing the LRS, then pointing it at an existing LRS and awaiting synchronization. Organization needs will be met modularly, without complicated configuration or data translation.”

I still see that possible future, and LRS + reporting hybrids are an important part of it.