At Saltbox, we’re always excited about DevLearn, and this year was exceptionally thrilling for us and the xAPI community. This year’s DevLearn took place at the MGM Grand in Las Vegas, which is fitting because we made sure that what happened in Vegas was tracked using the Experience API.
There was a sold-out crowd at xAPI camp, led by Megan Bowe and Aaron Silvers from Making Better, where learning leaders presented big challenges, ideas, tools, tips, and lots more. Aaron and Megan also introduced xAPI Quarterly, a new site with articles about Experience API topics. One of the most interesting issues—certainly one near to our heart—was the future of the Experience API and DISC, a newly announced consortium to move the specification forward. If you missed this one, don’t worry, it’s happening again and again and again, so get involved.
Russell Duhon, CTO at Saltbox, presented at xAPI Camp on “5 Projects Outside the Authoring Toolbox” using xAPI that L&D can pick up and run with today with the help of 1 developer for a month of two. He even built some awesome highly anticipated prototypes to help people get started. See the presentation, related resources, and prototypes.
The Morning Buzzzzzzzz
For the first time, we led a “Morning Buzz” discussion about measuring the impact of training on business outcomes. We learned quite a bit about the challenges and current approaches to this topic. Over 40 people attended this “intimate” discussion at 8:30 am, coffees in hand!
Here were a few of the topics covered during the discussion:
- What does the notion of “business impact” mean to you in your specific context?
- How do you measure business impact today?
- How is business impact different than ROI? Or is it?
- Are there any specific tools/methodologies you use or processes you follow to measure business impact today?
- What are the challenges you face or have faced in measuring business impact?
Here a few interesting talking points and ideas from this discussion:
- It’s hard to get senior leadership to look at L&D/HR as a strategic partner. Most times, L&D is called in too late to make an impact. Leadership changes compound this issue.
- Presenting evidence of impact is hard. Not sure where to start.
- Outcomes attendees are measuring today: Customer retention; New customer growth; Content output; Cost to business based on time lost; Quantifying FTE lost per annum.
- Impact vs. ROI: ROI answers “how do we get additional resources?” Impact answers “how have we contributed to a measurable increase in something the business cares about?”
- Training is a reactive process. Must have a clear five-year plan. Adjust and iterate accordingly.
- To understand what you should be measuring, you have to get feedback from customers.
- There are deep challenges to getting data: Business has a hard time articulating what they want to impact. Data is not readily available.
- Disparate systems. Data is everywhere. Different data stores. HR silo data.
- Be careful of vanity metrics.
If any of these topics are of interest to you, we have experience working with L&D leaders who are solving the problem of designing for and measuring business impact. Get in touch.
The Expo Hall
In the Expo hall, we found some old friends and made quite a few new ones. Some of the new Exhibitors that really got our attention included Skillaware (performance support solution and learning analytics), Conductrr (scenario-based learning and storytelling), and Weejee Learning (eLearning experience provider—game mechanics for learning, simulations, content development, xAPI expertise). Plus, some of our old favorites: RISC LMS (first CMI-5 prototype implementation), LearnUpon LMS (leading the way in connecting the LMS to the LRS), Hapyak (interactive video learning), and Trek (the experiential learning platform from Cognitive Advisors).
Special shout out to our fellow LRS peeps at RISC, Learning Locker, and Watershed, who represented at the Expo Hall…and then Roary from HT2 got kidnapped, then shenanigans followed. Oh Roary.
This year, there were eight sessions (we knew of) related to the Experience API, which shows a growing interest from the community to learn more about the affordances, challenges, and possibilities of leveraging the xAPI to achieve L&D’s program and organizational performance goals.
We presented how the University of Ottawa Faculty of Medicine is using the xAPI and LRS to improve patient safety. Tune in to our blog, full case study coming soon!
DemoFest! (say it fast 10 times, very fast)
DemoFest was awesome and hectic at the same time. I think we demo’d our solution to over 400 people, and it felt like a few hours passed in minutes. You can see all the winners by category. Our demo was “Experience Paths” analysis, which showed how people experience CRM-tool training in detail, analyzing the flow they took through each step of the simulation (designed in Storyline by Sara Walters). We were able to show where people backtrack, skip, and retrace their steps, what optional resources they use, and the comparative paths people take through the content vs. the entire cohort. We demonstrated how to make it easy to find where content could be improved, and how to design, intervene, and augment specific people’s experiences.
Sara was a great partner for this project with an exceptional understanding of instructional design, interpreting data, and turning them into insights. We look forward to working with her again in the near future. Overall, we loved the pace and were super happy when, after a five-minute demo, people were already deriving their own insights about the content and how people experienced the simulation. It was very rewarding and confirmed the hard work we had done.
One final note: The next time you are at the MGM Grand, you must have dinner at Crush. It was fabulous. A great end to a highly educational and productive conference. Thank you eLearning Guild! We can’t wait until next year.