xAPI implementing bookmarking and progress
by Brett Andrew 16/7/2024
What we were pretty sure we could complete in a few hours, as usual when it comes to difficult to read and non-existintent documentation and non-public 3rd party systems, we had to throw rocks at the xAPI from Articulate until something came back that looked like what we were after.
Last year we wrote a .net function that takes an xAPI course and uploads it to our server and unzips it and we can then serve each page relatively from our APIs. This year a client asked us to capture bookmarking and progress for xAPI courses, easy right. We could capture the data coming out of courses already, so surely its easy just to set this data when a user resumes..... nope. This is what we found out by watching the APIs
Technology: Courses are created in Articulate Rise 360, you can get a 30 day trial and you can grab one of their many templates out of the box.
Backend: .Net Core 8
Frontend: React, we are using this 3rd party xAPIWrapper to provide the tools to launch our course.
When you pass in an endpoint to an xAPI course, it will automatically send GET/PUT and POST statements at that endpoint. Some GET/PUT and POST statements you can use to capture lots of data, the GET calls, is where the course tries to read in three values to see if this course should start at a certain point. You can set the bookmark and the data back into the report (still trying to get progress to automate but at least the bookmark is being set and the majority of the menu items show progressed).
When a course first loads, it asks the LMS for three thing in the GET statement : suspend_data, bookmark and cumulative_time.
suspend_data : data that is compressed and only the course itself can read, send it back as it was sent to you. Completing our research, if you send this back exactly how you receive it, it sets up the progress on the page, including header and side menu progress.
bookmark : this is literally the anchor on where to go e.g. #course2 as a user progresses, this bookmark is not sent as part of the verb, it is in a put statement. the value is like this #/lessons/Cm5O
cumulative_time : this is the milliseconds the user has been taking this course, you can put this value back in when it asks for it
This is the process is as follows:
We create an LRS in memory and only save for some events, which seems to be efficient enough.
The xAPI course itself will first call GET GetActivity(course), this is where you send back the right data so it resumes from where it left off. See the function below on how to capture the data.
The xAPI course itself will then issue PUT SetActivity(course) statements, in here is where we capture duration, storage data and the bookmark, we store this is memory, and when we trigger a bookmark persist to database. So if a user resumes, the bookmark and the storage data is what allows the user to resume progress from where they left off.
The POST PostStatement(course, query, object) is called when a verb event occurs like passed, failed, completed, started. Here we are only capturing when a course is completed and storing that record, in our system we monitor the value completed to go and mark this activity as complete fo the user if it is autocompleted.
You should be able to this out for yourself now, here is our source code from the moment we started seeing the course being marked as complete and the ability to restart the course. It won't match your systems code exactly but there is enough detail here for you to be able to see how we capture these in .net.
/// <summary>
/// use this to convert xAPI statements into simple to use data in the model xAPIStatement
/// </summary>
/// <param name="jsonData"></param>
/// <param name="Course"></param>
/// <returns></returns>
private UserActivityxAPICourseData ProcessStatement(ref IHelper iHelper, string jsonData, string Course)
{
dynamic data = Newtonsoft.Json.JsonConvert.DeserializeObject(jsonData);
UserActivityxAPICourseData statement = new UserActivityxAPICourseData();
// See if there was one already stored, continue to edit it
// User session level, use course to differentiate between multiple courses
var objecttest = iHelper.GetSetting("xAPIStatement" + Course);
if (objecttest != null && objecttest != "")
{
// Populate the existing 'statement' object with values from the JSON string
Newtonsoft.Json.JsonConvert.PopulateObject(objecttest, statement);
}
else
{
///////////////////////////////////////////////////////////////////
statement.Course = Course;
if (data != null)
{
if (data.context != null && data.context.registration != null)
{
//statement.ItemID = data.context.registration;
}
if (data.verb != null && data.verb.id != null)
{
statement.Verb = data.verb.id;
}
if (data.actor != null && data.actor.name != null)
{
statement.Username = data.actor.name;
}
//Result
if (data.result != null)
{
if (data.result.completion != null)
{
statement.Completed = data.result.completion;
}
// Parse duration
if (data.result.duration != null)
{
try
{
var duration = System.Xml.XmlConvert.ToTimeSpan((string)data.result.duration);
if (duration > TimeSpan.Zero)
{
int totalMinutes = (int)duration.TotalMinutes;
int totalSeconds = duration.Seconds;
statement.DurationMinutes = totalMinutes;
statement.DurationSeconds = totalSeconds;
}
}
catch { }
}
}
if (data["object"] != null)
{
var Activity = "";
if (data["object"].id != null)
{
Activity += data["object"].id;
}
if (data["object"].objectType != null)
{
Activity += data["object"].objectType;
}
statement.Activity = Activity;
statement.ActivityData = Newtonsoft.Json.JsonConvert.SerializeObject(data["object"]);
}
}
// Check if progress exists
if (data.result != null && data.result.extensions != null && (data.result.extensions["http://w3id.org/xapi/cmi5/result/extensions/progress"] != null || data.result.extensions["https://w3id.org/xapi/cmi5/result/extensions/progress"] != null))
{
if (data.result.extensions["http://w3id.org/xapi/cmi5/result/extensions/progress"] != null)
{
statement.Progress = data.result.extensions["http://w3id.org/xapi/cmi5/result/extensions/progress"].ToString();
}
if (data.result.extensions["https://w3id.org/xapi/cmi5/result/extensions/progress"] != null)
{
statement.Progress = data.result.extensions["https://w3id.org/xapi/cmi5/result/extensions/progress"].ToString();
}
}
iHelper.SetSetting("xAPIStatement" + Course, Newtonsoft.Json.JsonConvert.SerializeObject(statement));
return statement;
}
//XAPI Course
[Produces("application/json")]
[HttpGet("[action]")]
public basicresponse GetxAPISessionInfo(string Course)
{
basicresponse response = new basicresponse();
response.success = true;
response.message = "";
IHelper iHelper = (IHelper)HttpContext.RequestServices.GetService(typeof(IHelper));
UserActivityxAPICourseData statement = new UserActivityxAPICourseData();
////////////////////////////compare with session////////////////////
///
//see if there was one already stored, continue to edit it
//user session level, use course to differentiate between multiple courses
var objecttest = iHelper.GetSetting("xAPIStatement" + Course);
if (objecttest != null && objecttest != "")
{
response.success = true;
response.dataobject = Newtonsoft.Json.JsonConvert.DeserializeObject<UserActivityxAPICourseData>(objecttest);
}
else
{
response.success = true;
response.message = "No Data found";
}
///////////////////////////////////////////////////////////////////
return response;
}
/// <summary>
/// This is the function that the xAPI course will call and pass in details of started, completed and passed and failed not always in one go, use verb to dissect.
/// We process each request in order.
/// This is a POST Statement, where the P
/// </summary>
/// <param name="course"></param>
/// <param name="query"></param>
/// <param name="statement"></param>
/// <returns></returns>
[HttpPut("XAPI/{course}/{*query}")]
[HttpPost("XAPI/{course}/{*query}")]
public IActionResult PostStatement(string course, string query, [FromBody] object statement)
{
// Your code to handle the statement goes here
IHelper iHelper = (IHelper)ControllerContext.HttpContext.RequestServices.GetService(typeof(IHelper));
IUser iUser = (IUser)ControllerContext.HttpContext.RequestServices.GetService(typeof(IUser));
UserActivityxAPICourseData Test = ProcessStatement(ref iHelper, Newtonsoft.Json.JsonConvert.SerializeObject(statement), course);
if (Test != null)
{
if (Test.Verb != null && Test.Verb.Length > 0 && Test.Username != null && Test.Username.Length > 0)
{
BasicUser ThisUser = iUser.GetUserBasic(Test.Username);
if (ThisUser.Status == "Active" && ThisUser.isDeleted != true)
{
bool savetodisk = false;
switch (Test.Verb)
{
case "http://adlnet.gov/expapi/verbs/attempted":
iHelper.DebugLog("xAPI", "Attempted: " + Test.Verb);
break;
case "http://adlnet.gov/expapi/verbs/launched":
iHelper.DebugLog("xAPI", "Launched: " + Test.Verb);
savetodisk = true;
break;
case "http://adlnet.gov/expapi/verbs/experienced":
iHelper.DebugLog("xAPI", "Experienced: " + Test.Verb);
break;
case "http://adlnet.gov/expapi/verbs/progressed":
iHelper.DebugLog("xAPI", "Progressed: " + Test.Progress + ". Completed:" + Test.Completed + ". Test:" + Newtonsoft.Json.JsonConvert.SerializeObject(Test) + " Raw Statement: " + Newtonsoft.Json.JsonConvert.SerializeObject(statement));
if (Validation.GetInteger(Test.Progress, 0) > 0)
{
iHelper.SetSetting("Progress", Newtonsoft.Json.JsonConvert.SerializeObject(statement));
}
savetodisk = true;
break;
case "http://adlnet.gov/expapi/verbs/completed":
case "http://adlnet.gov/expapi/verbs/passed":
iHelper.DebugLog("xAPI", "Passed: " + Test.Completed + ". " + Test.Verb);
Test.Completed = true; //make sure we update this
savetodisk = true;
break;
case "http://adlnet.gov/expapi/verbs/failed":
iHelper.DebugLog("xAPI", "Failed: " + Test.Completed + ". " + Test.Verb);
savetodisk = true;
break;
case "http://adlnet.gov/expapi/verbs/answered":
iHelper.DebugLog("xAPI", "Answered: " + Test.Activity + ". " + Test.Verb);
break;
case "http://adlnet.gov/expapi/verbs/exited":
iHelper.DebugLog("xAPI", "Exited: " + Test.Activity + ". " + Test.Verb);
savetodisk = true;
break;
default:
iHelper.DebugLog("xAPI", "Not captured? " + Test.Verb + ". JSON: " + Newtonsoft.Json.JsonConvert.SerializeObject(Test) + ". Raw data: " + query + " : " + Newtonsoft.Json.JsonConvert.SerializeObject(statement));
break;
}
if (savetodisk)
{
//save course session to disk here!
IEducation iEducation = (IEducation)ControllerContext.HttpContext.RequestServices.GetService(typeof(IEducation));
iHelper.DebugLog("xAPI", "Saved (statement) " + Newtonsoft.Json.JsonConvert.SerializeObject(Test));
iEducation.SaveUserActivityxAPICourseData(Test);
}
}
else
{
//user not active
iHelper.DebugLog("xAPI", "User not active or deleted. Activity not captured. " + Test.Verb);
}
}
iHelper.DebugLog("xAPI", "Set Statement: " + Test.Verb);
}
//record the progress, bookmark and activity data against this user, if important update the master record.
return Ok();
}
/// <summary>
/// The xAPI course will send data to this endpoint constantly when it is running
/// We dont want to log every call to the database, instead update session memory and trigger database updates
/// only after specific events such bookmark.
/// </summary>
/// <param name="course"></param>
/// <returns></returns>
[HttpPut("XAPI/{course}/activities/state")]
public async Task<IActionResult> SetActivity(string course)
{
// Access query parameters
var stateId = HttpContext.Request.Query["stateId"].ToString();
var activityId = HttpContext.Request.Query["activityId"].ToString();
var agent = HttpContext.Request.Query["agent"].ToString();
var registration = HttpContext.Request.Query["registration"].ToString();
var version = HttpContext.Request.Query["version"].ToString();
// Read the body of the request
string requestBody;
using (var reader = new StreamReader(Request.Body))
{
requestBody = await reader.ReadToEndAsync();
}
// Your code to handle the statement goes here
IHelper iHelper = (IHelper)ControllerContext.HttpContext.RequestServices.GetService(typeof(IHelper));
//see if we have the data already
UserActivityxAPICourseData thisStatement = new UserActivityxAPICourseData();
basicresponse TestSession = this.GetxAPISessionInfo(course);
//we have this in session memory, relax.
if (TestSession.success == true && TestSession.dataobject != null)
{
thisStatement = (UserActivityxAPICourseData)TestSession.dataobject;
}
bool savedata = false;
bool updatesession = false;
switch (stateId)
{
case "bookmark":
thisStatement.BookmarkData = requestBody;
savedata = true;
updatesession = true;
break;
case "suspend_data":
// Retrieve the progress from the data store
thisStatement.SuspendData = requestBody;
updatesession = true;
savedata = true;
break;
case "cumulative_time": // Retrieve the progress from the data store
int milliseconds = Validation.GetInteger(requestBody, 0);
int totalSeconds = milliseconds / 1000;
thisStatement.DurationMinutes = totalSeconds / 60; //calculates the total minutes.
thisStatement.DurationSeconds = totalSeconds % 60; //calculates the remaining seconds.
updatesession = true;
break;
default:
//iHelper.DebugLog("xAPI", "Course: " + course + ". Get Activity Data sent x " + stateId + " : " + activityId + " : " + agent + " : " + registration + " : " + version);
break;
}
if (updatesession)
{
iHelper.SetSetting("xAPIStatement" + course, Newtonsoft.Json.JsonConvert.SerializeObject(thisStatement));
}
if (savedata)
{
iHelper.DebugLog("xAPI", "Saved (activity) " + Newtonsoft.Json.JsonConvert.SerializeObject(thisStatement));
IEducation iEducation = (IEducation)ControllerContext.HttpContext.RequestServices.GetService(typeof(IEducation));
iEducation.SaveUserActivityxAPICourseData(thisStatement);
}
iHelper.DebugLog("xAPI", "Set Activity: " + stateId);
return Ok();
}
[Produces("application/json")]
[HttpGet("XAPI/{course}/activities/state")]
public object GetActivity(string course)
{
// Access query parameters
var stateId = HttpContext.Request.Query["stateId"].ToString();
var activityId = HttpContext.Request.Query["activityId"].ToString();
var agent = HttpContext.Request.Query["agent"].ToString();
var registration = HttpContext.Request.Query["registration"].ToString();
var version = HttpContext.Request.Query["version"].ToString();
// Your code to handle the statement goes here
IHelper iHelper = (IHelper)ControllerContext.HttpContext.RequestServices.GetService(typeof(IHelper));
//see if we have the data already
UserActivityxAPICourseData thisStatement = new UserActivityxAPICourseData();
basicresponse TestSession = this.GetxAPISessionInfo(course);
//we have this in session memory, relax.
if (TestSession.success==true && TestSession.dataobject != null)
{
thisStatement = (UserActivityxAPICourseData)TestSession.dataobject;
}
switch (stateId)
{
case "bookmark":
if (thisStatement.BookmarkData != null)
{
iHelper.DebugLog("xAPI", "Get Suspend Data. Data returned : " + thisStatement.BookmarkData);
return thisStatement.BookmarkData;
}
break;
case "suspend_data":
// Retrieve the progress from the data store
if (thisStatement.SuspendData != null) {
iHelper.DebugLog("xAPI", "Get Suspend Data. Data returned : " + Newtonsoft.Json.JsonConvert.SerializeObject(thisStatement.SuspendData));
return thisStatement.SuspendData;
}
break;
case "cumulative_time": // Retrieve the progress from the data store
int CumulativeTime = 0;
if (thisStatement.DurationMinutes!=null && thisStatement.DurationMinutes > 0)
{
CumulativeTime += Validation.GetInteger(thisStatement.DurationMinutes,0) * 60 * 1000;//mins to milliseconds
}
if (thisStatement.DurationSeconds != null && thisStatement.DurationSeconds > 0)
{
CumulativeTime += Validation.GetInteger(thisStatement.DurationSeconds, 0) * 1000;//seconds to milliseconds
}
iHelper.DebugLog("xAPI", "Get Cumalative Time Data. Data returned : " + CumulativeTime);
return CumulativeTime;//cumulative time in ms
break;
default:
iHelper.DebugLog("xAPI", "Course: " + course + ". Not used " + stateId + " : " + activityId + " : " + agent + " : " + registration + " : " + version);
break;
}
return Ok();
}public class UserActivityxAPICourseData
{
public string tablename = "[Education_UserActivityxAPICourseData]";
public string type = "UserActivityxAPICourseData";
public int ItemID { get; set; }
public int? UserActivityID { get; set; }
public bool? isDeleted { get; set; }
public string Verb { get; set; }
public string Username { get; set; }
public string Course { get; set; }
public bool? Completed { get; set; }
public int? DurationMinutes { get; set; }
public int? DurationSeconds { get; set; }
public string Progress { get; set; }
public string Activity { get; set; }
public string ActivityData { get; set; }
public string BookmarkData { get; set; }
public string SuspendData { get; set; }
public DateTime? CreatedDate { get; set; }
public int? CreatedBy { get; set; }
public DateTime? LastModifiedDate { get; set; }
public int? LastModifiedBy { get; set; }
public Guid? ItemGuid { get; set; }
}Mid-size business and building an LMS?
If you are building an LMS because its too expensive, think again, go see mition.com - we built it! Mition comes with an LMS that works beautifully with xAPI courses for free with a standard monthly subscription. It also includes 18 other modules like website, members, invoicing, events. It is API first and mobile responsive. Maybe you don't need to build an LMS?
Powered by mition